Show HN: Python Monitoring for LLMs, OpenAI, Inference, GPUs

• graphsignal-python

  30 201 8.1 Python

  Graphsignal Tracer for Python

  

We've built it for apps that use LLMs and other ML models. The lightweight Python agent autoinstruments OpenAI, LangChain, Banana, and other APIs and frameworks. Basically by adding one line of code you'll be able to monitor and analyze latency, errors, compute and costs. Profiling using CProfile, PyTorch Kineto or Yappi can be enabled if code-level statistics are necessary.

Here is a short demo screencast for a LangChain/OpenAI app: https://www.loom.com/share/17ba8aff32b74d74b7ba7f5357ed9250

In terms of data privacy, we only send metadata and statistics to https://graphsignal.com. So no raw data, such as prompts or images leave your app.

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

Suggest a related project