Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more β
Top 14 TypeScript llmops Projects
-
promptfoo
Test your prompts, models, and RAGs. Catch regressions and improve prompt quality. LLM evals for OpenAI, Azure, Anthropic, Gemini, Mistral, Llama, Bedrock, Ollama, and other local & private models with CI/CD integration.
-
SurveyJS
Open-Source JSON Form Builder to Create Dynamic Forms Right in Your App. With SurveyJS form UI libraries, you can build and style forms in a fully-integrated drag & drop form builder, render them in your JS app, and store form submission data in any backend, inc. PHP, ASP.NET Core, and Node.js.
-
pezzo
πΉοΈ Open-source, developer-first LLMOps platform designed to streamline prompt design, version management, instant delivery, collaboration, troubleshooting, observability and more.
-
llm.report
π llm.report is an open-source logging and analytics platform for OpenAI: Log your ChatGPT API requests, analyze costs, and improve your prompts.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
GPTRouter
Smoothly Manage Multiple LLMs (OpenAI, Anthropic, Azure) and Image Models (Dall-E, SDXL), Speed Up Responses, and Ensure Non-Stop Reliability.
-
langtrace
Langtrace π is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. ππ»π
-
empirical
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application (by empirical-run)
-
craftgen
Integrating AI into every workflow with our open-source, no-code platform, powered by the actor model for dynamic, graph-based solutions.
-
openllmetry-js
Sister project to OpenLLMetry, but in Typescript. Open-source observability for your LLM application, based on OpenTelemetry
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Great to know this!
I have come across Portkey's Open-source AI Gateway which kind of does the same.
https://github.com/portkey-ai/gateway
It looks like with more LLM adoption, resiliency and cost related aspects take off sooner than expected unlike other technological trends in the past.
I'm also thinking that there is a chance if something like this could help build a better RAG pipeline or evals for the GenAI App. Because end of the day you want to reduce hallucinations but still get good generative responses.
Project mention: Ask HN: How does deploying a fine-tuned model work | news.ycombinator.com | 2024-04-23- Fireworks: $0.20
If you're looking for an end-to-end flow that will help you gather the training data, validate it, run the fine tune and then define evaluations, you could also check out my company, OpenPipe (https://openpipe.ai/). In addition to hosting your model, we help you organize your training data, relabel if necessary, define evaluations on the finished fine-tune, and monitor its performance in production. Our inference prices are higher than the above providers, but once you're happy with your model you can always export your weights and host them on one of the above!
Project mention: Top Open Source Prompt Engineering Guides & Toolsπ§ποΈπ | dev.to | 2024-05-02Pezzo is a cloud-native LLMOps platform. You can observe and monitor your AI operations, troubleshoot issues, collaborate and manage your prompts in one place, and instantly deliver AI changes.
Project mention: Rebuff: Detect and protect against prompt injection attacks in LLMs | news.ycombinator.com | 2023-07-05
Project mention: Show HN: Monitoring your OpenAI usage without a third party | news.ycombinator.com | 2023-10-04
Project mention: Show HN: GPT Router β Open-Source API Gateway for LLMs (OpenAI, Anthropic, etc.) | news.ycombinator.com | 2023-12-22
Lmstudio (that they linked) is definitely not open source, and doesn't even offer a pricing model for business use.
Llmstudio is, but I suspect that was a typo in their comment. https://github.com/TensorOpsAI/LLMStudio
- Typescript - https://github.com/dosco/llm-client/
One specific feature I would like to see is to expose the prompt templates as config and give greater control to the developers in incorporating our βversionsβ of templates for CoT, Fewshots etc. This will also help with the adoption of DSPy to languages other than English.
Finally, I was able to understand what DSPy is adding to its prompts and what sort of optimizations it is doing by using Langtrace, an open source tracing tool i am building that can be setup with just 2 lines of code.
https://github.com/Scale3-Labs/langtrace
Project mention: Show HN: Empirical β test framework for JavaScript developers building with LLMs | news.ycombinator.com | 2024-05-22
Check it out craftgen.ai or https://github.com/craftgen/craftgen
Project mention: Show HN: Made Open Source NPM like package ecosytem for reusable Prompts | news.ycombinator.com | 2023-12-15
TypeScript llmops related posts
-
Iterate on LLMs Faster
-
OpenAI: Improvements to the fine-tuning API and expanding our cus
-
Doku: Open-source LLM observability platform
-
Show HN: Open-Source LLM Observability and Export to Grafana, Datadog etc.
-
AI Infrastructure Landscape
-
Adding a streaming run function to the Assistants API
-
Launch HN: Talc AI (YC S23) β Test Sets for AI
-
A note from our sponsor - InfluxDB
www.influxdata.com | 1 Jun 2024
Index
What are some of the best open-source llmops projects in TypeScript? This list will help you:
Project | Stars | |
---|---|---|
1 | gateway | 4,830 |
2 | promptfoo | 3,100 |
3 | OpenPipe | 2,401 |
4 | pezzo | 1,872 |
5 | rebuff | 958 |
6 | llm.report | 847 |
7 | app | 449 |
8 | GPTRouter | 390 |
9 | LLMstudio | 215 |
10 | langtrace | 152 |
11 | empirical | 137 |
12 | craftgen | 84 |
13 | sugarcane-ai | 46 |
14 | openllmetry-js | 33 |
Sponsored