promptfoo
github-pr-summary
| promptfoo | github-pr-summary | |
|---|---|---|
| 21 | 8 | |
| 3,100 | 203 | |
| 28.1% | 0.5% | |
| 9.9 | 7.7 | |
| 2 days ago | 10 months ago | |
| TypeScript | Rust | |
| MIT License | - |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
promptfoo
- Google CodeGemma: Open Code Models Based on Gemma [pdf]
- AI Infrastructure Landscape
- Promptfoo – Testing and Evaluation for LLMs
-
Show HN: Prompt-Engineering Tool: AI-to-AI Testing for LLM
Super interesting. We've been experimenting with [promptfoo](https://github.com/promptfoo/promptfoo) at my work, and this looks very similar.
- GitHub – promptfoo/promptfoo: Test your prompts
-
I asked 60 LLMs a set of 20 questions
In case anyone's interested in running their own benchmark across many LLMs, I've built a generic harness for this at https://github.com/promptfoo/promptfoo.
I encourage people considering LLM applications to test the models on their _own data and examples_ rather than extrapolating general benchmarks.
This library supports OpenAI, Anthropic, Google, Llama and Codellama, any model on Replicate, and any model on Ollama, etc. out of the box. As an example, I wrote up an example benchmark comparing GPT model censorship with Llama models here: https://promptfoo.dev/docs/guides/llama2-uncensored-benchmar.... Hope this helps someone.
- Ask HN: Prompt Manager for Developers
- DeepEval – Unit Testing for LLMs
- Show HN: Knit – A Better LLM Playground
- Show HN: CLI for testing and evaluating LLM outputs
github-pr-summary
-
Rust boosts LLM app development: Make a serverless Japanese Learning bot in mins
Excited to share my journey in creating a Telegram bot that learns Japanese and answers Q&As, all thanks to https://flows.network/ built in Rust
-
Ask HN: Prompt Manager for Developers
Maybe you can try https://flows.network/, which supports environment variables when creating an app. You can also manage the environment variables later with their UI. Click on https://flows.network/flow/createByTemplate/Telegram-ChatGPT, you can see you could set up system_prompt.
-
Launch HN: Sweep (YC S23) – A bot to create simple PRs in your codebase
I'm wondering what will happen if we let ChatGPT review these PRs created by ChatGPT.
Yes, We made a small tool to help developer review their PR. Seems a great supplement for Sweep AI.
Build your own PR review bot in 3 minutes here: https://github.com/flows-network/github-pr-summary
- Flows.network: Add eyes, ears, memory& hands to LLMs with serverless functions
- Create your own ChatGPT bot (in Rust) to review & summarize GitHub Pull Request.
- Create and deploy your own ChatGPT bot to review & summarize GitHub Pull Request.
- Rust API for OpenAI workflows. Enable ChatGPT on your own GitHub repo to review & summarize Pull Request.
- A ChatGPT Bot (in Rust) to Review and Summarize GitHub Pull Request
What are some alternatives?
shap-e - Generate 3D objects conditioned on text or images
sweep - Sweep: open-source AI-powered Software Developer for small features and bug fixes.
prompt-engineering - Tips and tricks for working with Large Language Models like OpenAI's GPT-4.
wasmedge-quickjs - A high-performance, secure, extensible, and OCI-complaint JavaScript runtime for WasmEdge.
WizardLM - Family of instruction-following LLMs powered by Evol-Instruct: WizardLM, WizardCoder and WizardMath
sweep-ai
chat-ui - Open source codebase powering the HuggingChat app
review-any-pr-with-chatgpt - GPT4-based code review for ANY public Pull Request on GitHub
litellm - Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
PromptPal - A Prompt Manager that focuses on On-Premise and developer experience.
ChainForge - An open-source visual programming environment for battle-testing prompts to LLMs.
landing-page