promptfoo
cloudseeder
promptfoo | cloudseeder | |
---|---|---|
5 | 6 | |
328 | 65 | |
- | - | |
10.0 | 8.8 | |
11 months ago | 3 days ago | |
TypeScript | Shell | |
MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
promptfoo
-
Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
Jumping in because I'm a big believer in (1) local LLMs, and (2) evals specific to individual use cases.
[0] https://github.com/typpo/promptfoo
- Meta Llama 3
-
Launch HN: Talc AI (YC S23) – Test Sets for AI
Congrats on the launch!
I've been interested in automatic testset generation because I find that the chore of writing tests is one of the reasons people shy away from evals. Recently landed eval testset generation for promptfoo (https://github.com/typpo/promptfoo), but it is non-RAG so more simplistic than your implementation.
Was also eyeballing this paper https://arxiv.org/abs/2401.03038, which outlines a method for generating asserts from prompt version history that may also be useful for these eval tools.
-
GPT-Prompt-Engineer
Thanks for the promptfoo mention. For anyone else who might prefer deterministic, programmatic evaluation of LLM outputs, I've been building promptfoo: https://github.com/typpo/promptfoo
Example asserts include basic string checks, regex, is-json, cosine similarity, etc.
cloudseeder
-
SIMPle TerminAL – Read Online Terminal Output in Fyne Golang
Submission: This is a simple terminal for 'output/read-only' usage in a Fyne GoLang app. You can basically run a command and it will feel like its running in a terminal and the output will go to said terminal. This strips out all ansi, escape and control codes.
It's part of Cloud Seeder [1] which we wrote open source and in a way so that everyone can use the parts [2][3] easily.
[1] https://github.com/ipv6rslimited/cloudseeder - 1 click host with hands free upgrades your favorite self hosted appliance
[2] https://github.com/ipv6rslimited/configurator - make a config screen from a json
[3] https://github.com/ipv6rslimited/tray - make a system tray from a json
- Cloud Seeder: Self Host with a Click on Your Home PC – Golang
-
Ollama v0.1.33 with Llama 3, Phi 3, and Qwen 110B
Ollama is really organized - it relies on llama but the UX and organization it provides makes it legit. We recently made a one-click wizard to run Open WebUI and Ollama together, self hosted and remotely accessible but locally hosted [1]
[1] https://github.com/ipv6rslimited/cloudseeder
- Open Source One Click Server Appliances on Your Self Hosted Home Computer
-
Show HN: We Built Cloud Seeder – One Click Self-Hosting with Auto Updates
Salutations HN,
We're excited to introduce Cloud Seeder, a tool designed to make self-hosting easy.
Problem Identification:
We realized there were many discussions about self-hosting but they never transitioned into action. After surveying potential users, two major obstacles emerged:
1. Lack of an external IP address.
2. The complexity of setting up and maintaining a server.
Our Solution - Cloud Seeder:
To make self-hosting straightforward, we developed Cloud Seeder. It's a platform that supports one-click deployment for over 20+ different appliances (and growing upon request), ranging from Mastodon (a federated social network that communicates with Meta's Threads) to Nextcloud (a robust, self-hosted alternative to major collaboration platforms).
Cloud Seeder is fully open source. The code is available on GitHub [1].
We believe Cloud Seeder will play a crucial role in fostering a more decentralized internet.
We hope you'll give this a try and start self hosting!
We also include a Configurator [2] and Tray [3] on github which lets you dynamically (from just a json) create dialog boxes and system trays. We use this in our application and thought it might be useful to people who don't want to spend time on GUI but would love to reach people with such UX.
[1] https://github.com/ipv6rslimited/cloudseeder
[2] https://github.com/ipv6rslimited/configurator
[3] https://github.com/ipv6rslimited/tray
What are some alternatives?
rebuff - LLM Prompt Injection Detector
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
gpt-engineer - Specify what you want it to build, the AI asks for clarification, and then builds it.
TensorRT-LLM - TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT-LLM also contains components to create Python and C++ runtimes that execute those TensorRT engines.
ChainForge - An open-source visual programming environment for battle-testing prompts to LLMs.
mlx - MLX: An array framework for Apple silicon
gateway - A Blazing Fast AI Gateway. Route to 100+ LLMs with 1 fast & friendly API.
configurator - Configurator provides a GUI interface for your application, dynamically generated from a JSON file.
shap-e - Generate 3D objects conditioned on text or images
tray - Generate a system tray menu from a JSON file.
sugarcane-ai - npm like package ecosystem for Prompts 🤖
llama-chat - Implements a simple REPL chat with a locally running instance of Ollama.