litellm
ollama-webui
| litellm | ollama-webui | |
|---|---|---|
| 29 | 14 | |
| 9,506 | 5,789 | |
| 13.5% | - | |
| 10.0 | 9.8 | |
| 6 days ago | 4 months ago | |
| Python | Svelte | |
| GNU General Public License v3.0 or later | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
litellm
-
RAG with llama.cpp and external API services
Next, we'll show how an Embeddings database can integrate with external API services via LiteLLM .
-
Anthropic launches Tool Use (function calling)
There are a few libs that already abstract this away, for example:
- https://github.com/BerriAI/litellm
- https://jxnl.github.io/instructor/
- langchain
It's not hard for me to imagine a future where there is something like the CNCF for AI models, tools, and infra.
-
Ask HN: Python Meta-Client for OpenAI, Anthropic, Gemini LLM and other API-s?
Hey, are you just looking for litellm - https://github.com/BerriAI/litellm
context - i'm the repo maintainer
-
Voxos.ai – An Open-Source Desktop Voice Assistant
It should be possible using LiteLLM and a patch or a proxy.
https://github.com/BerriAI/litellm
- Show HN: Talk to any ArXiv paper just by changing the URL
-
Integrate LLM Frameworks
This article will demonstrate how txtai can integrate with llama.cpp, LiteLLM and custom generation methods. For custom generation, we'll show how to run inference with a Mamba model.
-
Is there any open source app to load a model and expose API like OpenAI?
I use this with ollama and works perfectly https://github.com/BerriAI/litellm
-
OpenAI Switch Kit: Swap OpenAI with any open-source model
Another abstraction layer library is: https://github.com/BerriAI/litellm
For me the killer feature of a library like this would be if it implemented function calling. Even if it was for a very restricted grammar - like the traditional ReAct prompt:
Solve a question answering task with interleaving Thought, Action, Observation usteps. Thought can reason about the current situation, and Action can be three types: - LibreChat
- LM Studio – Discover, download, and run local LLMs
ollama-webui
- Run copilot locally
- Show HN: I made an app to use local AI as daily driver
-
Exploring Podman: A More Secure Docker Alternative
I'm a podman beginner, trying to install ollama-webui(1) using Podman on M2 MBA.
I started up Podman Desktop, and did a terminal command "docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main" based on Github's instructions, but it gave a error message something about "host".
Do you know what is the problem and how do I overcome this?
If I run the above command using Docker Desktop, it runs and installs Ollama-WebUI just fine.
(1) https://github.com/ollama-webui/ollama-webui ("Installing with Docker")
Thank you.
- Mixtral: Mixture of Experts
- Run Mistral 7B on M1 Mac
-
OpenAI's New Strategy
I set up Ollama in a docker container (really easy) and I use the Ollama web-ui here, that's very much like ChatGPT, and I have that in a container as well.
- chatgpt alternative
- How can I configure the same settings I get in the llama.cpp webUI with Ollama on macOS?
-
LibreChat
The primary use case here seems to be that it might be possible to use this tool to spend <$20/mo for the same feature set as ChatGPT+. It does not currently make any effort to support locally-hosted open source models, which is what I would have assumed from its name.
If you're interested in a fully Libre LLM stack, I've had fun lately with ollama [0] and ollama-webui [1]. It was pretty trivial to take ollama-webui's docker-compose file and set up a locally-running chat server with Mistral 7B. Trying out different models and prompts was likewise very easy to get started with.
Mistral isn't anything like as good as GPT-4, but it's Apache licensed and fully local, which meets my definition of Libre. I'll continue to use both while the FOSS stacks catch up, but it's fun to keep up with the progress on the open source stuff as tooling develops.
[0] https://github.com/jmorganca/ollama
[1] https://github.com/ollama-webui/ollama-webui
- LM Studio – Discover, download, and run local LLMs
What are some alternatives?
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
code-llama-for-vscode - Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
FastChat - An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.
LibreChat - Enhanced ChatGPT Clone: Features OpenAI, Assistants API, Azure, Groq, GPT-4 Vision, Mistral, Bing, Anthropic, OpenRouter, Vertex AI, Gemini, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. More features in development
LocalAI - :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.
llamafile - Distribute and run LLMs with a single file.
dify - Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
open-webui - User-friendly WebUI for LLMs (Formerly Ollama WebUI)
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
stable-diffusion-webui - Stable Diffusion web UI
libsql - libSQL is a fork of SQLite that is both Open Source, and Open Contributions.
MindMac - Issue Tracker for elegant client for MacOS