llama-gpt
llm-mlc
| llama-gpt | llm-mlc | |
|---|---|---|
| 7 | 3 | |
| 10,402 | 172 | |
| 2.2% | - | |
| 7.4 | 5.1 | |
| 30 days ago | about 2 months ago | |
| TypeScript | Python | |
| MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
llama-gpt
- FLaNK Stack Weekly 28 August 2023
-
Continue with LocalAI: An alternative to GitHub's Copilot that runs locally
wodner if you can pair with https://github.com/getumbrel/llama-gpt
-
Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2
I put up a draft PR to demo how to run it on a GPU: https://github.com/getumbrel/llama-gpt/pull/11
It breaks other things like model downloading, but once I got it to a working state for myself, I figured why not put it up there in case its useful. If I have time, I'll try to rework it a little bit with more parameters and less dockerfile repetition to fit the main project better.
- llama-gpt - A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device
llm-mlc
-
LLM now provides tools for working with embeddings
I'm still iterating on that. Plugins get complete control over the prompts, so they can handle the various weirdnesses of them. Here's some relevant code:
https://github.com/simonw/llm-gpt4all/blob/0046e2bf5d0a9c369...
https://github.com/simonw/llm-mlc/blob/b05eec9ba008e700ecc42...
https://github.com/simonw/llm-llama-cpp/blob/29ee8d239f5cfbf...
I'm not completely happy with this yet. Part of the problem is that different models on the same architecture may have completely different prompting styles.
I expect I'll eventually evolve the plugins to allow them to be configured in an easier and more flexible way. Ideally I'd like you to be able to run new models on existing architectures using an existing plugin.
-
Show HN: LlamaGPT – Self-hosted, offline, private AI chatbot, powered by Llama 2
What is the advantage of this versus running something like https://github.com/simonw/llm , which also gives you options to e.g. use https://github.com/simonw/llm-mlc for accelerated inference?
-
Show HN: LLMs can generate valid JSON 100% of the time
I'm quite impressed with Llama 2 13B - the more time I spend with it the more I think it might be genuinely useful for more than just playing around with local LLMs.
I'm using the MLC version (since that works with a GPU on my M2 Mac) via my https://github.com/simonw/llm-mlc plugin.
What are some alternatives?
ollama - Get up and running with Llama 3, Mistral, Gemma, and other large language models.
llm-gpt4all - Plugin for LLM adding support for the GPT4All collection of models
serge - A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.
can-ai-code - Self-evaluating interview for AI coders
gpt4all - gpt4all: run open-source LLMs anywhere
outlines - Structured Text Generation
trulens - Evaluation and Tracking for LLM Experiments
TypeChat - TypeChat is a library that makes it easy to build natural language interfaces using types.
seamless_communication - Foundational Models for State-of-the-Art Speech and Text Translation
ad-llama - Structured inference with Llama 2 in your browser
prettymapp - 🖼️ Create beautiful maps from OpenStreetMap data in a streamlit webapp
llama.cpp - LLM inference in C/C++