basaran
chatbot-ui
basaran | chatbot-ui | |
---|---|---|
22 | 63 | |
1,281 | 26,925 | |
- | - | |
10.0 | 9.4 | |
5 months ago | 5 days ago | |
Python | TypeScript | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
basaran
- OpenLLM
-
Langchain and self hosted LLaMA hosted API
What are the current best "no reinventing the wheel" approaches to have Langchain use an LLM through a locally hosted REST API, the likes of Oobabooga or hyperonym/basaran with streaming support for 4-bit GPTQ?
-
Run and create custom ChatGPT-like bots with OpenChat
Disclaimer: I am curating LLM-tools on github [1]
A few thoughts:
* allow for custom endpoint URLs, this way people can use open source LLMs with a fake openAI API backend like basaran[2] or llama-api-server[3]
* look into better embedding methods for info-retrieval like InstructorEmbeddings or Document Summary Index
* Don't use a single embedding per content item, use multiple to increase retrieval quality
1 https://github.com/underlines/awesome-marketing-datascience/...
2 https://github.com/hyperonym/basaran
3 https://github.com/iaalm/llama-api-server
-
1-Jun-2023
open-source alternative to the OpenAI text completion API (https://github.com/hyperonym/basaran)
- Introducing Basaran: self-hosted open-source alternative to the OpenAI text completion API
- Basaran is an open-source alternative to the OpenAI text completion API
-
Ask HN: What's the best self hosted/local alternative to GPT-4?
Guanaco-65B[0] using Basaran[1] for your OpenAI compatible API. You can use any ChatGPT front-end which lets you change the OpenAI endpoint URL.
[0] An fp4 finetune of LLaMA-30B by Tim Dettmers
[1] https://github.com/hyperonym/basaran
-
Are all the finetunes stupid?
For lm-eval, I think you'd either need to take GPTQ's inference script and shim it into a model: https://github.com/EleutherAI/lm-evaluation-harness/tree/master/lm_eval/models or you might be able to use a project like https://github.com/hyperonym/basaran and then you could use the gpt3 model...
-
Using the API in Node
There are also: - Basaran repo: "Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models". "...Compatibility with OpenAI API and client libraries..."; - llama-cpp-python repo: "Simple Python bindings for @ggerganov's llama.cpp library...". "...OpenAI-like API...".
-
Researcher looking for help with how to prepare a finetuning dataset for models like Bloomz and Cerebras-GPT
I want to start with a totally freely available model, so again, that excludes things like LLaMA where the weights are only available through a wait list. The two models that most get my attention and (I think, and hope) fit my criteria of open availability are Cerebras-GPT (13b) and Bloomz (7b). The tools to process and fine-tune that seem most feasible to me, from my limit knowledge, are xturing and basaran.
chatbot-ui
-
AI programming tools should be added to the Joel Test
One of the first things we did when GPT-4 became available was talk to our Azure rep and get access to the OpenAI models that they'd partnered with Microsoft to host in Azure. Now, we have our own private, not-datamined (so they claim, contractually) API endpoint and we use an OpenAI integration in VS Code[1] to connect to, allowing anyone in the company to use it to help them code.
I also spun up an internal chat UI[2] to replace ChatGPT so people can feel comfortable discussing proprietary data with the LLM endpoint.
The only thing that would make it more secure would be running inference engines internally, but I wouldn't have access to as good of models, and I'd need a _lot_ of hardware to match the speeds.
[1] - https://marketplace.visualstudio.com/items?itemName=AndrewBu...
[2] - https://github.com/mckaywrigley/chatbot-ui (legacy branch)
-
Ask HN: Has Anyone Trained a personal LLM using their personal notes?
[3] https://github.com/mckaywrigley/chatbot-ui
-
Show HN: I made an app to use local AI as daily driver
Thank you for the work.
Please take this in a nice way: I can't see why I would use this over ChatbotUI+Ollama https://github.com/mckaywrigley/chatbot-ui
Seem the only advantage is having it as MacOS native app and only real distinction is maybe fast import and search - I've yet to try that though.
ChatbotUI (and other similar stuff) are cross-platform, customizable, private, debuggable. I'm easily able to see what it's trying to do.
-
ChatGPT for Teams
You can make a privacy request for OpenAI to not train on your data here: https://privacy.openai.com/
Alternatively, you could also use your own UI/API token (API calls aren't trained on). Chatbot UI just got a major update released and has nice things like folders, and chat search: https://github.com/mckaywrigley/chatbot-ui
- Chatbot UI 2.0
- webui similar to chatgpt
-
They made ChatGPT worse at coding for some reason, and it’s caused me to look at alternative AI options
Also chatbotUI is great https://github.com/mckaywrigley/chatbot-ui it has a ui similar to chatgpt
-
Please Don't Ask If an Open Source Project Is Dead
> The comment I screenshotted is passive-aggressive at best, and there's no really good way to ask "is this repo dead" without being passive-aggressive. My day-to-day job that actually pays me a salary wouldn't ever provide a bulleted list of the reasons I suck, let alone a project I develop in my spare time.
There is nothing passive-aggressive about that comment. There is nothing problematic about it at all. Nobody's calling you slurs or making demands. I see one guy who might as well be a Mormon Boy Scout from Canada. "Is this repo dead" is not passive-aggressive, just ineloquent. Fuck my eyes until the jelly leaks out my ears if a courteous and professionally-written question constitutes "applying pressure and being rude" these days.
I don't know what a "bulleted list of the reasons [you] suck" has to do with anything (I don't see where anybody sent you one) but you're coming across as someone who invites people to your garage sale and then brandishes a shotgun and starts screaming when they set foot on your property.
> I’ve never seen any discussions or articles about whether it’s appropriate to ask if an open source repository is dead. Is there an implicit contract to actively maintain any open source software you publish? Are you obligated to provide free support if you hit a certain star amount on GitHub or ask for funding through GitHub Sponsorships/Patreon? After all, most permissive open source code licenses like the MIT License contain some variant of “the software is provided ‘as is’, without warranty of any kind.”
Here's an example of why everyone should ask if an open source project is dead:
https://github.com/mckaywrigley/chatbot-ui/issues
A number of issues complain about it leaking OpenAI keys. Nobody's figured out how, but it'd be nice to know if anybody's working on it, if it's worth submitting a PR, if it should be forked, if it's worth bothering with at all. This code is a massive liability in its current state. Its creator is absent. It warrants questions being asked about its future. Yeah, it's as-is software, but it's not an affront to your mother's virtue when someone asks if your shit still works or if you have plans to fix it.
> I’ve had an existential crisis about my work in open source AI on GitHub, particularly as there has been both increasingly toxic backlash against AI and because the AI industry has been evolving so rapidly that I flat-out don’t have enough bandwidth to keep up
Herein lies the problem? You sound overwhelmed. I've been there myself. I don't know what your year's been like but you genuinely might want to get away from the screen and get some fresh air. This is a good time of year to do it, since things generally slow down at work.
- I need help with getting an API
- I need help with getting an api
What are some alternatives?
text-generation-inference - Large Language Model Text Generation Inference
BetterChatGPT - An amazing UI for OpenAI's ChatGPT (Website + Windows + MacOS + Linux)
openai-chatgpt-opentranslator - Python command that uses openai to perform text translations
gpt4all - gpt4all: run open-source LLMs anywhere
AutoGPTQ - An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.
Flowise - Drag & drop UI to build your customized LLM flow
NeMo-Guardrails - NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.
text-generation-webui - A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
llm-foundry - LLM training code for Databricks foundation models
chatgpt-clone - Enhanced ChatGPT Clone: Features OpenAI, Bing, PaLM 2, AI model switching, message search, langchain, Plugins, Multi-User System, Presets, completely open-source for self-hosting. More features in development [Moved to: https://github.com/danny-avila/LibreChat]
alpaca.cpp - Locally run an Instruction-Tuned Chat-Style LLM
chatgpt-retrieval-plugin - The ChatGPT Retrieval Plugin lets you easily find personal or work documents by asking questions in natural language.