Mixtral: Mixture of Experts

• ollama-webui

  14 5,789 9.8 Svelte

  Discontinued ChatGPT-Style WebUI for LLMs (Formerly Ollama WebUI) [Moved to: https://github.com/open-webui/open-webui]

  

On mac silicon:

https://ollama.ai/

ollama pull mixtral

For a chatgpt-esk web ui

https://github.com/ollama-webui/ollama-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Navigate to http://localhost:3000

You can also use ollama in langchain.

• ollama

  220 69,806 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

On mac silicon:

https://ollama.ai/

ollama pull mixtral

For a chatgpt-esk web ui

https://github.com/ollama-webui/ollama-webui

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v ollama-webui:/app/backend/data --name ollama-webui --restart always ghcr.io/ollama-webui/ollama-webui:main

Navigate to http://localhost:3000

You can also use ollama in langchain.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• CogVLM

  16 5,355 9.0 Python

  a state-of-the-art-level open visual language model | 多模态预训练模型

  

CogVLM is very good in my (brief) testing: https://github.com/THUDM/CogVLM

The model weights seem to be under a non-commercial license, not true open source, but it is "open access" as you requested.

• llamafile

  38 16,040 9.6 C++

  Distribute and run LLMs with a single file.

  

• LLMstudio

  1 215 9.3 TypeScript

  Framework to bring LLM applications to production

  

Lmstudio (that they linked) is definitely not open source, and doesn't even offer a pricing model for business use.

Llmstudio is, but I suspect that was a typo in their comment. https://github.com/TensorOpsAI/LLMStudio

• llama.cpp

  788 59,389 10.0 C++

  LLM inference in C/C++

> The output quality is not "ruined" at all.

That was my experience as well.

I also tried 2-bit version, and it was horrible.

However, there is a new approach in the works[1] (merged yesterday) which works surprisingly well with 2.10 bits per weight (12.3 GB model size)

[1] https://github.com/ggerganov/llama.cpp/pull/4773

Suggest a related project