M2 Ultra can run 128 streams of Llama 2 7B in parallel

• llama.cpp

  791 59,389 10.0 C++

  LLM inference in C/C++

• whisper.coreml

  1 11 8.5 Python

  Robust Speech Recognition via Large-Scale Weak Supervision

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• more-ane-transformers

  4 43 7.0 Python

  Run transformers (incl. LLMs) on the Apple Neural Engine.

• ollama

  222 71,334 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

They are very much uncensored, give it a try yourself: `ollama run llama2-uncensored` [0]

It will be happy to curse, talk about religions, help you cook illicit substances or do all sorts of other stuff.

[0] https://ollama.ai/

Suggest a related project