How is ExLlama so good? Can it be used with a more feature rich UI?

This page summarizes the projects mentioned and recommended in the original post on /r/LocalLLaMA
hardware-buttons scrape-images linkedin-bot

Scout Monitoring - Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

  • text-generation-webui

    A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

    There's a PR here for ooba with some instructions: Add exllama support (janky) by oobabooga · Pull Request #2444 · oobabooga/text-generation-webui (github.com)

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo

  • KoboldAI

  • exllama

    A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights. (by 0cc4m)

  • magi_llm_gui

    A Qt GUI for large language models

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Gloe v0.6.0 released Your Code as a Flow

    1 project | news.ycombinator.com | 26 Jun 2024

  • A Better Way to Code: Documentation Driven Development

    3 projects | news.ycombinator.com | 26 Jun 2024

  • Nuitka Is a Python Compiler

    1 project | news.ycombinator.com | 26 Jun 2024

  • Show HN: R2R V2 – A open source RAG engine with prod features

    2 projects | news.ycombinator.com | 26 Jun 2024

  • Show HN: TF-GPT – a TensorFlow implementation of a decoder-only transformer

    1 project | news.ycombinator.com | 26 Jun 2024

Did you konow that Python is
the 1st most popular programming language
based on number of metions?