All AI Models, from 3B to 13B running at ~0.5 tokens/s, what could be causing this?

This page summarizes the projects mentioned and recommended in the original post on /r/LocalLLaMA
AI API chatgpt-api llm Localhost

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • local.ai

    🎒 local.ai - Run AI locally on your PC!

    • Sidenote: can you try out localai.app and see if it's faster than oobabooga on your end? (It's all CPU inferencing as well, but just curious if there's any speed gain).

  • text-generation-webui

    A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

    • follow these steps https://github.com/oobabooga/text-generation-webui/blob/main/docs/llama.cpp-models.md if you installed using the one click installer then you have to activate the conda environment before following the steps at least on linux anyways.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Why does GPT4all respond so slowly on my machine?

    2 projects | /r/LocalLLaMA | 12 Jul 2023

  • Show HN: Medical LLM on Par with Google Med-Palm 98% Usmle Accuracy

    2 projects | news.ycombinator.com | 23 Mar 2024

  • Show HN: LLM Code Interpreter

    1 project | news.ycombinator.com | 4 Sep 2023

  • Show HN: Build your own code interpreter for ChatGPT

    1 project | news.ycombinator.com | 8 Aug 2023

  • Better Code Interpreter for ChatGPT

    1 project | news.ycombinator.com | 7 Aug 2023