Ask HN: Affordable hardware for running local large language models?

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com
llama llm

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • llama.cpp

    LLM inference in C/C++

    • Yes, Metal seems to allow a maximum of 1/2 of the RAM for one process, and 3/4 of the RAM allocated to the GPU overall. There’s a kernel hack to fix it, but that comes with the usual system integrity caveats. https://github.com/ggerganov/llama.cpp/discussions/2182

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • RAG with llama.cpp and external API services

    2 projects | dev.to | 31 May 2024

  • Ask HN: I have many PDFs – what is the best local way to leverage AI for search?

    10 projects | news.ycombinator.com | 30 May 2024

  • Deploying llama.cpp on AWS (with Troubleshooting)

    1 project | dev.to | 28 May 2024

  • Devoxx Genie Plugin : an Update

    6 projects | dev.to | 28 May 2024

  • Mistral Fine-Tune

    2 projects | news.ycombinator.com | 25 May 2024