Rust llm

Open-source Rust projects categorized as llm
Edit details
Topics: Rust AI openai chatgpt Machine Learning
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 23 Rust llm Projects

  • burn

    Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.

    • Project mention: 3 years of fulltime Rust game development, and why we're leaving Rust behind | news.ycombinator.com | 2024-04-26

    You can use libtorch directly via `tch-rs`, and at present I'm porting over to Burn (see https://burn.dev) which appears incredibly promising. My impression is it's in a good place, if of course not close to the ecosystem of Python/C++. At very least I've gotten my nn models training and running without too much difficulty. (I'm moving to Burn for the thread safety - their `Tensor` impl is `Sync` - libtorch doesn't have such a guarantee.)

    Burn has Candle as one of its backends, which I understand is also quite popular.

  • llm

    An ecosystem of Rust libraries for working with large language models

    • Project mention: Open-sourcing a simple automation/agent workflow builder | /r/ChatGPTPro | 2023-10-07

    We're open-sourcing a project that lets you build simple automations/agent workflows that use LLMs for different tasks. Kinda like Zapier or IFTTT but focused on using natural language to accomplish your tasks.It's super early but we'd love to start getting feedback to steer it in the right direction. It currently supports OpenAI and local models through llm.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • postgresml

    The GPU-powered AI application database. Get your app to market faster using the simplicity of SQL and the latest NLP, ML + LLM models.

    • Project mention: PostgresML | /r/programming | 2023-08-30

  • aichat

    All-in-one AI-Powered CLI Chat & Copilot that integrates 10+ AI platforms, including OpenAI, Azure-OpenAI, Gemini, VertexAI, Claude, Mistral, Cohere, Ollama, Ernie, Qianwen...

    • Project mention: Show HN: A shell CLI tool to predict your next command enhanced by LLM and RAG | news.ycombinator.com | 2024-04-16

    thanks for sharing this, I have been using aichat (https://github.com/sigoden/aichat) and shell_gpt for a while. Let's see how it works.

  • text-embeddings-inference

    A blazing fast inference solution for text embeddings models

    • Project mention: HuggingFace text-generation-inference is reverting to Apache 2.0 License | news.ycombinator.com | 2024-04-08

    Worth noting that this also impacts the great https://github.com/huggingface/text-embeddings-inference, which allows anyone to run state of the art embeddings with great performance.

  • aici

    AICI: Prompts as (Wasm) Programs

    • Project mention: Google Gemini: Context Caching | news.ycombinator.com | 2024-05-16

    To me, context caching is only a subset of what is possible with full control over the model. I consider this a more complete list: https://github.com/microsoft/aici?tab=readme-ov-file#flexibi...

    Context caching only gets you “forking generation into multiple branches” (i.e. sharing work between multiple generations)

  • smartgpt

    A program that provides LLMs with the ability to complete complex tasks using plugins.

    • Project mention: Smartgpt: A program that provides LLMs ability with complex tasks using plugins | news.ycombinator.com | 2023-08-28

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • pgvecto.rs

    Scalable, Low-latency and Hybrid-enabled Vector Search in Postgres. Revolutionize Vector Search, not Database.

    • Project mention: My binary vector search is better than your FP32 vectors | dev.to | 2024-03-25

    To evaluate the performance metrics in comparison to the original vector approach, we conducted benchmarking using the dbpedia-entities-openai3-text-embedding-3-large-3072-1M dataset. The benchmark was performed on a Google Cloud virtual machine (VM) with specifications of n2-standard-8, which includes 8 virtual CPUs and 32GB of memory. We used pgvecto.rs v0.2.1 as the vector database.

  • mistral.rs

    Blazingly fast LLM inference.

    • Project mention: Mistral.rs: Fast LLM Engine | news.ycombinator.com | 2024-04-25

  • llm-chain

    `llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks

    • Project mention: are there anytools or frameworks similar to "langchain" or "llamaindexbut implemented or designed in a language other than python? | /r/LocalLLaMA | 2023-06-28

  • floneum

    A toolkit for controllable, private AI on consumer hardware in rust

    • Project mention: Dioxus 0.5: Web, Desktop, Mobile Apps in Rust | news.ycombinator.com | 2024-03-28

    It is pretty good. I am working on an application that uses SVGs as a way to draw a workflow editor UI with Dioxus: https://github.com/floneum/floneum

  • llama-node

    Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.

    • Project mention: Tell HN: Rust Is the Superglue | news.ycombinator.com | 2023-07-15

    You can practice your Rust skills by writing performant and/or gluey extensions for higher-level language such as NodeJS (checkout napi-rs) and Python or complementing JS in the browser if you target Webassembly.

    For instance, checkout Llama-node https://github.com/Atome-FE/llama-node for an involved Rust-based NodeJS extension. Python has PyO3, a Rust-Python extension toolset: https://github.com/PyO3/pyo3.

    They can help you leverage your Rust for writing cool new stuff.

  • plock

    From anywhere you can type, query and stream the output of an LLM or any other script

    • Project mention: Show HN: NotesOllama – I added local LLM support to Apple Notes (through Ollama) | news.ycombinator.com | 2024-02-21

    I'm a big fan of this space and have been hacking on it too. With a few tricks you can use LLMs or anything else you can call from a script, via input capture and simulation, and clipboard. And it can be cross platform!

    Here's the project, if it might interest anyone.

    https://github.com/jasonjmcghee/plock

  • oatmeal

    Terminal UI to chat with large language models (LLM) using different model backends, and integrations with your favourite editors!

    • Project mention: FLaNK Stack 26 February 2024 | dev.to | 2024-02-26

  • ai00_server

    A localized open-source AI server that is better than ChatGPT.

    • Project mention: People who've used RWKV, whats your wishlist for it? | /r/LocalLLaMA | 2023-12-09

  • fltr

    Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

    • Project mention: Show HN: Like grep but for natural questions. Mixtral 8x7B – 28 tok/s on 8GB GPU | news.ycombinator.com | 2024-03-13

  • indexify

    A scalable realtime and continuous indexing and structured extraction engine for Unstructured Data to build Generative AI Applications

    • Project mention: How I got my first Rust job by doing open-source | dev.to | 2024-04-30

    Around 2 weeks ago now, someone opened an issue on OasysDB to integrate it to his platform, Indexify, an open-source platform to extract and process various unstructured data from different sources for generative AI apps in real-time.

  • tenere

    🔥 TUI interface for LLMs written in Rust

    • Project mention: Tenere supports llamacpp and ollama as back ends now | news.ycombinator.com | 2024-02-09

  • smolrsrwkv

    A relatively basic implementation of RWKV in Rust written by someone with very little math and ML knowledge. Supports 32, 8 and 4 bit evaluation. It can also directly load PyTorch RWKV models.

  • opentau

    Using Large Language Models for Gradual Type Inference

    • Project mention: Opentau – Using Large Language Models for Gradual Type Inference | news.ycombinator.com | 2023-08-13

  • signway

    Bring the power of pre-signed URLs to your apps. Signway is a gateway for redirecting ephimeral signed URLs to the requested API

    • Project mention: Show HN: Signed URLs Gateway – The Missing Infra Piece for Building Modern Apps | news.ycombinator.com | 2023-11-20

  • Orphic

    A natural language shell interface for *nix systems

  • memex

    Super-simple, fully Rust powered "memory" (doc store + semantic search) for LLM projects, semantic search, etc. (by spyglass-search)

    • Project mention: Playing around w/ a more structured LLM workflow | /r/LocalLLaMA | 2023-08-21

    It's currently working with OpenAI (which you can play with here: https://cloud.spyglass.fyi/playground). We're still ironing out some bugs before we can release an open-source version that works the llama models (via our other utility: https://github.com/spyglass-search/memex).

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust llm related posts

  • How I got my first Rust job by doing open-source

    3 projects | dev.to | 30 Apr 2024

  • My binary vector search is better than your FP32 vectors

    1 project | dev.to | 25 Mar 2024

  • Show HN: Like grep but for natural questions. Mixtral 8x7B – 28 tok/s on 8GB GPU

    1 project | news.ycombinator.com | 13 Mar 2024

  • Indexify -Scalable, realtime, continuous indexing engine–Unstructured Data to AI

    1 project | news.ycombinator.com | 6 Mar 2024

  • Like grep but for natural language questions. Based on Mixtral 8x7B

    1 project | news.ycombinator.com | 5 Mar 2024

  • Show HN: Kalosm an embeddable framework for pre-trained models in Rust

    3 projects | news.ycombinator.com | 28 Feb 2024

  • Show HN: Mistral LLM w Assistants API and Action tool 4 autonomous requests

    1 project | news.ycombinator.com | 13 Feb 2024
  • A note from our sponsor - SaaSHub
    www.saashub.com | 16 May 2024
    SaaSHub helps you find the best software and product alternatives Learn more →

Index

What are some of the best open-source llm projects in Rust? This list will help you:

Project Stars
1 burn 7,223
2 llm 5,954
3 postgresml 5,483
4 aichat 2,906
5 text-embeddings-inference 2,055
6 aici 1,756
7 smartgpt 1,732
8 pgvecto.rs 1,429
9 mistral.rs 1,625
10 llm-chain 1,184
11 floneum 987
12 llama-node 849
13 plock 421
14 oatmeal 400
15 ai00_server 388
16 fltr 343
17 indexify 250
18 tenere 234
19 smolrsrwkv 91
20 opentau 89
21 signway 79
22 Orphic 52
23 memex 45

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com