Rust Inference

Open-source Rust projects categorized as Inference
Edit details
Topics: Rust Onnx Machine Learning AI WASM
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

Top 8 Rust Inference Projects

  • aici

    AICI: Prompts as (Wasm) Programs

    • Project mention: Google Gemini: Context Caching | news.ycombinator.com | 2024-05-16

    To me, context caching is only a subset of what is possible with full control over the model. I consider this a more complete list: https://github.com/microsoft/aici?tab=readme-ov-file#flexibi...

    Context caching only gets you “forking generation into multiple branches” (i.e. sharing work between multiple generations)

  • ort

    A Rust wrapper for ONNX Runtime (by pykeio)

    • Project mention: AI Inference now available in Supabase Edge Functions | dev.to | 2024-04-16

    To solve this, we built a native extension in Edge Runtime that enables using ONNX runtime via the Rust interface. This was made possible thanks to an excellent Rust wrapper called Ort:

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • blindai

    Confidential AI deployment with secure enclaves :lock:

  • snips-nlu-rs

    Snips NLU rust implementation

  • onnxruntime-rs

    Rust wrapper for Microsoft's ONNX Runtime (version 1.8)

  • llama-dfdx

    LLaMa 7b with CUDA acceleration implemented in rust. Minimal GPU memory needed!

  • meteorite

    A fast and simple web server for machine learning models (by deploifai)

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • mediapipe-rs

    A Rust library for mediapipe tasks for WasmEdge WASI-NN.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust Inference related posts

  • Google Gemini: Context Caching

    1 project | news.ycombinator.com | 16 May 2024

  • AI Inference Now Available in Supabase Edge Functions

    1 project | news.ycombinator.com | 16 Apr 2024

  • Meta's Segment Anything written with C++ / GGML

    4 projects | news.ycombinator.com | 5 Sep 2023

  • Machine Learning Inference Server in Rust?

    4 projects | /r/rust | 21 Mar 2023

Index

What are some of the best open-source Inference projects in Rust? This list will help you:

Project Stars
1 aici 1,797
2 ort 629
3 blindai 493
4 snips-nlu-rs 337
5 onnxruntime-rs 262
6 llama-dfdx 94
7 meteorite 8
8 mediapipe-rs 0

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com