Text-generation-inference Alternatives

Similar projects and alternatives to text-generation-inference

• text-generation-webui

  876 36,552 9.9 Python text-generation-inference VS text-generation-webui

  A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

• llama.cpp

  773 57,463 10.0 C++ text-generation-inference VS llama.cpp

  LLM inference in C/C++

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Open-Assistant

  329 36,647 8.3 Python text-generation-inference VS Open-Assistant

  OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

  

• ollama

  199 62,615 9.9 Go text-generation-inference VS ollama

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

• llama

  184 53,227 8.1 Python text-generation-inference VS llama

  Inference code for Llama models

  

• lapce

  178 32,854 9.7 Rust text-generation-inference VS lapce

  Lightning-fast and Powerful Code Editor written in Rust

  

• Graal

  156 19,807 10.0 Java text-generation-inference VS Graal

  GraalVM compiles Java applications into native executables that start instantly, scale fast, and use fewer compute resources 🚀

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• LocalAI

  83 19,862 9.9 C++ text-generation-inference VS LocalAI

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

  

• FLiPStackWeekly

  80 14 9.9 text-generation-inference VS FLiPStackWeekly

  FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...

• Mage

  77 7,050 9.9 Python text-generation-inference VS Mage

  🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai

  

• exllama

  64 2,594 9.0 Python text-generation-inference VS exllama

  A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.

• llama-cpp-python

  55 6,475 9.8 Python text-generation-inference VS llama-cpp-python

  Python bindings for llama.cpp

• onnxruntime

  54 12,736 10.0 C++ text-generation-inference VS onnxruntime

  ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

  

• vllm

  31 18,571 9.9 Python text-generation-inference VS vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  

• refact

  34 1,422 9.8 JavaScript text-generation-inference VS refact

  WebUI for Fine-Tuning and Self-hosting of Open-Source Large Language Models for Coding

  

• server

  24 7,356 9.5 Python text-generation-inference VS server

  The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  

• llm

  23 2,954 9.4 Python text-generation-inference VS llm

  Access large language models from the command-line (by simonw)

• openvino

  17 5,962 10.0 C++ text-generation-inference VS openvino

  OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference

  

• optimum

  8 2,157 9.5 Python text-generation-inference VS optimum

  🚀 Accelerate training and inference of 🤗 Transformers and 🤗 Diffusers with easy to use hardware optimization tools

  

• basaran

  22 1,281 10.0 Python text-generation-inference VS basaran

  Discontinued Basaran is an open-source alternative to the OpenAI text completion API. It provides a compatible streaming API for your Hugging Face Transformers-based text generation models.

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives