Is there any open source app to load a model and expose API like OpenAI?

• llm-api

  2 145 6.6 Python

  Run any Large Language Model behind a unified API

1b5d/llm-api: Run any Large Language Model behind a unified API (github.com)

• litellm

  28 8,696 10.0 Python

  Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)

  

I use this with ollama and works perfectly https://github.com/BerriAI/litellm

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• LocalAI

  83 20,346 9.9 C++

  :robot: The free, Open Source OpenAI alternative. Self-hosted, community-driven and local-first. Drop-in replacement for OpenAI running on consumer-grade hardware. No GPU required. Runs gguf, transformers, diffusers and many more models architectures. It allows to generate Text, Audio, Video, Images. Also with voice cloning capabilities.

  

• text-generation-inference

  29 7,995 9.6 Python

  Large Language Model Text Generation Inference

  

• server

  24 7,414 9.5 Python

  The Triton Inference Server provides an optimized cloud and edge inferencing solution. (by triton-inference-server)

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project