Top 4 C++ llm Projects
-
infinity
The AI-native database built for LLM applications, providing incredibly fast full-text and vector search
-
cortex
Drop-in, local AI alternative to the OpenAI stack. Multi-engine (llama.cpp, TensorRT-LLM). Powers 👋 Jan (by janhq)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
vectordb
Epsilla is a high performance Vector Database Management System. Try out hosted Epsilla at https://cloud.epsilla.com/
Project mention: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search | news.ycombinator.com | 2024-05-30Within our open source RAG product RAGFlow(https://github.com/infiniflow/ragflow), Elasticsearch is currently used instead of other general vector databases, because it can provide hybrid search right now. Under the default cases, embedding based reranker is not required, just RRF is enough, while even if reranker is used, keywords based retrieval is also a MUST to be hybridized with embedding based retrieval, that's just what RAGFlow's latest 0.7 release has provided.
On the other hand let me introduce another database we developed, Infinity(https://github.com/infiniflow/infinity), which can provide the fastest hybrid search, you can see the performance here(https://github.com/infiniflow/infinity/blob/main/docs/refere...), both vector search and full-text search could perform much faster than other open source alternatives.
From the next version(weeks later), Infinity will also provide more comprehensive hybrid search capabilities, what you have mentioned the 3-way recalls(dense vector, sparse vector, keyword search) could be provided within single request.
Jan incorporates a lightweight, built-in inference server called Nitro. Nitro supports both llama.cpp and NVIDIA's TensorRT-LLM engines. This means many open LLMs in the GGUF format are supported. Jan's Model Hub is designed for easy installation of pre-configured models but it also allows you to install virtually any model from Hugging Face or even your own.
I helped create this using https://www.epsilla.com/ which is an incredibly good vector search tool. Our long covid guidance got 10x better when we switched to it!
Got a web version here, check it out and let us know if it’s helpful to you:
C++ llms related posts
Index
What are some of the best open-source llm projects in C++? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | infinity | 1,868 |
| 2 | cortex | 1,661 |
| 3 | vectordb | 874 |
| 4 | pyllamacpp | 59 |
Sponsored