Top 13 Python preprocessing Projects

• ragflow

  10 8,245 9.7 Python

  RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding.

  

Project mention: Better RAG Results with Reciprocal Rank Fusion and Hybrid Search | news.ycombinator.com | 2024-05-30

Within our open source RAG product RAGFlow(https://github.com/infiniflow/ragflow), Elasticsearch is currently used instead of other general vector databases, because it can provide hybrid search right now. Under the default cases, embedding based reranker is not required, just RRF is enough, while even if reranker is used, keywords based retrieval is also a MUST to be hybridized with embedding based retrieval, that's just what RAGFlow's latest 0.7 release has provided.

On the other hand let me introduce another database we developed, Infinity(https://github.com/infiniflow/infinity), which can provide the fastest hybrid search, you can see the performance here(https://github.com/infiniflow/infinity/blob/main/docs/refere...), both vector search and full-text search could perform much faster than other open source alternatives.

From the next version(weeks later), Infinity will also provide more comprehensive hybrid search capabilities, what you have mentioned the 3-way recalls(dense vector, sparse vector, keyword search) could be provided within single request.

• igel

  11 3,080 1.1 Python

  a delightful machine learning tool that allows you to train, test, and use models without writing code

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• MLBox

  1 1,481 0.0 Python

  MLBox is a powerful Automated Machine Learning python library.

• NVTabular

  1 1,016 5.4 Python

  NVTabular is a feature engineering and preprocessing library for tabular data designed to quickly and easily manipulate terabyte scale datasets used to train deep learning based recommender systems.

  

• nnAudio

  1 968 5.3 Python

  Audio processing by using pytorch 1D convolution network

• voicesmith

  1 207 10.0 Python

  [WIP] VoiceSmith makes training text to speech models easy.

• courlan

  0 70 7.5 Python

  Clean, filter and sample URLs to optimize data collection – includes spam, content type and language filters

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• pytorch-VideoDataset

  1 67 0.0 Python

  Tools for loading video dataset and transforms on video in pytorch. You can directly load video files without preprocessing.

• podium

  1 60 0.0 Python

  Podium: a framework agnostic Python NLP library for data loading and preprocessing

  

• cpip

  1 39 3.7 Python

  CPIP - a C/C++ preprocessor implemented in Python.

• VHDLproc

  1 24 0.0 Python

  VHDLproc is a VHDL preprocessor

• riffusion-scripts

  1 0 5.3 Python

  Scripts to aid in preparing training data for riffusion.

• videotooimage

  1 0 3.6 Python

  📽 A Python pip package for converting videos to sequential image frames

Project mention: 🎥 Worried about video preprocessing for ML?! | dev.to | 2024-05-14

Python preprocessing related posts

• LEAF: A Learnable Frontend for Audio Classification

  1 project | /r/speechtech | 26 Jan 2021

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Python projects with our weekly report!