Show HN: Open-source Rule-based PDF parser for RAG

• llmsherpa

  6 970 6.6 Jupyter Notebook

  Developer APIs to Accelerate LLM Projects

  

I wrote about split points and the need for including section hierarchy in this post: https://ambikasukla.substack.com/p/efficient-rag-with-docume...

All this is automated in the llmsherpa parser https://github.com/nlmatics/llmsherpa which you can use as an API over this library.

• nlm-ingestor

  3 823 7.1 Python

  This repo provides the server side code for llmsherpa API to connect. It includes parsers for various file formats.

  

Here's another notebook from the repo with examples: https://github.com/nlmatics/nlm-ingestor/blob/main/notebooks...

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• txtai

  356 7,080 9.3 Python

  💡 All-in-one open-source embeddings database for semantic search, LLM orchestration and language model workflows

  

Nice project! I've long used Tika for document parsing given it's maturity and wide number of formats supported. The XHTML output helps with chunking documents for RAG.

Here's a couple examples:

- https://neuml.hashnode.dev/build-rag-pipelines-with-txtai

- https://neuml.hashnode.dev/extract-text-from-documents

Disclaimer: I'm the primary author of txtai (https://github.com/neuml/txtai).

• grobid

  12 3,094 9.2 Java

  A machine learning software for extracting information from scholarly documents

• paperetl

  12 317 6.3 Python

  📄 ⚙️ ETL processes for medical and scientific papers

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project