Rust Arrow

Open-source Rust projects categorized as Arrow

Top 13 Rust Arrow Projects

  • polars

    Dataframes powered by a multithreaded, vectorized query engine, written in Rust

  • Project mention: Big Data Is Dead | news.ycombinator.com | 2024-05-27
  • datafusion

    Apache DataFusion SQL Query Engine

  • Project mention: Velox: Meta's Unified Execution Engine [pdf] | news.ycombinator.com | 2024-03-25

    Python's Substrait seems like the biggest/most-used competitor-ish out there. I'd love some compare & contrast; my sense is that Substrait has a smaller ambition, and more wants to be a language for talking about execution rather than a full on execution engine. https://github.com/substrait-io/substrait

    We can also see from the DataFusion discussion that they too see themselves as a bit of a Velox competitor. https://github.com/apache/arrow-datafusion/discussions/6441

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • roapi

    Create full-fledged APIs for slowly moving datasets without writing a single line of code.

  • Project mention: Full-fledged APIs for slowly moving datasets without writing code | news.ycombinator.com | 2023-10-25
  • datafusion-ballista

    Apache Arrow Ballista Distributed Query Engine

  • Project mention: DataFusion Comet: Apache Spark Accelerator | news.ycombinator.com | 2024-05-31

    But why. Just ditch Spark and use https://github.com/apache/datafusion-ballista directly.

  • parquet-wasm

    Rust-based WebAssembly bindings to read and write Apache Parquet data

  • Project mention: FLaNK AI Weekly for 29 April 2024 | dev.to | 2024-04-29
  • datafusion-comet

    Apache DataFusion Comet Spark Accelerator

  • Project mention: DataFusion Comet: Apache Spark Accelerator | news.ycombinator.com | 2024-05-31
  • duckdb-rs

    Ergonomic bindings to duckdb for Rust

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • pqrs

    Command line tool for inspecting Parquet files

  • biobear

    Work with bioinformatic files using Arrow, Polars, and/or DuckDB

  • s2protocol-rs

    Starcraft 2 Protocol Replay Reader

  • Project mention: New version of s2protocol-rs SC2Replay parsing crate | /r/starcraft2 | 2023-10-06
  • fastexcel

    A Python wrapper around calamine (by ToucanToco)

  • myval

    Lightweight Apache Arrow data frame for Rust

  • vortex

    A toolkit for working with compressed array data (by spiraldb)

  • Project mention: Ask HN: Who is hiring? (May 2024) | news.ycombinator.com | 2024-05-01

    Fulcrum | Software Engineer | London or New York | ONSITE | Full-Time

    Fulcrum is building next generation storage platform for diverse data of the future. We believe users will need to process non tabular and tabular data together and we need to develop new methods to support them.

    We develop Vortex (our core storage primitive) in the open https://github.com/spiraldb/vortex and currently are looking to hire more people to our 5 person team to help build our product.

    Tech: Rust, Python, Zig

    Reach out to me at hn[at]fulcrum[dot]so

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Rust Arrow related posts

Index

What are some of the best open-source Arrow projects in Rust? This list will help you:

Project Stars
1 polars 26,779
2 datafusion 5,266
3 roapi 3,105
4 datafusion-ballista 1,327
5 parquet-wasm 476
6 datafusion-comet 461
7 duckdb-rs 384
8 pqrs 257
9 biobear 133
10 s2protocol-rs 101
11 fastexcel 74
12 myval 61
13 vortex 48

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com