Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 parallel-computing Open-Source Projects
-
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
swifter
A package which efficiently applies any function to a pandas dataframe or series in the fastest available manner (by jmcarpenter2)
-
kokkos
Kokkos C++ Performance Portability Programming Ecosystem: The Programming Model - Parallel Execution and Memory Abstraction
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
awesome-machine-learning-in-compilers
Must read research papers and links to tools and datasets that are related to using machine learning for compilers and systems optimisation
-
Arraymancer
A fast, ergonomic and portable tensor library in Nim with a deep learning focus for CPU, GPU and embedded devices via OpenMP, Cuda and OpenCL backends
-
-
-
-
Kratos
Kratos Multiphysics (A.K.A Kratos) is a framework for building parallel multi-disciplinary simulation software. Modularity, extensibility and HPC are the main objectives. Kratos has BSD license and is written in C++ with extensive Python interface. (by KratosMultiphysics)
-
-
-
-
-
-
-
Hyperactive
An optimization and data collection toolbox for convenient and fast prototyping of computationally expensive models.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: The Way We Are Building Event-Driven Applications Is Misguided | news.ycombinator.com | 2024-05-28> The set-theory approach is hard to do, but promising. Each object that wants something has a small set of the things it wants. There's a big pool of such sets. There's also a big pool of the items you have, which changes constantly. It's easy to express what you need to fetch and which objects are now ready to go as set intersection and difference operations. But you need representations for big sparse sets which can do set operations fast. Probably B-trees, or something in that space.
Incremental updates to dynamic dependency graphs is a familiar problem for build tooling. I personally have used the taskflow C++ library (https://github.com/taskflow/taskflow) to great effect.
> Microsoft Research fooled around with this concept years ago in a different context. The idea was to have a database which supported pending SQL queries. The query would return new results when the database changed such that the results of the query changed. The goal was to to support that for millions of pending queries. Financial traders would love to have that. It's a very hard scaling problem. Don't know how that came out.
Incremental view maintenance is an active area of research. The likes of Noria and Materialize have done this with SQL, and the pg_ivm Postgres extension looks promising. Not sure if there is an equivalent implementation geared towards entity-component systems, though.
Project mention: Creando Subtítulos Automáticos para Vídeos con Python, Faster-Whisper, FFmpeg, Streamlit, Pillow | dev.to | 2024-04-29
It is a small DSL written using macros at https://github.com/mratsim/Arraymancer/blob/master/src/array....
Nim has pretty great meta-programming capabilities and arraymancer employs some cool features like emitting cuda-kernels on the fly using standard templates depending on backend !
https://github.com/topics/datalog?l=rust ... Cozo, Crepe
Crepe: https://github.com/ekzhang/crepe :
> Crepe is a library that allows you to write declarative logic programs in Rust, with a Datalog-like syntax. It provides a procedural macro that generates efficient, safe code and interoperates seamlessly with Rust programs.
Looks like there's not yet a Python grammar for the treeedb tree-sitter: https://github.com/langston-barrett/treeedb :
> Generate Soufflé Datalog types, relations, and facts that represent ASTs from a variety of programming languages.
Looks like roxi supports n3, which adds `=>` "implies" to the Turtle lightweight RDF representation: https://github.com/pbonte/roxi
FWIW rdflib/owl-rl: https://owl-rl.readthedocs.io/en/latest/owlrl.html :
> simple forward chaining rules are used to extend (recursively) the incoming graph with all triples that the rule sets permit (ie, the “deductive closure” of the graph is computed).
ForwardChainingStore and BackwardChainingStore implementations w/ rdflib in Python: https://github.com/RDFLib/FuXi/issues/15
Fast CUDA hashmaps
Gdlog is built on CuCollections.
GPU HashMap libs to benchmark: Warpcore, CuCollections,
https://github.com/NVIDIA/cuCollections
https://github.com/NVIDIA/cccl
https://github.com/sleeepyjack/warpcore
/? Rocm HashMap
DeMoriarty/DOKsparse:
https://grass.osgeo.org/
GRASS GIS offers powerful raster, vector, and geospatial processing engines in a single integrated software suite. It includes tools for terrain and ecosystem modeling, hydrology, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It comes with a temporal framework for advanced time series processing and a Python API for rapid geospatial programming. GRASS GIS has been optimized for performance and large geospatial data analysis.
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
parallel-computing related posts
-
Forkrun: Runs multiple inputs through a command in parallel using bash coprocs
-
Forkrun – A pure-bash function for parallelizing loops
-
Symbolics.jl
-
Minimal implementation of Mamba, the new LLM architecture, in 1 file of PyTorch
-
Coarrays
-
Awesome research papers on ML in Compilers
-
I made a Python package to do adaptive learning of functions in parallel [P]
-
A note from our sponsor - InfluxDB
www.influxdata.com | 1 Jun 2024
Index
What are some of the best open-source parallel-computing projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Taskflow | 9,649 |
| 2 | Joblib | 3,698 |
| 3 | CTranslate2 | 2,916 |
| 4 | swifter | 2,478 |
| 5 | kokkos | 1,755 |
| 6 | mfem | 1,591 |
| 7 | Vc | 1,421 |
| 8 | awesome-machine-learning-in-compilers | 1,352 |
| 9 | Arraymancer | 1,314 |
| 10 | Symbolics.jl | 1,301 |
| 11 | adaptive | 1,117 |
| 12 | elmerfem | 1,106 |
| 13 | pyopencl | 1,034 |
| 14 | Kratos | 970 |
| 15 | future | 936 |
| 16 | accelerate | 888 |
| 17 | cccl | 854 |
| 18 | grass | 779 |
| 19 | dolfinx | 678 |
| 20 | oneMKL | 577 |
| 21 | OpenTimer | 516 |
| 22 | Hyperactive | 495 |
| 23 | post-me | 484 |
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com