Intel QuickAssist Technology Zstandard Plugin for Zstandard

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com
Compression Huffman Library C Zlib

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • zlib-ng

    zlib replacement with optimizations for "next generation" systems.

  • qatlib

    • https://github.com/intel/qatlib/blob/main/quickassist/includ...

    It also seems to suggest that it could accelerate hyperscan, but considering the hardware and software are both from Intel and they don't integrate them, maybe it doesn't work or is theoretical.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • QAT-ZSTD-Plugin

    • Interesting that Intel's code for this includes numerous references to LZ4, as if that's the actual algorithm the hardware originally aimed to accelerate.. So seems like LZ4 and ZSTD are quite similar?

    https://github.com/intel/QAT-ZSTD-Plugin/blob/main/src/qatse...

  • isa-l

    Intelligent Storage Acceleration Library

    • For using accelerators (QAT, I assume, because the article is about this) from the CLI, then QATzip[3] comes with a command line tool qatzip which can be used as described in the project ReadMe. I didn't test it, though, as I have no QAT-enabled device.

    [1] https://github.com/intel/isa-l/tree/master/igzip

  • rapidgzip

    Gzip Decompression and Random Access for Modern Multi-Core Machines

  • QATzip

    Compression Library accelerated by Intel® QuickAssist Technology

  • TurboBench

    Compression Benchmark

    • - https://github.com/powturbo/TurboBench/issues/43

    [1] https://github.com/powturbo/TurboBench

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • FiniteStateEntropy

    New generation entropy codecs : Finite State Entropy and Huff0

    • It's obsolete. It's limited to 32KB LZ window with huffman coding. Zstd can use a much larger window (8MB recommended) and a much better entropy coder: https://github.com/Cyan4973/FiniteStateEntropy

  • lib842

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts