DwarFS – The Deduplicating Warp-Speed Advanced Read-Only File System

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • dwarfs

    A fast high compression read-only file system for Linux, Windows and macOS

  • It's in the man page[1], you put the file at the filesystem creation:

    $ mkdwarfs -i /path/dir -o image.dwarfs

    [1]: https://github.com/mhx/dwarfs/blob/main/doc/mkdwarfs.md

  • dwarffs

    A FUSE filesystem that allows tools like gdb to look up debug info files via HTTP

  • Not to be confused with dwarffs – https://github.com/edolstra/dwarffs.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • setup

  • seaweedfs

    SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  • Whoops: WebDAV:

    https://news.ycombinator.com/item?id=39417503

    SeaweedFS supports WebDAV. https://github.com/seaweedfs/seaweedfs/wiki/WebDAV

    I'm not able to find if both/restic supports mounting backups as WebDAV, but in theory there's nothing stopping you.

    It's 100% user space (expose a rest service) and supported by a bunch of file-browsers with a bit of a network aware component to it as well.

  • kopia

    Cross-platform backup tool for Windows, macOS & Linux with fast, incremental backups, client-side end-to-end encryption, compression and data deduplication. CLI and GUI included.

  • I think Kopia would be great for your use case

    https://kopia.io/

    It has a great system to snapshot files but only store data if it's changed. I use it in an environment where I can't use something like zfs to snapshot data because I don't have the ability to make decisions about what filesystem we're using. It's been amazing, love it so much!

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Ask HN: What distributed file system would you use in 2024?

    8 projects | news.ycombinator.com | 10 May 2024
  • Ask HN: Open-source Windows 11 backup solutions

    4 projects | news.ycombinator.com | 4 Apr 2024
  • I Backup

    2 projects | news.ycombinator.com | 27 Feb 2024
  • DuckDB + dbt for a serverless event correlation pipeline?

    1 project | /r/dataengineering | 7 Dec 2023
  • Very strange behavior/bug - devices stuck together

    1 project | /r/bcachefs | 7 Dec 2023