Changing Tires at 100mph: A Guide to Zero Downtime Migrations

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • gh-ost

    GitHub's Online Schema-migration Tool for MySQL

  • Actually I never tried but I was scared by the small print of GH not using RDS themselves [1] and Ghost relying on lower-level features that might be not easily available in RDS. Also I had the impression you have to setup a normal non-RDS replica attached to your RDS master?

    [1] https://github.com/github/gh-ost/blob/master/doc/rds.md

  • pg-osc

    Easy CLI tool for making zero downtime schema changes and backfills in PostgreSQL

  • Postgres has some less robust tooling, at least both of these carry the "experimental" tag:

    https://news.ycombinator.com/item?id=29825520

    https://github.com/shayonj/pg-osc

    But, still, they do it for you.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • reshape

    An easy-to-use, zero-downtime schema migration tool for Postgres

  • Anybody interested in this subject might also be interested in a tool for Postgres I’ve been working on, Reshape: https://github.com/fabianlindfors/reshape. It aims to fully automate away all the pain and manual steps zero-downtime migrations normally requires :)

  • database-lab-engine

    DBLab enables 🖖 database branching and ⚡️ thin cloning for any Postgres database and empowers DB testing in CI/CD. This optimizes database-related costs while improving time-to-market and software quality. Follow to stay updated.

  • litestream

    Streaming replication for SQLite.

  • I imagine that GP means a hardware failure severe enough that it permanently borks the disk drive of the machine. In that case vanilla SQLite would indeed lose data written since the time of the last offsite backup. MySQL/Postgres/etc typically have streaming replication which means you can promote one of the replicas and not lose any (or at least much less) data. There are some new-ish projects like Litestream (https://litestream.io/) that provide a similar functionality for SQLite by streaming the WAL file changes to S3.

  • ghost_adapter

    Run ActiveRecord migrations through gh-ost

  • At least for Rails there are several gems available (ie https://github.com/WeTransfer/ghost_adapter or https://github.com/departurerb/departure) that seamlessly hook into the existing migration system and will run all eligible migrations through gh-ost or pt-osc as needed. You're right that it's not free but it isn't all that far off either.

    That said, online schema migrations are a specialized tool designed for very big tables that take hours to run an ALTER TABLE on. If all your tables are small enough that alterations take less than a second or so, don't bother and just block the table for a bit. It's fine.

  • departure

    Percona's pt-online-schema-change runner for ActiveRecord migrations.

  • At least for Rails there are several gems available (ie https://github.com/WeTransfer/ghost_adapter or https://github.com/departurerb/departure) that seamlessly hook into the existing migration system and will run all eligible migrations through gh-ost or pt-osc as needed. You're right that it's not free but it isn't all that far off either.

    That said, online schema migrations are a specialized tool designed for very big tables that take hours to run an ALTER TABLE on. If all your tables are small enough that alterations take less than a second or so, don't bother and just block the table for a bit. It's fine.

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • janet

    A dynamic language and bytecode vm

  • The "boots up our application within a second on a Raspberry Pi" comment in the FAQ[0] is actually about my development setup, as I don't own a fancy computer myself. It was mainly meant to illustrate the fast startup time of the Janet[1] language. In fact, that startup time is so fast, you can run your app as a CGI script. That has some more possible advantages, see the FAQ for more details.

    [0] FAQ: https://withoutdistractions.com/cv/faq#technology

    [1] Janet: https://janet-lang.org/

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Eradicating N+1s: The Two-Phase Data Load and Render Pattern in Go

    4 projects | news.ycombinator.com | 28 May 2024
  • Ask HN: What tool(s) do you use to code review and deploy SQL scripts?

    5 projects | news.ycombinator.com | 14 Apr 2024
  • Show HN: Tsynamo – Type-friendly DynamoDB query builder for TypeScript

    3 projects | news.ycombinator.com | 10 Apr 2024
  • Features I wish PostgreSQL had as a developer

    1 project | news.ycombinator.com | 5 Apr 2024
  • DBeaver – open-source Database client

    10 projects | news.ycombinator.com | 10 Mar 2024