Scheduling in Data Engineering

• rocketry

  32 3,194 0.6 Python

  Modern scheduling library for Python

I'm the developer behind Rocketry, a Pythonic scheduling engine. I originally developed it for other needs than data engineering but as some have said they have started to replace Airflow with it, I thought perhaps some of you might like it as well. I'm also interested in your opinions.

• rocketry-with-fastapi

  6 81 0.0 JavaScript

  Example of how to create a scheduler with an API

I don't think Rocketry will achieve the same level of adaptation as Airflow and it is missing many advanced features Airflow has such as built-in UI (working on such), executors for containers etc. but considering how versatile it is and how powerful the scheduling is (basically logical statements), I think it could be interesting for those data engineers who need to customize their setup or need something smaller in scale.

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

Suggest a related project