Concurrent-ruby (async) S3 files download

This page summarizes the projects mentioned and recommended in the original post on dev.to

Scout Monitoring - Rennaisance engineers rejoice! 1 gem 5 min to app monitoring
5-minute onboarding. No sales team. Devs in the support channels. No DevOps team required. Get the free app insights every engineer deserves with Scout Monitoring.
www.scoutapm.com
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
  • Concurrent Ruby

    Modern concurrency tools including agents, futures, promises, thread pools, supervisors, and more. Inspired by Erlang, Clojure, Scala, Go, Java, JavaScript, and classic concurrency patterns.

  • Let’s say we need to traverse through thousands of files in our S3 Storage in a Ruby app. Let’s say we have a bunch of logs there that we need to read every day and process. If we just use a straightforward approach, like opening, reading, and processing every file one by one, our solution will work, but It will take a lot of time to process. So we need to improve the speed. Here ruby-concurrent gem is our helper https://github.com/ruby-concurrency/concurrent-ruby. Why do we need this gem? Because It’s simpler to use than Threads and this gem has a lot more features in It. In this article, we will use Concurrent::Promises.future as the most common use of concurrent code. Because reading a file from S3 is an IO operation, we can get a huge benefit in speed if we gonna use concurrent code doing HTTP requests. Remember that concurrency will not give you speed improvements if in every Promise or Thread you will do any calculations. Because of Ruby GIL, every thread will be blocked until calculations are finished.

  • Scout Monitoring

    Rennaisance engineers rejoice! 1 gem 5 min to app monitoring. 5-minute onboarding. No sales team. Devs in the support channels. No DevOps team required. Get the free app insights every engineer deserves with Scout Monitoring.

    Scout Monitoring logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Ruby class pattern to work with API requests with built-in async approach

    8 projects | dev.to | 16 May 2024
  • A Tour of Go Examples in Ruby

    1 project | news.ycombinator.com | 16 Mar 2024
  • Exploring concurrent rate limiters, mutexes, semaphores

    2 projects | dev.to | 11 Sep 2023
  • EventMachine Performance Spikes

    2 projects | /r/ruby | 5 Sep 2023
  • My Adventure with Async Ruby

    1 project | news.ycombinator.com | 3 Sep 2023