Migrate 5 TB S3 bucket from one AWS account to another

• s5cmd

  11 2,359 7.3 Go

  Parallel S3 and local filesystem execution tool.

  

In other threads, I have seen this tool mentioned in another thread for 200 TB and about half a billion files, so your challenge should barely make it break a sweat: https://github.com/peak/s5cmd

• boto3

  37 8,728 9.7 Python

  AWS SDK for Python

  

Alternatively, you could create a Python script using either Boto3 or her asynchronous sister, aioBoto3 that will spin through the contents of the origin bucket and move it over to the destination.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• aioboto3

  2 664 6.2 Python

  Wrapper to use boto3 resources with the aiobotocore async backend

Alternatively, you could create a Python script using either Boto3 or her asynchronous sister, aioBoto3 that will spin through the contents of the origin bucket and move it over to the destination.

Suggest a related project