[D] What is the recommended approach to training NN on big data set?

This page summarizes the projects mentioned and recommended in the original post on /r/MachineLearning
Tensorflow Uber Machine Learning Machinelearning Mpi

Scout Monitoring - Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com
featured
InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

  • ecosystem

    Integration of TensorFlow with other open-source frameworks

    • I have a big data set that I would like to train on, so my thought is that I am going to do distributed training , but I am currently setting up MultiWorkerMirroredStrategy on tensorflow and i find it hard to use even with https://github.com/tensorflow/ecosystem/tree/master/spark/spark-tensorflow-distributor

  • horovod

    Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet.

    • And in case scaling is really important to you. May I suggest you look into Horovod?

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Discussion Thread

    1 project | /r/neoliberal | 10 Feb 2023

  • Anyone know of any papers or models for segmenting satellite images of a city into things like roads, buildings, parks, etc?

    1 project | /r/MLQuestions | 25 Apr 2022

  • Embedding Python

    1 project | /r/fortran | 21 Feb 2022

  • [D] PyTorch Distributed Training Libraries: What are the current options?

    3 projects | /r/MachineLearning | 7 Dec 2021

  • [D] GPU buying recommendation

    1 project | /r/MachineLearning | 17 Jul 2021