machine_learning   11929

« earlier    

[1706.03292] Poseidon: An Efficient Communication Architecture for Distributed Deep Learning on GPU Clusters
Deep learning models can take weeks to train on a single GPU-equipped machine, necessitating scaling out DL training to a GPU-cluster. However, current distributed DL implementations can scale poorly due to substantial parameter synchronization over the network, because the high throughput of GPUs allows more data batches to be processed per unit time than CPUs, leading to more frequent network synchronization. We present Poseidon, an efficient communication architecture for distributed DL on GPUs. Poseidon exploits the layered model structures in DL programs to overlap communication and computation, reducing bursty network communication. Moreover, Poseidon uses a hybrid communication scheme that optimizes the number of bytes required to synchronize each layer, according to layer properties and the number of machines. We show that Poseidon is applicable to different DL frameworks by plugging Poseidon into Caffe and TensorFlow. We show that Poseidon enables Caffe and TensorFlow to achieve 15.5x speed-up on 16 single-GPU machines, even with limited bandwidth (10GbE) and the challenging VGG19-22K network for image classification. Moreover, Poseidon-enabled TensorFlow achieves 31.5x speed-up with 32 single-GPU machines on Inception-V3, a 50% improvement over the open-source TensorFlow (20x speed-up).
machine_learning  TensorFlow 
yesterday by amy
UC Irvine Machine Learning Repository
UC Irvine Machine Learning Repository!

We currently maintain 394 data sets as a service to the machine learning community. You may view all data sets through our searchable interface.
machine_learning  dataset 
yesterday by slavko
AWS public datasets
AWS hosts a variety of public datasets that anyone can access for free.
Previously, large datasets such as satellite imagery or genomic data have required hours or days to locate, download, customize, and analyze. When data is made publicly available on AWS, anyone can analyze any volume of data without needing to download or store it themselves. These datasets can be analyzed using AWS compute and data analytics products, including Amazon EC2, Amazon Athena, AWS Lambda and Amazon EMR.
machine_learning  dataset 
yesterday by slavko
Datasets « Deep Learning
These datasets can be used for benchmarking deep learning algorithms
database  data  ai  datasets  machine_learning 
2 days ago by rrraul

« earlier    

related tags

abtesting  academia  ai  aiux  algorithmic_fairness  algorithms  alpha  apple  artificial_intelligence  audio  automation  biomimetics  bookmarks_menu  britain  causality  cnn  collections  computaional_advertising  computational_thinking  computer  coreml  critique  data-science  data  data_science  database  dataset  datasets  deep_learning  deeplearning  design  development  directory  economics  education  ethical_ai  ethics  evolution  evolutionary  face_detection  facebook  fake_news  feature_engineering  fill_probability  for_friends  framework  future  gafa  gan  geography  gesture-recognizers  github  google-home  google  google_tensorflow  graduate_education  growth_hacking  hcds  i_remain_skeptical  ios  iphone  janzing.dominik  java  javascript  julia  kottke  learning_struggles  learning_to_learn  library  limit_order_book  machine-learning  machinelearning  margarita  market_making  market_microstructure  math  metal  microprice  ml  muckrock_crowdsourcing_ideas  muckrock_feature_ideas  neural_networks  nlp  nytimes  paper  phd  pmz  prediction  productivity  programming  programming_evolution  python  re:adafaepov  reading  recommendations  recsys_eval  research  security  shared  simulation  small_data  social_media  software  speech-recognition  statistics  swift  tensorflow  to:nb  to_read  tools  tutorials  ux  video  vision  visualization  voice-recognition 

Copy this bookmark:



description:


tags: