« earlier · Making neural nets uncool again
“This is great news! Here has been my experience using I had been training a deep learning network using Keras with Tensorflow, for diagnosing medical images - and it took me several months of hard work - tweaking parameters, training, and testing to get acceptable levels of accuracy for our models. And then last month, I switched to (their pre-release version) and I was blown away - my models trained faster, and I matched and finally exceeded accuracy levels achieved with my earlier models. And I accomplished what had taken several months in Keras, in just a few days! And the biggest reasons for it were in my view,'s learning rate finder, the differential learning rates, and Test Time augmentation - all which are advanced features built into And the other great thing is that uses the best defaults automatically, and it trains much, much faster than Keras / TF for some reason.
So I can't wait to try the new release out. I think has set a new bar for deep learning frameworks in terms of speed and ease of use. Thank you for all your great work!” sits on top of PyTorch, making it even easier to use.  pytorch 
11 weeks ago by Sylphe
Ten Techniques Learned From
1. Use the library
2. Don’t use one learning rate, use many
3. How to find the right learning rate
4. Cosine annealing
5. Stochastic Gradient Descent with restarts
6. Anthropomorphise your activation functions
7. Transfer learning is hugely effective in NLP
8. Deep learning can challenge ML in tackling structured data
9. A game-winning bundle: building up sizes, dropout and TTA
10. Creativity is key
dl  ml  deeplearning  transferlearning  learningrate 
august 2018 by drmeme

« earlier    

related tags

adam  adamw  ai  algorithm  algorithms  ami  anaconda  arguments  art  attention  augmentation  autoencoder  autograd  automl  averaging  awd-lstm  aws  batch-normalization  batch-size  beam-search  beam  benjamin-franklin  blogs  cam  capsule  capsules  career  categorical  class-activation  classification  classifier  cli  cloud  cluster  cnn  cnns  code  colab  collaborative-filtering  collaborative  competition  conda  content  convolutional-neural-networks  convolutions  corpus  course-notes  course  courses  cuda  cyclical-learning-rate  dae  data-augmentation  data  dataset  dawnbench  decision-trees  deep-learning  deeplearning  denoising  deploy  deployment  differential-learning-rates  discussion  dl  download  dropbox  dropout  education  embedding  ensemble  fastai  feature-importance  filtering  free  gan  gcloud  gdrive  generative-adversarial-networks  github  google  gpu  gradient-boosting  health  healthcare  heat-map  heatmap  hinton  hype  hyperparameter-optimization  image-classification  image  imagenet  images  individual-conditional-expectation  inference  install  interpretability  interpretation  jigsaw  job  jupyter-notebook  jupyter  k-nearest-neighbours  kaggle  keras  knn  lang:en  language-modeling  learning-rate-annealing  learning-rate-finder  learning-rate  learning  learningrate  lecture  libraries  library  link  lm  long-short-term-memory_networks  lstm  machine-learning  machine_learning  machinelearning  math  metaphor  ml  model  models  movie  movies  multi-modal  naive-bayes  nearest-neighbour  network-architecture  neural-net  neural-networks  neural-style-transfer  nlp  nmslib  nn  normalization  notes  now.js  object-detection  occlusion  optimizer  optimizers  overfitting  packages  parameter  partial-dependence  performance  pil  pillow  programming  python  pytorch  qrnn  random-forest  randomforests  rankgauss  reading  recommendation  rectified-linear-unit  recurrent-neural-networks  reinforcement-learning  resnet  resources  rnn  rnns  ruder  sae  sagemaker  script  sdae  search  seq2seq  setup  sgd  sgdr  shortcut-connections  single-shot  spam  spell-check  spelling  ssd  statistics  structure  structured-data  summarisation  summarise  super-convergence  svm  swa  tasks  teaching  technique  techniques  tensorboard  test-time-augmentation  testing  text_classification  time-series-data  tips  training  transfer-learning  transferlearning  tree-interpreter  tuning  tutorial  unit-test  unit-testing  validation  verge  vggcam  videos  visualization  waterfall-charts  weight  wikipedia  windows  winner  word-embedding  word2vec  work  xgboost 

Copy this bookmark: