**resnet**88

[1806.10909] ResNet with one-neuron hidden layers is a Universal Approximator

5 days ago by arsyed

We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation functions can uniformly approximate any Lebesgue integrable function in d dimensions, i.e. ℓ1(ℝd). Because of the identity mapping inherent to ResNets, our network has alternating layers of dimension one and d. This stands in sharp contrast to fully connected networks, which are not universal approximators if their width is the input dimension d [Lu et al, 2017; Hanin and Sellke, 2017]. Hence, our result implies an increase in representational power for narrow deep networks by the ResNet architecture.

resnet
neural-net
universal-approximator
5 days ago by arsyed

Under the hood: Facebook Marketplace powered by artificial intelligence - Facebook Code

9 weeks ago by danhon

"To understand the relationship between buyer activity and product content, the system also incorporates a model for the buyer, created with embeddings using the demographic information from the person’s Facebook profile and keywords from searches within Marketplace." - you know what this means, and it should not come as a surprise

facebook
ai
machinelearning
deeplearning
cnn
via:gnat
deeptext
lumos
resnet
marketplace
entityextraction
9 weeks ago by danhon

Residual Networks | Sleeba Paul

resnet
residual
deep-learning
tutorial

july 2018 by nharbour

Dreamer | Learner | Storyteller

july 2018 by nharbour

[1605.07146] Wide Residual Networks

may 2018 by arsyed

Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR, SVHN, COCO, and significant improvements on ImageNet. Our code and models are available at this https URL

resnet
wide-resnet
neural-net
may 2018 by arsyed

tensorflow-resnet: ResNet model in TensorFlow

april 2018 by lxp121

py code of tensorflow (resnet)

github
tensorflow
resnet
april 2018 by lxp121

[1708.07120] Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates

april 2018 by arsyed

In this paper, we show a phenomenon, which we named "super-convergence", where residual networks can be trained using an order of magnitude fewer iterations than is used with standard training methods. The existence of super-convergence is relevant to understanding why deep networks generalize well. One of the key elements of super-convergence is training with cyclical learning rates and a large maximum learning rate. Furthermore, we present evidence that training with large learning rates improves performance by regularizing the network. In addition, we show that super-convergence provides a greater boost in performance relative to standard training when the amount of labeled training data is limited. We also derive a simplification of the Hessian Free optimization method to compute an estimate of the optimal learning rate. The architectures and code to replicate the figures in this paper are available at github.com/lnsmith54/super-convergence.

neural-net
sgd
resnet
training
performance
leslie-smith
learning-rate
april 2018 by arsyed

Decoding the ResNet architecture // teleported.in

february 2018 by rishaanp

A blog where I share my intuitions about artificial intelligence, machine learning, deep learning.

resnet
shortcut-connections
network-architecture
convolutional-neural-networks
cnns
deep-learning
fast.ai
from pocket
february 2018 by rishaanp

Yet Another ResNet Tutorial (or not) – Apil Tamang – Medium

february 2018 by rishaanp

The purpose of this article is to expose the most fundamental concept driving the design and success of ResNet architectures. Many blogs and articles go on and on describing how this architecture is…

ResNet
neural-networks
network-architecture
from pocket
february 2018 by rishaanp

**related tags**

Copy this bookmark: