Krizhevsky, Sutskever, Hinton, ImageNet Convolutional Network, 2012

<p>Jun 30, 2015 09:03 PM</p>

<p>open access; &nbsp;key paper;</p>

<p>Univ of Toronto; &nbsp;2012;</p>

<p>We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes.</p>

<p>On the test data, we achieved top-1 and top-5 error rates of 37.5%
and 17.0% which is considerably better than the previous state-of-the-art.&nbsp;</p>

<p>The neural network, which has <strong>60 million parameters and 650,000 neurons</strong>, consists&nbsp; of <strong>five convolutional layers,</strong> some of which are followed by<strong> max-pooling layers,&nbsp;</strong>and <strong>three fully-connected layers with a final 1000-way softmax.</strong></p>

<p>To make training faster, we used non-saturating neurons and&nbsp;
a very efficient GPU implementation of the convolution operation.
&nbsp;To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called &ldquo;dropout&rdquo; that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%,compared to 26.2% achieved by the second-best entry.</p>

<p></p>

<p>Intro: &nbsp;small &nbsp;datasets are relativ. easy. &nbsp; need larger sets to expose shortcomings.</p>

<p>ImageNet: &nbsp;15 millions labeled high-res images in over 22K categories.</p>

<p>our models need a lot of prior know.: &nbsp;CNNs &nbsp;one class of models;</p>

<p>CNNS have much fewer connections and params;</p>

<p>&nbsp;
&nbsp;</p>