Part 01 - Data and Architecture
309K subscribers
SUBSCRIBED
Neural Networks Demystified Part 1: Data and Architecture @stephencwelch Supporting Code: https://github.com/stephencwelch/Neur...In this short series, we will build and train a complete Artificial Neural Network in python. New videos every other friday. Part 1: Data + Architecture Part 2: Forward Propagation Part 3: Gradient Descent Part 4: Backpropagation Part 5: Numerical Gradient Checking Part 6: Training Part 7: Overfitting, Testing, and Regularization
Caption author (Russian)
Caption author (Spanish)
Caption author (Chinese (Taiwan))
Caption author (Ukrainian)
Caption author (Dutch)
Caption author (Korean)
Caption authors (German)
Caption author (French)
Caption author (Indonesian)
Caption authors (Portuguese)
Caption author (Lithuanian)
00:00
let's say you want to predict some output value y
00:03
given some input value x. For example
00:06
maybe you want to predict your score on test based on how many hours you sleep and
00:10
how many hours you study
00:11
the night before. To use a machine learning approach
00:15
we first need some data. Let's say for the last three tests
00:19
you recorded your number of hours studying, you number of hours sleeping,
00:22
and your score on the test. We'll use the programming language Python
00:26
to store data in two-dimensional "numpy" arrays. Now that we have some data
00:31
we're going to use it to train a model to predict how well you'll do
00:35
on your next test,
00:36
based on how many hours you sleep and how many hours you study. This is called a
00:40
supervised regression problem.
00:42
It's supervised because our examples have inputs
00:45
and outputs. It's a regression problem because we're predicting your test
00:49
score, which is a
00:50
continuous output. If, we were predicting your letter grade
00:54
this would be called a classification problem and not a regression problem.
00:58
There are an overwhelming number of models within machine learning
01:02
here we're going to use a particularly interesting one
01:05
called an artificial neural network these guys are loosely based on how the
01:09
neurons in your brain work
01:11
and had been particularly successful recently at solving really big
01:14
and really hard problems
01:17
before we throw our data into the model we need to account for the differences in
01:20
the units of our data.
01:22
Both of our inputs are on hours, but our
01:25
output is a test score, scaled between 0 and 100.
01:28
Neural networks are smart, but not smart enough to guess the units of our data.
01:33
It's kinda like asking our model to compare apples to oranges
01:36
when most learning models really only want to compare apples to apples.
01:40
The solution is to scale our data, thus our model will only see standardized units.
01:45
Here we're going to take advantage of the fact that all our data is positive
01:49
and simply divide by the maximum value for each variable
01:52
effectively scaling a result between 0 and 1.
01:57
Now we can build our neural net. We know our network must have two inputs,
02:01
and one output because these are the dimensions of our data.
02:04
We'll call or output layer y hat, because it's an estimate of y,
02:08
but not the same as y. Any layer between our input and output layers is called a
02:13
hidden layer
02:13
recently researchers have built networks with many many many hidden layers
02:18
these are known as deep belief networks giving rise to the term deep learning
02:22
here going to use one hidden layer with three hidden units
02:26
but if we wanted to build a deep neural network we would just stack a bunch of these
02:29
layers together.
02:30
In neural net visuals, circles represent neurons
02:35
and lines represent synapses
02:37
Synapses have a really simple job, they take a value from their input, multiply it by a
02:42
specific weight
02:43
and output the result. Neurons are a little more complicated
02:46
their job is to add together the output from other synapses
02:50
and apply an activation function. Certain activation functions allow neural
02:55
nets to model complex
02:56
nonlinear patterns that simpler models may miss. For our neural net we'll use
03:01
sigmoid activation functions.
03:03
Next we'll build out our neural net in Python
English
###