Press "Enter" to skip to content

Let’s code a neural network in plain JavaScript


good Monday morning today we’re going to
code a neural network from scratch in
JavaScript
no libraries no assumptions about
knowing math because I don’t know any
math I don’t know statistics I don’t
know AI I only have a vague very vague
idea about what our neural network is
from some YouTube videos I watched I
have been procrastinating on learning
machine learning for so long let’s just
do it I am MC J and you are watching fun
[Music]
today’s show is sponsored by pusher who
sha makes it easy to add real-time
communication and collaboration features
to your app chats pub/sub mobile push
and Pheebs check them out at push r dot
fun fun dot-com that link is also
in the episode description Sweden is
very warm so my energy levels might be
steadily dropping until I die in this
episode alright so last week I watched a
video by Daniel Schiffman on the basics
of implementing a neural network I like
Schiffman I wanted to get started with
with machine learning and I just figured
that hey neural networks that sounds
cool it’s start with that I streamed it
on Twitch
I sometimes stream on Twitch so if you
don’t follow me on Twitch you should at
twitch.tv slash fun fun function twitch
is still a little bit of a hobby but I
really like doing it it was a great deal
of fun didn’t turn out too well though
it’s in terms of results I didn’t really
complete the neural network but I’m not
sleeping around with it a little bit
more and I think I have something to
show you from what I’ve learned so in
this machine learning series I’m going
to try to piggyback a lot on the fact
that I don’t know anything I don’t know
are much Python I don’t know much about
math I don’t know much about statistics
I don’t know a lot about probability I
don’t know a lot about machine learning
and AI so I’m going to be utilizing that
beginner’s mind 7 and try to use the
space from where I just when I learned
something I’m going to try to still
remember what it was like before I
learned it and I’m going to use that to
to make these videos for you so I’m kind
of making making these videos for myself
but like two days ago
hopefully that will turn out into
interesting content or it might just be
very very messy this is a new and
experimental way of doing videos for me
so we’ll leave your comments down below
after you’ve watched it where you
thought about it and what how I can make
it better or if I should just throw it
away anyway let’s get going have a look
at this
I’m using observable notebooks because
they are amazing not getting sponsored
or anything they’re just these in really
handy interactive notebooks it’s kind of
like jsfiddle but you can actually store
data and oh they’re so good I won’t go
into a high-level explanation of what a
neural network is honestly because
there’s already a pretty much the
perfect video on that exists already by
three blue one brown but what is a
neural network I’ve linked it here just
look at that first before viewing this
video if you’re unsure what a neural
network is it’s going to give you a good
sort of high-level overview of what it
is he also goes into a couple of Matthew
details that it’s they’re going to be
scary but don’t worry about it we’re not
gonna go into that just try to you know
like oh there’s math so our goal is to
make out like an ultra ultra simple
neural network like the silliest little
neural network like it’s not even a
network it’s just one brain cell one
neuron perceptron whatever like words
they use we’re not going to use those
words where we’re trying to we’re gonna
try to focus here on getting getting the
principle right so I’m not gonna spend a
lot of time in words I’m not going to
spend a lot of time in the the brain
analogies wouldn’t accidents in New York
like blood that’s not really important I
don’t mean to on all the
interesting research that has been done
on that but I find that getting getting
confused and into the the math and the
the brain science of things kind of is
kind of distracting has been distracting
for me when learning these things I just
want to get to the essence of it and
understand how it solves problems after
that well then we can go dive into the
interesting aspects of what lies beyond
and how it how it connects into
everything but for now we’re interested
in looking at what a neural network is
and how it works so we’re going to build
a little silly neural network and the
purpose of the this little silly neural
network is to classify
on what side of a coordinate system that
a point is so you just imagine a
coordinate system and this is like line
between it and then it’s a bunch of
points and then the neural network we
will teach the neural network to figure
out if a point is on this side of the of
the coordinate system or on the line
going through the coordinate system or
on this side it’s not a very smart AI
that we’re building but you know baby
steps building Skynet is a it’s a later
part in this series okay let’s begin by
we’re gonna generate a bunch of random
points you know like points with XY
coordinates in a coordinate system in
machine learning there’s a lot of
coordinate systems that’s a little bit
of math and it’s a little bit of
geometry there which might give you
flashbacks from from high school or
whatever but like try to breathe the
first thing that we’re going to do is to
pull in rammed up this is one of the
nice things about observable is that you
can just pull in our NPM modules it’s
really nice so let’s create some random
points random points I’m going to use
the are range function and the orange
function gives us like numbers from 0 to
5 Z like it just gives us you know the
numbers one two yeah I’m really not
interested in the actual numbers here I
just want an array that is 100 long so
that I can do math on it and then I can
really care what is in the array I just
want you you know do I get assigned a
random number there and this was like
during the stream at one of the stream
viewers actually wrote this function for
me so it was nice and interactive which
way and there’s and we want the numbers
to range from minus 1 to 1 so you see
here like
we get like you see like trenches from
minus one to two 180th right numbers by
the way you might take offense to this
syntax here this is observable an
observable thing this is not strictly a
global variable like like playing
JavaScript interpret it this is actually
creating an observable so when if I do
like random I can do random points dot
length here and ask random points there
so whenever I change to this range here
you want to watch this number here let’s
change this to 200 that will
automatically update so that this is how
this is a basic item or thing in
observable and how you declare them so
it’s it’s it’s at first it seems like a
disgusting syntax but it’s actually
really really nice to deal with anyway
this is not actually points it yet it’s
as you see it’s it’s just just numbers
but we pour at point is an x-coordinate
and y-coordinate so that is it’s what we
won’t we won’t and x and y-coordinate so
am trying to be really really happy even
though it is so warm who we now have
some X&Y coordinates that’s nice
alright I want to make this as visual
and and interactive as possible so that
you don’t need to work execute a lot of
code in your head and keep abstractions
in your head so we’re gonna in the
series we’re going to try to as much as
possible constantly visualize what the
hell is going on so I’m going to draw
the coordinate system and we’re gonna
draw it with SVG what mpj bear with me
okay
it drew something but it draws
everything in the top left corner and
that is because you know the the random
points that I just keep it with between
minus 1 and 1 and I don’t know why I did
that
that was just dumb let’s not do that
instead I want to keep them between like
how big the the the coordinate system is
so I’m going to like make these
variables so I’m going to kill this x
max and I’m going to call this that Y
max and we’re going to why why max 400
and I’m going to also do x max 400 by
the way this when I after I finish this
episode I’m going to put this a notebook
public so you can find it in the app so
description it’s not gonna have this
Earl it’s gonna have another early in
the after it’s polish but you’re gonna
be fine it and you can play around with
it yourself while you’re doing this or
you can just create it from scratch and
for a while anyway
let’s see look up X max here X Max here
and the random points has to be
generated from this so X is going to be
0 to X Max and it’s going to be C 0 to Y
max and what is there and then it didn’t
work at all
it not whatsoever so this look like
what this actually look this actual will
look fine okay inspect an SVG the SVG it
only draws one circle and then there’s
circle inside the circle and then a
circle inside this you can watch
inception I didn’t want inception
oh it’s because I’m not closing the tag
there we go okay radius 5 is is too big
radius 3 seems more sane this shows you
a lot of the power with observable let’s
let me just when I three generate points
here you see that it’s updating
everything is mmm observing than
reactive and nice super cool I love
observable remember that I said that we
wanted to start classifying we’re on the
what side of the coordinate system the
the circles are or L not the circles the
points that we are going circles for we
have the data and then we have the
visualization of the data anyway I’m
going to draw a line come on
no oh yes ok let’s note I want it to be
purple so we’re going to call this team
a team one and we weren’t told this team
minus 1 so 1 minus 1 and I’m going to
write a little function that we call
team and it’s going to be take a point
and if the point a point X is above a
point point why it’s it’s going to be
team team 1 otherwise it’s going to be t
minus y and then I’m actually going to
use this function here to give it a
different color so that we make sure
all right so it turns out that this team
is team -1 and this is team 1 now the
tricky thing with teaching machine
learning is that the examples that you
need to use in order to understand what
the hell is happening are so simple that
you might as well have used code and
that is that is definitely the case in
in this in this case so if we look at
this this we have already solved the
problem here by using this little team
function that is that with this team
function takes a point and then it
actually act actually correctly
classifies a point so in this problem
this is a very very very simple problem
so it’s it because it has only two
inputs so it has an X and a y-coordinate
that that’s all that determines what how
where in what team appoint and Suppan
and the logic is also very simple it’s
just if if X is bigger than then Y then
it ends up in one team or where another
and in real life we have much more
complicated classification problems for
instance let’s say that you have a bunch
of data on house prices hospice is a
classical problem for this so every row
might be like a sale or a house so
instead of these things these random
points x and y coordinates it might be
just houses in in a long list and they
they each each table row of these houses
they have like like a like a square like
how big the apartment is might have
square meters or swear feet or whatever
you use in alien land mytab the where it
is located what area it is located it
might have like a the year that the
building was was built the number of
rooms and it might have like 20 of these
might serve in more complicated problems
there might be hundreds of these things
in this problem that we’re looking at
here it’s the X&Y the coordinates but in
an another it’s another problem it might
be aspects of a house that effects the
price such as size and and and location
and in another problem might be flowers
it might be petals and petals size like
when classifying images of animals it
might be like have pointy ears they have
like if they have whiskers and what we
want to do is that we want to take all
of these inputs or traits or whatever we
want to call them and we want to gobble
them together somehow with some logic
and some not really lawyer like if we
did it with programming we would use
logic we used a lot of if statements and
or perhaps if the house prices this and
like if it’s the location is this then
this meant effect and like but it it
works in this simple case but in the
house pricing case it’s it’s gonna be
extremely complicated and it’s just code
is not very suitable if we created a
housing price predictor we would take
all these housing prices and garble them
together and somehow produce an output
price based on all their inputs so with
the housing prices we wouldn’t be able
to visualize this weather with a 2d
coordinate system because there would be
way more dimensions to the problem than
there are coordinates in real life like
it wouldn’t like it would be not
three-dimensional it would be like 13
dimensions and humans can visualize that
that is when machine learning becomes
very abstract and very useful but that’s
not what we’re gonna do today or the
reason that I explained that is to give
you an idea of why we use machine
learning and have you like you have to
use your imagination a little bit to
understand the why of this because but
in this case where a problem is very
pedagogically simple ins used these two
points let’s create an AI we’ve got
create a function call guess that does
exactly the same thing as team but
instead of using logic to determine
where where point is instead of using
math it’s going to use machine learning
so let’s just start yes it’s just bear
with me here yes it’s a function guess
it’s a function that takes Waits and it
takes inputs and you know inputs it’s
like oh that’s a general term for things
oh my god it’s gonna call it point
because the point is our inputs in this
case remember then the inputs in this
case is just two points on a on a
coordinate system and x and y y but in a
neural netting might be like housing
prices or like cat’s ears or whiskers it
might be like a list of thing and that’s
called the input but in this case the
point is it’s fine otherwise we might
get confused if I try to think too much
about the abstraction yeah yeah let’s
just okay so let’s start there we are
going to need some weights because
otherwise we’ve got computer I’m gonna
what are the weights the weights are
kind of the brain state of our AI the
the weights is what we’re going to be
multiplying every everything with and
the weights they’re gonna start out
random the AIS should just have some
weights and then it’s going to we’re
going to train the AI we’re going to
have the AI guess based on its random
weights what a given coordinate is what
what team a given coordinate is and it’s
gonna get wrong because the the the
weights are all random it’s gonna just
multiply it by its weights and it’s
gonna be pretty off but we are then
going to give it the correct
position because we know that position
in in this case we have a function that
gives us the correct answer that the
team but in in real life we would have
like the actual sale prices for houses
that we could give and when we give it
that you can see how wrong was I okay
I was this wrong then I adjust my weight
a little bit towards that wrongness if
that’s the gist of it don’t worry if
you’re confused that it’s normal this is
not intuitive in any way but let’s just
bear with me I’m just trying to give you
a sense of where we’re going before I
start coding so just try to follow
whoo yes okay we have some random
weights it’s an X and it’s a Y okay this
is the weights this is the initial state
of our little AI and this is how its its
brain looks like inside it’s pretty
nonsensical and it will be continue to
be nonsensical but it’s even more
sensible now because it’s just two
random numbers
so if weights are is the brain of our AI
then then guess is there a scuttle that
we’re going to put that brain in and
that is why it take takes weights as an
argument so how does guess use the
weights to think I’m going to show you
that but first I’m going to switch shirt
and probably switch lighting and hair do
and maybe complexion as well because I’m
going out and meeting some friends and
continuing this later one two
hello I’m mpj from the not-too-distant
future I’m indicating this by this hat
I’m sitting around had vivid editing
editing this video and I realize it’s
very long so I’ve split it into multiple
parts so cliffhanger please remember
that this episode was kindly sponsored
by pusher so please check them out at
pusher top front of action column you
can subscribe here so that you don’t
miss the next episode on on Monday or if
you are watching this from the future
then you can watch the next episode
right here and otherwise this is gonna be just a recommended video I think
Please follow and like us: