what is going on everybody welcome to
part 24 of our go language programming
tutorial series in this tutorial what
we’re in an attempt to do is apply what
we’ve been learning up to this point
with channels and go routines
concurrency in general to our web app to
see if we can make it go a little
quicker because right now it’s pretty
slow so what I’ve done is I’ve just kind
of run what we have up to this point and
if you press f12 in Google Chrome you’ll
get the kind of developer I forget what
you call this thing but anyways you’ll
get this thing if you can come to
let’s see performance here and then go
ahead and just hit this button start
profiling and reload this page wait for
it and in general this is more for like
front-end type stuff but we can actually
use it to time the the back end so here
we can see that this whole thing
actually took a really long time I don’t
think this is actually how long it
normally takes but what we can do is we
can trim off that part and we can see in
idle time alone we spent four thousand
two hundred and twenty three point five
milliseconds I’m gonna go ahead and run
this one more time though because I feel
like that was actually more than we
typically spend so I’m not really sure
if that was a fluke or what but the idea
is in that that idle time that’s time
that the browser spends doing basically
nothing because it’s waiting so this
time it’s thirty eight forty five
anyways that idle time let me run it one
more time while I just explain is is the
time that the browser spent it made the
request and then it’s just waiting for a
response so that’s just pure idle time
by the browser and the reason why we’re
having that idle time is because our web
server every time we look love how this
number just keeps dropping every time
anyway every time we visit this slash
AGG page its repo lling all of the
information from all of the site maps so
every time we pull it it’s gonna do all
that code again and that’s why we have
this idle time yet still dropping this
is really frustrating I’d like to see
just a really static one
at least whenever I did the testing it
was about 2800 to 3300 so hopefully this
won’t have dropped further yeah okay so
fine so let’s say right now we’re
running about 30 153 milliseconds so
just remember that we’ll come back and
we’ll run again
with on another tab and then we can kind
of compare them and we’re done so so now
let’s apply everything that we’ve
learned up to this point so to do that
you should just have handy if you don’t
go to the text-based version this
tutorial I have all the starting scripts
that you’ll need to follow along but you
should have this tutorial from just one
tutorial go the previous tutorial you
should have the latest version of our
news aggregator all ready to go sorry
it’s not quite up to standards again run
go format if you if you care so and then
obviously the the HTML okay so what I’m
going to do is I’m just going to start
modifying this new zag go to use go
routines channels and synchronization so
the first thing that we’re gonna go
ahead and do is let’s go ahead and
import sync and then let’s define our
you know far WG sync dot wait group
group and then now we’re gonna come into
basically this is the the main function
that we’re dealing with so one other
thing I forgot to do is close our
response body so just at some point you
just want to close that so response stop
body dot close I was forgot to do that
one of the viewers was like hey how come
you’re not doing that anymore because I
was being stupid so so make sure you
close it I’m not gonna worry about this
one for now because we’re gonna change
some things here but I mean really you
shouldn’t you should be closing it here
as well so response not body not close I
guess I’ll do it
we’re gonna move all this code so that’s
why I didn’t really care but anyways so
now inside of our news aggregator
handler basically we want
we’re gonna separate out these two
things but for now basically our go
routine the thing that we want to spread
out basically is this code here the code
that grabs the response the code that
unmarshal Xin to our end variable and
then obviously close the response that’s
the code that we need to run as a good
routine but also as a channel to return
the values that we are Marshall into our
n variable here so what I’m gonna do is
we’re gonna create a new function and I
guess I’ll put it I’ll just put it right
here so we’re gonna call this funk news
routine and for now we’re gonna
definitely need to pass a location of
the string type if we’re going to mimic
this function entirely so first we’re
just trying to mimic the function and
then we’ll convert it to an actual
channel in full so I’m gonna go ahead
and we’ll take all this here and we’ll
just cut it will come up here paste and
let me just tab this over if you’re in
sublime it should be control opening
square brackets that should fix it okay
but we’re gonna have a problem we
unmarshal into this news value now you
could pass the news object basically
through the function there’s a couple of
reasons why you wouldn’t want to do that
when we’re doing this concurrently but
mostly we’re gonna be returning these
the values as part of a channel so
instead what we’re going to do actually
is we’re just going to take Bar n news
from here cut come up here paste done
then what we want to do is we can come
down here and we could just say go and
then we run news routine so go news
routine and then we pass location now of
course part of the problem is this isn’t
really a channel right so we need to
first we need to create a channel so
let’s go ahead and do what call this Q
and that’s going to be able to make
we’re gonna make a channel of a news
type and then for the buffer we’ll just
say like 30 I mean you can put anything
it just needs to be bigger than the
amount that you’re going to be using so
you could say 500 or something I’m just
going to put 30 it’s totally fun so
that’s gonna be our channel so the
channel that we’re gonna then pass to
our our news routine is going to be
basically what we’ll do is it’ll be C
Chan news capital N and then here we’re
gonna pass Q as the channel to that go
routine now the other thing that we need
to do is we need to actually put the
values on the channel inside that good
routine so coming up to the news routine
basically channel use the channel
operator and then end so we’re gonna
send the value of and over to the
channel okay so once we have that
basically every value so we’ll have some
number of these news types that have
been unpacked into n so we’re basically
I have some number of these news types
and then we iterate over those news
types so so to handle for this what
we’re gonna do is basically we want to
take basically we’re gonna take this out
because that’s going to send it over to
the channel right and then coming over
here we are gonna want to do this
iteration over N and keywords and stuff
but we don’t just have a single end
we’ve got a cluster of ends right and
this cluster of ends is coming over the
channel so actually we need to iterate
over the channel and then we’ve got this
this news type basically so to iterate
over the channel we already know how to
do that so like fourth element colon
equal range over the Q Channel and in
fact let me just
cut this come here Elam will be that and
you know news type so now let’s just
take this and replace n with Elam
element okay so we’ve got so that way
we’re actually iterating over those news
maps getting populated there’s news maps
so everything’s pretty much the same the
only major difference here is rather
than a single news type that we’re going
to iterate over that has all of the
values we’re gonna have a bunch of these
news types and so we need to iterate
over those because each one’s on its own
little channel basically so now what we
need to do is handle for the
synchronization so we’ve already got our
our weight group up here so the only
other thing we need to do is make sure
as we iterate around we need to wait
group add one okay we also need to after
this for for loop we need to wait group
dot wait but then once we’re done we
need to close the queue and then we need
to defer the wait group done so defer
wait group done in the actual news
routine itself I think that’s everything
I guess I’ll just consider myself lucky
if that turns out to be everything so
I’m sure I miss something but let’s go
ahead and I broke the running news AG
let’s go ahead and rerun go run news AG
go on line 12 after top-level
declaration line twelve so here
oh I see so it’s just it’s a sink top
weight group type let’s try it again
save that come over here I was treating
like a function okay I’ll allow it
so come over here let’s refresh and
let’s open up the developer tools so
this is the old one I just went ahead
and left it up and then I just kind of
trimmed make sure we don’t have any
added idle time and then this is our new
one so let me just refresh first of all
real quick as you can see when we
refresh I mean it has the data there
like almost instantly compared to before
anyways but let’s actually get a real
time out of it wait for the rest okay so
I mean you can tell right out of the
gate this is definitely gonna be shorter
than than the other one
so this time our idle time is 454
milliseconds as compared to before where
we had 30 162 so a massive difference
compared to before so yeah that’s more
than six times quicker so so all we did
to get that that game was just using go
routines with channels and that’s it
really so pretty cool we were able to
cut that time down quite a bit now 454
millisecond response time still isn’t
anything we necessarily want to write
home about a a couple of things just to
consider is like for example some of
this time is going to be Washington
Post’s fault right
so for example how long what’s the
response time when we query the actual
like the actual sitemap index
what’s that response time and then also
what’s the response time for the slowest
sitemap in the list so at least when I
checked them it was like a about 70
millisecond response time for the
sitemap index and at least one of the
sitemaps I checked so I would say you
know you might be able to trim off let’s
say 150 milliseconds leaving us with
about 300 milliseconds that are hours
that we need to figure out what to do
with one thing that we could do to is
rather than like we could just on-site
load we could do this part
we could load like this the sitemap
index information we could actually do
this outside of the news a gambler
chances are they’re not gonna add a new
site map as often as they’re gonna get
new data so we want to load the new data
for the user basically every time they
access this page because new news is
coming out constantly
but how frequently is a new site map
gonna come out probably not very
frequently right that’s a little
different but to be honest now I mean I
reload and bam I mean the data’s there
like instantly compared to compared to
before I’m kind of surprised it’s
actually a 300 millisecond it seems
quicker than that to me but I guess not
I don’t know anyway that’s a lot of data
also I mean the other things that we
have going on are like the bigger gains
that we could actually make here
probably aren’t any more in our go code
that’s really more to do with kind of
front-end web development which I guess
could be our go code for example like
this table even though it does have
pagination it’s like instant right
because all the data is actually there
if we were to view the source all of the
table data is there right and that’s why
the search is so nice and so quick
because the data is already there it’s
just using some JavaScript and that’s it
so so to speed this up one of the things
that we could do in theory is like we
could like if we scroll this over oh my
head’s in the way but yeah so you can
see most of the time spent here is on
rendering there’s some sort of plane
flying over which is mostly for this
table and mostly because we have so much
information so one thing we could do is
add real pagination or something and not
actually have you know 1400 entries at
any one given time being passed because
that’s a lot of information we’ve got
the title we’ve got the URL data and
then all the keyword data that’s being
passed and shown on this page which just
simply is taking up a lot of time that’s
a lot of stuff so anyway at least for
now that’s a huge improvement over our
previous code we’ve got things running
much much much quicker so cool
if you guys have any questions comments
whatever feel free to leave them below
if you’ve got any and
movements anything that you can think of
to actually speed this up get it going
faster share with us otherwise that’s
all for now hope you guys have enjoyed
the ghost series and I will see you guys
in another tutorial
part 24 of our go language programming
tutorial series in this tutorial what
we’re in an attempt to do is apply what
we’ve been learning up to this point
with channels and go routines
concurrency in general to our web app to
see if we can make it go a little
quicker because right now it’s pretty
slow so what I’ve done is I’ve just kind
of run what we have up to this point and
if you press f12 in Google Chrome you’ll
get the kind of developer I forget what
you call this thing but anyways you’ll
get this thing if you can come to
let’s see performance here and then go
ahead and just hit this button start
profiling and reload this page wait for
it and in general this is more for like
front-end type stuff but we can actually
use it to time the the back end so here
we can see that this whole thing
actually took a really long time I don’t
think this is actually how long it
normally takes but what we can do is we
can trim off that part and we can see in
idle time alone we spent four thousand
two hundred and twenty three point five
milliseconds I’m gonna go ahead and run
this one more time though because I feel
like that was actually more than we
typically spend so I’m not really sure
if that was a fluke or what but the idea
is in that that idle time that’s time
that the browser spends doing basically
nothing because it’s waiting so this
time it’s thirty eight forty five
anyways that idle time let me run it one
more time while I just explain is is the
time that the browser spent it made the
request and then it’s just waiting for a
response so that’s just pure idle time
by the browser and the reason why we’re
having that idle time is because our web
server every time we look love how this
number just keeps dropping every time
anyway every time we visit this slash
AGG page its repo lling all of the
information from all of the site maps so
every time we pull it it’s gonna do all
that code again and that’s why we have
this idle time yet still dropping this
is really frustrating I’d like to see
just a really static one
at least whenever I did the testing it
was about 2800 to 3300 so hopefully this
won’t have dropped further yeah okay so
fine so let’s say right now we’re
running about 30 153 milliseconds so
just remember that we’ll come back and
we’ll run again
with on another tab and then we can kind
of compare them and we’re done so so now
let’s apply everything that we’ve
learned up to this point so to do that
you should just have handy if you don’t
go to the text-based version this
tutorial I have all the starting scripts
that you’ll need to follow along but you
should have this tutorial from just one
tutorial go the previous tutorial you
should have the latest version of our
news aggregator all ready to go sorry
it’s not quite up to standards again run
go format if you if you care so and then
obviously the the HTML okay so what I’m
going to do is I’m just going to start
modifying this new zag go to use go
routines channels and synchronization so
the first thing that we’re gonna go
ahead and do is let’s go ahead and
import sync and then let’s define our
you know far WG sync dot wait group
group and then now we’re gonna come into
basically this is the the main function
that we’re dealing with so one other
thing I forgot to do is close our
response body so just at some point you
just want to close that so response stop
body dot close I was forgot to do that
one of the viewers was like hey how come
you’re not doing that anymore because I
was being stupid so so make sure you
close it I’m not gonna worry about this
one for now because we’re gonna change
some things here but I mean really you
shouldn’t you should be closing it here
as well so response not body not close I
guess I’ll do it
we’re gonna move all this code so that’s
why I didn’t really care but anyways so
now inside of our news aggregator
handler basically we want
we’re gonna separate out these two
things but for now basically our go
routine the thing that we want to spread
out basically is this code here the code
that grabs the response the code that
unmarshal Xin to our end variable and
then obviously close the response that’s
the code that we need to run as a good
routine but also as a channel to return
the values that we are Marshall into our
n variable here so what I’m gonna do is
we’re gonna create a new function and I
guess I’ll put it I’ll just put it right
here so we’re gonna call this funk news
routine and for now we’re gonna
definitely need to pass a location of
the string type if we’re going to mimic
this function entirely so first we’re
just trying to mimic the function and
then we’ll convert it to an actual
channel in full so I’m gonna go ahead
and we’ll take all this here and we’ll
just cut it will come up here paste and
let me just tab this over if you’re in
sublime it should be control opening
square brackets that should fix it okay
but we’re gonna have a problem we
unmarshal into this news value now you
could pass the news object basically
through the function there’s a couple of
reasons why you wouldn’t want to do that
when we’re doing this concurrently but
mostly we’re gonna be returning these
the values as part of a channel so
instead what we’re going to do actually
is we’re just going to take Bar n news
from here cut come up here paste done
then what we want to do is we can come
down here and we could just say go and
then we run news routine so go news
routine and then we pass location now of
course part of the problem is this isn’t
really a channel right so we need to
first we need to create a channel so
let’s go ahead and do what call this Q
and that’s going to be able to make
we’re gonna make a channel of a news
type and then for the buffer we’ll just
say like 30 I mean you can put anything
it just needs to be bigger than the
amount that you’re going to be using so
you could say 500 or something I’m just
going to put 30 it’s totally fun so
that’s gonna be our channel so the
channel that we’re gonna then pass to
our our news routine is going to be
basically what we’ll do is it’ll be C
Chan news capital N and then here we’re
gonna pass Q as the channel to that go
routine now the other thing that we need
to do is we need to actually put the
values on the channel inside that good
routine so coming up to the news routine
basically channel use the channel
operator and then end so we’re gonna
send the value of and over to the
channel okay so once we have that
basically every value so we’ll have some
number of these news types that have
been unpacked into n so we’re basically
I have some number of these news types
and then we iterate over those news
types so so to handle for this what
we’re gonna do is basically we want to
take basically we’re gonna take this out
because that’s going to send it over to
the channel right and then coming over
here we are gonna want to do this
iteration over N and keywords and stuff
but we don’t just have a single end
we’ve got a cluster of ends right and
this cluster of ends is coming over the
channel so actually we need to iterate
over the channel and then we’ve got this
this news type basically so to iterate
over the channel we already know how to
do that so like fourth element colon
equal range over the Q Channel and in
fact let me just
cut this come here Elam will be that and
you know news type so now let’s just
take this and replace n with Elam
element okay so we’ve got so that way
we’re actually iterating over those news
maps getting populated there’s news maps
so everything’s pretty much the same the
only major difference here is rather
than a single news type that we’re going
to iterate over that has all of the
values we’re gonna have a bunch of these
news types and so we need to iterate
over those because each one’s on its own
little channel basically so now what we
need to do is handle for the
synchronization so we’ve already got our
our weight group up here so the only
other thing we need to do is make sure
as we iterate around we need to wait
group add one okay we also need to after
this for for loop we need to wait group
dot wait but then once we’re done we
need to close the queue and then we need
to defer the wait group done so defer
wait group done in the actual news
routine itself I think that’s everything
I guess I’ll just consider myself lucky
if that turns out to be everything so
I’m sure I miss something but let’s go
ahead and I broke the running news AG
let’s go ahead and rerun go run news AG
go on line 12 after top-level
declaration line twelve so here
oh I see so it’s just it’s a sink top
weight group type let’s try it again
save that come over here I was treating
like a function okay I’ll allow it
so come over here let’s refresh and
let’s open up the developer tools so
this is the old one I just went ahead
and left it up and then I just kind of
trimmed make sure we don’t have any
added idle time and then this is our new
one so let me just refresh first of all
real quick as you can see when we
refresh I mean it has the data there
like almost instantly compared to before
anyways but let’s actually get a real
time out of it wait for the rest okay so
I mean you can tell right out of the
gate this is definitely gonna be shorter
than than the other one
so this time our idle time is 454
milliseconds as compared to before where
we had 30 162 so a massive difference
compared to before so yeah that’s more
than six times quicker so so all we did
to get that that game was just using go
routines with channels and that’s it
really so pretty cool we were able to
cut that time down quite a bit now 454
millisecond response time still isn’t
anything we necessarily want to write
home about a a couple of things just to
consider is like for example some of
this time is going to be Washington
Post’s fault right
so for example how long what’s the
response time when we query the actual
like the actual sitemap index
what’s that response time and then also
what’s the response time for the slowest
sitemap in the list so at least when I
checked them it was like a about 70
millisecond response time for the
sitemap index and at least one of the
sitemaps I checked so I would say you
know you might be able to trim off let’s
say 150 milliseconds leaving us with
about 300 milliseconds that are hours
that we need to figure out what to do
with one thing that we could do to is
rather than like we could just on-site
load we could do this part
we could load like this the sitemap
index information we could actually do
this outside of the news a gambler
chances are they’re not gonna add a new
site map as often as they’re gonna get
new data so we want to load the new data
for the user basically every time they
access this page because new news is
coming out constantly
but how frequently is a new site map
gonna come out probably not very
frequently right that’s a little
different but to be honest now I mean I
reload and bam I mean the data’s there
like instantly compared to compared to
before I’m kind of surprised it’s
actually a 300 millisecond it seems
quicker than that to me but I guess not
I don’t know anyway that’s a lot of data
also I mean the other things that we
have going on are like the bigger gains
that we could actually make here
probably aren’t any more in our go code
that’s really more to do with kind of
front-end web development which I guess
could be our go code for example like
this table even though it does have
pagination it’s like instant right
because all the data is actually there
if we were to view the source all of the
table data is there right and that’s why
the search is so nice and so quick
because the data is already there it’s
just using some JavaScript and that’s it
so so to speed this up one of the things
that we could do in theory is like we
could like if we scroll this over oh my
head’s in the way but yeah so you can
see most of the time spent here is on
rendering there’s some sort of plane
flying over which is mostly for this
table and mostly because we have so much
information so one thing we could do is
add real pagination or something and not
actually have you know 1400 entries at
any one given time being passed because
that’s a lot of information we’ve got
the title we’ve got the URL data and
then all the keyword data that’s being
passed and shown on this page which just
simply is taking up a lot of time that’s
a lot of stuff so anyway at least for
now that’s a huge improvement over our
previous code we’ve got things running
much much much quicker so cool
if you guys have any questions comments
whatever feel free to leave them below
if you’ve got any and
movements anything that you can think of
to actually speed this up get it going
faster share with us otherwise that’s
all for now hope you guys have enjoyed
the ghost series and I will see you guys
in another tutorial
Please follow and like us:
Be First to Comment