what’s going on everybody welcome to
part 13 of the go language tutorial
series in this tutorial what we’re doing
is just building on the last tutorial
with our news aggregator web app idea in
the previous tutorial we were talking
about looping and how we could loop over
this the information that we’re pulling
which in this case is just a bunch of
sitemaps now what we want to do is
actually visit those site maps and then
pull the information from there which is
actually the load you know the actual
articles themselves their titles and
keywords stuff like that so that’s what
we’re gonna be doing here we also need
to clean this up a little bit I kind of
was really explicit with every part that
we wanted to break down here but what we
can do is actually we can we can
simplify this quite a bit so it’s all
just one struct which is gonna make
sense you wouldn’t want to I mean you
can break them down is this far if you
really wanted to the problem is as this
program grows that’s going to get super
problematic like well like to do the
next the next site map and all the
information that we want from there
which will be titles keywords and the
location you’re gonna end up with like
15 structs if you’re not careful so so
let’s go ahead and clean this up at
least a little bit so one of the things
that we could do right out of the gate
is like if you’ll notice here sitemap
index is a you know it’s luke has just
one value in it which is locations which
is a slice of the location type well if
we look at the location type it turns
out that’s just a string type and so all
we really need to do at this point would
be to take rather than have locations be
a location type we can actually just
make that a string type and that solves
this problem here well to an extent we
also need to reference the tag itself so
its location inside of the sitemap tag
so to handle for that you just do a
greater than sign and go to the next tag
so this means the location tag
underneath the sitemap tag so so now if
you just make that slight change we can
actually get rid of both the location
struct but we can also get rid of this
string method
we won’t actually need that anymore
because now it’s a string type as we
iterate over locations so that
definitely simplifies things quite a bit
for us so now we want to talk a little
bit about what happens when we go and
visit one of the site maps from here so
let me go ahead and just pull one up
real quick just so we can kind of see
what we’re working with
so I’ll just pull up the politics
sitemap and I’ll just bring that over so
here is that let me just make it fit so
as you can see here this time everything
is contained in the the most parent tag
is URL set and then inside of URL set
you’ve got URL inside of the URL tag we
have the location we’ve got n colon news
the publication you’ve got title
keywords all that and the n colon news
is something I’m I don’t really see too
too much I’m guessing they’re just
calling this the like why they’re using
n colon I don’t really know something’s
like super familiar with XML and you
want to let us know why they do n colon
rather than just news because basically
this is still a news tag it’s just like
a news tag of of n which I guess is you
know new I’m guessing is news but it
makes no sense to me why didn’t you just
call the tag news so if anybody can
answer that question let me know anybody
from Washington Post watching anyway
[Music]
wondering where all their new traffic is
coming from and moving along so anyways
this is what we want to parse so the
structure is a little different so just
again just you can pull this up the you
can go to the text-based version I have
a sample of this so if things do change
you’re in luck I got a sample just like
I did before that you could still work
with or you can adapt this to your own
but basically we know we’re gonna have
to go inside the URL set inside the URL
one of the things we are interested in
is the location tag but we’re also gonna
be interested in from inside URL inside
the news tag we also want title in
probably key words so quite a bit more
structure that we want to pull in and
that’s why we want to really get our
struck
perfect because otherwise we’re gonna
like I was saying before end up with
like 30 struts probably more like like
10 struts or something like that but it
would be kind of absurd so so that’s our
sitemap index struct let’s go ahead and
build a new struct and that one are
basically a new type and that’s going to
be type news and of course it’ll be a
struct and then inside news this one’s
going to have titles which will be a
slice in the string form and the
location of this utility here is gonna
be it’s gonna be XML tags and then like
we were discussing it’s the URL and then
it’s within remember it was in colon
news but for some reason I’m not even
sure about the official name for
something like this would be but you
don’t use that you’re just gonna say
okay it’s in the news tag and then it’s
inside I’m sorry we are going to do
keywords but title and I think title was
under case let me confirm yeah okay yeah
it looks like it should be under cased
or a lower case rather okay so we have
titles now we do want alright now we
want to do keywords so keywords of the
string form and again XML and then this
one basically the same path actually
it’s just rather than title it was
keywords so keywords and then finally we
have the actual locations so locations
don’t forget capital L locations so it
does get exported string and then this
one stop it this one is located under
XML and then this one was like it was
just URL and then the location and just
remember like one of the like location
was at the same level as News was it’s
just news content happens to contain
other other tags that we were also
interested in so that’s our news struct
now let’s go ahead and go into our main
function here now and basically
everything here is is good for us
the only difference is basically we want
to visit
now rather than just print out the
location we actually want to visit that
location instead so to do that we would
just basically do the exact same thing
that we’ve done you know up here right
so so what we’re gonna do is where we’ve
defined and probably probably you should
define your bars like at the top here so
that’s going to do that and then also
bar in news and then so this will
[Music]
unmarshal into s then we start iterating
through and then basically we’re just
going to do the exact same thing here so
I’m actually just gonna copy actually
let’s just copy all three of these lines
because we need to do all three of those
things so just paste that in and then
now rather than HTTP GET that means
sitemap what we’re gonna do is get the
the location whatever that happens to be
as we iterate around bytes are gonna
stay the same and then this time rather
than on marshaling into the sitemap
index we’re gonna unmarshal into our
news struct so now we should wind up
with pretty much all that same
information that we have the only issue
now is how we’re gonna start going
through that information because at this
point like when we just had locations
okay that was just one value we just
loop through that value and that’s all
we really wanted to do but now in each
of these sitemaps what we’re getting is
title location and keywords well what do
we actually want to do with that we want
to display that on our web app and so at
least to me my desire at this point is I
want to put this into some sort of key
and values system so in you know in
Python that would be a dictionary in
Django that’s gonna be a map so in the
next tutorial what we’re gonna be
talking about is how we can you know
iterate through this and build a map
that we can then pass basically to our
our web app and then we’ll use that to
display all the information that we want
to show on the page so anyways that’s
what we’re gonna be doing in the next
tutorial if you have any questions
comments concerns whatever up to this
point feel free to leavin below
otherwise I will see you in the next
toriel
part 13 of the go language tutorial
series in this tutorial what we’re doing
is just building on the last tutorial
with our news aggregator web app idea in
the previous tutorial we were talking
about looping and how we could loop over
this the information that we’re pulling
which in this case is just a bunch of
sitemaps now what we want to do is
actually visit those site maps and then
pull the information from there which is
actually the load you know the actual
articles themselves their titles and
keywords stuff like that so that’s what
we’re gonna be doing here we also need
to clean this up a little bit I kind of
was really explicit with every part that
we wanted to break down here but what we
can do is actually we can we can
simplify this quite a bit so it’s all
just one struct which is gonna make
sense you wouldn’t want to I mean you
can break them down is this far if you
really wanted to the problem is as this
program grows that’s going to get super
problematic like well like to do the
next the next site map and all the
information that we want from there
which will be titles keywords and the
location you’re gonna end up with like
15 structs if you’re not careful so so
let’s go ahead and clean this up at
least a little bit so one of the things
that we could do right out of the gate
is like if you’ll notice here sitemap
index is a you know it’s luke has just
one value in it which is locations which
is a slice of the location type well if
we look at the location type it turns
out that’s just a string type and so all
we really need to do at this point would
be to take rather than have locations be
a location type we can actually just
make that a string type and that solves
this problem here well to an extent we
also need to reference the tag itself so
its location inside of the sitemap tag
so to handle for that you just do a
greater than sign and go to the next tag
so this means the location tag
underneath the sitemap tag so so now if
you just make that slight change we can
actually get rid of both the location
struct but we can also get rid of this
string method
we won’t actually need that anymore
because now it’s a string type as we
iterate over locations so that
definitely simplifies things quite a bit
for us so now we want to talk a little
bit about what happens when we go and
visit one of the site maps from here so
let me go ahead and just pull one up
real quick just so we can kind of see
what we’re working with
so I’ll just pull up the politics
sitemap and I’ll just bring that over so
here is that let me just make it fit so
as you can see here this time everything
is contained in the the most parent tag
is URL set and then inside of URL set
you’ve got URL inside of the URL tag we
have the location we’ve got n colon news
the publication you’ve got title
keywords all that and the n colon news
is something I’m I don’t really see too
too much I’m guessing they’re just
calling this the like why they’re using
n colon I don’t really know something’s
like super familiar with XML and you
want to let us know why they do n colon
rather than just news because basically
this is still a news tag it’s just like
a news tag of of n which I guess is you
know new I’m guessing is news but it
makes no sense to me why didn’t you just
call the tag news so if anybody can
answer that question let me know anybody
from Washington Post watching anyway
[Music]
wondering where all their new traffic is
coming from and moving along so anyways
this is what we want to parse so the
structure is a little different so just
again just you can pull this up the you
can go to the text-based version I have
a sample of this so if things do change
you’re in luck I got a sample just like
I did before that you could still work
with or you can adapt this to your own
but basically we know we’re gonna have
to go inside the URL set inside the URL
one of the things we are interested in
is the location tag but we’re also gonna
be interested in from inside URL inside
the news tag we also want title in
probably key words so quite a bit more
structure that we want to pull in and
that’s why we want to really get our
struck
perfect because otherwise we’re gonna
like I was saying before end up with
like 30 struts probably more like like
10 struts or something like that but it
would be kind of absurd so so that’s our
sitemap index struct let’s go ahead and
build a new struct and that one are
basically a new type and that’s going to
be type news and of course it’ll be a
struct and then inside news this one’s
going to have titles which will be a
slice in the string form and the
location of this utility here is gonna
be it’s gonna be XML tags and then like
we were discussing it’s the URL and then
it’s within remember it was in colon
news but for some reason I’m not even
sure about the official name for
something like this would be but you
don’t use that you’re just gonna say
okay it’s in the news tag and then it’s
inside I’m sorry we are going to do
keywords but title and I think title was
under case let me confirm yeah okay yeah
it looks like it should be under cased
or a lower case rather okay so we have
titles now we do want alright now we
want to do keywords so keywords of the
string form and again XML and then this
one basically the same path actually
it’s just rather than title it was
keywords so keywords and then finally we
have the actual locations so locations
don’t forget capital L locations so it
does get exported string and then this
one stop it this one is located under
XML and then this one was like it was
just URL and then the location and just
remember like one of the like location
was at the same level as News was it’s
just news content happens to contain
other other tags that we were also
interested in so that’s our news struct
now let’s go ahead and go into our main
function here now and basically
everything here is is good for us
the only difference is basically we want
to visit
now rather than just print out the
location we actually want to visit that
location instead so to do that we would
just basically do the exact same thing
that we’ve done you know up here right
so so what we’re gonna do is where we’ve
defined and probably probably you should
define your bars like at the top here so
that’s going to do that and then also
bar in news and then so this will
[Music]
unmarshal into s then we start iterating
through and then basically we’re just
going to do the exact same thing here so
I’m actually just gonna copy actually
let’s just copy all three of these lines
because we need to do all three of those
things so just paste that in and then
now rather than HTTP GET that means
sitemap what we’re gonna do is get the
the location whatever that happens to be
as we iterate around bytes are gonna
stay the same and then this time rather
than on marshaling into the sitemap
index we’re gonna unmarshal into our
news struct so now we should wind up
with pretty much all that same
information that we have the only issue
now is how we’re gonna start going
through that information because at this
point like when we just had locations
okay that was just one value we just
loop through that value and that’s all
we really wanted to do but now in each
of these sitemaps what we’re getting is
title location and keywords well what do
we actually want to do with that we want
to display that on our web app and so at
least to me my desire at this point is I
want to put this into some sort of key
and values system so in you know in
Python that would be a dictionary in
Django that’s gonna be a map so in the
next tutorial what we’re gonna be
talking about is how we can you know
iterate through this and build a map
that we can then pass basically to our
our web app and then we’ll use that to
display all the information that we want
to show on the page so anyways that’s
what we’re gonna be doing in the next
tutorial if you have any questions
comments concerns whatever up to this
point feel free to leavin below
otherwise I will see you in the next
toriel
Please follow and like us:
Be First to Comment