what’s going on everybody welcome to
part two of the Python reddit API
wrapper or prawn tutorial mini-series in
this tutorial what were we talking about
is at least beginning to parse comments
so like I said but at the end of the
last video comments represent a
different kind of challenge for a
variety of reasons mainly it’s just the
fact that comments aren’t you know
perfectly in order there it’s a tree of
data it’s not a linear form of data so
anyways I’m going to go ahead and remove
a subreddit that subscribe but the rest
of this stuff can remain so just
underneath this let’s go ahead and
continue so the first thing we could do
is first of all I want to limit this to
there’s there there are two stickies so
I’m just going to limit this to three
just so we don’t go you know so we just
do one submission for now and now I’m
going to come down here and we can
reference the comments by just saying
comments equals submission dot
comments so this gives us the comments
so now we can do is to say for comment
in comments we can go ahead let’s go
ahead let’s like print 20 times this but
let give us some separation and then
what we’re going to do is we’re going to
print comment but just like a submission
the comments are like these objects like
the perot object and the object is just
going to have the ID so then you
reference an attribute and one of the
attributes is body for the body of that
comment and then what we’re going to say
is so that’s our that’s our comment so
we can at least iterate through comments
that way so for example let’s just run
that real quick this here your shirt
here so these are like all our you know
comments now let me pull up that what
they’re just close out of it I guess I
closed out of it
[Music]
pull over mine so that was why so
there’s six comments total here but some
of these are like replies like for
example if you’re unfamiliar do yourself
a favor and look into pandas so for
example if you made me look for this
army okay okay anyway it’s not here okay
so what we have to do is iterate through
it at least I’m pretty sure it’s not
there so these would be just like top
levels I’m pretty sure I just want to be
a hundred percent sorry for wasting your
time anyway so I think I closed again I
cus I’m bad at closing things anyway I’m
pretty sure it’s not there so what we
need to do is get the replies so now we
could say you know for reply so for or
rather prot what we should do is we
there might not be any replies so then
we could say if lend comment dot replies
is greater than zero and again if you
didn’t know replies existed you could
have done a Durer on comment Abadi or
you can read the documents anyway
if when comment our replies is greater
than zero so we have some replies then
order is a for reply in comment dot
replies hmm we get loops that’s not a
that’s a thank you anyway
we can print then let’s just say like
for blog that’s why and also we got body
on that
okay so here you get a it’s just me
reply really great high-quality reply
yeah okay so oh and here’s another reply
I was like this really isn’t another one
yet so this is the this is that comma I
just searched for a second ago so there
we caught that reply about pandas but
then I think I close this let me open it
again
someone complained I wanted my videos
like I just murder my Enter key it’s
true uh okay if you’re there you go so
so pan is looking to pandas but then
there’s another comment underneath that
right so then we would have to be like
um you know we did we’d have to just
basically okay and then at this plant
reply we could say okay if when reply
dot replies is greater than there but
you have no idea how deep down the
rabbit hole the comment tree things go
right so that’s that’s slightly
problematic then so the solution is we
can actually say submission comments we
can add dot lists to these and this will
list out your all of the comments so dot
list I believe is purely a Python reddit
API wrapper so purely a prof. um ssin
allottee that’s not something that’s
actually available to you in the Python
alright it’s not something that’s
actually available to you even the
reddit API but anyways that doesn’t
matter
let me go ahead and close this so we’ve
got a nice clean thing and then also we
uh we kind of want to do like print
comment body we don’t really want to do
the replies so let’s just do that to
cancel this real quick
so in this case we’ve run through all of
them so here you go here’s a the
second-level reply now unfortunately we
have no absolutely no idea the
contextual data for this like we don’t
really know where this this was in the
whole thing so for example you know you
wouldn’t really know that this was in
reply to you know which reply it was to
now
what list does is basically it takes all
the top-level comments list those out
then it goes down to the second level
comments lists all those out then third
level and so on so one option you have
is rather than comment body what you
could say is you can also grab like you
could you can grab a print the parent ID
and that would be comment dot parent now
do you note that’s not an attribute
that’s an actual new API call which in
my opinion is super unfortunate I wish
that was supplied and I don’t think
that’s a mistake I believe that’s that’s
just in reddit and I realize not every
comment is going to necessarily have a
parent but pretty much every comment
would write like you know the parent is
the actual submission or the parent is
another comment so and these are like
little tiny ID strings like I really
think that should be included but it’s
not it’s a new API call
so anyway comment ID so comment that
parent and rather than that this one is
just comment ID which just is actually
an attribute so huh crazy I can’t
remember if a submission I’m pretty sure
like the submission contains the
subreddit ID so I love to give wrong
though anyway that’s okay so now what we
could do is get the parent ID in the
comment idea of every comment and then
what we could do is print the comment
body
and then you’ve got the parent ID in the
comments idea of everything now from
that point you could begin to do some
pretty cool stuff but the first thing I
want to show you is right let’s say
let’s say we don’t do Python and instead
we do news so very very popular
subreddit and if this doesn’t work I’ll
do like politics or something but we
should hit an error here let’s go there
we go here we go there’s error so if you
use the dot list and you actually do
iterate through all comments chances are
eventually you’re going to wind up with
this stupid error so more comments
object has no attribute parent ok so
what’s happening there is like on really
long comment chains so like for example
let me go to the news subreddit that
would be this one marijuana company buys
entire US time to create cannabis from
the municipality that’s going to have
lots of comments so for example right
away you can see here this like load
more comments that’s a more comments
object and actually even though red it
looks super simple they’re going to that
till you click this I’m pretty sure
you’re making a new call like it’s an
actual call to their database same thing
would like continue this thread that’s a
new call it’s going to reload that data
like all this data is not loaded on your
page load that would be nuts you never
load the page so anyways if you wanted
to continue iterating through those
comments you would need to also either
handle with a you know an exception or
something like that or one option you
have is to replace the mores so for
example coming down here comments that
list one option you have is so you could
you can just use dot replace more kind
of starting to add a little too many um
a little too many things here but let’s
just do
I’ll do I’ll add the dot list down here
and then what we’ll say is dot replace
underscore more and then for now we’ll
say limit equals zero but at some point
you will run into limits with the
replace more like there’s only so many
more it will add I think it’s 30 or
something like that which is so fond of
comments because like each replace more
will load in a bunch of comments but
just keep that in mind like you’re you
you’re going to run out eventually
but it won’t air if you do run out of
the option to continue replacing instead
it’s just going to toss them so you
won’t hit an actual error anymore so
anyways let’s let’s go ahead and run
this real quick and probably I should
remove the parent call that’s going to
slow me down
Walt hmm let’s see submission dot
comments okay replace more hmm
okay fine fine fine one one okay dot
list and then we’ll come over here
comments that replace more okay so first
we we’ve converted it to list form which
then creates this more comment object
and now we can replace them I just did
it backwards this should work that’s
still going to be a lot of queries to
the API but hopefully we’ll get through
it are you kidding me please what have I
done what have I done
comments dot replace more so comment
equals submission that comments
I think I had it right the first time so
comments equals submission comments
please
so where is a submission comments that
replace more limit equals 0 now for
comment in comments let’s see no.4
comment in submission comments I really
feel like I should have been able to
string that someone can comment below
what the fix should have been because I
don’t see why I wasn’t able to string
those together but obviously messing up
something so for comment in submission
comments that list let me try that drink
some more coffee 1 Matic there we go
not a problem that’s going forever
though I’m going to go I’m just going to
break that pencil
API calls eventually it would probably
throttle me anyway as you can see now
we’ve got all the parent IDs the comment
IDs everything’s hunky-dory we’re doing
great so go ahead and close this out so
so that’s how you can iterate through
all the comments and all that now now
the question is you know how might you
rebuild that comment tree right because
at some point right like you’ve got to
rebuild that tree so for example one
option you could have is like build a
dictionary or something like that and
then each of like the you know like the
parent you’ve got a parent ID and then
the parent content and then all the
replies so a parent ID content all
replies parent ID kind of all replies
and if you did that you could rebuild
the tree yourself now I’m not going to
go ahead and go through all that I don’t
really see too much point covering that
in video but if you are interested in
that you can go to part 2 of this
tutorial series on Python programming
and there’ll be an example there if
you’re interested in truly rebuilding
those comment trees that’s one way you
could do it that’s how I would do it
anyway if you have a better way I’m sure
somebody could come up with a better way
anyways so now in the next tutorial
we’re going to talk about is basically
just streaming from reddit so this has
all been like historical grabbing from
reddit but there’s also a way you can
actually just stream data from reddit so
anyways that’s all going to be doing in
the next tutorial if you’ve got
questions comments concerns whatever
feel free to them below otherwise I will
see you in the next trip
part two of the Python reddit API
wrapper or prawn tutorial mini-series in
this tutorial what were we talking about
is at least beginning to parse comments
so like I said but at the end of the
last video comments represent a
different kind of challenge for a
variety of reasons mainly it’s just the
fact that comments aren’t you know
perfectly in order there it’s a tree of
data it’s not a linear form of data so
anyways I’m going to go ahead and remove
a subreddit that subscribe but the rest
of this stuff can remain so just
underneath this let’s go ahead and
continue so the first thing we could do
is first of all I want to limit this to
there’s there there are two stickies so
I’m just going to limit this to three
just so we don’t go you know so we just
do one submission for now and now I’m
going to come down here and we can
reference the comments by just saying
comments equals submission dot
comments so this gives us the comments
so now we can do is to say for comment
in comments we can go ahead let’s go
ahead let’s like print 20 times this but
let give us some separation and then
what we’re going to do is we’re going to
print comment but just like a submission
the comments are like these objects like
the perot object and the object is just
going to have the ID so then you
reference an attribute and one of the
attributes is body for the body of that
comment and then what we’re going to say
is so that’s our that’s our comment so
we can at least iterate through comments
that way so for example let’s just run
that real quick this here your shirt
here so these are like all our you know
comments now let me pull up that what
they’re just close out of it I guess I
closed out of it
[Music]
pull over mine so that was why so
there’s six comments total here but some
of these are like replies like for
example if you’re unfamiliar do yourself
a favor and look into pandas so for
example if you made me look for this
army okay okay anyway it’s not here okay
so what we have to do is iterate through
it at least I’m pretty sure it’s not
there so these would be just like top
levels I’m pretty sure I just want to be
a hundred percent sorry for wasting your
time anyway so I think I closed again I
cus I’m bad at closing things anyway I’m
pretty sure it’s not there so what we
need to do is get the replies so now we
could say you know for reply so for or
rather prot what we should do is we
there might not be any replies so then
we could say if lend comment dot replies
is greater than zero and again if you
didn’t know replies existed you could
have done a Durer on comment Abadi or
you can read the documents anyway
if when comment our replies is greater
than zero so we have some replies then
order is a for reply in comment dot
replies hmm we get loops that’s not a
that’s a thank you anyway
we can print then let’s just say like
for blog that’s why and also we got body
on that
okay so here you get a it’s just me
reply really great high-quality reply
yeah okay so oh and here’s another reply
I was like this really isn’t another one
yet so this is the this is that comma I
just searched for a second ago so there
we caught that reply about pandas but
then I think I close this let me open it
again
someone complained I wanted my videos
like I just murder my Enter key it’s
true uh okay if you’re there you go so
so pan is looking to pandas but then
there’s another comment underneath that
right so then we would have to be like
um you know we did we’d have to just
basically okay and then at this plant
reply we could say okay if when reply
dot replies is greater than there but
you have no idea how deep down the
rabbit hole the comment tree things go
right so that’s that’s slightly
problematic then so the solution is we
can actually say submission comments we
can add dot lists to these and this will
list out your all of the comments so dot
list I believe is purely a Python reddit
API wrapper so purely a prof. um ssin
allottee that’s not something that’s
actually available to you in the Python
alright it’s not something that’s
actually available to you even the
reddit API but anyways that doesn’t
matter
let me go ahead and close this so we’ve
got a nice clean thing and then also we
uh we kind of want to do like print
comment body we don’t really want to do
the replies so let’s just do that to
cancel this real quick
so in this case we’ve run through all of
them so here you go here’s a the
second-level reply now unfortunately we
have no absolutely no idea the
contextual data for this like we don’t
really know where this this was in the
whole thing so for example you know you
wouldn’t really know that this was in
reply to you know which reply it was to
now
what list does is basically it takes all
the top-level comments list those out
then it goes down to the second level
comments lists all those out then third
level and so on so one option you have
is rather than comment body what you
could say is you can also grab like you
could you can grab a print the parent ID
and that would be comment dot parent now
do you note that’s not an attribute
that’s an actual new API call which in
my opinion is super unfortunate I wish
that was supplied and I don’t think
that’s a mistake I believe that’s that’s
just in reddit and I realize not every
comment is going to necessarily have a
parent but pretty much every comment
would write like you know the parent is
the actual submission or the parent is
another comment so and these are like
little tiny ID strings like I really
think that should be included but it’s
not it’s a new API call
so anyway comment ID so comment that
parent and rather than that this one is
just comment ID which just is actually
an attribute so huh crazy I can’t
remember if a submission I’m pretty sure
like the submission contains the
subreddit ID so I love to give wrong
though anyway that’s okay so now what we
could do is get the parent ID in the
comment idea of every comment and then
what we could do is print the comment
body
and then you’ve got the parent ID in the
comments idea of everything now from
that point you could begin to do some
pretty cool stuff but the first thing I
want to show you is right let’s say
let’s say we don’t do Python and instead
we do news so very very popular
subreddit and if this doesn’t work I’ll
do like politics or something but we
should hit an error here let’s go there
we go here we go there’s error so if you
use the dot list and you actually do
iterate through all comments chances are
eventually you’re going to wind up with
this stupid error so more comments
object has no attribute parent ok so
what’s happening there is like on really
long comment chains so like for example
let me go to the news subreddit that
would be this one marijuana company buys
entire US time to create cannabis from
the municipality that’s going to have
lots of comments so for example right
away you can see here this like load
more comments that’s a more comments
object and actually even though red it
looks super simple they’re going to that
till you click this I’m pretty sure
you’re making a new call like it’s an
actual call to their database same thing
would like continue this thread that’s a
new call it’s going to reload that data
like all this data is not loaded on your
page load that would be nuts you never
load the page so anyways if you wanted
to continue iterating through those
comments you would need to also either
handle with a you know an exception or
something like that or one option you
have is to replace the mores so for
example coming down here comments that
list one option you have is so you could
you can just use dot replace more kind
of starting to add a little too many um
a little too many things here but let’s
just do
I’ll do I’ll add the dot list down here
and then what we’ll say is dot replace
underscore more and then for now we’ll
say limit equals zero but at some point
you will run into limits with the
replace more like there’s only so many
more it will add I think it’s 30 or
something like that which is so fond of
comments because like each replace more
will load in a bunch of comments but
just keep that in mind like you’re you
you’re going to run out eventually
but it won’t air if you do run out of
the option to continue replacing instead
it’s just going to toss them so you
won’t hit an actual error anymore so
anyways let’s let’s go ahead and run
this real quick and probably I should
remove the parent call that’s going to
slow me down
Walt hmm let’s see submission dot
comments okay replace more hmm
okay fine fine fine one one okay dot
list and then we’ll come over here
comments that replace more okay so first
we we’ve converted it to list form which
then creates this more comment object
and now we can replace them I just did
it backwards this should work that’s
still going to be a lot of queries to
the API but hopefully we’ll get through
it are you kidding me please what have I
done what have I done
comments dot replace more so comment
equals submission that comments
I think I had it right the first time so
comments equals submission comments
please
so where is a submission comments that
replace more limit equals 0 now for
comment in comments let’s see no.4
comment in submission comments I really
feel like I should have been able to
string that someone can comment below
what the fix should have been because I
don’t see why I wasn’t able to string
those together but obviously messing up
something so for comment in submission
comments that list let me try that drink
some more coffee 1 Matic there we go
not a problem that’s going forever
though I’m going to go I’m just going to
break that pencil
API calls eventually it would probably
throttle me anyway as you can see now
we’ve got all the parent IDs the comment
IDs everything’s hunky-dory we’re doing
great so go ahead and close this out so
so that’s how you can iterate through
all the comments and all that now now
the question is you know how might you
rebuild that comment tree right because
at some point right like you’ve got to
rebuild that tree so for example one
option you could have is like build a
dictionary or something like that and
then each of like the you know like the
parent you’ve got a parent ID and then
the parent content and then all the
replies so a parent ID content all
replies parent ID kind of all replies
and if you did that you could rebuild
the tree yourself now I’m not going to
go ahead and go through all that I don’t
really see too much point covering that
in video but if you are interested in
that you can go to part 2 of this
tutorial series on Python programming
and there’ll be an example there if
you’re interested in truly rebuilding
those comment trees that’s one way you
could do it that’s how I would do it
anyway if you have a better way I’m sure
somebody could come up with a better way
anyways so now in the next tutorial
we’re going to talk about is basically
just streaming from reddit so this has
all been like historical grabbing from
reddit but there’s also a way you can
actually just stream data from reddit so
anyways that’s all going to be doing in
the next tutorial if you’ve got
questions comments concerns whatever
feel free to them below otherwise I will
see you in the next trip
Please follow and like us:
Be First to Comment