continue our graph QL journey we’re
going to do graph QL caching using data
loader I am your host M PJ and you are
[Music]
all right so we’re continuing to evolve
our little graph QL goodreads API
wrapper if you just happen to run into
this episode this is a part of a little
series you can check out the earlier
episodes in a playlist by clicking there
alright let’s have a look so this is our
application it’s the same from earlier
weeks I don’t quite remember where we
were so yes let’s just start it node
serve J s and let’s have a look so let’s
just run this non-white space before
first tag okay this is because the XML
is it’s not valid and that is because we
we don’t have a valid API key because I
regenerated these from last time because
I showed them on-screen and I didn’t
want everybody in the entire world to
use my API key
so let’s regenerate that clicky clicky
weighty weighty are you ready longer and
then developer key and then login then
there’s the API key we were set that
just for kicks then I’ll copy that we’ll
paste that in here in a real-world
application you wouldn’t hard-coded you
would probably use it in a process
variable or something like that but the
point of this video is to show how to do
graph do that stuff not to do
architecture I digress let’s restart the
server and try that again
ie refresh the page but I didn’t really
have to okay so this works and we can do
wery other things okay so what are we
gonna do today is we’re going to add
caching to this thing but I want to
complicate things a little bit first I
want to add the ability to query authors
on the books so here like right here I
want the ability to add author we don’t
have that now because we can’t query
feel author on type author because no
hang on
we don’t want authors on authors because
authors don’t have authors but books
have authors might have multiple authors
co-authors if I ever wrote a book I
would need a co-author because I have no
self discipline at all
let’s go back to the app and see where
we might do this
[Music]
scrolling scrolling scrolling scrolling
okay where we have the book type here
let me remove the sidebar you know what
we have this annoying console.log here
I’m going to delete that do that so we
are rid of that stop the server cool no
console.log here even if we run that oh
well yeah it wouldn’t like with we still
have breakage ah
cannot find query in query field authors
on type book again let’s go to the book
and start defining an author’s field and
this is going to be a tie and it’s going
to be a list okay so let’s go down to
author type and see how that looks
yeah so author type has something
similar as books books property and of
type graph journalist book type that’ll
copy paste that because I hate typing
but we don’t want books to have a list
of books we want a list of authors so
you want the author type here in there
and so how do we resolve authors well
first of all we need a result method
result result result is going to get an
XML blob you can see we can cheat a
little bit here and look at how what the
ISBN looks like so has this copy that
after this thing’s become mysterious
because we don’t know what the author’s
XML looks like so we’re gonna go to the
API again and this is the okay so we
have this is the book element here let
me make that a bit bigger car so we have
this book element here that corresponds
to this book here and it’s the first
element here because XML the way XML is
structure you might have multiple
elements so theoretically XML all the
weight structure it could have multiple
books but it only has one book element
so we grab the first book element and
that’s this one and it’s what happens to
be the only book element of course and
we screw down screwed and screwed down
scroll down until we find author’s
author hairs authors cool it’s the next
and eleventh that
turn has several elements there are
several elements of authors or in my top
several so it’s a Walters it has an ID
property so what we’re going to do that
we are going to grab the author element
and you know what we’re gonna do this
and I’m going to call them and no hang
on
authors and then yeah the first author’s
government and then we want the author
elements in that XML it’s hard now we
want to grab the IDS and for each author
[Music]
all right there Jesus Christ I don’t
even remember I remember my own hotkeys
and we want to map over them and for
each element we want to grab the ID tag
element I mean and we want the first one
because this is what it looks like it’s
we want the first ID tag and I think
this just gives us the value inside of
it so this will give us an array of
strings which are the IDS and now blur
to get those we want to look a little
bit at how the author type gets its it’s
like them so it will return a promise
that resolves to an array of all four
books but so we want to do this
exactly this but we don’t want the book
we want the officers so I’m going to
steal this is this line here
bump it boom paste it in there and then
we’re going to go down to where we load
the author’s gonna steal this as well
don’t panic too much about the
duplication we’re going to deal with
that yes are we missing one parenthesis
and I think we might have it no wait we
have our X ID here we can’t have that
we’re gonna have this ID
there it goes here I don’t know if this
will work let’s restart and see okay go
back to graphic ul and run this and see
what happens feel authors of time author
must have a selection of subfields you
mean authors okay maybe I did expect
syntax a request what expected name
alright
ha that actually worked great okay so as
you can see we can now fetch the authors
of a book and we can also actually do
this one level down so the author’s can
actually now we can now check fetch the
books of those authors and then fetch
the names of those books no sorry the
title of those books and then we can
fetch the authors again to create a
source of their authors and the names of
those authors to create an incredibly
inefficient query which takes a long
time and probably will rate limit me on
the Goodreads okay
oh nice cool we now have a terribly
inefficient API so that’s what we’re
gonna start with how can we make this ah
non insanely slow before we start
dealing with that I want to remove some
duplication because we have some serious
duplication we have something that is
exactly the same here which is this code
here let me make that smaller which is
used to fetch the author in the authors
array is exactly the same as this code
down here which fetches the author for
the root root root element so we’re
going to extract this out into a a
you know what it’s gonna be something
like this it’s going to be fetch you you
out sir for some reason I’m having a big
trouble spelling author today it’s
probably due to me having the onset
stroke I copy that and up here perhaps
we define comest fetch over third Jesus
that is that it’s not good my spelling
okay maybe this maybe maybe this works
hang on hang on to let’s restart the
server and see if things let’s run that
Oryx is not defined boo
no okay I need to that it’s because of
this let me let me make this bigger
again let me make it bigger let me make
it bigger right right
and let me resize again this is
great stuff big font so scrolling down
yeah I need to change this to be this
like only argument goes here restarting
again see what we get running it running
it rather yeah yeah still very very slow
very slow of course
okay
refactoring worked I’m gonna use the
fetch author at the other place where we
have duplication which is here and now
since this is a function with just one
argument the ID I can pass this just
right into map here boom and that will
work
presumably yes it did overall this is a
great technique back map map is such a
fantastic function it’s so universally
applicable it’s it’s a great concept but
it becomes even greater when you have a
lot of when you deal with functions like
this but just take single argument and
does stuff it makes it really really
like really nice to deal with everything
becomes magical with unary functions as
they are cold I think all right so now
how do we make this not slow as but now
well what I want to do is that I’m going
to sorry sorry sorry that I’m going to
install something called the data loader
I’m going to instantiate that so or well
I need to require it first so almost
data loader with wire quiet oh this is
object-oriented so I think I need to
data loader over
if you’re wondering what my nice plugin
for determining package sizes I don’t
remember what that is
so in abled extensions it’s called
something like import costs
it’s called import cost so now you know
right so now we’re going to create our
author loader or third loader author
loader let’s call it authored over and
it’s a new data loader and this data
loader the constructor per data loader
takes one argument a function which is a
function that takes a G’s object which
is just an array of keys and it’s
supposed to return a promise with an
array for it values for those keys which
encases our authors so we already have
the logic for that down in downing it
downing down in here in authors I’m
gonna steal this thing here yep paste at
it so it’s a promise that all on door
it’s the IDS but it’s not I need its
keys what this could be called anything
I guess but let’s we’re fine with that
and now we’re going to be using this
here instead of actually calling fetch
author directly we’re going to be using
the data loader because the data loader
growth or loader is going to do the do
defection for us and cache it if it
feels like it
what was this I think this is just old
load and then I I think it’s gonna load
Manny don’t money it’s got a little
money so let’s serve and and run this to
it seems to work but it’s still very
slow so let’s use it in more places so
let’s scroll down here so instead of
fetch author here we’re gonna use our
author loader author loader
don’t load because we’re just loading
one thing that’s so lazy I don’t want to
type anything I just want to copy and
paste let’s restart it and see if it’s
it’s faster a little bit faster but the
thing is that we’re still fetching the
books here so we need to extract this
out as well
I believe we want to grab this here and
we are going to call this fetch book and
write this up here it’s home stretch
book paste that in here see this is
exactly the same function yeah I think
yeah let’s restart that just to see that
things work I’m keep saying yeah I don’t
ah I’m just talking talking and
practicing a bit you just keep talking
because eventually I need to do
streaming because I promise to do
streaming to the patrons love you
patrons but I haven’t gotten around to
it because streaming is super scary but
you just have to keep talking anyway
alright
so ah yes it works my refactoring worked
now we need to create a book loader as
well otherwise we won’t have any book
cash book loader new data loader and
that works the same way just gonna steal
this here it’s pasty paste
and we want to use fetch book instead of
fetch author so book loader is the data
loader that yes uses fetch book let’s
see if we can restart this no hang on we
need to we need to do that won’t do
anything because we actually need to use
the book loader as well so instead of
calling fetch book directly we’re going
to use our book loader and many and
we’re gonna pass in the IDS into that
and at least the whole hop let’s try
that again what is happening cool cool
cool cool
alright that ran pretty slow let’s just
run it again
Bam Bam Bam very fast now this means
that we have these these author loaders
and the book the author loader and the
book loader they are instantiated here
when the application starts and then
they live on for that for that duration
which is actually fine in the case of
the Goodreads API because books don’t
change around a lot but in the case of
another API we probably wouldn’t want
the cache to live on for this long we
kind of probably want the cache to live
on request so that if the data loader
sees request for the same ID in in the
same request we don’t want to do that
same request again but the next request
we probably want to throw away the cache
because things might have changed under
us so let’s take these out and we’re
gonna move them hang out to to the
outside here outside in the Indus serve
instead of the schema moving out of the
schema and we are going to give
bump bump bump bump bump
so graph QL HTTP yeah we might give it a
like this but we can also give it a
function that receives a a request
object and this way it will create this
thing this configuration object here
every time it gets a request so we can
create these this author load the author
loader and the book loader on every
request and we’re gonna we’re gonna pass
those in as a context object context the
context this is just an object you can
have anything we’re gonna you’re gonna
just pass these into that object by the
way these have no special naming
significant or or anything they are just
my name’s I could call them what losers
if I if I wanted to but I don’t want I
want to call them this we’re gonna go
back into the schema so how do we access
these things right now before we just
have then had them right here in the you
sort of global scope but we’re now going
to grab them here and you the second
argument is the orgs but there’s also a
third argument that is provided to you
by the graph QL expressing and that is
the context object and that is passed
your context so and remember that we put
the author loader in the context object
so yes the context dot author loader in
step and we do the same thing in the
books over here and again the second or
third argument is the context and then
we also do that here in the resolver for
the route author context
and context here as well
ha and that should give us let me run
that data loader is not defined no that
makes sense hang on hang on hang on
we need to grab the data loader out into
this into the circle yes let’s try that
again let’s run that BAM
everything broke fetch is not defined
okay I need to move every goddamn thing
out I know we need fetch out I think as
well try that let’s start the server
again let’s run it like oh god still
failing
6 and well is not too fine let’s it’s
good ok yes no sorry about that
Can I grab that as well hopefully I
won’t need you to stew okay well let’s
see oh god what happened
no I needed did you do as well neither
that ok i oh yeah I use util to promise
if I the XML to the GIS library running
it again please work
alright cool it’s taking a lot of time
the first time and now I press again
it’s gonna take a little bit longer
because it’s it’s doing its caching for
a quest but it’s probably not gonna take
quite as long as if I had done this from
completely cold cash anyway I hope you
get the idea the app level caching is
really fast once it’s warmed up but
might be stale request level caching
less likely to be stale but also not as
fast of course because it proves the
cache out what more often that is how
caching in graph QL using data loader
works if you have any questions and you
are a patron of the show you have access
to the Fun Fun forum and there is a
dedicated thread for
this particular episode you can find
that but clicking there or in the
episode description if you are not a
patron you can post a comment down below
that is it for today you have just
watched an episode of fun fun function I
release these every Monday morning Oh
800 GMT if you don’t want to wait until
next Monday morning you can check out
this episode that machine learning rumba
lubbers have selected for me my PJ until
next one [Music]