watching fun fun function this week’s
video is going to be about unit tests
versus integration tests I would like to
start talking about automated tests in
general when you first start out doing
software development one of the first
concepts that you’re going to be very
aware of is bucks you start writing some
piece of software and it works fine but
you didn’t quite think of this thing so
you fix that and then you test around a
bit and oh I you need to fix this case
as well and after while the application
grows and the there are quite a bit of
test cases that you have to go through
every time you make a change to make
sure that you didn’t break anything a
fancy word for that is regression
testing you basically check that your
software has not progressed that a
feature that previously worked now is
broken or maybe it has started working
in another way which the user will not
like so let’s say that you start out
doing your regression testing manually
because your app is small it doesn’t
have a lot of features it’s basically
perhaps I’ve got ten twenty test cases
that you have to go through every time
you release but as months go by and you
do active development on your software
that number grows it grows from from
twenty cases to thirty to forty and
you’re probably up in the hundreds after
a couple of months at least by now you
might have created an excel sheet that
tracks all the manual regression testing
that you have to do and every time you
do a release you walk through this
spreadsheet then you do the manual
testing and each test is surprisingly
complicated too because you have to
create this fresh user and simulate the
fresh state in the application and then
work from there and even if you have a
nice tool for creating test users that
can do that quickly and setting the
application in a certain state this ends
up taking at least a couple of minutes
per per test case which means that
you’re basically having at least a
person or maybe even a couple of persons
spending more than a day on doing just
the manual regression testing and then
to make things even more complicated you
realize that your customers have
different environments where they run
your app for instance they might have
our different operating systems or
different devices different mobile
phones or they might be running them in
different web browsers and this affects
your tests so this list of your hundred
regression tests is actually multiplied
by the number of operating systems you
support and also multiplied by the
number of devices you support creating
this combinatorial explosion you find
that even if you have full-time people
employed just to do regression testing
you will find that they are overwhelmed
and if they’re not overwhelmed they will
eventually be overwhelmed because this
list of 100 test cases will keep growing
that is just the way software works it
it’s very rare that you remove features
from software at least compared to how
often you add things to software so the
answer to this problem is of course to
have machines do the bulk of your
regression testing for you you cannot
completely replace human testers with
automated tests partly because there are
some things that only humans can see and
also because test cases needs to be
invented in the first place and humans
are way better at that than machines
doing so-called exploratory testing but
the bulk of your regression testing can
and should be done by machines what I
want to say here is
that automated tests is absolutely
critical to the production of quality
software so this this might be harsh to
say but if you don’t use automated tests
I am willing to bet quite a bit of money
that your software isn’t very good if we
are building software that people are
relying on to work we must have
automated tests next up I’m going to
define what I mean by a unit test and a
and an integration test because there’s
a lot of confusion and different usages
of the terminology when it comes to
automated testing first we’re going to
talk about the definition of a unit test
so let’s say that you are building an
application applications normally has at
least some attempt that the Norka
texture your application might have a
like let’s say that this is a view layer
view the view might have some other
layer that it’s interacting with let’s
call it the controller but it can be
anything like some people called models
some people called logic somebody might
call it core these are just words
there’s no official definition or
anything for them even though some
people might give you the impression
that there is and then maybe you have
some kind of database your controller is
writing and reading from the database
and um maybe you have ah some you know
payment system as well this is code that
is generally written by you
and this is external component the view
of your application might be built out
of multiple smaller parts that we might
be calling components or modules or
whatever if you’re using react for
example these would be your react
component you know the things that you
define with JSX and in the same way your
controller or business logic layer or
whatever
is also made up of multiple smaller
parts if you’re using Redux for this
layer for example these components would
be your action creators and your
reducers the exact terminology isn’t
important really the important takeaway
here as is that your application is made
out of these parts whatever they may be
these units that we can write unit tests
for and test in isolation the unit tasks
that we write they are only concerned
with this friend here this part it
interacts with the payment system and
it’s being used by a couple of
components in the view but when we’re
unit testing we are not concerned with
this system or these systems we’re only
concerned with this system this part so
what we do in our unit tests is that
this input here we’re not actually user
reproducing real inputs we are just
putting in fake inputs into the
controller and in the unit test we’re
not giving this part a real payment
system we are giving it a fake payment
system a so called mock payment system
that we expect the this part to do
certain perform certain actions on and
then we inspect our fake payment mock
spy thing to see what calls have been
made on it and then just verify that
this part did the right thing to it by
the way if you’re interested in learning
more about the concept of mocking I have
made a video about that that you can
find in the episode description this
message was brought to you by former MP
J the takeaway here is that testing
these things these parts individually
and isolated that is the definition of a
unit test okay so with that in mind what
is the definition of an integration test
a disclaimer here is that multiple
people are going to give you different
answers to the question of what an
integration test is I’m going to give
you one so remember our hypothetical
application from before when we write an
integration test we are writing a task
that is concerned with the entirety of
the application or at least multiple
components we might write an integration
test that looks like this that covers
some component in the view and some some
component in the controller or whatever
you call this
and that also touches the database in
fact the integration test might even an
integration test might even cover
multiple controller components an
example of an integration test might be
something that just punches through the
entire application for instance might
have to go to the item page and put an
item into the cart and then view the
cart these things they touch a lot of
stuff
unlike the unit test these things the
integration tests they tend to use a
real database that you set up and spin
up and create for thee for the unit for
the integration test you can’t actually
use the real database and the proper
real payment provider because then you
would be making payments and stuff each
test you run but they are the very heavy
fakes like it’s probably a real database
that you’re spinning up and populating
with sample data for instance and up
here it’s probably something that looks
like a real browser such as something
that is controlled with selenium or
maybe a phantom phantom Gaius thing up
here is is something that are very much
looks to the system like a real user
that goes around clicking in the
application that is how I define an
integration test something that touches
multiple components in one fell swoop
you don’t necessarily have to you
real databases and stuff some parts of
an integration test might be might be
mocked integration tasks it’s a fuzzy
definition but for this video it’s a
test that touch with multiple components
now that we have defined what unit tests
and integration tests are and what they
do let’s talk about the downsides of
unit tests there are two downsides to
unit tests and the first one is that
they do not test the contract what does
that mean well these view components
here they expect the controller the
controller to look a certain way they
expect it to have methods and properties
and you know stuff like they expect this
school to look a certain way in the same
way this this part here expects a
payment system to look a certain way it
might expect a payment module to have
you know like a method called pay
whatever this is sort of the contract
between these parts of the application
this contract needs to be in place for
us to be able to do unit testing and
it’s what we exploit to do it for
instance are we when we pass our mock
payment service into the this this part
here we make sure that that mock payment
service implement the same contract or
interface or whatever you would like to
call it as the real payment service this
controller part it doesn’t really care
if it’s being passed a real payment
service or a mock payment service the
important part is that it implements the
payment contract in statically typed
languages like Java you would you would
probably solve this by having the
payment service and the mock payment
service both implement like the payment
service interface but in JavaScript we
don’t have to do that we just make sure
that they have the same methods to call
and this is the weakness of of unit
tests
because they don’t test this interaction
the unit test will test that this this
part of you parts make a call a certain
call to the controller and the unit
tests of the controller tests that the
controller does certain things such as
calling the payment system based on
based on inputs that it gets from from
the view but this is all based on like
the assumption that we’re getting the
contract right both tests for example if
you make a mistake implementing our
payment service mock or if we make
mistake implement when we write the fake
inputs into the controller our tests
will not catch these errors even though
over you test path an integration tests
on the other hand it will just go that
it’s not concerned with this at all it
just tests the applications straight
through and yeah it it will catch those
errors that’s the first downside of unit
tests it’s that they do not test the
contracts but integration tests do the
second downside with unit test is that
you have to create contracts when you
start writing unit tests in your career
it’s going to it’s just gonna annoy you
that you find yourself that you have to
write new boilerplate just to make your
code testable these contracts that I
talked about like these interactions
here and this this separation it has to
be there in order to write unit tests if
the payment system is not cleanly
separated from the controller parts for
instance if you say that you just have
the payment system inside of these parts
and call it directly then it’s not
really possible to write a unit test
because well it’s just not a unit is
just stuff jumbled together
this is not a problem that integration
tests have they just test the
application and then they inspect the
database it does not matter to the
integration test if these parts are not
properly
Rize they just go through and checks I
talked before in that in statically
typed languages you would create an
interface that both the payment service
and the mock payment service inherited
from or implemented that is a problem
that you probably to be honest only have
and it’s code that you only have to
write because you want to unit as your
code and you also have to create
mechanisms in order to get your mock
payment system into the controller parts
this is the mechanism that people talk
about when they refer to dependency
injection while it is annoying that you
have to create these layers of
separation it also is a benefit because
it forces you to modularize your code if
you have code that is modularized but
you have not previously been writing
unit tests you will probably find that
when you do that your code is not at all
as modular as you thought it wants it’s
actually pretty jumbled together and not
cleanly separated at all and writing
unit tests have helped you expose that
problem but it is work you have to do it
in order to you unit testing like unit
tests have definitely the downside that
you will have to create these contracts
now at this point in their careers a lot
of developers almost all developers
actually find themselves asking can’t I
just use only integration tests
I mean integration tests seem to test
all of this and they don’t seem to have
any of the drawbacks of the unit test
well couldn’t we just use integration
tests to test all of our app then we
just have real databases we don’t have
to bother with this mocking we don’t
have to bother with dependency injection
and and making being afraid of making
mistakes and our contract why don’t we
use just
raishin test and almost all developers
do try to do that at some point in their
career but they will then learn that
there are downsides to integration tests
as well the first one is that
integration tests are more expensive
than unit tests the second problem with
integration tests is that they cannot
simulate errors and the third downside
with integration tests is that they
don’t tell you where the problem is what
do I mean by expensive integration tests
are expensive in three ways
and the first way that integration tests
are expensive is that they are slower to
run in order to facilitate this thing
you need to spin up a real browser and
you need to spin up a real database you
like if you’re careful you you might be
able to do this relatively fast like if
you have a some kind of database that is
very quick to create and teardown
MongoDB as for instance very very good
at this you can do it in just a couple
of hundred of milliseconds and if you
have not a full fledged browser but the
browser like phantom Jas which is also
pretty fast to spin up then you can run
them reasonably quickly but no matter
how you cut it they are very very very
slow running compared to this thing you
can easily run hundreds of unit tests in
the time it takes to run a single one of
these this makes a lot of difference
because ah while you are developing you
want to be able to run the test as fast
as possible preferably every time you
hit command save and if you have tests
that takes seconds or even minutes or
maybe even hours to run you’re not going
to do that you’re going to defer that to
like the build process and maybe it’s
even going to be too slow for even your
build process to start deferring it to
even later you really want to test suite
that you
run quickly and not something that will
take hours so an integration test is
often hundreds of times slower to run
than a unit test the second reason why
integration tests are expensive is that
they are pretty brittle so integration
tests like they cover a lot of systems
which is in one way it’s good but
there’s also a lot of moving parts here
you have a browser you have all these
systems and you have a database that
you’re tearing up tearing down these
systems might also be on separate
network so you have like a network
network aspect to them as well and the
more of these little things that you
introduce the more things there are that
can go wrong and because we run these
things thousands of times and because
statistics the more things will go wrong
anyone who has written these things they
know that these things are very very
hard to get to run ah with 100%
reliability so integration test a lot
more brittle than unit test because
there are a lot more moving parts the
third and final reason why integration
tests are more expensive than unit tests
is that they are simply harder to write
at first glance it seems easy to write a
unit test that has yes remote controls a
browser and clicks around in in your app
and then inspects the database but in
practice it’s really complicated because
you do this thing and you just realize
that huh nothing is working and I don’t
really know why it’s just silently
failing like I do these clicks and it’s
not ending up the I’m not getting the
correct output in the database and the
error is somewhere in this application
and the application is huge these things
usually have some kind of screen
shuffling functionality that you can use
for debugging but it’s often hard to get
stuck traces and stuff out of these
things and perhaps you cannot log into
this system but
then you have problem getting that out
and also we couple with the fact that
there’s a lot of system and say here and
sometimes sometimes things go wrong
which means that you have intermittent
problems and you have no real way to
figure those intermittent problems out
unless you have some kind of logging
infrastructure integration tests require
a lot more work and skill to write them
unit tests it’s an order of magnitude
harder because integration tests are so
expensive
you simply cannot create enough of them
to test your entire app if you’re
actually trying to cover all of your
possible test cases using integration
tests you will find yourself a is
paralyzed with the amount of time and
effort that it requires to debug and
maintain and and write your integration
tests they are simply too expensive to
use for all of your testing but that’s
not the only downside with integration
test there’s also the fact that
integration tests cannot simulate errors
if we go back and look at our head
sample app your system needs to be error
resilient right
if the payment provider for some reason
is down your application cannot simply
crash it needs to be able to handle that
error gracefully or the database might
have a network error it might be a
network area here between these like you
need the ability to fake errors and unit
tests really shine at this this is very
hard to do in an integration test it’s a
it might even be impossible depending on
what your setup looks like simulating
errors is actually one of the really
good arguments for unit testing because
it’s also one of the things that manual
regression testing like human based our
regression testing is also very bad at
so unit tests
are almost your only line of defense
against those errors but the most
troublesome downside with integration
tests it’s that they don’t tell you
where the problem is I touched on this a
little bit before when I talked about
how integration tests are hard to write
when you run a test like an integration
test and and it fails somewhere
sometimes you’re lucky and you get a
stack trace with a clear a clear
description on where the error is but
more often than not is just gonna fail
and fail in a cryptical way which sends
you on a wild needle-in-a-haystack chase
through all of these systems trying to
find out where things went wrong this is
even worse if the error is intermittent
because then you will you can’t actually
let deterministically run it and debug
the application you have to rely on
Long’s and then you need to have very
good logging in your integration test
framework which it you should usually
don’t have because that’s hard so we
cannot simply replace unit test with
integration test because of the
downsides of integration tests
integration tests are expensive they
cannot simulate errors and they don’t
tell you where the problem is you have
probably figured out the moral of the
story already but you need both
integration tests and unit tests when
discussing these things that young
developers Jesus that sounded arrogant
but it is often young developers often
pose the theoretical question like me
but if you had to choose which is more
important integration tests or unit
tasks it is like asking it which one
would you prefer to have removed your
heart or your brain well the brain needs
the body but no they are both new
needed in order to create quality
software you you cannot skimp on one of
them anyway let’s summarize we talked
about unit test versus situation tests I
sometimes talk about why automated tests
are critical to produce quality software
I present the definition of unit tests
and I then presented a tradition of
interventionist I totally with I assure
you there’s a control no matter how you
test our testing these things nor in
parts of your application in isolation
while intervention tasks are more like
coming through the entire app touching
many different systems or a little bit
about the downside of unit tests one was
that they don’t test contracts talk
about how contracts are the
communication between your application
parts the second downside to unit test
is that you have to create those
contract in the first place and finally
I talked about why you can’t simply
replace unit tests with integration
tests because of the downsides of
integration tests which is that they are
expensive and they cannot simulate
errors and they don’t tell you where the
problem is if you liked this video
please do share it will be falling if
you are that colleague welcome you have
yours– watched an episode on fan fan
function I release these every Monday
morning
Oh 800 GMT please press below and check
out the channel and see if it’s
something that you like and maybe even
subscribed I and then DJ until next Monday morning stay tuned