integration is when a software team has
a habit of doing multiple merges per day
and they have an automated verification
system in place to check those messages
for problems our team does this in order
to avoid wasting time hunting down
problems in monster merges that is what
we are going to talk about today I and
npj and you are watching fun fun
function today’s video is going to be
about why teams do continuous
integration I am going to make a video
after this one showing how to get
started with continuous integration
using a circle CI who are sponsoring
this video
however when learning something it is
very important to know why you are
learning it that is why this video is
going to focus on what continuous
integration is and why teams are doing
it and that is to avoid the monster
merge merging code into master almost
always results in some problems and
finding a problem in a code base that is
often like finding a needle in a
haystack especially if that haystack is
this monster merge with lots of commits
in it the philosophy of a team that does
continuous integration is to keep these
haystacks small by doing multiple merges
per day but in order to pull that off
the overhead of every merge the time the
cost must be low to achieve that our CI
team uses an automated verification
system that checks every merge for
problems quickly and automatically and
that is basically a summary of this
video that’s what we are going to talk
about today in today’s video monster
merges is the villain multiple merges
per day is the hero and the automated
verification system is the Batmobile
alright let’s meet our villain I want to
talk about time waste
a little bit when I say time waste I am
referring to time that you are spending
on things that are not really producing
value have you ever had this feeling you
you come home after a day of work and
you you’re tired because you have worked
really hard but you don’t feel satisfied
and the reason you don’t feel satisfied
is because very little came out of it
you spent all day working and all you
have to show for it is some
little label going green when clicked or
some really tiny thing like that if you
have felt this pain ah I want you to
remember this pain and keep it with you
during this episode because that is
essentially why we do CI now of course
there are a hundred different things
that can waste time in software
development and CI actually helps with
several of them but for this episode in
order to keep us focused I want to focus
on one one time waster that I think is
particularly sinister and that is
monster merges as I mentioned before the
team that does continuous integration
they try to reduce the time wasted on
merge problems by doing multiple murders
per day but for a minute I would like to
explore what happens if you do the
opposite what happens when a team does
these weekly monster merges as a
software developer I find that it at
least for me very easy to fall into the
trap of picking a task and then going
into my soul for days and sometimes even
weeks and then I emerge out of my cave
with this monster pile of code changes
it’s monster merge and these monster
merges they I wouldn’t say always but
very often they create these crosser
bumps of time waste in your team where
they are just dreadful
let me look you through
how that happens imagine this we obvious
emerged out of our coat cave with our
monster merge and the first time waster
that we will encounter is that code has
changed under our feet and this happened
because it was very likely to happen
because every hour that we spend in our
code cave the probability of code
changing under our feet goes up and we
spent a lot of hours in that cave this
this merge touches a lot of code so
there is rich conflict galore so we have
to go back into our code cave and
restructure our monster and we pray that
the code doesn’t change under our feet
when we’re away this time but of course
when we come back we find that the code
has changed again and we have to go back
to the cave again but after a few hours
of merge conflict refactoring hell we
finally get rid of all the merge
conflicts and it’s built and we can
start looking for regressions however
because the universe is a cold and
unfair place we find a regression almost
immediately sometimes we are lucky and
the regression comes with a nice error
message exception and a line number but
we are not lucky because the universe
hates programmers and wants to our
lights up so there is no error message
really no hint about the origin of this
error at all at this point you are
probably like I’m going to go home drink
whiskey play fallout 4 and murder a
village after several hours of very
boring work we finally find the culprit
we apply effects we merge that fix and
then we go home carrying with us a sense
of low self-esteem because we got so
little done in so much time at all this
waste that comes with doing these
monster merges that is what a team that
does continuous integration is trying to
reduce by instead doing multiple merges
per day the CI team has established a
habit of disciplining themselves
to break the work into small changes and
merging those changes several times per
day for example if let’s say that you’re
fixing a bug instead of fixing that bug
as one big change you spend a little bit
of time thinking about how can we break
this into parts
perhaps it’s you find out that there is
one segment where you want to fix some
typos add some comments or just
basically make the code better then
there is a some actual refactoring that
you need to do in order to apply the fix
and then finally there is the fix then
you push each of these changes
individually I’m not sure if that
example resonates with you if it doesn’t
like the idea is to break your work
units into small enough units that you
can do many of them per day by changing
code in this way we are in and out very
fast and that means that there is a
reduced chance of the codebase changing
under our feet and when it does change
under our feet less of our work will
have to be redone because our work units
are small I want to stress here that CI
is not some magic thing that shields us
from this problem it does reduce it to a
certain extent and when it happens CI
also provides a good level of damage
control so even with CI in place our
merges will still break a lot they will
not break as often as without the ice
but they will still break but when they
do we will waste less time hunting down
problems because our haystacks are small
so I’ve been speaking very highly of CI
so far so many of you might be sensing
that there is a big hairy butt coming
around the corner what is the problem
with CI or as an American would say the
challenge with CI mpj if multiple
murders per day is so good then why
don’t you marry it why don’t all teams
do CI
first of all many things do you see I I
did a very scientific Twitter poll where
I asked my audience if the team that
they primarily work in uses CI and it
turns out that slightly more than half
do I was actually a bit surprised that
it was that high but when I thought
about it it does make sense
ci has a very good and very clear return
on investment so it does make sense that
it is relatively commonplace but but but
but but they’re big hairy butts that we
tore the challenge getting CI in place
is not a zero cost investment because
you need an automated verification
system in place you must have this in
order to be able to merge multiple times
per day if you don’t have this yes
cannot merge fast enough if you don’t
have a computer helping out with testing
your systems all testing will have to be
done by humans don’t get me wrong here
computer testing cannot replace human
testing completely we still have to take
responsibility for the quality of our
merges but it really really saves a huge
amount of time when you have a system
that takes every merge and builds it and
runs a few thousand tests on it before
letting it into master that is really
nice it makes me more effective when I
know that the computer tests a few
basics then I can relax my manual
testing off those things and then move
that time to two other things that I
know that the computer is really bad a
testing for example checking the things
that I know that the computer cannot
fully test it like a computer can only
test like the CSS disability tags while
I can actually see if a button is behind
something or hidden it if it’s actually
reaching my optic nerve also if I can
relax on the manual regression testing a
bit that means I can spend more time do
in exploratory testing yes what if I
select a ton of items in this this
selection area and I also mix the types
of the selected stuff that kind of thing
and while that kind of testing is
something that humans are better at we
should also remember that there are
kinds of testing that computers are not
better than humans that computers have
one particular edge over humans in that
they are very diligent and that makes
them very good at protecting the
integrity of the master branch as a
human I might you know simply have a bad
day and I forget to build before merging
or I might feel overconfident or lazy or
some combination of both and is kept
doing the regression test excel sheet
this time because it’s such a small
change but there was this little thing I
was not thinking about and I end up
breaking the master for everyone wasting
everyone’s time and automated
verification system will never make
those mistakes and this is how an
automated verification system saves time
which gives us the ability to do these
multiple merges per day because the
system has made me as a human more
effective because I know that our shared
master is building and I also know that
it has undergone this particular set of
tests that we have and I also know that
those same tests will be running on my
merge before letting it in so I can be
slightly less paranoid about about
merging I still have to be careful but I
can allow myself to move faster all
right let’s summarize what is continuous
integration and what problems stop it’s
all monster merges merchant code into
master almost always results in some
problems and finding those problems in
code can be like finding a needle in a
haystack especially if that haystack is
a monster merge multiple merges per day
the philosophy of a continuous
integration team is to keep those
haystacks small by having this
amid of doing multiple mergers per day
in order to pull that off the overhead
of each merge needs to be kept low and
to achieve that our team uses an
automated verification system that
checks every merge quickly and
automatically and that’s my view of CI
and there are many other aspects to it
so please if you have any reflections
please post them down below this video
is sponsored by circle CI I am very
happy about that partially because it
allows me to scale back my day job and
spend more time on Frontline function
but mostly I am happy because circle CI
is my favorite product in this space and
they’ve been that for a long time
prior to this sponsorship when I first
started using CI many many years ago it
used to be very frustrating but the
tooling has improved tremendously in the
last few years and in my opinion circle
CI really pushes the boundaries in in
polish and ease of use Circle C I also
have a very generous premium tier I mean
you still have to pay if you are like an
excitable team but if it’s just you and
a friend you will do absolutely fine on
the feet here forever and I really like
that you can get started by going to
Circle C i.com slack yci that link is
also in the episode description you have
just watched an episode of fun fun
function I really seize every Monday
morning o 800 GMT you can turn on
notifications in the YouTube app if
you’re forgetful if you don’t want to
wait that long you can check out this
video that machine learning voodoo has
selected for you I am npj until next Monday morning stay curious