[Music]
okay I guess we’ll start thanks very
much for deciding to be in this talk
rather than something else I’m Goku and
what I’ll be talking about is how do
service deployments impact how we design
and deploy our systems I think one of
the really interesting things that is
happening at the moment is this whole
buzzword is exploding and lots of people
are approaching it from a technical
perspective where is it
stateless is it functions is it platform
as a service and as an industry we like
dealing with technical stuff that’s what
we do tend to do but I think that the
whole serverless thing whatever you
decide to call it is much much much more
important to look from a financial
perspective so the way we are
approaching things in terms of designing
and deploying applications things we
take for granted today I have been built
based on our experiences in the last 20
30 years deploying systems and many of
the constraints that exists in the last
20 30 years no longer applied to
deploying with a SS lambda which means
that many things that we now take for
granted as best practices just are
solutions for constraints that no longer
apply and that was really really
interesting for me kind of when we
started getting our head around lambda
deployments and that’s kind of what I
want to talk about what are the things
that we now think are best practices but
are actually just solutions for problems
that are no longer applicable and kind
of in terms of what I’m gonna talk about
just as an example I develop a
collaboration up that helps people do
mind maps online and in February 2016
we started migrating from Heroku to a SS
lambda it
took us about one year to move
everything because we did gradually
during that year we increased them now
kind of decreased the hosting costs by
about 1/2 and while at the same time
adding a bunch of new services and our
number of active users increased by 50%
in the same period so kind of all in all
my estimate is that we saved around 66%
or 2/3 on our hosting costs and that’s
really really interesting if you look at
it from a perspective running a small
business
so after kind of I started publishing on
this a guy called Robert Charlie got in
touch with me he wanted to do a
scientific paper on this and together we
wrote a proper scientific paper Martin
talked about science papers yesterday so
you know all my professors at the
university of finally gonna be proud of
me and know that all the alcohol kind of
was worth something at the end and you
can kind of download this it’s it’s
called the kind of economic an
architectural impact of cephalus is
horrible but that’s okay so you can get
a lot more on the numbers that I talked
about there but I realized as we were
doing research for this that actually
our results were not even that
interesting because we talked to people
that saved something like 99% on their
hosting costs by moving away from other
platforms to – lambda’
Heroku is reasonably cost efficient
anyway and moving or moving from older
generations of cloud hosting or moving
from on-premise hosting to lambda has an
even bigger potential to kind of do
stuff so the key thing there I I think
that is really important to consider is
kind of the way lambda is priced and in
my experience is mostly with AWS lambda
although pretty much all the other cloud
providers are copying features and
models now so the way a double is lambda
is priced kind of fundamentally changes
the incentives for deployments
fundamentally changes the incentives for
good architecture and I think Martin’s
talked about yesterday about engineering
and how that’s working within
constraints
I think cost is one of the major
constraints we have to work in
and the pricing model kind of
fundamentally changes really important
so kind of in terms of the pricing lamda
prices stuff per request and per 100
millisecond increments in a processor in
a virtual memory pack so these two
things are really really important to
consider because they change how we pay
for for kind of what we’re using and
they start charging things not in terms
of reserved capacity but in terms of
actual usage on a hundred millisecond
increments so it doesn’t matter whether
you have five VMs 500 VMs if you run
three boxes or 5000 boxes to process
something all that matters is how many
requests came in and how long they took
to execute under what memory conditions
so this whole buzzword of services is is
horrible because of course there are
servers out there and things like that
but as somebody came up with a real nice
definition on Twitter the other day is
saying that kind of that the thing is
serverless
if you’re only paying for actual usage
if you’re not paying for what you have
to plan as a reserved capacity so kind
of historically that’s not been like
that historically good architecture
optimized for reserved resources and I
you know 20 years ago in my previous
life I worked on trading platforms that
were deployed on immortal Hardware I was
never supposed to die the storage cost
more than my house the processors were
insanely expensive and everything was
duplicated replicated replicated because
it’s never ever ever supposed to die but
once you have a machine like that you
optimize for using what you’ve bought
you kind of you bundle stuff onto it you
put everything you can to run there and
you’re very very careful not to exceed
the capacity of that because if you do
then you know adding a couple of more
processors or adding a bit more storage
requires
an insane amount of cost so then 2006
Amazon kind of came out with the idea
that you can get a virtual machine
running in about five or ten minutes
which was completely insane at the time
I worked for a big telecoms provider
where it took them nine months to
provision a virtual machine internally
and you know now comes Amazon you can
get the VM for about ten minutes and and
that there was amazing but kind of it
did not fundamentally change how we are
thinking about deployments because you
got five virtual machines you’re paying
for five virtual machines so you’re
gonna bundle everything into those five
virtual machines and we teach people
good software design like decoupling
isolation and and all those brilliant
software design practices and then
because you have five virtual machines
then you put your payment servers and
your log and your monitoring system all
on the five virtual machines where they
start interacting with each other and we
had this problem we’re kind of deploying
lots of different payment services to
reserve VMs one of the VMS did not clean
up the temp space correctly now one of
the payment services did not clean up
after itself and all the time it filled
up the temp space Linux starts really
really misbehaving when you fill up the
temp space so that kind of VM started
going crazy all the payment services
that machine went down although they
were designed to be decoupled and
isolated and everything and that when
that machine went down there was a
cascade all you know the remaining four
machines started filling up that time
space very very quickly and everything
kind of imploded
so um although you know he teach people
about good design as decoupling we end
up deploying stuff to save cash because
we need to reserve capacity and the next
generation of cloud deployments Google
App Engine Heroku and things like that
they moved a lot of the responsibilities
over to the cloud providers like
provisioning like monitoring and things
like that but the whole thing remained
you were paying for dinos on Heroku for
example we our app was deployed on
Heroku we
we have about 20 or 30 different
exporters in two different formats and
if I’m going to run a primary and a
failover for each of these exporters on
separate isolated VMs plus for some
exporters I need a lot more capacity and
for others for some one in five or six
VMs that means that for 30 exporters I
need a hundred VMs to run on if I really
want to make it reliable and isolated
but some of those exporters like
markdown I wrote for myself nobody else
uses that some of those exporters like
PDF I use it all the time so you know
I’ll put the market on exporter on the
same block of machines as the PDF I’m
not going to create a separate block of
machines to save money and you know of
course we made a stupid mistake and some
of those things started interfering with
each other because they were on the same
machine now technically we have a
solution for this and we had a solution
for this it’s containers it’s isolation
it’s things like that but as a you know
average company it’s very very difficult
to dedicate the resources to manage
everything there so what lambda does
Islam that kind of provides that as a
service so we start moving away from
reserved capacity to utilize capacity it
doesn’t matter how many VMs you need
Amazon is going to scale this and
recharge you for how much you’ve used
and that’s kind of the request based
pricing that’s really really interesting
so what that means is that there’s no
more financial incentive for me to put
the markdown in the PDF exporter on the
same VM even worse because they have
different memory needs markdown doesn’t
need any memory PDF needs a lot because
it’s using go script and go script is a
memory hog putting those two things on
the same VM I will end up paying for a
markdown for a high memory watermark
that’s a lot more than I need if I
separate them so the financial
incentives here are actually for people
to unbundle the apps in all these small
isolated modules it’s really really
interesting because said historically it
was completely different so one of the
things that I think kind of people
should think about more and it’s not
microservices it’s not kind of functions
you can you can buzz for this any way
you want but I think what we started
thinking about is really
decoupling different tasks that have
different memory constraints different
CPU constraints different deployment
needs and thinking about well you know
if it doesn’t matter how many VMs this
runs on I don’t care about it being
separately deployed and that’s very very
liberating and it’s completely opposite
of what we’ve been taught to do a lot of
kind of many practices like Bluegreen
deployments many practices like kind of
concentrated monitoring and things like
that they did no longer that problematic
and that opens up some really
interesting possibilities for example
moving from Heroku to lambda we’ve kind
of deleted a bunch of code that was –
dealing with kind of you know figuring
out what’s going on on a particular VM
figuring out how these things interact
with do they interact with each other so
a lot of this infrastructure code is
gone so the next thing that kind of I
think historically happened and no
longer applies is generally good
architecture because of the whole idea
of you know reserved stuff optimized for
failovers and what that meant was really
complex state management you wanted to
have a warm failover machine or a couple
of warm fill of machines that can take
the load in you know very quickly if
something fails so data needs to be
replicated cache needs to be replicated
data needs to be synchronized across
replicated machines then this kind of
cache invalidation policies people need
to care about this there’s a lot of
really complex code to do to make this
work really really well but from a
perspective of paying per request kind
of failover machines are not doing
anything so kind of unless the receiving
request you’re not paying for them and
then the big question becomes do you
actually need to plan for failover like
that and another really interesting
constraint that I think Canova’s is
changing with the whole lambda
architecture is kind of the time to
recover that was historically a big
driver for a lot of architectural
practices is no longer that important
and that opens up some really
interesting possibilities what I think
is becoming a lot more important is the
time to start
so when we talk about the time to start
I’ll just show you something very
quickly let’s see if the gods of the
Internet help us here so ok so if I do
[Music]
and I do constable let’s be nice
so this is kind of a a web api that on
any request to slash hello will reply
with hi there it’s not that’s kind of
complicated
and if I’ve not made a stupid mistake we
should be able to do something like so
I’m using an open-source tool for
deployments here that actually we’ve
open sourced while building it for
mindmap the first one that we created
was something like 30 lines of
JavaScript code and about 250 lines of
shell scripts to deploy and then I
realized that kind of the risk is no
longer in the code the risk is in the
deployment so we wanted to have properly
unit tested deployment and configuration
and and we kind of just packaged up all
the scripts so this thing is now on on a
URL on Amazon and if I grab this URL so
I’ve created a slash hello so if I do
curl hello I get to hi there
so now this is a kind of thing that’s
running on one VM but what’s really
interesting about this is that if all of
you start attacking this thing now it
will perfectly handle the load if
nobody’s using it they don’t have to pay
for anything if we have a massive
massive spike now but then kind of you
know the spike goes out that’s perfectly
fine as well it will scale up and scale
down plus with this I get a bunch of
things automatically like monitoring
logging security provisioning all the
stuff that kind of people want to deal
with the operations so I’ve created this
thing called go to Copenhagen so if I go
to my lambdas now
come on internet okay so we have a knob
that’s not the right account
goodgoodgood
so let’s see Amazon four one three six
so for now you’ve seen my code come on
come on so if I go to lambda now I have
no idea what they’ve changed here so
this is the okay III you can see how
often I use this anomaly use the command
line D so what I want you to say is kind
of with with this you get a bunch of
operations things for free immediately
and why is this suffering bustle and
things that doesn’t matter
let’s functions functions functions
functions page to go to Copenhagen okay
so here it is and then I have kind of
lots of stuff around monitoring I have
my kind of errors throttles I have logs
in cloud watch I have pretty much
everything I need from an Operations
perspective already provided to me so
there’s this whole buzzword kind of that
lambda is moving from DevOps to no ops
and Simon worldly who’s a researcher in
the UK said kind of your company’s not
done DevOps yet just don’t bother skip a
whole generation of things and move to
something like this and it’s easy
fair enough it’s not completely killing
DevOps but now I said you know you have
all the operations stuff already I have
a log I have kind of when it started
when it ended how much memory it used
and this is amazingly good for
optimizing stuff if you look at kind of
the typical architecture and and and
stuff like that we normally had figuring
out how much money a particular function
cost you to run was almost impossible
people are optimizing based on a gut
feel they were optimizing whole
applications now I know exactly for this
particular task how much money I’m
spending so I can decide do I need to
kind of invest in optimizing that or not
so that’s really really really
interesting and kind of in terms of time
to start here’s some here’s some
empirical numbers that I got these are
not confirmed by the AWS a Tablas
doesn’t publish any numbers about this
but these are kind of my numbers that we
got generally so for a completely new
instance of a deployed up so if you
start hitting this stuff now and this
instance that it’s running their content
load with the JavaScript
this lady takes less than one second to
get the new instance if you want to do a
new deployment a new version like I’ve
just done it takes about three or four
seconds to kind of set everything up so
that’s the infrastructure stuff now
what’s left is how long our app takes to
connect to the database load up the
cache load up the data and things that’s
I think kind of unlike the previous
architectures where we were optimizing
for quick failover what we need to start
thinking about is optimizing for quick
start lazy loading everything kind of
making sure that we don’t add a lot of
this stuff because if we can then does
this stop just kind of starts and dies
on its own now there’s a whole buzzword
here where people are talking about how
we need to optimize we need to write
this stuff in a stateless way because
there’s a big confusion of buzzwords and
some people talk about services function
as a service than in a functional
programming and functional programming
is stateless and this you know people
throw buzzwords around lambdas are not
stateless don’t have to anybody tell you
that lambdas are a stateful container
just having no idea how many containers
you’re running once the container starts
you have no control over when is it
going to stop and whether the next
request from the same user is going to
hit the same container or a completely
different container so rather than
developing for stateless what I think we
should be thinking about this developing
kind of for sure nothing designing for
sure nothing when the VM load is
perfectly fine to start caching things
in that VM that are not user specific
that’s how we save a lot of time on
processing requests that’s how we save a
lot of cash because you know that there
will be a VM running it’s just that you
mustn’t ever catch anything that is
specific to that particular request the
state of a user session is not there the
state of a user session needs to be
somewhere else and we’ll talk about
quite an interesting option where to
push the user state later but kind of
it’s it’s not stateless people that have
never kind of deployed any of this talk
about lambdas being stateless they’re
not and and I think you lose a lot of
money by treating them like stateless so
the next thing that I think kind of
historically happened is it was really
really difficult to replicate production
it was you know people had a production
system with expensive storage or even if
it’s on the cloud with good VMs and then
that cost a lot so you wouldn’t want to
pay the same amount for your staging you
didn’t want to pay the same amount for
the testing so typically kind of
performance tests unless you are Lmax
would run on a kind of a smaller copy of
data on a smaller copy of the production
and things like that they were not
really relevant they were not reliable
and generally that made it really really
difficult to claim anything about
production usage and production
performance based on the testing system
now kind of I have a couple of friends
from Australia and and they they claim
that I don’t know if this is true or not
the time that kind of Facebook is
continuously broken in Australia because
Facebook is really good with kind of
running business experiments on all
their users and kind of Australia’s the
market that’s big enough to be
statistically significant it’s English
so they don’t have to waste a lot of
time preparing software for that but
Facebook doesn’t really care too much
about Australians they’re laid-back
people anyway so they can do experiments
in Australia and then when they figure
what works what doesn’t work they can
translate that to the US now when you
think about something like that that’s
incredibly useful but up until five
years ago or up until three years ago
that was available to you if your Google
or Facebook running experiments like
that is you feel expensive you need a
complete copy of production for
something like that and you need the
relevant copy of production you need to
synchronize the data between these
things what one of them kind of best
examples of how important that is is a
story called 40 shades of blue they’re
absolutely loved because it’s one of
those rare things where you can read
both sides of the story online forty
shades of blue led to kind of that the
head of design at Google wanted to
change the color of the links on the
homepage for ads and the developers
challenged him a bit
and he wanted to change it to a
particular blue collar overnight they
ran 40 different colors of blue and
measured how much people are clicking on
that and then as a result of that
depending whose side you read this guy
either quitter was fired because the
difference was something like 250
million dollars between the current
color and his color if you expand it to
a whole whole year and 100% of the users
so kind of things like that are
incredibly important to test but for
most people out there it was almost
impossible to do it because production
copy cost so much now when we started
doing lambda I had this brilliant best
idea in the world and I wanted to
integrate wiki data graph of knowledge
with our app so that when you press a
question mark it automatically opens up
related terms it was going to be amazing
it was going to be brilliant it’s going
to be fantastic and and my business
partner said no that’s a shitty idea
nobody’s going to use that let’s not
waste time doing that so you know I kind
of generally don’t like to ask for
permission like to ask for forgiveness
so I decided I’m going to do that anyway
and he said I know you’re going to do it
anyway do not touch the
production code because if you put this
in we’re going to have people you know
screaming that we can’t support the
right performance we’re gonna be bottom
like here we’re gonna and I said okay
and then I realized kind of with lambdas
you don’t pay for stuff if you people
don’t use them so I kind of spent two
days knocked up a quick lambda version
that was extended with this deployed a
completely separate copy of our
production and sent 20% of our users
there and you know lo and behold I
proved that it was a idea nobody
wants to use it so we deleted that code
with our disturbing uh production so but
this is this is some you know something
that traditionally we’d get into a fight
who’s right who’s wrong guy would do it
anyway and then that code would stay in
the production and kind of you know cost
us more to maintain it was amazing
because we could run a quick experiment
so because we’re paying for requests
we’re not paying for reserve capacity it
was exactly the same amount of money to
send 80% of users to one version 20% to
another or to send everybody to the same
version or to have a version for
Copenhagen or to have a version for
go-to or to have a version for anything
you want and actually kind of what this
starts getting us to think about is
moving away from thinking about
production to thinking about multiple
versions and multi versioning in lambda
is incredibly well done I you know for
my scenes like everybody else have done
an Orion framework a logging framework
and a data multi version in framework
everybody’s done that when they young
otherwise you don’t you know you don’t
get to call yourself a developer and
then you end up kind of suffering
through maintaining that for 10
years but that’s okay so a data multi
versioning request multi versioning
service multi versioning is really
really difficult to do well at scale and
Amazon probably have done it for their
own needs and they’ve just exposed a T
lambda so every lambda function gets a
numerical version every time you deploy
it and you can direct the calls for a
particular function either to the last
non-deployed version or to a particular
numerical deployment or you can assign
aliases to a numerical deployment like
production testing staging so when we
started doing this we started really
kind of doing oh I have a production
version a testing version and staging
version and then we realized why I’ll
just save a version to test this feature
oh I’ll have a version to kind of you
know when we’re doing something
experimental and 5% of our users really
really need this feature but it’s not
ready for everybody else we can deploy a
version for them and give it to them and
test it on them early and it’s multi
version is incredibly well done so I
think this is a really interesting thing
to start thinking about kind of how do
we bundle our tasks because it has a
massive impact on how we design how we
deploy if we are going to design for a
multi version universe where you know at
the same time several versions of a
wrapper running and different things are
communicating with even I think the
whole domain driven design concept of
aggregates becomes incredibly more
important because we want to make sure
that kind of you know all the data for a
particular version of this object
travels together and because we are kind
of don’t want to clog the communication
things that we want to make sure
aggregates are actually relatively
minimal so that we can push them around
and we can load them quickly but the
movie to land has gotten us to thinking
a lot more about what are our actual
aggregates where the aggregate
boundaries what needs to be in a
particular version consistent with
itself what can just kind of be
different versions and and be okay so I
think that’s a completely interesting
kind of phenomenon that are not really
seen in my code before that and because
we’ve designed for multi versioning now
up front as we’re migrating we can do
lots of crazy things and the stuff that
was the traditional available to you
know companies that make billions and
billions of dollars we can do now and
we’re a two-person team and that that’s
I think amazing as a kind of for what we
can do from the platform so the next
really interesting thing driven by the
pricing model of AWS lambda is that
different services charge for different
things so you can save quite a lot of
money by playing arbitrage available yes
against AWS that’s amazing
and remember kind of lambda is as a
processing service charges for the
number of requests in time so if you can
delegate work from lambda to stuff that
does not charge for the number of
requests in time you can save a lot of
cash for example kind of API gateway
charges for the bytes being transferred
the number of requests at the same time
kind of s3 that’s the storage system
just charges for transfer it doesn’t
care about the number of requests so one
example of that is that we own Heroku
and before we started really thinking
about this we would let people upload
files for export to a server where the
server would communicate with the
storage and then the server would save
stuff through storage it with rondo
converter it would kind of upload it
back from the storage to the user what
that means is that during that whole
time the server is busy we’re paying for
the server now if you’re uploading a
hundred megabyte file the transfer from
you to
Amazon and the transfer for Amazon to
you is actually the most amount of time
the conversion time is relatively quick
so we’re paying kind of for this
operation for a long time we’re because
there’s three only pays s3 only charges
for transfer if we can get people to
upload directly to s3 and download
directly from s3 we have reduced our
server costs by a significant amount so
some other service that are interesting
to consider like kognito the Amazon
authentication and session service only
charges for the number of users it
doesn’t charge for the number of
requests those users make it doesn’t
charge for the capacity of the sessions
it only charges for the number of users
so moving session state into kognito
it’s a really really interesting way of
kind of playing arbitrage with Amazon
versus Amazon so kind of as a kind of an
example I’ll show you later there’s also
this thing called the IOT gateway that
we abuse massively IOT gateway is
designed to get low-power devices to
talk to each other but we’ve built kind
of real-time collaboration directly
through we’re building real-time
collaboration directly through IOT
gateway which kind of because it only
charges for the number of messages it
doesn’t charge for processing time it
doesn’t charge for data transfer again
you can arbitrage things nicely so kind
of we started moving a lot more for
thinking about applications and and kind
of managed apps like Heroku like Google
App Engine to really the glue between
different platform services for me
lambda is mostly about what’s missing
from amazon’s platform and how do i kind
of glue those things things together and
what’s the kind of what’s the real
business value of my code because it’s
unlikely that developing a queue
interface system is where i can provide
the most value i can provide value
developing kind of the small bits of
processing those queue messages and
that’s really really interesting so kind
of just as an example that this is kind
of how to get people to use s3 directly
from a browser so on a server we have
something like this the s3 is the Amazon
API SDK so you get a request for upload
file you populated with you know where
it needs to go you limit the file size
you
kind of provides security stuff and then
you get a signature and then returned
that to the browser that takes you know
20 milliseconds then the browser spends
ten minutes uploading the big file to s3
where you’re just paying for transfer
then kind of this thing goes very
quickly you go back and people download
it directly from s3 from a signed URL so
amazon being amazon of course there’s
this five different ways of authorizing
requests
there’s signed urls there’s the cig v4
that they do for kind of approaches this
is giving people access to kognito so
this lots and lots of ways how you can
get people to access directly one of
these front-end services without going
through a traditional server so kind of
in to really benefit from that
financially I think what we started
thinking about a lot more is give the
platform the roles that were
traditionally associate with the server
process things like being the Gateway
keeper things like being the
orchestrator things like keeping kind of
sensation storage if you push that away
from the lambda that’s kind of the
processor to the other parts of the
platform you can save a lot of cash now
here’s why I think this is so insane in
September our app had something like
400,000 active users it’s it’s not you
know Google but it’s not Mickey Mouse as
well and just so that I’m not cheating
this is V live Amazon page so if I go to
my billing dashboard and I look at my
costs for where is it the September bill
bill details September so for four
hundred thousand active users in
September we have paid 53 cents
for lambda now beat that with your
hosting costs and we you know they of
course there’s there’s some other
services like we paid four dollars for
data transfer and then we paid something
for API gateway and something for dynamo
and things like that but all in all that
the bill was a hundred bucks for a you
know four hundred thousand active users
that are kind of collaborating in real
time now this is insane
completely insane if you look at kind of
stuff equivalent stuff I was doing ten
years ago that this is completely
completely insane and you know add up
all the multi versioning and everything
else they provide for almost not free
but included in the price that’s that’s
why I think this is gorgeous and and and
you know fantastic in so many ways so
kind of the another thing that started
happening here as we started thinking
about more and more of arbitrage in
different services against each other we
realized that kind of what engrained in
my head for the last 30 years is do not
trust users to talk to Becky and
resources like users are not allowed to
talk to storage directly users are never
ever ever allowed to connect your
database directly they have to go
through a gatekeeper they have to go
through a server because on the server
we discard invalid requests we validate
stuff we you know everything before the
server we don’t trust everything after
the server we trust that’s how we you
know did the whole web logic thing
evolved we have application servers we
have storage we have clients and I think
kind of especially if you start to use
the platform on Amazon none of these
things are actually physically back-end
resources anymore s3 is available over
HTTP dynamo that’s the databases
available over HTTP the fact that we are
not letting users talk to you directly
doesn’t mean it’s not available if
somebody kind of guesses the name
it’s there Amazon is making it available
and because of that kind of Amazon is
actually implementing really really good
request level authorization policies
each single request going from lambda to
a database is authorized because if they
have no idea if your lambda is talking
to the database if somebody else Islam
days talking to derivative is and it’s
not your database anywhere it’s their
database and it’s kind of things like
this are really really interesting from
a perspective of thinking about well you
know if it’s authorized per request
what’s the damage in actually kind of
authorizing clients to go there so
Amazon gives you three or four different
mechanisms for authorizing these
requests including say including saying
that this user is only allowed to write
to this particular key in the database
and only allowed to read from these keys
in the database hierarchically or this
user is only allowed to read from this
part of the queue and post to these
parts of the queue and the same
authorization policies exactly the same
apply as if you talk to from the client
there then if you go through the server
and then I realized well you know we’re
just introducing latency by putting a
server in the middle we’re just paying
more and we started kind of using this
like mad my brain still does not allow
me to or let users connect to the
database because I’ve been a server-side
developer for 20 years and it’s just
wrong
but we’re letting people talk to the
storage directly well I think people
talk to the queues and and kind of
things like so you know maybe two years
from now I’ll I’ll come into the talk
and say no no everybody’s you know
talking to the database and all our data
is stolen and it’s horrible or you know
but generally if you think about it’s
not your database it’s Amazon’s database
and it’s available using HTTP so it’s
not back-end it’s content or its
middlewares of some kind so kind of we
started moving away from kind of
three-tier models to smart clients not
not dumb terminal smart terminals where
things connect directly so here’s a URL
and this is a
a prototype we’ve developed for a chat
app that works on all browsers and works
on all mobile devices and things like
that and kind of it
the source code is on github you can
look it up afterwards so kind of connect
to this from your mobile phone or
something like that and then you can
kind of log in as a guest what that
means is that we are now getting a
authorization ID from Cognito I can make
this login with the username and
password log in through Google login
through Facebook log in through many
many different ways do two-factor
authentication for a conference kind of
show it’s you know open and then once I
have this I can actually use their API
directly from a browser to talk to any
resource I want that I’m allowed to talk
to so in this case we are talking
directly to if you cannot get into the
server listing just pump it up we are
talking to the IOT gateway the IOT
gateway is designed to kind of exchange
messages between low-power devices but
it’s actually allowing you to have a
WebSocket interface as well how cool is
that so you can use WebSockets on demand
managed that paid $5 per million
messages so peanuts and they’re
completely managed so you know we can
get a million people connecting to this
now or we can get five people connecting
to this and it’s it’s all done
operationally the source code is 30
lines of code so it is just completely
insane what we can do now with this
stuff and how much it costs and I you
know ten years ago I remember kind of
evaluating lots of different push
mechanisms where they were doing
degrading of a flash they were doing
long pole they were doing all these
amazing things and I think the best
thing we’ll choose they were asking for
something like a hundred thousand quid a
month I can get this now for five
dollars on a million messages if
nobody’s using it
I don’t pay if people are using it and
probably making money of it so you know
perfectly fine to pay but there’s no
upfront cost there’s no monthly
maintenance cost this is insane
it’s completely since I think kind of
this thing changes how we approach what
we reserve what we what we kind of do
and I said you just format see a nice ah
good good good good good
so um so yeah I said that is you know
from here you can just go to go to the
Lincoln and you can get the the source
code for this so kind of in that respect
what I want to say is good engineering
who the architecture is driven by
constraints cost is one of the key
constraints we have to deal with and
deploying on lambda fundamentally
changes the cost structure so lots of
stuff that you know have evolved as best
practices over the last 20 years kind of
no longer apply and our challenge as a
community over the next five 10 years is
going to be to figure out what are you
know just the shackles of the old world
that we’re running with I mean in it you
know I I can talk what’s wrong about
lambda 4 you know days on end and this
is not a silver bullet it doesn’t solve
all the problems but I think it’s a
really really interesting perspective if
you can you know run a four hundred
thousand uses and pay fifty three cents
for the whole thing it’s just insane and
I think kind of the financial incentives
of that are going to get pretty much
everything that can run in lambda to run
in lambda over the next five years and
that’s why you know although most people
here I assume are not really deploying
things in production for lambda yet this
will comes to start investigating that
and in particular running cheap
experiments is amazing if you need to
run a kind of cheap stupid experiment
and you don’t know if it’s going to work
out or not this is brilliant for that
and then even if you do and kind of
on-premise deployment later then you
know what to integrate and and and what
to throw away so kind of I think one of
the key things that was you know a mind
shift for us is
is to start letting clients connect to
back-end resources because there are no
back-end resources anymore and what that
means is that all of the sudden your app
is not really running just on fifty or
five or five hundred virtual machines
it’s running on four hundred thousand
client processes as well because you can
push if you let people talk to Becky and
resources you can push orchestration you
can push state to the client and all the
stuff that kind of is difficult to
manage in the distributed architectures
push it to the clients machine where the
client is a single client having a
single state which simplifies things
significantly so instead of kind of that
that’s why we pay so little for lambda
our app does not run on the VMS what
runs on the VMS is a glue between
different back-end services our app
actually runs on four hundred thousand
client processes that we do not have to
pay for and that’s kind of the really
really interesting mind shift here so
kind of as two URLs for for more info
this first one is my blog where I post a
lot about this stuff because I’m
incredibly excited about it the second
one is the open source tool for
deployment that I’ve shown you that kind
of simplify stuff if you do in
JavaScript that’s pretty much it thank
you very much I hope I kind of tickled
your imagination at least of it
and if anybody’s posted any questions I
guess we can talk about that now do we
have any questions where’s the I can
read it loud okay lovely how do you feel
about locking kind of that that’s a
really interesting question and
unlocking is is problematic on several
levels lots of people talk about locking
in terms of locking with libraries
locking with code if you use Oracle then
you know you use Oracle’s libraries with
lambda because the platform calls you
not the other way around you’re actually
not locked into the lambda API at all
there aren’t it’s it’s moving away from
there would be trivial the big problem
is you locked into the platform if you
really want to get the benefits of
lambda then you’re letting clients talk
to storage directly I think lines talk
to the database correctly and that’s
where the locking happens now for us
we’ve kind of we we’ve decide
commercially that going really for
Amazon and and using Amazon for
everything is a good commercial decision
it gives us the risk Obama’s are not
working but in my experience kind of
they’re pretty solid and they’re much
much better doing the OP stand I can so
I I know that there are some tools like
the serverless framework that allow you
to deploy to multiple clouds and do kind
of this hybrid thing but I guess the big
problem then like you know doing a
database independent deployment is you
get to use the the least common
denominator of the whole thing and
you’re never really using the platform
what it is so kind of that’s a
commercial decision that I guess
everybody needs to make on their own but
there’s definitely things like you know
ports and adapters or hexagonal
architecture and things that you can
design stuff so if you do actually
decide to move on to different cloud
provider you know you will I I don’t
I’ve never worked with a company where
the whole investment in being able to
move from the primary database paid off
because they never moved away from the
primary database so it’s a commercial
decision there’s only two of us building
this thing so I’d rather spend stuff
delivering successful features than
building kind of an abstract software
system but you know if you have five
kind of developers why not keep them
busy so so the this disorder locked your
kind of can I get Apple pay on your
phone as well when you unlock it okay so
we have how do you keep the state we
decision so we tend to push a lot to the
clients directly and and we tend to keep
the state in either incognito or in the
users browsers kognito does automatic
synchronization across devices we don’t
tend to use that a lot because our our
state is typically kind of the document
you’re working on and we don’t need to
keep a lot of that but kognito is a
pretty good way of kind of synchronizing
stuff across devices using dynamo using
something like that to keep kind of the
state per user is also pretty good
because you can configure dynamo to
allow users to write only to a
particular key so you ko or or a sub key
so you can write only to your own state
and read only from your own state for
example that
that would be a possibility mm-hmm if
I’m if I’m P dose so uh III don’t know
that’s never happened to us kind of
lambda n API gateway allow you to
throttle things you can just configure
throttling so you can say that you know
I want to run up to a thousand
concurrent functions of this or up to I
think the limit by default is a thousand
per function but then you can increase
it or with the API gate where you can do
throttling based on an API key an
authorization key or kind of generally
on an API endpoint so you can configure
throught link so you can with the
monitoring they have and things like
that spot if you are kind of being DDoS
again my assumption is that Amazon will
protect against DDoS much much better
than I can code that I don’t know kind
of about anybody in the audience whether
you feel you can build a better DDoS
defense system than Amazon but certainly
you can configure it to be to get an
early warning and then figure out what
to do from there so you don’t have to
spend millions and millions and millions
if you get the dust at the moment the
biggest disadvantage is that lambda
functions are limited to about five
minutes run
so anything that takes longer than five
minutes you need to split into multiple
executions which means you can’t keep an
open socket we were trying to develop
something that talks to the Twitter API
s and Twitter doesn’t have a push API or
you know if you’re on mortal you cannot
get the push API you need to connect the
socket and get them to kind of stream
stuff to you and with lambda that’s not
I mean it’s possible but you need to
kind of load it every five minutes and
then disconnect in the render state so
for something like that I would still
use ECS another kind of reasonable
disadvantage for many people is that
there’s this virtually no SLA the lambda
or not virtual is actually no isolation
and that they don’t offer any date and
they don’t of
Neela’s yet so our experience is that
kind of the u.s. East one the region
that is overloaded with everything
because that’s the first one that
started occasionally gets kind of
hiccups where we get a bit of delay but
we’ve never really had a full outage
since February 2016 when we started kind
of moving to this that doesn’t it’s not
going to happen and I think you know as
Lomb that becomes more and more
important I assume they will start
providing a slice for it at some point
so that’s an interesting limitation B I
guess those would be those would be the
two key key limitations for people so I
think we ran out of time I don’t know if
we have kind of I’ll be around you know
you can pick me up in in in in the
corridor and then we’ll talk about this
thing more I need to get other people to
set up thank you very much
[Applause]
okay I guess we’ll start thanks very
much for deciding to be in this talk
rather than something else I’m Goku and
what I’ll be talking about is how do
service deployments impact how we design
and deploy our systems I think one of
the really interesting things that is
happening at the moment is this whole
buzzword is exploding and lots of people
are approaching it from a technical
perspective where is it
stateless is it functions is it platform
as a service and as an industry we like
dealing with technical stuff that’s what
we do tend to do but I think that the
whole serverless thing whatever you
decide to call it is much much much more
important to look from a financial
perspective so the way we are
approaching things in terms of designing
and deploying applications things we
take for granted today I have been built
based on our experiences in the last 20
30 years deploying systems and many of
the constraints that exists in the last
20 30 years no longer applied to
deploying with a SS lambda which means
that many things that we now take for
granted as best practices just are
solutions for constraints that no longer
apply and that was really really
interesting for me kind of when we
started getting our head around lambda
deployments and that’s kind of what I
want to talk about what are the things
that we now think are best practices but
are actually just solutions for problems
that are no longer applicable and kind
of in terms of what I’m gonna talk about
just as an example I develop a
collaboration up that helps people do
mind maps online and in February 2016
we started migrating from Heroku to a SS
lambda it
took us about one year to move
everything because we did gradually
during that year we increased them now
kind of decreased the hosting costs by
about 1/2 and while at the same time
adding a bunch of new services and our
number of active users increased by 50%
in the same period so kind of all in all
my estimate is that we saved around 66%
or 2/3 on our hosting costs and that’s
really really interesting if you look at
it from a perspective running a small
business
so after kind of I started publishing on
this a guy called Robert Charlie got in
touch with me he wanted to do a
scientific paper on this and together we
wrote a proper scientific paper Martin
talked about science papers yesterday so
you know all my professors at the
university of finally gonna be proud of
me and know that all the alcohol kind of
was worth something at the end and you
can kind of download this it’s it’s
called the kind of economic an
architectural impact of cephalus is
horrible but that’s okay so you can get
a lot more on the numbers that I talked
about there but I realized as we were
doing research for this that actually
our results were not even that
interesting because we talked to people
that saved something like 99% on their
hosting costs by moving away from other
platforms to – lambda’
Heroku is reasonably cost efficient
anyway and moving or moving from older
generations of cloud hosting or moving
from on-premise hosting to lambda has an
even bigger potential to kind of do
stuff so the key thing there I I think
that is really important to consider is
kind of the way lambda is priced and in
my experience is mostly with AWS lambda
although pretty much all the other cloud
providers are copying features and
models now so the way a double is lambda
is priced kind of fundamentally changes
the incentives for deployments
fundamentally changes the incentives for
good architecture and I think Martin’s
talked about yesterday about engineering
and how that’s working within
constraints
I think cost is one of the major
constraints we have to work in
and the pricing model kind of
fundamentally changes really important
so kind of in terms of the pricing lamda
prices stuff per request and per 100
millisecond increments in a processor in
a virtual memory pack so these two
things are really really important to
consider because they change how we pay
for for kind of what we’re using and
they start charging things not in terms
of reserved capacity but in terms of
actual usage on a hundred millisecond
increments so it doesn’t matter whether
you have five VMs 500 VMs if you run
three boxes or 5000 boxes to process
something all that matters is how many
requests came in and how long they took
to execute under what memory conditions
so this whole buzzword of services is is
horrible because of course there are
servers out there and things like that
but as somebody came up with a real nice
definition on Twitter the other day is
saying that kind of that the thing is
serverless
if you’re only paying for actual usage
if you’re not paying for what you have
to plan as a reserved capacity so kind
of historically that’s not been like
that historically good architecture
optimized for reserved resources and I
you know 20 years ago in my previous
life I worked on trading platforms that
were deployed on immortal Hardware I was
never supposed to die the storage cost
more than my house the processors were
insanely expensive and everything was
duplicated replicated replicated because
it’s never ever ever supposed to die but
once you have a machine like that you
optimize for using what you’ve bought
you kind of you bundle stuff onto it you
put everything you can to run there and
you’re very very careful not to exceed
the capacity of that because if you do
then you know adding a couple of more
processors or adding a bit more storage
requires
an insane amount of cost so then 2006
Amazon kind of came out with the idea
that you can get a virtual machine
running in about five or ten minutes
which was completely insane at the time
I worked for a big telecoms provider
where it took them nine months to
provision a virtual machine internally
and you know now comes Amazon you can
get the VM for about ten minutes and and
that there was amazing but kind of it
did not fundamentally change how we are
thinking about deployments because you
got five virtual machines you’re paying
for five virtual machines so you’re
gonna bundle everything into those five
virtual machines and we teach people
good software design like decoupling
isolation and and all those brilliant
software design practices and then
because you have five virtual machines
then you put your payment servers and
your log and your monitoring system all
on the five virtual machines where they
start interacting with each other and we
had this problem we’re kind of deploying
lots of different payment services to
reserve VMs one of the VMS did not clean
up the temp space correctly now one of
the payment services did not clean up
after itself and all the time it filled
up the temp space Linux starts really
really misbehaving when you fill up the
temp space so that kind of VM started
going crazy all the payment services
that machine went down although they
were designed to be decoupled and
isolated and everything and that when
that machine went down there was a
cascade all you know the remaining four
machines started filling up that time
space very very quickly and everything
kind of imploded
so um although you know he teach people
about good design as decoupling we end
up deploying stuff to save cash because
we need to reserve capacity and the next
generation of cloud deployments Google
App Engine Heroku and things like that
they moved a lot of the responsibilities
over to the cloud providers like
provisioning like monitoring and things
like that but the whole thing remained
you were paying for dinos on Heroku for
example we our app was deployed on
Heroku we
we have about 20 or 30 different
exporters in two different formats and
if I’m going to run a primary and a
failover for each of these exporters on
separate isolated VMs plus for some
exporters I need a lot more capacity and
for others for some one in five or six
VMs that means that for 30 exporters I
need a hundred VMs to run on if I really
want to make it reliable and isolated
but some of those exporters like
markdown I wrote for myself nobody else
uses that some of those exporters like
PDF I use it all the time so you know
I’ll put the market on exporter on the
same block of machines as the PDF I’m
not going to create a separate block of
machines to save money and you know of
course we made a stupid mistake and some
of those things started interfering with
each other because they were on the same
machine now technically we have a
solution for this and we had a solution
for this it’s containers it’s isolation
it’s things like that but as a you know
average company it’s very very difficult
to dedicate the resources to manage
everything there so what lambda does
Islam that kind of provides that as a
service so we start moving away from
reserved capacity to utilize capacity it
doesn’t matter how many VMs you need
Amazon is going to scale this and
recharge you for how much you’ve used
and that’s kind of the request based
pricing that’s really really interesting
so what that means is that there’s no
more financial incentive for me to put
the markdown in the PDF exporter on the
same VM even worse because they have
different memory needs markdown doesn’t
need any memory PDF needs a lot because
it’s using go script and go script is a
memory hog putting those two things on
the same VM I will end up paying for a
markdown for a high memory watermark
that’s a lot more than I need if I
separate them so the financial
incentives here are actually for people
to unbundle the apps in all these small
isolated modules it’s really really
interesting because said historically it
was completely different so one of the
things that I think kind of people
should think about more and it’s not
microservices it’s not kind of functions
you can you can buzz for this any way
you want but I think what we started
thinking about is really
decoupling different tasks that have
different memory constraints different
CPU constraints different deployment
needs and thinking about well you know
if it doesn’t matter how many VMs this
runs on I don’t care about it being
separately deployed and that’s very very
liberating and it’s completely opposite
of what we’ve been taught to do a lot of
kind of many practices like Bluegreen
deployments many practices like kind of
concentrated monitoring and things like
that they did no longer that problematic
and that opens up some really
interesting possibilities for example
moving from Heroku to lambda we’ve kind
of deleted a bunch of code that was –
dealing with kind of you know figuring
out what’s going on on a particular VM
figuring out how these things interact
with do they interact with each other so
a lot of this infrastructure code is
gone so the next thing that kind of I
think historically happened and no
longer applies is generally good
architecture because of the whole idea
of you know reserved stuff optimized for
failovers and what that meant was really
complex state management you wanted to
have a warm failover machine or a couple
of warm fill of machines that can take
the load in you know very quickly if
something fails so data needs to be
replicated cache needs to be replicated
data needs to be synchronized across
replicated machines then this kind of
cache invalidation policies people need
to care about this there’s a lot of
really complex code to do to make this
work really really well but from a
perspective of paying per request kind
of failover machines are not doing
anything so kind of unless the receiving
request you’re not paying for them and
then the big question becomes do you
actually need to plan for failover like
that and another really interesting
constraint that I think Canova’s is
changing with the whole lambda
architecture is kind of the time to
recover that was historically a big
driver for a lot of architectural
practices is no longer that important
and that opens up some really
interesting possibilities what I think
is becoming a lot more important is the
time to start
so when we talk about the time to start
I’ll just show you something very
quickly let’s see if the gods of the
Internet help us here so ok so if I do
[Music]
and I do constable let’s be nice
so this is kind of a a web api that on
any request to slash hello will reply
with hi there it’s not that’s kind of
complicated
and if I’ve not made a stupid mistake we
should be able to do something like so
I’m using an open-source tool for
deployments here that actually we’ve
open sourced while building it for
mindmap the first one that we created
was something like 30 lines of
JavaScript code and about 250 lines of
shell scripts to deploy and then I
realized that kind of the risk is no
longer in the code the risk is in the
deployment so we wanted to have properly
unit tested deployment and configuration
and and we kind of just packaged up all
the scripts so this thing is now on on a
URL on Amazon and if I grab this URL so
I’ve created a slash hello so if I do
curl hello I get to hi there
so now this is a kind of thing that’s
running on one VM but what’s really
interesting about this is that if all of
you start attacking this thing now it
will perfectly handle the load if
nobody’s using it they don’t have to pay
for anything if we have a massive
massive spike now but then kind of you
know the spike goes out that’s perfectly
fine as well it will scale up and scale
down plus with this I get a bunch of
things automatically like monitoring
logging security provisioning all the
stuff that kind of people want to deal
with the operations so I’ve created this
thing called go to Copenhagen so if I go
to my lambdas now
come on internet okay so we have a knob
that’s not the right account
goodgoodgood
so let’s see Amazon four one three six
so for now you’ve seen my code come on
come on so if I go to lambda now I have
no idea what they’ve changed here so
this is the okay III you can see how
often I use this anomaly use the command
line D so what I want you to say is kind
of with with this you get a bunch of
operations things for free immediately
and why is this suffering bustle and
things that doesn’t matter
let’s functions functions functions
functions page to go to Copenhagen okay
so here it is and then I have kind of
lots of stuff around monitoring I have
my kind of errors throttles I have logs
in cloud watch I have pretty much
everything I need from an Operations
perspective already provided to me so
there’s this whole buzzword kind of that
lambda is moving from DevOps to no ops
and Simon worldly who’s a researcher in
the UK said kind of your company’s not
done DevOps yet just don’t bother skip a
whole generation of things and move to
something like this and it’s easy
fair enough it’s not completely killing
DevOps but now I said you know you have
all the operations stuff already I have
a log I have kind of when it started
when it ended how much memory it used
and this is amazingly good for
optimizing stuff if you look at kind of
the typical architecture and and and
stuff like that we normally had figuring
out how much money a particular function
cost you to run was almost impossible
people are optimizing based on a gut
feel they were optimizing whole
applications now I know exactly for this
particular task how much money I’m
spending so I can decide do I need to
kind of invest in optimizing that or not
so that’s really really really
interesting and kind of in terms of time
to start here’s some here’s some
empirical numbers that I got these are
not confirmed by the AWS a Tablas
doesn’t publish any numbers about this
but these are kind of my numbers that we
got generally so for a completely new
instance of a deployed up so if you
start hitting this stuff now and this
instance that it’s running their content
load with the JavaScript
this lady takes less than one second to
get the new instance if you want to do a
new deployment a new version like I’ve
just done it takes about three or four
seconds to kind of set everything up so
that’s the infrastructure stuff now
what’s left is how long our app takes to
connect to the database load up the
cache load up the data and things that’s
I think kind of unlike the previous
architectures where we were optimizing
for quick failover what we need to start
thinking about is optimizing for quick
start lazy loading everything kind of
making sure that we don’t add a lot of
this stuff because if we can then does
this stop just kind of starts and dies
on its own now there’s a whole buzzword
here where people are talking about how
we need to optimize we need to write
this stuff in a stateless way because
there’s a big confusion of buzzwords and
some people talk about services function
as a service than in a functional
programming and functional programming
is stateless and this you know people
throw buzzwords around lambdas are not
stateless don’t have to anybody tell you
that lambdas are a stateful container
just having no idea how many containers
you’re running once the container starts
you have no control over when is it
going to stop and whether the next
request from the same user is going to
hit the same container or a completely
different container so rather than
developing for stateless what I think we
should be thinking about this developing
kind of for sure nothing designing for
sure nothing when the VM load is
perfectly fine to start caching things
in that VM that are not user specific
that’s how we save a lot of time on
processing requests that’s how we save a
lot of cash because you know that there
will be a VM running it’s just that you
mustn’t ever catch anything that is
specific to that particular request the
state of a user session is not there the
state of a user session needs to be
somewhere else and we’ll talk about
quite an interesting option where to
push the user state later but kind of
it’s it’s not stateless people that have
never kind of deployed any of this talk
about lambdas being stateless they’re
not and and I think you lose a lot of
money by treating them like stateless so
the next thing that I think kind of
historically happened is it was really
really difficult to replicate production
it was you know people had a production
system with expensive storage or even if
it’s on the cloud with good VMs and then
that cost a lot so you wouldn’t want to
pay the same amount for your staging you
didn’t want to pay the same amount for
the testing so typically kind of
performance tests unless you are Lmax
would run on a kind of a smaller copy of
data on a smaller copy of the production
and things like that they were not
really relevant they were not reliable
and generally that made it really really
difficult to claim anything about
production usage and production
performance based on the testing system
now kind of I have a couple of friends
from Australia and and they they claim
that I don’t know if this is true or not
the time that kind of Facebook is
continuously broken in Australia because
Facebook is really good with kind of
running business experiments on all
their users and kind of Australia’s the
market that’s big enough to be
statistically significant it’s English
so they don’t have to waste a lot of
time preparing software for that but
Facebook doesn’t really care too much
about Australians they’re laid-back
people anyway so they can do experiments
in Australia and then when they figure
what works what doesn’t work they can
translate that to the US now when you
think about something like that that’s
incredibly useful but up until five
years ago or up until three years ago
that was available to you if your Google
or Facebook running experiments like
that is you feel expensive you need a
complete copy of production for
something like that and you need the
relevant copy of production you need to
synchronize the data between these
things what one of them kind of best
examples of how important that is is a
story called 40 shades of blue they’re
absolutely loved because it’s one of
those rare things where you can read
both sides of the story online forty
shades of blue led to kind of that the
head of design at Google wanted to
change the color of the links on the
homepage for ads and the developers
challenged him a bit
and he wanted to change it to a
particular blue collar overnight they
ran 40 different colors of blue and
measured how much people are clicking on
that and then as a result of that
depending whose side you read this guy
either quitter was fired because the
difference was something like 250
million dollars between the current
color and his color if you expand it to
a whole whole year and 100% of the users
so kind of things like that are
incredibly important to test but for
most people out there it was almost
impossible to do it because production
copy cost so much now when we started
doing lambda I had this brilliant best
idea in the world and I wanted to
integrate wiki data graph of knowledge
with our app so that when you press a
question mark it automatically opens up
related terms it was going to be amazing
it was going to be brilliant it’s going
to be fantastic and and my business
partner said no that’s a shitty idea
nobody’s going to use that let’s not
waste time doing that so you know I kind
of generally don’t like to ask for
permission like to ask for forgiveness
so I decided I’m going to do that anyway
and he said I know you’re going to do it
anyway do not touch the
production code because if you put this
in we’re going to have people you know
screaming that we can’t support the
right performance we’re gonna be bottom
like here we’re gonna and I said okay
and then I realized kind of with lambdas
you don’t pay for stuff if you people
don’t use them so I kind of spent two
days knocked up a quick lambda version
that was extended with this deployed a
completely separate copy of our
production and sent 20% of our users
there and you know lo and behold I
proved that it was a idea nobody
wants to use it so we deleted that code
with our disturbing uh production so but
this is this is some you know something
that traditionally we’d get into a fight
who’s right who’s wrong guy would do it
anyway and then that code would stay in
the production and kind of you know cost
us more to maintain it was amazing
because we could run a quick experiment
so because we’re paying for requests
we’re not paying for reserve capacity it
was exactly the same amount of money to
send 80% of users to one version 20% to
another or to send everybody to the same
version or to have a version for
Copenhagen or to have a version for
go-to or to have a version for anything
you want and actually kind of what this
starts getting us to think about is
moving away from thinking about
production to thinking about multiple
versions and multi versioning in lambda
is incredibly well done I you know for
my scenes like everybody else have done
an Orion framework a logging framework
and a data multi version in framework
everybody’s done that when they young
otherwise you don’t you know you don’t
get to call yourself a developer and
then you end up kind of suffering
through maintaining that for 10
years but that’s okay so a data multi
versioning request multi versioning
service multi versioning is really
really difficult to do well at scale and
Amazon probably have done it for their
own needs and they’ve just exposed a T
lambda so every lambda function gets a
numerical version every time you deploy
it and you can direct the calls for a
particular function either to the last
non-deployed version or to a particular
numerical deployment or you can assign
aliases to a numerical deployment like
production testing staging so when we
started doing this we started really
kind of doing oh I have a production
version a testing version and staging
version and then we realized why I’ll
just save a version to test this feature
oh I’ll have a version to kind of you
know when we’re doing something
experimental and 5% of our users really
really need this feature but it’s not
ready for everybody else we can deploy a
version for them and give it to them and
test it on them early and it’s multi
version is incredibly well done so I
think this is a really interesting thing
to start thinking about kind of how do
we bundle our tasks because it has a
massive impact on how we design how we
deploy if we are going to design for a
multi version universe where you know at
the same time several versions of a
wrapper running and different things are
communicating with even I think the
whole domain driven design concept of
aggregates becomes incredibly more
important because we want to make sure
that kind of you know all the data for a
particular version of this object
travels together and because we are kind
of don’t want to clog the communication
things that we want to make sure
aggregates are actually relatively
minimal so that we can push them around
and we can load them quickly but the
movie to land has gotten us to thinking
a lot more about what are our actual
aggregates where the aggregate
boundaries what needs to be in a
particular version consistent with
itself what can just kind of be
different versions and and be okay so I
think that’s a completely interesting
kind of phenomenon that are not really
seen in my code before that and because
we’ve designed for multi versioning now
up front as we’re migrating we can do
lots of crazy things and the stuff that
was the traditional available to you
know companies that make billions and
billions of dollars we can do now and
we’re a two-person team and that that’s
I think amazing as a kind of for what we
can do from the platform so the next
really interesting thing driven by the
pricing model of AWS lambda is that
different services charge for different
things so you can save quite a lot of
money by playing arbitrage available yes
against AWS that’s amazing
and remember kind of lambda is as a
processing service charges for the
number of requests in time so if you can
delegate work from lambda to stuff that
does not charge for the number of
requests in time you can save a lot of
cash for example kind of API gateway
charges for the bytes being transferred
the number of requests at the same time
kind of s3 that’s the storage system
just charges for transfer it doesn’t
care about the number of requests so one
example of that is that we own Heroku
and before we started really thinking
about this we would let people upload
files for export to a server where the
server would communicate with the
storage and then the server would save
stuff through storage it with rondo
converter it would kind of upload it
back from the storage to the user what
that means is that during that whole
time the server is busy we’re paying for
the server now if you’re uploading a
hundred megabyte file the transfer from
you to
Amazon and the transfer for Amazon to
you is actually the most amount of time
the conversion time is relatively quick
so we’re paying kind of for this
operation for a long time we’re because
there’s three only pays s3 only charges
for transfer if we can get people to
upload directly to s3 and download
directly from s3 we have reduced our
server costs by a significant amount so
some other service that are interesting
to consider like kognito the Amazon
authentication and session service only
charges for the number of users it
doesn’t charge for the number of
requests those users make it doesn’t
charge for the capacity of the sessions
it only charges for the number of users
so moving session state into kognito
it’s a really really interesting way of
kind of playing arbitrage with Amazon
versus Amazon so kind of as a kind of an
example I’ll show you later there’s also
this thing called the IOT gateway that
we abuse massively IOT gateway is
designed to get low-power devices to
talk to each other but we’ve built kind
of real-time collaboration directly
through we’re building real-time
collaboration directly through IOT
gateway which kind of because it only
charges for the number of messages it
doesn’t charge for processing time it
doesn’t charge for data transfer again
you can arbitrage things nicely so kind
of we started moving a lot more for
thinking about applications and and kind
of managed apps like Heroku like Google
App Engine to really the glue between
different platform services for me
lambda is mostly about what’s missing
from amazon’s platform and how do i kind
of glue those things things together and
what’s the kind of what’s the real
business value of my code because it’s
unlikely that developing a queue
interface system is where i can provide
the most value i can provide value
developing kind of the small bits of
processing those queue messages and
that’s really really interesting so kind
of just as an example that this is kind
of how to get people to use s3 directly
from a browser so on a server we have
something like this the s3 is the Amazon
API SDK so you get a request for upload
file you populated with you know where
it needs to go you limit the file size
you
kind of provides security stuff and then
you get a signature and then returned
that to the browser that takes you know
20 milliseconds then the browser spends
ten minutes uploading the big file to s3
where you’re just paying for transfer
then kind of this thing goes very
quickly you go back and people download
it directly from s3 from a signed URL so
amazon being amazon of course there’s
this five different ways of authorizing
requests
there’s signed urls there’s the cig v4
that they do for kind of approaches this
is giving people access to kognito so
this lots and lots of ways how you can
get people to access directly one of
these front-end services without going
through a traditional server so kind of
in to really benefit from that
financially I think what we started
thinking about a lot more is give the
platform the roles that were
traditionally associate with the server
process things like being the Gateway
keeper things like being the
orchestrator things like keeping kind of
sensation storage if you push that away
from the lambda that’s kind of the
processor to the other parts of the
platform you can save a lot of cash now
here’s why I think this is so insane in
September our app had something like
400,000 active users it’s it’s not you
know Google but it’s not Mickey Mouse as
well and just so that I’m not cheating
this is V live Amazon page so if I go to
my billing dashboard and I look at my
costs for where is it the September bill
bill details September so for four
hundred thousand active users in
September we have paid 53 cents
for lambda now beat that with your
hosting costs and we you know they of
course there’s there’s some other
services like we paid four dollars for
data transfer and then we paid something
for API gateway and something for dynamo
and things like that but all in all that
the bill was a hundred bucks for a you
know four hundred thousand active users
that are kind of collaborating in real
time now this is insane
completely insane if you look at kind of
stuff equivalent stuff I was doing ten
years ago that this is completely
completely insane and you know add up
all the multi versioning and everything
else they provide for almost not free
but included in the price that’s that’s
why I think this is gorgeous and and and
you know fantastic in so many ways so
kind of the another thing that started
happening here as we started thinking
about more and more of arbitrage in
different services against each other we
realized that kind of what engrained in
my head for the last 30 years is do not
trust users to talk to Becky and
resources like users are not allowed to
talk to storage directly users are never
ever ever allowed to connect your
database directly they have to go
through a gatekeeper they have to go
through a server because on the server
we discard invalid requests we validate
stuff we you know everything before the
server we don’t trust everything after
the server we trust that’s how we you
know did the whole web logic thing
evolved we have application servers we
have storage we have clients and I think
kind of especially if you start to use
the platform on Amazon none of these
things are actually physically back-end
resources anymore s3 is available over
HTTP dynamo that’s the databases
available over HTTP the fact that we are
not letting users talk to you directly
doesn’t mean it’s not available if
somebody kind of guesses the name
it’s there Amazon is making it available
and because of that kind of Amazon is
actually implementing really really good
request level authorization policies
each single request going from lambda to
a database is authorized because if they
have no idea if your lambda is talking
to the database if somebody else Islam
days talking to derivative is and it’s
not your database anywhere it’s their
database and it’s kind of things like
this are really really interesting from
a perspective of thinking about well you
know if it’s authorized per request
what’s the damage in actually kind of
authorizing clients to go there so
Amazon gives you three or four different
mechanisms for authorizing these
requests including say including saying
that this user is only allowed to write
to this particular key in the database
and only allowed to read from these keys
in the database hierarchically or this
user is only allowed to read from this
part of the queue and post to these
parts of the queue and the same
authorization policies exactly the same
apply as if you talk to from the client
there then if you go through the server
and then I realized well you know we’re
just introducing latency by putting a
server in the middle we’re just paying
more and we started kind of using this
like mad my brain still does not allow
me to or let users connect to the
database because I’ve been a server-side
developer for 20 years and it’s just
wrong
but we’re letting people talk to the
storage directly well I think people
talk to the queues and and kind of
things like so you know maybe two years
from now I’ll I’ll come into the talk
and say no no everybody’s you know
talking to the database and all our data
is stolen and it’s horrible or you know
but generally if you think about it’s
not your database it’s Amazon’s database
and it’s available using HTTP so it’s
not back-end it’s content or its
middlewares of some kind so kind of we
started moving away from kind of
three-tier models to smart clients not
not dumb terminal smart terminals where
things connect directly so here’s a URL
and this is a
a prototype we’ve developed for a chat
app that works on all browsers and works
on all mobile devices and things like
that and kind of it
the source code is on github you can
look it up afterwards so kind of connect
to this from your mobile phone or
something like that and then you can
kind of log in as a guest what that
means is that we are now getting a
authorization ID from Cognito I can make
this login with the username and
password log in through Google login
through Facebook log in through many
many different ways do two-factor
authentication for a conference kind of
show it’s you know open and then once I
have this I can actually use their API
directly from a browser to talk to any
resource I want that I’m allowed to talk
to so in this case we are talking
directly to if you cannot get into the
server listing just pump it up we are
talking to the IOT gateway the IOT
gateway is designed to kind of exchange
messages between low-power devices but
it’s actually allowing you to have a
WebSocket interface as well how cool is
that so you can use WebSockets on demand
managed that paid $5 per million
messages so peanuts and they’re
completely managed so you know we can
get a million people connecting to this
now or we can get five people connecting
to this and it’s it’s all done
operationally the source code is 30
lines of code so it is just completely
insane what we can do now with this
stuff and how much it costs and I you
know ten years ago I remember kind of
evaluating lots of different push
mechanisms where they were doing
degrading of a flash they were doing
long pole they were doing all these
amazing things and I think the best
thing we’ll choose they were asking for
something like a hundred thousand quid a
month I can get this now for five
dollars on a million messages if
nobody’s using it
I don’t pay if people are using it and
probably making money of it so you know
perfectly fine to pay but there’s no
upfront cost there’s no monthly
maintenance cost this is insane
it’s completely since I think kind of
this thing changes how we approach what
we reserve what we what we kind of do
and I said you just format see a nice ah
good good good good good
so um so yeah I said that is you know
from here you can just go to go to the
Lincoln and you can get the the source
code for this so kind of in that respect
what I want to say is good engineering
who the architecture is driven by
constraints cost is one of the key
constraints we have to deal with and
deploying on lambda fundamentally
changes the cost structure so lots of
stuff that you know have evolved as best
practices over the last 20 years kind of
no longer apply and our challenge as a
community over the next five 10 years is
going to be to figure out what are you
know just the shackles of the old world
that we’re running with I mean in it you
know I I can talk what’s wrong about
lambda 4 you know days on end and this
is not a silver bullet it doesn’t solve
all the problems but I think it’s a
really really interesting perspective if
you can you know run a four hundred
thousand uses and pay fifty three cents
for the whole thing it’s just insane and
I think kind of the financial incentives
of that are going to get pretty much
everything that can run in lambda to run
in lambda over the next five years and
that’s why you know although most people
here I assume are not really deploying
things in production for lambda yet this
will comes to start investigating that
and in particular running cheap
experiments is amazing if you need to
run a kind of cheap stupid experiment
and you don’t know if it’s going to work
out or not this is brilliant for that
and then even if you do and kind of
on-premise deployment later then you
know what to integrate and and and what
to throw away so kind of I think one of
the key things that was you know a mind
shift for us is
is to start letting clients connect to
back-end resources because there are no
back-end resources anymore and what that
means is that all of the sudden your app
is not really running just on fifty or
five or five hundred virtual machines
it’s running on four hundred thousand
client processes as well because you can
push if you let people talk to Becky and
resources you can push orchestration you
can push state to the client and all the
stuff that kind of is difficult to
manage in the distributed architectures
push it to the clients machine where the
client is a single client having a
single state which simplifies things
significantly so instead of kind of that
that’s why we pay so little for lambda
our app does not run on the VMS what
runs on the VMS is a glue between
different back-end services our app
actually runs on four hundred thousand
client processes that we do not have to
pay for and that’s kind of the really
really interesting mind shift here so
kind of as two URLs for for more info
this first one is my blog where I post a
lot about this stuff because I’m
incredibly excited about it the second
one is the open source tool for
deployment that I’ve shown you that kind
of simplify stuff if you do in
JavaScript that’s pretty much it thank
you very much I hope I kind of tickled
your imagination at least of it
and if anybody’s posted any questions I
guess we can talk about that now do we
have any questions where’s the I can
read it loud okay lovely how do you feel
about locking kind of that that’s a
really interesting question and
unlocking is is problematic on several
levels lots of people talk about locking
in terms of locking with libraries
locking with code if you use Oracle then
you know you use Oracle’s libraries with
lambda because the platform calls you
not the other way around you’re actually
not locked into the lambda API at all
there aren’t it’s it’s moving away from
there would be trivial the big problem
is you locked into the platform if you
really want to get the benefits of
lambda then you’re letting clients talk
to storage directly I think lines talk
to the database correctly and that’s
where the locking happens now for us
we’ve kind of we we’ve decide
commercially that going really for
Amazon and and using Amazon for
everything is a good commercial decision
it gives us the risk Obama’s are not
working but in my experience kind of
they’re pretty solid and they’re much
much better doing the OP stand I can so
I I know that there are some tools like
the serverless framework that allow you
to deploy to multiple clouds and do kind
of this hybrid thing but I guess the big
problem then like you know doing a
database independent deployment is you
get to use the the least common
denominator of the whole thing and
you’re never really using the platform
what it is so kind of that’s a
commercial decision that I guess
everybody needs to make on their own but
there’s definitely things like you know
ports and adapters or hexagonal
architecture and things that you can
design stuff so if you do actually
decide to move on to different cloud
provider you know you will I I don’t
I’ve never worked with a company where
the whole investment in being able to
move from the primary database paid off
because they never moved away from the
primary database so it’s a commercial
decision there’s only two of us building
this thing so I’d rather spend stuff
delivering successful features than
building kind of an abstract software
system but you know if you have five
kind of developers why not keep them
busy so so the this disorder locked your
kind of can I get Apple pay on your
phone as well when you unlock it okay so
we have how do you keep the state we
decision so we tend to push a lot to the
clients directly and and we tend to keep
the state in either incognito or in the
users browsers kognito does automatic
synchronization across devices we don’t
tend to use that a lot because our our
state is typically kind of the document
you’re working on and we don’t need to
keep a lot of that but kognito is a
pretty good way of kind of synchronizing
stuff across devices using dynamo using
something like that to keep kind of the
state per user is also pretty good
because you can configure dynamo to
allow users to write only to a
particular key so you ko or or a sub key
so you can write only to your own state
and read only from your own state for
example that
that would be a possibility mm-hmm if
I’m if I’m P dose so uh III don’t know
that’s never happened to us kind of
lambda n API gateway allow you to
throttle things you can just configure
throttling so you can say that you know
I want to run up to a thousand
concurrent functions of this or up to I
think the limit by default is a thousand
per function but then you can increase
it or with the API gate where you can do
throttling based on an API key an
authorization key or kind of generally
on an API endpoint so you can configure
throught link so you can with the
monitoring they have and things like
that spot if you are kind of being DDoS
again my assumption is that Amazon will
protect against DDoS much much better
than I can code that I don’t know kind
of about anybody in the audience whether
you feel you can build a better DDoS
defense system than Amazon but certainly
you can configure it to be to get an
early warning and then figure out what
to do from there so you don’t have to
spend millions and millions and millions
if you get the dust at the moment the
biggest disadvantage is that lambda
functions are limited to about five
minutes run
so anything that takes longer than five
minutes you need to split into multiple
executions which means you can’t keep an
open socket we were trying to develop
something that talks to the Twitter API
s and Twitter doesn’t have a push API or
you know if you’re on mortal you cannot
get the push API you need to connect the
socket and get them to kind of stream
stuff to you and with lambda that’s not
I mean it’s possible but you need to
kind of load it every five minutes and
then disconnect in the render state so
for something like that I would still
use ECS another kind of reasonable
disadvantage for many people is that
there’s this virtually no SLA the lambda
or not virtual is actually no isolation
and that they don’t offer any date and
they don’t of
Neela’s yet so our experience is that
kind of the u.s. East one the region
that is overloaded with everything
because that’s the first one that
started occasionally gets kind of
hiccups where we get a bit of delay but
we’ve never really had a full outage
since February 2016 when we started kind
of moving to this that doesn’t it’s not
going to happen and I think you know as
Lomb that becomes more and more
important I assume they will start
providing a slice for it at some point
so that’s an interesting limitation B I
guess those would be those would be the
two key key limitations for people so I
think we ran out of time I don’t know if
we have kind of I’ll be around you know
you can pick me up in in in in the
corridor and then we’ll talk about this
thing more I need to get other people to
set up thank you very much
[Applause]
Please follow and like us:
Be First to Comment