[Applause]
this is my cellphone and my cellphone is
very important to me
I read my emails on it I talked my
friends on Facebook on it I checked the
weather before I go outside and I look
at funny cat pictures my phone is very
important to me because it touches so
many different aspects of my life my
phone is also very important to a lot of
companies because by looking at my
internet browser history they can learn
about who I am and it’s not just my
phone it’s my laptop
it’s my iPad it’s anything I use to
connect to the Internet companies are
interested in what I’m doing and it’s
not just me it’s every single one of you
every time you connect the Internet
companies want to know who are you what
are you interested in welcome to the
world of big data the world is changing
and you need to understand how and why
because you’re part of it we currently
live in the information age and there is
one rule here knowledge is power
this is a common saying you’ve probably
heard it before but I need you to be
convinced beyond a shadow of a doubt
that this is truth so I’m going to
convince you entire books have been
written about this idea one of the most
common one of the most popular is the
art of war it’s this fundamental concept
that you can overcome a difference in
power by leveraging a difference in
knowledge so okay it’s popular that
doesn’t mean that’s right
so let’s look at a case study you’ve
probably heard of America
let’s go back to the Revolutionary War
you have a brand new militia these
scarcely trained people and they are
going to fight the British Empire and
they’re highly trained veteran redcoats
how can you possibly win you start using
guerrilla tactics you start out
maneuvering them outsmarting them and
suddenly you have a new nation on your
hands here’s a new example this one’s my
favorite Batman this is a super hero
with no superpowers how does that work
he’s just a regular guy but by
leveraging knowledge by using technology
he can accomplish superhuman feats
knowledge is power and the reason that
I’m making this point the reason I’m
trying to drive this home over and over
is because we live in a time when there
is more information than ever before
never has knowledge been so readily
available in 2010 there were over 50
billion photographs on Facebook that
number has skyrocketed three years later
in 2013 there were over 350 billion
that’s huge that is mind-boggling Lee
large in fact there is no other word I
can use to describe it other than big
big is in fact the perfect word this is
not a coincidence how many people have
heard the term big data before it is a
popular term and it is going to become
more popular in the very near future
what is Big Data well some of the
problems that we are facing today are so
large that they cannot be solved using
traditional analysis techniques why why
not they’ve worked before what’s
different there are two big differences
that define a big data problem from a
regular problem
the first is volume how much data is
there this is Google and last year they
processed over 1 trillion
searches 1 trillion that’s a lot of
information you can clearly learn
something from that but if you’re going
to do that you need to be able to look
at all that information you need to
store it somewhere computers have gotten
better and the first computer had so
little memory but if you took a
photograph with your cell phone today it
would be too large there’s too much data
there to be stored in the first computer
so we’ve clearly made some progress but
that’s a trillion searches up there you
can’t store that no laptop or in 10
laptops you need to change how you
organize your data
what’s more traditional algorithms don’t
work anymore will work in the sense that
they’ll get you an answer but it’s going
to take you 200 years so we need not
just new hardware I need new ways to
store data we need new algorithms to
process it faster that is a volume big
data problem there’s another kind of big
data problem that’s velocity this is a
picture of the Large Hadron Collider it
is a 17 mile long track that straddles
the border between France and
Switzerland and it accelerates subatomic
particles to near the speed of light and
then smashes them together that’s really
cool however when you do that you
produce a lot of information really
really quickly it is very much like if
you are a student in class you’re trying
to take notes on what the teacher is
saying but the teacher has a lot of
really good information and they seem to
talk really really fast and they don’t
even need to breathe and they just keep
going and going and going and going and
you can’t get all of it so what do you
do you filter out the most important
stuff you listen and this is exactly
what they do at CERN the organization
that runs the Large Hadron Collider
in real-time they filter out what’s most
important and what’s the least important
because they can’t capture everything
and so they need to decide what to do
with it all
that’s a velocity big data problem so ok
big data it’s kind of a big deal why is
it happening why now
for one we have made some advances in
the ability to store information and the
algorithms we use but we also have the
Internet that whole part where there’s
more information than ever before that
Facebook that Google we are creating
information and now the Large Hadron
Collider probably does not impact you
everyday so let’s pick an example of
something that might pop up more in your
daily life like advertisements that was
a pop up ads by the way thank you thank
you
so why advertising for one thing it’s
something we’ve all encountered
advertising is pervasive for another
thing it’s something that companies are
very very interested in it’s very easy
for them to see the value here if you
have better ads more people buy your
stuff you make more money
boom boom boom we like it so Wow is how
is all of this new information how is
big data impacting advertising and how
does it pertain to you
the first kind of advertisements were
completely untargeted let’s say you put
an advertisement on a billboard you
don’t know anything about who’s driving
by you can’t say anything to reach out
to someone driving by and so you just
have to appeal to as many people as
possible
it’s untargeted but let’s say that
you’re buying advertising space on TV
suddenly you know a little something
about your audience
Gillette should not be advertising
shaving cream on Cartoon Network and
Hasbro should not be trying to sell
play-doh during the Superbowl the people
the audience just isn’t interested you
know something about them and so you
control your advertisements
so we went from untargeted to targeted a
general demographic the next logical
step is targeting individual people this
is big data this is the future it’s
actually the present but it’s going to
become more and more important so what
is an advertisement that targets you on
the Internet
it’s called a banner ad and even if you
haven’t heard the term before I bet
you’re going to recognize it this is a
weather website you can see it today is
a forecast you can see some news
articles and that red box is an
advertisement that’s State Farm auto
insurance how did that get there this is
another weather website and I’ve got the
weekly forecast and sunglasses
why are those there how did they get
there this is a process known as
real-time bidding and there are billions
of these transactions every day and
every single one of them takes
milliseconds let’s walk through the
process because this happens every time
you visit a webpage with an
advertisement this is happening to you
this is someone using the Internet let’s
call them Bob Bob has just bought a
convertible and he wants to know what
today’s weather forecast is so that he
knows whether he can ride with the top
down or not so Bob goes onto the
internet and he visits weather.com which
has room for an advertisement on it in
the milliseconds before the page loads
where the calm says hey Bob here is
visiting I need someone to put an
advertisement on this page I want to
make some money someone give me an
advertisement
so whether calm reaches out to Google’s
double-click Ad Exchange
that’s what ad X stands for it’s ad
exchange and there are companies other
than Google doing this um but what Addax
does is it coordinates advertisers with
websites that need ad for
Huysmans so whether comm reaches out and
says addicts hey bob is visiting a
weather website and I really need an
advertisement about that large so
addicts reaches out to companies who are
looking to advertise to Bob this is the
first time that companies can really
leverage their knowledge about you as
individuals because addicts can’t reach
out to every advertiser on the planet
every time someone visits a webpage
there are too many so what it does is it
reaches out and it says all right you
you and you you all have Bob in your
target audience so I’m gonna ask you how
much are you willing to pay to advertise
to Bob so for companies it’s very
important to know individual people so
that they can have a list now some some
companies just target demographics I
only want men between 25 and 35 I only
want women under 25 you still need to
know something about your target
audience you still need some knowledge
about the individual visiting the
webpage so okay annex reaches out and
there are three advertiser isn’t for it
interested in talking with Bob let’s
call the green one
State Farm the State Farm happens to
know that Bob just bought a convertible
and so State Farm is willing to pay more
to advertise to Bob than their
competitors why because they know
something about Bob they know he has a
new convertible he probably needs auto
insurance he’s more likely than the
average person
so three advertisers are contacted which
one gets to place the ad there is a
bidding war there is a bidding war and
all three companies pay a certain offer
to pay a certain amount how much are
they willing to pay to put an
advertisement in front of Bob let’s say
State Farm pays $1 the second advertiser
offers $0.50 then the third advertiser
offers 25 cents this is a second price
auction so State Farm won by bidding $1
but they’re going to pay the second
price which is 50 Cent’s
this is the second place that companies
get to leverage all their knowledge of
you as an individual because how much
they bid determines whether they win the
auction but also whether they bankrupt
themselves if I’m paying more to put an
advertisement in front of you that I’m
likely to get back I’m losing money I’m
going out of business so this is why
companies care so much about what you’re
interested they care so much about
knowing who you are because they are
making and losing money based on how
well they know you so okay
State Farm wins the bid they cough up
fifty cents and they say hey this is the
advertisement I want Bob to see here you
go let’s put it on whether calm and this
took milliseconds Bob gets to see what
today’s weather is and he gets to see
the State Farm is offering him auto
insurance now the title of my talk is
called how big data is making the
internet smaller I want to clarify the
internet is not actually shrinking
however this is what I mean when I say
it is getting smaller Bob bought a
convertible and now he’s seeing auto
insurance popping up imagine that the
Internet is like a city there are an
incredible variety of people and an
infinite number of stores but when Bob
visits the Internet it’s gonna feel like
a small town because he’s gonna see the
same familiar faces over and over and
he’s gonna visit the same shops again
and again companies are heavily
interested in giving you the same
content over and over because they know
that’s what you’re interested in I know
that you like cars and auto insurance so
I’m gonna make sure that you see that
same stuff every time you visit a
website that’s how I make money this is
what I mean when I say the Internet is
getting smaller now this kind of can
feel like big brother is always watching
so I just want to quickly point out some
of the good things that come from this
kind of scrutiny and individual
attention because make no mistake you
could not use the internet the way you
currently do
if things were different so let’s talk
about cookies unfortunately I don’t mean
this kind of cookie I mean a small piece
of text whenever you visit a website the
website can give a cookie a small piece
of text to your web browser this is how
weather Calm knew that it was Bob
visiting Bob types in weather calm and
his web browser says hey I need to know
today’s weather information by the way I
have this cookie you gave me before now
you can remember Who I am
every time you visit a webpage and it
remembers your user name that’s a cookie
every time you click on a link and it
changes color and remembers that you’ve
been there before that’s also a cookie
every time you’re filling out shipping
information you’ve bought something
online you’re putting your name address
credit card information and then you
accidentally close the tab but when you
open it it’s all still there that’s a
cookie
in fact cookies are so ubiquitous that
they’re even part of our security
procedures how can a website know that
it is you the same person on the same
device logging in to this website where
there’s a cookie and that’s how it
guarantees that the same computer using
the same web browser is coming back so
where am I going with this is big data
good or is it bad what’s what’s the
conclusion here it’s neither good nor
bad
it’s simply big it’s a lot of
information it’s a lot of knowledge and
knowledge is power the reason I want you
all to know and to understand what’s
happening around you is because the
legislation is catching up with the
technology this is the way things work
right now but the policies that will
govern what is allowed and what is not
are still being written you need to be
able to answer the question what is my
privacy worth
how do I weigh my privacy versus
convenience versus scientific progress
knowledge is power and I hope today
you’ve learned something and you feel
empowered to answer that question
thank you
[Applause]
this is my cellphone and my cellphone is
very important to me
I read my emails on it I talked my
friends on Facebook on it I checked the
weather before I go outside and I look
at funny cat pictures my phone is very
important to me because it touches so
many different aspects of my life my
phone is also very important to a lot of
companies because by looking at my
internet browser history they can learn
about who I am and it’s not just my
phone it’s my laptop
it’s my iPad it’s anything I use to
connect to the Internet companies are
interested in what I’m doing and it’s
not just me it’s every single one of you
every time you connect the Internet
companies want to know who are you what
are you interested in welcome to the
world of big data the world is changing
and you need to understand how and why
because you’re part of it we currently
live in the information age and there is
one rule here knowledge is power
this is a common saying you’ve probably
heard it before but I need you to be
convinced beyond a shadow of a doubt
that this is truth so I’m going to
convince you entire books have been
written about this idea one of the most
common one of the most popular is the
art of war it’s this fundamental concept
that you can overcome a difference in
power by leveraging a difference in
knowledge so okay it’s popular that
doesn’t mean that’s right
so let’s look at a case study you’ve
probably heard of America
let’s go back to the Revolutionary War
you have a brand new militia these
scarcely trained people and they are
going to fight the British Empire and
they’re highly trained veteran redcoats
how can you possibly win you start using
guerrilla tactics you start out
maneuvering them outsmarting them and
suddenly you have a new nation on your
hands here’s a new example this one’s my
favorite Batman this is a super hero
with no superpowers how does that work
he’s just a regular guy but by
leveraging knowledge by using technology
he can accomplish superhuman feats
knowledge is power and the reason that
I’m making this point the reason I’m
trying to drive this home over and over
is because we live in a time when there
is more information than ever before
never has knowledge been so readily
available in 2010 there were over 50
billion photographs on Facebook that
number has skyrocketed three years later
in 2013 there were over 350 billion
that’s huge that is mind-boggling Lee
large in fact there is no other word I
can use to describe it other than big
big is in fact the perfect word this is
not a coincidence how many people have
heard the term big data before it is a
popular term and it is going to become
more popular in the very near future
what is Big Data well some of the
problems that we are facing today are so
large that they cannot be solved using
traditional analysis techniques why why
not they’ve worked before what’s
different there are two big differences
that define a big data problem from a
regular problem
the first is volume how much data is
there this is Google and last year they
processed over 1 trillion
searches 1 trillion that’s a lot of
information you can clearly learn
something from that but if you’re going
to do that you need to be able to look
at all that information you need to
store it somewhere computers have gotten
better and the first computer had so
little memory but if you took a
photograph with your cell phone today it
would be too large there’s too much data
there to be stored in the first computer
so we’ve clearly made some progress but
that’s a trillion searches up there you
can’t store that no laptop or in 10
laptops you need to change how you
organize your data
what’s more traditional algorithms don’t
work anymore will work in the sense that
they’ll get you an answer but it’s going
to take you 200 years so we need not
just new hardware I need new ways to
store data we need new algorithms to
process it faster that is a volume big
data problem there’s another kind of big
data problem that’s velocity this is a
picture of the Large Hadron Collider it
is a 17 mile long track that straddles
the border between France and
Switzerland and it accelerates subatomic
particles to near the speed of light and
then smashes them together that’s really
cool however when you do that you
produce a lot of information really
really quickly it is very much like if
you are a student in class you’re trying
to take notes on what the teacher is
saying but the teacher has a lot of
really good information and they seem to
talk really really fast and they don’t
even need to breathe and they just keep
going and going and going and going and
you can’t get all of it so what do you
do you filter out the most important
stuff you listen and this is exactly
what they do at CERN the organization
that runs the Large Hadron Collider
in real-time they filter out what’s most
important and what’s the least important
because they can’t capture everything
and so they need to decide what to do
with it all
that’s a velocity big data problem so ok
big data it’s kind of a big deal why is
it happening why now
for one we have made some advances in
the ability to store information and the
algorithms we use but we also have the
Internet that whole part where there’s
more information than ever before that
Facebook that Google we are creating
information and now the Large Hadron
Collider probably does not impact you
everyday so let’s pick an example of
something that might pop up more in your
daily life like advertisements that was
a pop up ads by the way thank you thank
you
so why advertising for one thing it’s
something we’ve all encountered
advertising is pervasive for another
thing it’s something that companies are
very very interested in it’s very easy
for them to see the value here if you
have better ads more people buy your
stuff you make more money
boom boom boom we like it so Wow is how
is all of this new information how is
big data impacting advertising and how
does it pertain to you
the first kind of advertisements were
completely untargeted let’s say you put
an advertisement on a billboard you
don’t know anything about who’s driving
by you can’t say anything to reach out
to someone driving by and so you just
have to appeal to as many people as
possible
it’s untargeted but let’s say that
you’re buying advertising space on TV
suddenly you know a little something
about your audience
Gillette should not be advertising
shaving cream on Cartoon Network and
Hasbro should not be trying to sell
play-doh during the Superbowl the people
the audience just isn’t interested you
know something about them and so you
control your advertisements
so we went from untargeted to targeted a
general demographic the next logical
step is targeting individual people this
is big data this is the future it’s
actually the present but it’s going to
become more and more important so what
is an advertisement that targets you on
the Internet
it’s called a banner ad and even if you
haven’t heard the term before I bet
you’re going to recognize it this is a
weather website you can see it today is
a forecast you can see some news
articles and that red box is an
advertisement that’s State Farm auto
insurance how did that get there this is
another weather website and I’ve got the
weekly forecast and sunglasses
why are those there how did they get
there this is a process known as
real-time bidding and there are billions
of these transactions every day and
every single one of them takes
milliseconds let’s walk through the
process because this happens every time
you visit a webpage with an
advertisement this is happening to you
this is someone using the Internet let’s
call them Bob Bob has just bought a
convertible and he wants to know what
today’s weather forecast is so that he
knows whether he can ride with the top
down or not so Bob goes onto the
internet and he visits weather.com which
has room for an advertisement on it in
the milliseconds before the page loads
where the calm says hey Bob here is
visiting I need someone to put an
advertisement on this page I want to
make some money someone give me an
advertisement
so whether calm reaches out to Google’s
double-click Ad Exchange
that’s what ad X stands for it’s ad
exchange and there are companies other
than Google doing this um but what Addax
does is it coordinates advertisers with
websites that need ad for
Huysmans so whether comm reaches out and
says addicts hey bob is visiting a
weather website and I really need an
advertisement about that large so
addicts reaches out to companies who are
looking to advertise to Bob this is the
first time that companies can really
leverage their knowledge about you as
individuals because addicts can’t reach
out to every advertiser on the planet
every time someone visits a webpage
there are too many so what it does is it
reaches out and it says all right you
you and you you all have Bob in your
target audience so I’m gonna ask you how
much are you willing to pay to advertise
to Bob so for companies it’s very
important to know individual people so
that they can have a list now some some
companies just target demographics I
only want men between 25 and 35 I only
want women under 25 you still need to
know something about your target
audience you still need some knowledge
about the individual visiting the
webpage so okay annex reaches out and
there are three advertiser isn’t for it
interested in talking with Bob let’s
call the green one
State Farm the State Farm happens to
know that Bob just bought a convertible
and so State Farm is willing to pay more
to advertise to Bob than their
competitors why because they know
something about Bob they know he has a
new convertible he probably needs auto
insurance he’s more likely than the
average person
so three advertisers are contacted which
one gets to place the ad there is a
bidding war there is a bidding war and
all three companies pay a certain offer
to pay a certain amount how much are
they willing to pay to put an
advertisement in front of Bob let’s say
State Farm pays $1 the second advertiser
offers $0.50 then the third advertiser
offers 25 cents this is a second price
auction so State Farm won by bidding $1
but they’re going to pay the second
price which is 50 Cent’s
this is the second place that companies
get to leverage all their knowledge of
you as an individual because how much
they bid determines whether they win the
auction but also whether they bankrupt
themselves if I’m paying more to put an
advertisement in front of you that I’m
likely to get back I’m losing money I’m
going out of business so this is why
companies care so much about what you’re
interested they care so much about
knowing who you are because they are
making and losing money based on how
well they know you so okay
State Farm wins the bid they cough up
fifty cents and they say hey this is the
advertisement I want Bob to see here you
go let’s put it on whether calm and this
took milliseconds Bob gets to see what
today’s weather is and he gets to see
the State Farm is offering him auto
insurance now the title of my talk is
called how big data is making the
internet smaller I want to clarify the
internet is not actually shrinking
however this is what I mean when I say
it is getting smaller Bob bought a
convertible and now he’s seeing auto
insurance popping up imagine that the
Internet is like a city there are an
incredible variety of people and an
infinite number of stores but when Bob
visits the Internet it’s gonna feel like
a small town because he’s gonna see the
same familiar faces over and over and
he’s gonna visit the same shops again
and again companies are heavily
interested in giving you the same
content over and over because they know
that’s what you’re interested in I know
that you like cars and auto insurance so
I’m gonna make sure that you see that
same stuff every time you visit a
website that’s how I make money this is
what I mean when I say the Internet is
getting smaller now this kind of can
feel like big brother is always watching
so I just want to quickly point out some
of the good things that come from this
kind of scrutiny and individual
attention because make no mistake you
could not use the internet the way you
currently do
if things were different so let’s talk
about cookies unfortunately I don’t mean
this kind of cookie I mean a small piece
of text whenever you visit a website the
website can give a cookie a small piece
of text to your web browser this is how
weather Calm knew that it was Bob
visiting Bob types in weather calm and
his web browser says hey I need to know
today’s weather information by the way I
have this cookie you gave me before now
you can remember Who I am
every time you visit a webpage and it
remembers your user name that’s a cookie
every time you click on a link and it
changes color and remembers that you’ve
been there before that’s also a cookie
every time you’re filling out shipping
information you’ve bought something
online you’re putting your name address
credit card information and then you
accidentally close the tab but when you
open it it’s all still there that’s a
cookie
in fact cookies are so ubiquitous that
they’re even part of our security
procedures how can a website know that
it is you the same person on the same
device logging in to this website where
there’s a cookie and that’s how it
guarantees that the same computer using
the same web browser is coming back so
where am I going with this is big data
good or is it bad what’s what’s the
conclusion here it’s neither good nor
bad
it’s simply big it’s a lot of
information it’s a lot of knowledge and
knowledge is power the reason I want you
all to know and to understand what’s
happening around you is because the
legislation is catching up with the
technology this is the way things work
right now but the policies that will
govern what is allowed and what is not
are still being written you need to be
able to answer the question what is my
privacy worth
how do I weigh my privacy versus
convenience versus scientific progress
knowledge is power and I hope today
you’ve learned something and you feel
empowered to answer that question
thank you
[Applause]
Please follow and like us: