here hi i would like to start by doing a
little a five second experiment i would
like for you to imagine a person in
front of you this person is an american
professor of computer science one two
three four five now i would like those
of you who imagine the men to raise your
hands yeah that’s the majority i think
and those of you who imagine the woman
please raise your hands i see a few
hands up that’s pretty awesome and you
just demonstrated this marvelous ability
of a human brain to take five words and
then imagine a complete person in front
of you so your brain has accumulated
over the years a large amount of data
and it just so happened that the
majority of the professors that we had
an engineering were probably men and
when you look at the names of the
textbooks which we studied there were
also probably men and this is why when
you try to generalize and the your brain
generalizes in this particular way you
imagine a male professor of computer
science and now I get slides which are
right and i just wanted to show my two
favorite of professors of computer
science and on your left this is an
elysian scale so she is a professor of
cryptology at Brown University and on
the left is an room face ski so she’s a
professor at University of Massachusetts
she’s ahead of the natural language
processing lab and they actually both
American but they’re naturalized
citizens which has to say like me they
came with their parents from Soviet
Union so a bit of an aunty stereotype if
you will so let’s take another example
it’s the first day of Swedish school a
woman comes to pick up her children
she’s in her late thirties she has black
hair she speaks English with a funny
accent the children she’s picking up
speak perfect Swedish if you’re a
teacher to assume she’s a nanny or
mother and turns out if you’re a teacher
you assume she’s a nanny
yeah and this woman is actually me and
that’s me in September picking up my
children from school and being a bit
surprised by this kind of categorization
but that’s yet another ability of our
marvelous ability of our human brains to
take a situation to take this fairly
complex person working in and to reduce
this person to to a few simple
the interesting thing is that it’s an
evolutionary characteristic so it used
to be very helpful for our caveman
ancestors to be able to see a person or
evaluate a situation super super quickly
and to decide whether it’s dangerous or
not but these days especially in
developed countries this is not
particularly helpful in fact a little
bit the reverse in some of the cases so
my nanny example is pretty benign but
let’s consider another example it’s 2014
12 year old kid is playing at a
playground he has a toy gun for some
reason the police gets called to the
scene and within two seconds of arriving
they shoot the kid he dies in the
hospital the name of the kiedis Tamir
rice and you probably heard the story
the kid is black and he unfortunately
belonged to one of the simplified
categories which has certain negative
association to itself within the minds
of most people now if I ask most of you
here whether you’re racist or whether
you think women can’t do programming
most of you would for sure say no
absolutely not but as it turned out the
majority of us have certain internal
beliefs certain unconscious beliefs
which rely this so eighty to ninety
percent of what happens in our brain as
it turns out is unconscious and it’s
really marvelous how we are able to do
things instantaneously well with our
brains how were able to evaluate the
situations but at the same time this
information this processing that we do
unconsciously based on the data that we
encountered throughout our lives
leads us wrong and then the question
also becomes are we able to figure out
if we have this type of unconscious
biases because I can’t ask you a
questions you’re obviously going to tell
me know and at the end of the 90s this
wonderful test was developed called
implicit association test that can
measure exactly this the way it works is
like this let’s say there are two
categories women and men and let’s say
we create two sets of words some of them
are about technology and others are
about home or home making now in
implicit association test you are asked
to categorized women and technology and
men and homemaking and then you measure
how long it takes and then you ask to
categorize women and homemaking and men
and technology and you measure how long
it takes and depending on whether one
takes longer than the other you know
that whether you have an implicit bias
toward one or the other thing now it’s
very embarrassing but it takes me twice
as long approximately twice as long to
categorize women together with
technology and it’s embarrassing because
I’ve been 12 years in the tech industry
and before that I’ve been at the
University for a rather long time and
more or less the same thing happens when
I try to see whether i associate black
people with good things or not so try it
out turns out that that most people have
this beliefs to one degree or another
it’s rare people who don’t and I find
that this is really eye-opening at the
same time this ability of our brains to
do this type of generalization to do
things very quickly and evaluate the
situations is really useful if you think
about driving a car everyday as you’re
driving you encounter new situations and
yet you’re able to judge what’s going on
in them to explain things to predict
where other cars are going and to figure
out what you should do next very very
quickly and we would like computers to
be able to do the same thing and much
that’s what machine learning algorithms
are supposed to be able to do they’re
supposed to be able to generalize in a
way that is similar to the human brains
so we’re machine learning algorithms are
useful is in situations where there are
no simple rules so you can adjust
program precisely the rules and in
situations where adaptation is required
so for example fraudulent credit card
transactions where the fraudsters keep
changing in what way they gain the
system’s it’s very helpful to be able to
adapt things so machine learning
algorithms are sort of like babies you
give them pieces of data and they learn
from this data what’s going on and then
you want to be able to give them an
entirely new piece of data and for them
to be able to explain it using the data
they’ve seen before and it’s very cool
because there are certain human biases
of which are many and many that machine
learning algorithms are very good at
dealing with for example there is
anchoring bias which says that the very
first piece of data the human brain see
will latch onto it and it becomes extra
important so that’s why you never want
to be the first to list a salary in the
negotiations during in during the job
negotiations but you know at the same
time and there are other types of things
where things become a little bit more
problematic so in general we feel that
machine’s they don’t have opinions and
therefore they’re not biased they can’t
stereotype you know they’re not they
they’re not human and exactly the same
way humans are but it’s us who creates
these things and it’s us who fit some
data and i’m going to show you three
different ways in which machine learning
algorithms can become if you would
biased so one of them is the underlying
data bias so if you give if you have
information about what kind of jobs
women hold you will discover that women
tend to hold a lot of more low-paying
jobs and if you use it as an input to
algorithm you need to be really well
aware that you may be perpetuating the
bias you may be training on things which
are not exactly like the things you want
to be looking in the future necessarily
or if you are trying to figure out whom
to give the mortgage and you use in 30
years of mortgage they
from us you might discover that your
algorithm is a little bit racist because
the underlying data reflects systemic
racism then we can just think about what
kind of questions do we ask our
algorithms to solve so where do you see
machine learning algorithms normally so
you go to youtube some of them some are
you on the daily basis and you know you
watch a video and then it suggests to
you the next video well that’s lovely or
your credit card company calls and says
oh you your credit card transaction was
denied because we think it’s fraudulent
well that’s kind of handy if it’s
actually fraudulent or you can provide
some information to the teachers and say
these particular kids in your class
they’re at risk of dropping out we can
tell this to you right now at the
beginning of the class so you can spend
a bit more of your valuable time with
them that’s pretty nice but you can also
ask approximately the same question but
now or take the same kids in the class
and build a model where we will tell you
which of those kids will become
criminals later in your life in their
lives and then you kind of got to wonder
whether you want to leave in this
minority report type world where you
know yeah the machines are kind of
ethical e blind but the questions we ask
have certain ethical implications and we
need to be very well aware of them a
third example is in terms of evaluation
of algorithms so sometime last year it
turns out that google images which is
incredibly good at putting images
together into categories of related
images was putting together black people
and apes into the same category
sometimes and if you think about it it’s
very likely that these algorithms were
developed by white software engineers
probably men statistically speaking and
it’s very likely that during the
evaluation phase they probably looked at
some numbers but certain things didn’t
come up because their albums or the
albums of the people who are evaluating
didn’t have these particular types of
data and this just speaks to the fact
that it’s very important to have it
group of people to be developing
technology because there are certain
things where you may get good numbers
and good air raids and stuff like this
but it’s very important to look to have
a diverse set of people actually
evaluating the data so there are ways to
deal with it in the same way that the
human brains are black boxes and we
don’t quite know eighty to ninety
percent of what’s happening in them
machine learning algorithms are also
usually black boxes but we right now
there’s a trend to keep it simple stupid
or kiss as a call or toward explainable
algorithm if you’re denied mortgage by
an algorithm oisin learning algorithm
you really want to know why so it’s good
to start looking where we want to have
machine learning models where we able to
go later and actually explain which
pieces of training data or what
particular weights made us deny your
mortgage so we could think about whether
we need to adjust this type of stuff or
at least so that there is an explanation
available we also really need to make
our industry a little bit more diverse
so this type of things like with google
images don’t really happen and i think
the ethical aspects of what type of
questions we ask and how and what kind
of things we do are going to become very
important also now why why is this
important so we talked about the human
biases and you know yeah if you confuse
me for my non-existent 90 at some point
that’s really not a big deal clearly
there a lot of if you’re a kid shot at a
playground you would probably feel
pretty horrible about human biases but
machine learning algorithms that we’re
developing right now they can affect
that very large numbers of people so the
impact is going to be huge and this
industry is really exploding right now
because of the amount of machine power
we are able to bring to this and the
amount of data were able to train the
algorithms own so it’s very important to
start looking at what kind of questions
we are answering how important there are
two people who lot people lies and
whether they’re biased and so I leave
you with a thought that we need to work
on our own
ISS but also as we develop technology of
the future especially machine learning
style technology we need to be really
aware that it can be biased in a way
that is very similar in some cases to
human brain also and that’s it for me [Applause]