
for 25 years I’ve been a academia
actually for more than 25 years working
on systems conversational agents that
can interact with you I started working
on machine translation and currently I’m
as I said working on embodied
conversational agents also known as
talking heads or virtual humans and as I
say I tried to work on machines that can
understand what you say but I’ve learned
a couple of lessons and the first lesson
is that machines can translate text
pretty decently without understanding
them now when we were working on machine
translation more than 25 years ago we
didn’t really follow this lesson we
analyzed the end of the sentence in the
source language with grammar rules world
knowledge dictionaries trying to get to
the semantics the meaning of the
sentence and then generate the source
language the target language utterance
from that meaning now that never did us
get it get us any far now machine
translation as you get it from Google
this doesn’t look at the meaning of the
words and why is that the principle is
as follows everything that we say or at
least parts the parts that compose the
things that what we say have been said
before and most of what has been said
before has been translated before and
all of this has now entered the internet
so it’s just a matter of looking things
up now we couldn’t do this 25 years ago
because 25 years ago the internet was
pretty empty but now there’s another
lesson I’ve learned and that is being
able to understand what you say is not
enough for a computer to engage with you
in
conversation and this is what I want to
talk to you a bit more about and I’ll
start with a movie from the first
experiment I did 17 years ago and you
see here a virtual agent Karen she was
called and the idea is that you could
book tickets for the theater by in this
case not speaking to her but typing in
natural language texts and I apologize
for the poor quality of the movie poor
quality of the graphics and poor quality
of the animations but I would like to
have you look at what she’s doing uh-huh
[Music]
momentum the calculator bomb in the
before selling a member gascon exhausted
okay so this was not a very lively
character but at least she did move her
head once in a while and she gazed away
and towards the conversational partner
now this was based on an algorithm that
was inspired by how people in
conversations deal with looking away and
towards their partner and the important
thing is here that when we implemented
this algorithm it was the right
algorithm which we tested with different
variations and it was right in this
following ways first of all when people
interacted with the good version of the
algorithm they completed the task
significantly faster than people who
interacted with another version and the
second thing is that they liked the
character much more than the other
versions and they were not completely
awkward so what is important as I said
is that for computers to engage with you
in a conversation and certainly these
more advanced ones at least this is not
really the advanced ones but at least
the embodied ones they need to do more
they need to have the right body
language but it’s also important for
them and we’ll come to that in a minute
is that they also understand your body
language and let’s see whether I can
start this movie nope okay so one of the
things that we looked at is particular
when you start creating these embodied
conversational agents you really get to
an idea of how the body language
actually works and one of the things
that has not been studied so much is how
people move their heads in particular
ways when they have a conversation and
when you look at this movie I want you
to think about the following things what
is it actually determining what why the
people move their heads in a way they
move their head
and this is a movie from I think 2005
it’s on the 3rd of October
that’s the day just before of course
World Animal Day and people here are
debating whether it’s ethically allowed
to eat and serve for a girl in a
restaurant I don’t need note though that
is not a fact Apophis clitoris odd but
matter out as you know that’s on the
card health is talking of doing negative
havona
at fellow no Hans life to keep that is
shame is an alternate that it is neither
for like my smack and ignorant ignorant
smarter development until a
multi-channel turned on reporters to
thank the veggie Shiksha yeah yeah that
is it is yell and do some jobs I
described don’t do to the total almost
onset of licorice I’m sure onset and
filmmaker can do and it isn’t it is
funny it’s funny like it’s not like a
product and action Franklin had Geiger
there’s Ellicott Alec a cafe but not
know if either in a console a $50
hopelessly clear about yeah
Builders zone for climbing for directed
on from Paducah dear we will octagon
yes of count in Dena but Miss Dorothy
dear convertible or do talyvel yeah I
have now there her Dakota to tennis
everyday affair father’s attorney
totality Lucy on the continent in the
boot come home about tomatillo made a
VSO Satanist is that the Hobart
phonology that are saying a knowing
behave which is Philip Lasser Slifer
neighbor dr. soft flower correct
dissonance that that that that I need
the other consumers to the Big Finish
okay so there’s a couple of things that
I think you see which you might not have
expected of course people move their
heads in this multi-party conversation
to look at their conversational partners
but what you also see and you see it
underlined here that the movement of the
heads underline the argument structure
of what you’re saying so they come along
with the discourse structure and the
things that you say but you can also I
think see that there’s something of the
personality and the situation coming
into these movements you see how the
first person blow is really under attack
and is defending himself whereas the
professor glitter is really composed and
sure of himself so it’s really important
that these elements that are important
that are involved in our conversations
are picked up by these conversational
agents and that they also react to it
and this is important because what you
see and what you might not have noticed
at a certain time in this movie you saw
also this guy and he made these
movements class smack and in very subtle
very tiny movements but what wheels
doing he was shaking his head slightly
and this was a congruent with what blue
was doing so what is important is that
this conversational agent should not
just pay attention to what you’re doing
and should not be able to express the
same things but it should be more or
less aligned in a very specific way so
this is one of the things that we’ve
been looking at quite closely in the
last couple of years and to show where
we got to I will show you a movie of a
more recent conversational agent that we
billed to get in a European project and
what we did there was exactly trying to
find out how the agent should react
immediately to the response to the
occurrences of the human user human
speaker so what we did is we also
created agents with different
personalities as I said and tried to try
to get the body language of the agent
aligned with its personality
and what I’ll show you here is a movie
of the guy called spike and spike is the
conversational agent that you will that
will pop up as soon as I start the movie
and this guy is a bit more aggressive
and the idea is actually yeah the main
purpose of this one was for emotion
elicitation we try to get people angry
we try to get them happy we try to get
them depressed by talking to an agent
they like the aggressive character quite
a lot
they liked happy character quite a lot
but they didn’t like to speak to the
depressed person or the depressed
character but here you see spike I would
like to see whether you can find out
whether we how we did the align belong
spike what’s your problem oh gosh
see you again my problem is I’ve got to
listen to you is that so tell me about
it yeah well you’re just this horrible
nasty person look at you haven’t even
saved give me a break yeah yeah oh
you’re so pragmatic Thanks well welcome
and I suppose you were going to tell me
nothing can possibly go wrong
och well things can go around but you
know if you stay sensible and get other
people’s advice and things and do other
things don’t go well did I yeah that’s
feeble
and I really don’t like people like you
okay so there’s lots of things going on
here so one of the things that we did we
had a lot of quick analysis of the
movements of the head we sometimes use
mimicry to to have the agent react in a
similar way as the as a person was doing
it
we had emotion analysis of the Foy’s and
of the facial expressions of the persons
and we tried to get the the character
react to that almost immediately and
with big force now it didn’t do a lot of
understanding of what the person said it
the the character had about 100
sentences it could say but we had it
build up in a certain way and we also
use some keyword spotting to get some
idea of what was being said and use that
so it’s quite important for these
conversational agents to read your body
language and to be able to express it
now what we’ve been building is systems
that can help people in various ways and
as I said and it’s also in the title
Fiona
der Modi in Dublin is working on a coach
for people who are giving a public
speech and preparing for it now
unfortunately or fortunately I don’t
know I wasn’t able to use the system
I’ve seen it demoed and it’s quite
interesting to see how the body
movements of the when you’re giving the
talk you can you can see your body
movements and you can really see how the
system will give feedback to whether
it’s right or not in and stuff like that
so I certainly hope for my next next
TED talk that I would be able to use it
okay but let me go back to the lessons
learned the first was that machines
constrained
like text pretty decently without
understanding them now this is an
insight that was only gained like say 10
years ago 25 years ago we thought very
differently so the second lesson is that
being able to understand what you say is
not enough for a computer to engage with
you in a conversation now this is what
I’m believing now but what will I do in
2030 I think that we don’t only have
translations of what we say on the
internet we’ve got a lot of material of
people having conversations with each
other and this material is constantly
enriched with annotations we get better
in machine learning and profiling people
and so I guess that every conversation
we have has been going on at least
partially before and if we’ve got enough
material we might build agents from all
of these pieces that are on the internet
and this is actually what we are now
starting to work on and perhaps it’ll
take – 2030 Atilla give a pretty decent
result or perhaps we can get there a bit
earlier I don’t know but this brings me
to I guess the most important lesson
that I’ve learned in these twenty five
plus years ago that look at the years
long and that is that as soon as you’ve
learned a lesson as soon as you’ve got
the idea that you know something you
should question it thank you you