[Applause]
[Music]
so seems like most of you in the
audience are familiar with that song but
for those of you who don’t know that was
the theme to Kuch Kuch Hota hey great
movie Shahrukh Khan Kajol thirty
nineties you haven’t seen it should
check it out but more broadly what you
just experienced both with your ears and
with your eyes is a new sort of
instrument it’s a data visualization
instrument and for the time being I’m
calling it the synth of is but that’s
not a very good name so I’m definitely
open to new suggestions so today I’m
going to show you how we can use the
basic principles of data visualization
to invent just such an instrument and
hopefully in the process I’m going to
show you how visualizing data can
enhance our understanding of the world
can help us create new ideas and help us
think new thoughts so first okay so for
the first interlude let’s just talk
about what is data visualization it’s
probably a term you’ve heard before
maybe it conjures images of charts or
graphs word clouds but at its core what
data visualization is is just the
process of mapping attributes of data to
visual attributes so we have attributes
of data we have visual attributes and
we’re going to connect them with each
other and the reason that we do this is
generally speaking is that our visual
systems the human visual system is
extremely powerful and capable of
processing a lot of information very
quickly and in fact automatically even
involuntarily for example when you look
at a bar chart you don’t need to think
or to concentrate to determine which bar
is the longest or to
some pretty good sense of what the
relative lengths of the bars are if you
were to look at that same data in a
table or in a spreadsheet it would be
necessary to read all of the values to
do some mental computation to determine
which one is the largest
and even more mental computation to
determine what their relative values are
and so when we visualize data what we’re
doing is we’re offloading that positive
task to our visual system and we let our
minds do more higher-level reasoning to
create new ideas and thinking your
thoughts okay but that’s all pretty
abstract right so data visualization is
the process of mapping attributes from
data to visual attributes but in the
context of the synth of is that
instrument that you just saw what does
that mean so we’re gonna map the
attributes of a musical note that’s our
data so when you have a song the notes
are the data you know an individual
music note is a data points and we’re
going to take the attributes of that
note and we’re going to map them to the
circle that’s what the synth OVAs does
so what are the attributes of a musical
note well probably the most important
one the one that is easiest to detect is
its pitch how high or low the note is
also important and easy to detect is its
duration so how long the note is played
for and we could imagine maybe
visualizing other attributes of a note
its volume how high or how loud or soft
it is
you could also potentially visualize its
timbre or its tone right so for example
notes that are played on an electric
guitar with a lot of distortion have a
different sound quality than those
played on a flute but for the purposes
of our demonstration today we’re just
going to be keeping it simple and
mapping pitch and duration
okay so we have pitch and duration those
are our attributes of the data and what
can we map them to visually in other
words what are the attributes of a
circle well probably the easiest one to
think of is its size
all right how big the circle is in
geometric terms what’s its radius you
also have color humans are pretty good
at detecting the differences in color
between objects and finally perhaps the
most powerful visible channel that we
have at our disposal position we’re very
good at detecting even subtle
differences with great precision of
objects on a two-dimensional plane so
we’re going to be taking advantage of
that with the symptom bits okay so now
that we have the theory out of the way
let’s get back to our instrument and see
how this works in practice all right so
what I’m going to do first is I’m just
going to remove all of the visual
encodings so we’re gonna have map the
node to a circle but there won’t be any
other visual encodings happening and we
can see what that looks like
[Applause]
okay well not very interesting not very
useful so what we can do at first is see
what happens if we create a linguistic
encoding right if we put the names of
the notes on to the circles okay so
that’s not useless it provides some
information we have the name of the note
on a C major scale but it definitely
lacks the kind of intuitive power of the
original demonstration so let’s make our
first visual encodings first one that
we’re gonna do is we’re gonna map the
pitch of the note to the circles size
and maybe you can imagine in your mind
what that might look like if we map the
pitch of the note to the circle size
so we’re getting somewhere
slightly better than the linguistic
encoding but let’s go further and map
the pitch of the note to the color of
the circle hey we’re gonna map pitch to
color alright so this is kind of nice
and by the way this is in data
visualization terminology this is called
a redundant encoding we have used two
visual attributes right namely the size
and the color of the circle to represent
one dimension of data just the pitch of
the note and there’s nothing inherently
wrong with redundant encodings but maybe
we can improve this visualization by
taking the size of the circle which is
currently mapped to the pitch of the
note and map it instead to the duration
of the note so we’re going to take the
size of the circle and map it to the how
okay so that’s looking pretty good and
we still have the sharpest arrow in our
quiver right the best maybe the best
encoding of all and that is the circles
position so since we have three visual
attributes to choose from but only two
data attributes we have to use position
now as a redundant encoding so first
let’s try mapping the notes duration to
the circles position if we’re mapping
the notes duration how long it is to the
position of the circle so you can see a
kind of exploding out and it looks cool
but it’s not particularly useful for us
it doesn’t really provide much insight
so what we’re going to do instead is
we’re going to map the notes pitch to
the circles position see what happens
[Applause]
all right and that looks a lot better to
me it’s uh has that sort of emotional
intuitive Miss that we wanted and in
fact now our visual encodings are so
powerful that the linguistic component
the the letters of the notes isn’t
really doing anything for us anymore so
we’re just gonna go ahead and get rid of
that
[Music]
[Applause]
okay so we’re back to where we began but
let’s try to go a little further so as
the famous computer scientist ben
shneiderman has said the purpose of
visualization is insight not pictures
and I think so far synthesis has
succeeded in producing some beautiful
pictures but it doesn’t provide as much
insight about the music that we have in
place now fortunately data visualization
can also be used to depict events that
happened in the past and that allows our
powerful visual systems to detect
patterns and gain insight so far we’ve
just been visualizing notes in real time
but what happens if we use the synth of
is to peer into the recent past okay so
this here is a visualization of all of
the notes that I’ve played since the
beginning of this talk and the encoding
is are exactly the same so the duration
of the note is mapped to the size of the
circle and the pitch of the note is
mapped to both the color and the
position of the circle and so if I play
one very long high note we should see a
big blue circle show up somewhere on the
so there it is taking up a lot of space
now so to get a better sense of this
visual recording device we’re gonna
reset our notes and watch what happens
when we play a tune from scratch okay
and I Star Wars fans in the house and
even if you haven’t seen Star Wars or
any of those movies you’ve probably
heard that melody many times before but
I think it’s fun to visualize it because
it’s such a common melody and it’s nice
to see it in a new way and let’s just
talk about this visualization here for a
minute so when I look at this viz I
notice that there seems to be some
relationship between the duration of the
notes and their pitches so what happens
if we use the most powerful encoding at
our disposal position to compare the
duration of the notes okay
falling off the screen there a little
bit but I think you get a decent sense
of what’s going on here
so the pattern becomes much more clear
when we use this kind of encoding the
longest notes we can see are those DS
which is where the melody ends and then
we have the low and high C’s just below
that that’s sort of either end of the C
major scale and the two and also has
some medium length G’s they’re sort of
kind of in purple and then the a C’s and
DS and then all the way to the left you
can see three very small seeds which I
played up above in between both melodies
when I repeated it okay and finally just
for fun I want to see what happens when
we do a triple irredundant encoding so
we’re also going to map color to
duration as well not very useful but I
just think it looks nice and it lets us
all right so suppose we want to now do
some more useful analysis we want to get
all scientific about this so suppose we
want to find out how many of each pitch
was played so we can start by just
grouping the notes together or grouping
all the different pitches together like
that and grouping like that and I’m
starting to get a decent at least the
beginning of a sense of how many of each
pitch was played but the size of the
circles is really throwing me off so I’m
just gonna get rid of that encoding
completely not gonna use size for
anything at all okay so this is maybe
slightly better but it’s also kind of
jumbled up so it’s very hard to detect
subtle differences between the number of
notes played in each pitch unless I
actually count them and the whole reason
that data visualization is powerful is
that it takes advantage of mental
processes that happen automatically
right without any effort at all and so
if I have to count it it kind of defeats
the purpose of the whole thing so to
accomplish this task we’re going to use
the most powerful chart of all that’s
right the bar chart
now at this point you might be thinking
for the last 10 minutes or so this guy’s
been showing us all these crazy moving
circles changing color now he wants us
to get excited about a bar chart what’s
the deal with that well I’m glad you
asked so brief history of the bar chart
now today bar charts are everywhere it
would be hard to even be a person in
2018 without ever having seen one which
is why it surprises a lot of people to
learn that the bar chart is at
a relatively recent invention so this
here is the first-ever bar chart as far
as historians are concerned it was
created by William Playfair and
published in his commercial and
political atlas in 1786 1786 so that
means that in India the Mughal Empire
was already on the decline it means that
Shah Jahan you know maybe he may have
built the Taj Mahal but he never in his
life saw a bar chart pretty wild and the
bar chart is deceptively simple but I
think the reason that we see it
everywhere the reason that it’s survived
the test of time the reason that it’s so
powerful is that it takes advantage of
the fact that we humans are really
really good at the detecting differences
in the lengths of objects particularly
when those objects are lined up on the
same baseline right and this is the
reason that bar charts are so much more
powerful than pie charts because pie
charts force us to compare objects
relative areas and that’s the thing that
we’re just not as good at doing
especially now with any precision so in
homage to William Playfair and to
Bollywood I’m going to end this talk by
turning Kuch Kuch Hota hey into a bar
[Applause]
and that as good could throw the hey Isaac
[Music]
so seems like most of you in the
audience are familiar with that song but
for those of you who don’t know that was
the theme to Kuch Kuch Hota hey great
movie Shahrukh Khan Kajol thirty
nineties you haven’t seen it should
check it out but more broadly what you
just experienced both with your ears and
with your eyes is a new sort of
instrument it’s a data visualization
instrument and for the time being I’m
calling it the synth of is but that’s
not a very good name so I’m definitely
open to new suggestions so today I’m
going to show you how we can use the
basic principles of data visualization
to invent just such an instrument and
hopefully in the process I’m going to
show you how visualizing data can
enhance our understanding of the world
can help us create new ideas and help us
think new thoughts so first okay so for
the first interlude let’s just talk
about what is data visualization it’s
probably a term you’ve heard before
maybe it conjures images of charts or
graphs word clouds but at its core what
data visualization is is just the
process of mapping attributes of data to
visual attributes so we have attributes
of data we have visual attributes and
we’re going to connect them with each
other and the reason that we do this is
generally speaking is that our visual
systems the human visual system is
extremely powerful and capable of
processing a lot of information very
quickly and in fact automatically even
involuntarily for example when you look
at a bar chart you don’t need to think
or to concentrate to determine which bar
is the longest or to
some pretty good sense of what the
relative lengths of the bars are if you
were to look at that same data in a
table or in a spreadsheet it would be
necessary to read all of the values to
do some mental computation to determine
which one is the largest
and even more mental computation to
determine what their relative values are
and so when we visualize data what we’re
doing is we’re offloading that positive
task to our visual system and we let our
minds do more higher-level reasoning to
create new ideas and thinking your
thoughts okay but that’s all pretty
abstract right so data visualization is
the process of mapping attributes from
data to visual attributes but in the
context of the synth of is that
instrument that you just saw what does
that mean so we’re gonna map the
attributes of a musical note that’s our
data so when you have a song the notes
are the data you know an individual
music note is a data points and we’re
going to take the attributes of that
note and we’re going to map them to the
circle that’s what the synth OVAs does
so what are the attributes of a musical
note well probably the most important
one the one that is easiest to detect is
its pitch how high or low the note is
also important and easy to detect is its
duration so how long the note is played
for and we could imagine maybe
visualizing other attributes of a note
its volume how high or how loud or soft
it is
you could also potentially visualize its
timbre or its tone right so for example
notes that are played on an electric
guitar with a lot of distortion have a
different sound quality than those
played on a flute but for the purposes
of our demonstration today we’re just
going to be keeping it simple and
mapping pitch and duration
okay so we have pitch and duration those
are our attributes of the data and what
can we map them to visually in other
words what are the attributes of a
circle well probably the easiest one to
think of is its size
all right how big the circle is in
geometric terms what’s its radius you
also have color humans are pretty good
at detecting the differences in color
between objects and finally perhaps the
most powerful visible channel that we
have at our disposal position we’re very
good at detecting even subtle
differences with great precision of
objects on a two-dimensional plane so
we’re going to be taking advantage of
that with the symptom bits okay so now
that we have the theory out of the way
let’s get back to our instrument and see
how this works in practice all right so
what I’m going to do first is I’m just
going to remove all of the visual
encodings so we’re gonna have map the
node to a circle but there won’t be any
other visual encodings happening and we
can see what that looks like
[Applause]
okay well not very interesting not very
useful so what we can do at first is see
what happens if we create a linguistic
encoding right if we put the names of
the notes on to the circles okay so
that’s not useless it provides some
information we have the name of the note
on a C major scale but it definitely
lacks the kind of intuitive power of the
original demonstration so let’s make our
first visual encodings first one that
we’re gonna do is we’re gonna map the
pitch of the note to the circles size
and maybe you can imagine in your mind
what that might look like if we map the
pitch of the note to the circle size
so we’re getting somewhere
slightly better than the linguistic
encoding but let’s go further and map
the pitch of the note to the color of
the circle hey we’re gonna map pitch to
color alright so this is kind of nice
and by the way this is in data
visualization terminology this is called
a redundant encoding we have used two
visual attributes right namely the size
and the color of the circle to represent
one dimension of data just the pitch of
the note and there’s nothing inherently
wrong with redundant encodings but maybe
we can improve this visualization by
taking the size of the circle which is
currently mapped to the pitch of the
note and map it instead to the duration
of the note so we’re going to take the
size of the circle and map it to the how
okay so that’s looking pretty good and
we still have the sharpest arrow in our
quiver right the best maybe the best
encoding of all and that is the circles
position so since we have three visual
attributes to choose from but only two
data attributes we have to use position
now as a redundant encoding so first
let’s try mapping the notes duration to
the circles position if we’re mapping
the notes duration how long it is to the
position of the circle so you can see a
kind of exploding out and it looks cool
but it’s not particularly useful for us
it doesn’t really provide much insight
so what we’re going to do instead is
we’re going to map the notes pitch to
the circles position see what happens
[Applause]
all right and that looks a lot better to
me it’s uh has that sort of emotional
intuitive Miss that we wanted and in
fact now our visual encodings are so
powerful that the linguistic component
the the letters of the notes isn’t
really doing anything for us anymore so
we’re just gonna go ahead and get rid of
that
[Music]
[Applause]
okay so we’re back to where we began but
let’s try to go a little further so as
the famous computer scientist ben
shneiderman has said the purpose of
visualization is insight not pictures
and I think so far synthesis has
succeeded in producing some beautiful
pictures but it doesn’t provide as much
insight about the music that we have in
place now fortunately data visualization
can also be used to depict events that
happened in the past and that allows our
powerful visual systems to detect
patterns and gain insight so far we’ve
just been visualizing notes in real time
but what happens if we use the synth of
is to peer into the recent past okay so
this here is a visualization of all of
the notes that I’ve played since the
beginning of this talk and the encoding
is are exactly the same so the duration
of the note is mapped to the size of the
circle and the pitch of the note is
mapped to both the color and the
position of the circle and so if I play
one very long high note we should see a
big blue circle show up somewhere on the
so there it is taking up a lot of space
now so to get a better sense of this
visual recording device we’re gonna
reset our notes and watch what happens
when we play a tune from scratch okay
and I Star Wars fans in the house and
even if you haven’t seen Star Wars or
any of those movies you’ve probably
heard that melody many times before but
I think it’s fun to visualize it because
it’s such a common melody and it’s nice
to see it in a new way and let’s just
talk about this visualization here for a
minute so when I look at this viz I
notice that there seems to be some
relationship between the duration of the
notes and their pitches so what happens
if we use the most powerful encoding at
our disposal position to compare the
duration of the notes okay
falling off the screen there a little
bit but I think you get a decent sense
of what’s going on here
so the pattern becomes much more clear
when we use this kind of encoding the
longest notes we can see are those DS
which is where the melody ends and then
we have the low and high C’s just below
that that’s sort of either end of the C
major scale and the two and also has
some medium length G’s they’re sort of
kind of in purple and then the a C’s and
DS and then all the way to the left you
can see three very small seeds which I
played up above in between both melodies
when I repeated it okay and finally just
for fun I want to see what happens when
we do a triple irredundant encoding so
we’re also going to map color to
duration as well not very useful but I
just think it looks nice and it lets us
all right so suppose we want to now do
some more useful analysis we want to get
all scientific about this so suppose we
want to find out how many of each pitch
was played so we can start by just
grouping the notes together or grouping
all the different pitches together like
that and grouping like that and I’m
starting to get a decent at least the
beginning of a sense of how many of each
pitch was played but the size of the
circles is really throwing me off so I’m
just gonna get rid of that encoding
completely not gonna use size for
anything at all okay so this is maybe
slightly better but it’s also kind of
jumbled up so it’s very hard to detect
subtle differences between the number of
notes played in each pitch unless I
actually count them and the whole reason
that data visualization is powerful is
that it takes advantage of mental
processes that happen automatically
right without any effort at all and so
if I have to count it it kind of defeats
the purpose of the whole thing so to
accomplish this task we’re going to use
the most powerful chart of all that’s
right the bar chart
now at this point you might be thinking
for the last 10 minutes or so this guy’s
been showing us all these crazy moving
circles changing color now he wants us
to get excited about a bar chart what’s
the deal with that well I’m glad you
asked so brief history of the bar chart
now today bar charts are everywhere it
would be hard to even be a person in
2018 without ever having seen one which
is why it surprises a lot of people to
learn that the bar chart is at
a relatively recent invention so this
here is the first-ever bar chart as far
as historians are concerned it was
created by William Playfair and
published in his commercial and
political atlas in 1786 1786 so that
means that in India the Mughal Empire
was already on the decline it means that
Shah Jahan you know maybe he may have
built the Taj Mahal but he never in his
life saw a bar chart pretty wild and the
bar chart is deceptively simple but I
think the reason that we see it
everywhere the reason that it’s survived
the test of time the reason that it’s so
powerful is that it takes advantage of
the fact that we humans are really
really good at the detecting differences
in the lengths of objects particularly
when those objects are lined up on the
same baseline right and this is the
reason that bar charts are so much more
powerful than pie charts because pie
charts force us to compare objects
relative areas and that’s the thing that
we’re just not as good at doing
especially now with any precision so in
homage to William Playfair and to
Bollywood I’m going to end this talk by
turning Kuch Kuch Hota hey into a bar
[Applause]
and that as good could throw the hey Isaac
Please follow and like us: