►
From YouTube: Smarter & Cuter Bots by Rachel White, Microsoft
Description
Smarter & Cuter Bots - Rachel White, Microsoft
Everyone loves a good twitter bot, and node lets you get one up and running very easily. But what if we made bots cuter and more intelligent? Instead of using text as fodder for this bot, I'll show you how to use Twitter's API, ImageMagick, and Microsoft's Face API to
manipulate selfies with cute overlays.
About Rachel White
Rachel is a self-taught programmer and is currently a Technical | Evangelist at Microsoft. She is currently working on multiple video | game projects, a VR cat cafe, and thinking about what IoT devices she | can build for her two black cats. Her other interests include glitch | art, 80s horror, and indie games.
A
Hello-
everyone,
like
he
just
said
I'm
here,
to
talk
to
you
today
about
smarter
and
cuter
bots,
specifically
Twitter
BOTS,
to
give
you
a
little
primer
in
case
you
don't
know
who
I
am
I
am
gonna.
Tell
you
a
little
bit
about
myself.
I'm.
Also
gonna
talk
really
fast,
but
you
can
ask
me
questions
later,
not
about
me
but
like
about
the
Twitter
bot.
A
My
name
is
Rachel
white
I'm
Ohio
on
Twitter
I'm,
a
tech
evangelist
at
Microsoft,
I,
really,
love
cats,
I
also
love
really
cute
things
and
I
also
love
Twitter
BOTS,
making
them
following
them,
especially
weird
ones.
They're
super
fun,
super
great
I'm,
really
a
big
fan
of
making
weird
or
you
know
a
typical
stuff
things
that
just
aren't
your
standard.
You
know
web
application
or
little
like
blinking
LED.
So
last
year,
at
node,
interactive
I
actually
spoke
about
Robo
kitty,
which
you
can
find
at
that
URL
and
it's
an
automated
cat
feeder.
A
That's
powered
by
a
node.js
application,
that's
running
using
johnny-five!
So
there's
like
a
website.
You
push
a
button.
It
dispenses
cat
food
cats
are
happy.
It
was
really
fun.
I
also
do
weird
glitch
art
in
my
spare
time,
so
these
are
some
lenticular
prints
that
I
made
with
generative
glitch
art
that
I
made
and
saved
frames
from
synthesized
video
I
also
make
video
games.
So
this
is
a
JavaScript
there's
a
theme:
it's
cats.
A
This
is
a
video
game
that
I
made
about
my
fat
cat
Rick,
who
just
likes
to
be
pet
and
play
with
things,
and
it's
in
JavaScript
using
a
library
called
phaser,
so
I
pretty
much
just
like
taking
the
existing
languages
and
frameworks
and
all
that
stuff
that
we
already
know
and
doing
fun
stuff
with
it.
So,
let's
get
back
to
cute
Twitter,
BOTS
I,
also
built
one
of
one
of
those.
This
one
is
is
really
cute.
It's
a
very
nice
and
cute
and
friendly
Twitter
bot.
It
just
wants
to
be
your
friend.
A
Basically
when
you
follow
it,
it
tweets
compliments
at
you
and
then,
if
you
reply
to
it,
it
sends
you
more
compliments
and
really
cute
cow
emoji.
It's
super
fun,
so
segwaying
from
the
cute
bot
that
I
built
before
we
get
into
the
actual
thought
that
we're
going
to
talk.
I
wanted
to
talk
about
other
notable
bot
types.
A
This
can
be
not
not
necessarily
Twitter
bots,
but
for
the
examples
we're
gonna
use,
Twitter,
bots,
there's
a
few
there's
a
Markov
chain,
BOTS,
there's
corpus,
fed,
bots,
so
bots
that
are
having
like
a
body
of
some
kind
of
text
that
is
feeding
it.
There's
also
generative
image,
bots
and
just
plain
weird
bots
that
don't
really
have
a
point
and
are
just
there
to
like
make
you
feel
weird
I
guess.
A
So,
let's
talk
about
Markov
chain,
BOTS,
really
quick.
What
is
it?
It
is
a
stochastic
model
describing
a
sequence
of
possible
events
in
which
the
probability
of
each
event
depends
only
on
the
state
attained
in
the
previous
event
makes
sense
right
so,
like
a
visualization
would
be
this
little
drawing
here
that
I
did.
It
is
basically
with
two
states
a
and
B
in
our
state
space.
A
There
are
four
possible
transitions,
not
two,
because
the
state
can
turn
can
transition
back
into
itself,
so
for
at
a
we
could
transition
to
B
or
stay
at
a,
and,
if
we're
at
B
we
could
transition
to
a
or
State
B
makes
makes
total
sense.
For
everyone
we're
all
on
the
same
page,
let
me
give
you
some
examples
of
it
and
use,
so
it
makes
a
little
bit
more
sense.
A
This
is
my
ebooks
account.
What
it
does
is.
It
goes
through
my
whole
entire
Twitter
account,
and
then
it
grabs
pieces
of
it
and
uses
rules
such
as,
like
you
know,
transitional
parts
of
speech
and
if
the
conditions
are
met,
it
likes
smooshes
it
together.
So
here's
some
of
my
favorite
ones,
I,
promised
that
these
are
not
original
tweets.
A
That
I
had
sent
I
had
a
night
full
of
so
many
things
that
I
wish
to
be
satire
or
that
I
want
to
be
satire,
and
my
favorite
I
can
be
provoked
by
a
bird
I
actually
had
to
copy
and
paste
the
last
one
and
make
sure
I
didn't
actually
say
that
and
I
did
it
I,
don't
even
know
what
it's
from,
but
it's
good.
So
that's,
usually
what
a
lot
of
Markov
chain
bots
are
used
for
with
Twitter
the
ebooks
accounts.
A
Next,
we're
going
to
talk
about
corpus
fed
not,
and
what
is
that
in
linguistics,
a
corpus,
plural,
corpora
or
text
corpus
is
a
large
and
structured
set
of
texts
nowadays,
usually
electronically,
sorted
and
processed.
There
used
to
do
statistical
analysis
and
hypothesis,
testing,
checking
occurrences
or
validating
linguistic
rules
within
a
specific
language
territory.
A
The
way
that
people
use
it
is
often
in
national
or
natural
language
processing,
but
it's
more
often
used
with
generic
twitter
bots
that
combine
or
utilize
tech
snippets
in
a
more
generalized
way,
darius,
whose
tiny
versions
on
twitter
has
this
amazing
github
repository,
that's
called
corpora
and
it
has
like
it's
all
user
contributed
and
it's
just
a
huge
list
of
everything
that
you
would
ever
need
and
it's
a
really
great
place
to
get
started.
If
you
wanted
to
generate
something.
A
The
q-pop
that
I
showed
you
earlier
is
corpus.
Fet
I
have
a
ton
of
Lists
that
I
mash
together
and
it's
super
cute
nice
people,
other
types
of
bots
that
people
often
use
are
generative
images
and
those
are
pretty
much
SVG
generative
through
tracery,
which
is
a
JavaScript
library
that
allows
you
to
find
and
replace
text
more
easily.
A
So
here's
some
generative
image
bots
that
I
really
love.
This
is
tiny
neighbor.
It
was
made
by
Amanda
Clausen.
It
was
made
through
tracery,
Find
and
Replace.
This
is
SVG
that
gets
generated
once
a
day,
I
believe
and
it
was
made
through
chief
BOTS
done
quick,
which
is
a
website
that
I'll
tell
you
about
a
bunch
more
of
those
there's
also
soft
landscapes.
So
v21
is
the
person
that
made
cheap
BOTS
done
quick
and
you've
seen
a
lot
of
these
pop
up
this
one's
really
popular,
because
it's
pretty.
A
This
is
an
example
of
the
tracery
code.
You
can
see
that
it
has
some
of
the
placeholders
where
it
says
like
gradient
points
and
gradient
12
and
a
bunch
of
other
stuff
that
gets
swapped
and
replaced
I
I,
don't
get
it
it's
really.
It
blows
my
mind
how
they
have
the
patience
to
like
figure
out
generative
SPG's.
A
It's
really
cool,
though,
and
now
just
like
weird
Twitter
BOTS,
and
what
are
those
oh
they're,
just
they're,
just
weird,
some
weird
stuff,
but
it's
it's
usually
a
lot
of
corpus,
fed
text,
random
images
and
general
nonsensical
markov
chains
for
the
purpose
of
being
weird
and
fun.
For
example,
there's
Clara
Mansur,
who
just
tweets
things
there's
also.
This
is
my
personal
favorite,
ominous
zoo.
A
It's
pretty
fun,
so
what
are
we
making
we're,
obviously
making
a
Twitter
bot,
we're
gonna,
utilize,
Twitter,
streaming,
api's,
graphics,
magic,
Microsoft's,
cognitive
services,
specifically
the
face
API
and
I'm
gonna.
Tell
you
a
little
bit
about
all
of
those
things.
Twit
is
really
fantastic.
It
is
a
NPM
package
that
allows
you
to
access.
Twitter's
API
is
in
a
more
friendly
way.
A
You
pretty
much
just
make
sure
that
you
have
note
an
NPM
installed,
go
to
the
TWiT
repository
sign
up
for
the
Twitter
app
for
auth
tokens
and
then
you're
ready
to
go
I
know
that's
a
lot
more
easy,
but
I
go
into
it
in
depth
later,
graphics,
magic,
we're
also
going
to
use.
It
is
a
tool
to
create
edit
compose
and
composite
images.
It's
super
versatile.
You
can
access
it
from
the
CLI.
A
You
can
use
C
C++,
Lua,
Perl,
PHP,
Python,
TCL,
ruby
windows,
net
or
windows,
come
programming
interfaces
and
there's
a
node
port
of
it,
which
is
great
because
that's
what
we're
talking
about
and
it's
called
GM,
but
it's
kind
of
a
pain
in
the
ass
to
use.
So
graphics.
Magic
is
a
fork
of
image
magic
which
is
written
in
Perl
and
it
it
is
very
particular
also
down
an
up.
Sampling
of
images
is
kind
of
poor
poor
quality
and
when
we're
putting
something
on
Twitter,
it's
already
going
to
lose
some
of
its
quality
anyway.
A
So
you
just
have
to
deal
with
the
fact
that
it's
going
to
be
a
little
rasterized.
It
requires
some
hacky
workarounds
to
get
it
to
do
exactly
what
you
want
it
to
do,
and
a
node.
You
have
to
do
a
ton
of
callbacks
to
do
multiple
compositing
and
the
thing
that
we're
building
has
like
six.
So
it's
really
ugly
to
write,
but
you
know
it
works
sometimes
and
two
hosted
on
a
server.
A
You
have
to
be
able
to
have
access
to
install
some
binaries
for
it,
which
is
you
know,
a
problem
if
people
are
doing
their
own
self
hosting
and
stuff.
So
now
we're
gonna
talk
about
Microsoft,
cognitive
services.
It's
rad
I'm,
not
just
saying
that,
because
I
work
there
I
swear
it's
pretty
good.
It's
a
whole
suite
of
machine
learning.
Rest
api's
like,
if
you
can
post
forum
and
hit
an
endpoint
you
can
use
any
of
the
api's
here.
A
Here
is
almost
all
of
them:
computer
vision,
content,
moderator,
emotion,
face
video,
Bing
speech,
custom,
recognition,
speaker
recognition,
it's
lots
of
big
things,
there's
a
ton
that
you
can
do
and
play
with.
It's
super
super
easy
to
use.
If
you
know
how
to
use
an
endpoint,
you
can
use
everything
and
for
today
we're
going
to
use
the
face
API
the
face.
Api
is
really
great
because
it
detects
one
or
more
human
faces
in
an
image
and
gets
back
face.
A
You
know,
there's
there's
a
lot
of
attributes
for
features
available
age,
gender
pose
smile,
facial
hair,
27
landmarks,
which
are
XY,
coordinates,
it's
pretty
handy.
So,
let's
try
it
out.
Let's
look
at
this
very
flattering.
Photo
of
myself
of
green
hair.
I
ran
it
through
the
face
API,
and
it
comes
back
with
a
rectangle
around
my
face
points
where
my
eyes
are
point
on
my
nose
point
around
my
mouth.
A
It's
really
easy
and
then
you
can
manipulate
that
as
you
see
fit,
I
don't
know
you're,
probably
like
alright.
What
exactly
are
we
making?
If
you
follow
me
on
Twitter,
you
already
know,
but
if
you
don't
much
like
Sailor
Moon
transforms
into
an
already
cute
cuter
version
of
her
cute
self.
We're
gonna
make
a
magical
and
cute
image
bot.
So
I
was
super
inspired
by
Japanese
Perry
booths.
What
they
do
is
you
go
in
takes
a
picture.
It
makes
you
extra
cute.
It
puts
cute
blush
on
your
cheeks
and
animal
faces
and
animal
noses.
A
So
that's
exactly
what
I
wanted
to
build.
So
now
it's
code
demo
time
alright,
so
I
am
going
to
show
you
today
how
some
of
those
things
work.
I
have
my
twit
thing
up
here.
I
hope
these
access
tokens
are
still
active
and
not
revoked,
and
then
we're
going
to
have
all
of
the
twit
access
tokens
which
you
get
from
like
app
twitter.com.
A
You
have
to
provide
your
phone
number,
it's
a
pain
and
then
all
it
is
to
post
is
T
post
statuses,
slash,
update
and
my
status
is
going
to
be
tweeting
this
from
a
bot.
During
my
talk,
if
node
interactive,
hi
friends
I
forgot
to
swap
that
out
last
time,
I
gave
this
talk
and
I
ended
up
tweeting
just
like
do,
and
it
was
weird
so,
let's
see
require
twit
that
is
twit
1j
s,
so
no
twit
1j
s,
please
work
am
I
on
the
Internet
yeah.
B
A
A
Oh,
that's
a
different
flattering
photo
of
myself
and
then
I'm
gonna
run
it
through
the
client
for
Project
Oxford,
which
is
a
really
awesome
repo
someone
made
I,
don't
even
I'm,
not
sure
who
made
it,
but
they
made
my
life
easier
for
these
demos,
so
we'll
do
node
based
op
Jas
and
then
it's
going
to
return
its
gonna
guess
my
age
and
it
guesses
me
older
than
I
am
which
is
kind
of
offensive.
I'll
be
honest,
but
it'll
also
return
a
whole
entire
object
with
the
XY
coordinates.
I
think
the
Internet
is
down.
A
Of
course
it
is
I
have
a
hot
spot
if
we
need
to,
but
I,
don't
think
that
there
is
time
so
all
right,
let's
go
to
the
last
one,
which
does
not
require
the
internet.
So,
oh
yeah!
Thank
you
tweeting
this
from
a
bot.
During
my
talk,
it
node
interactive,
hi,
friends,
yeah,
it
worked
cool
and
then
we
go
up
here
and
that's
still
waiting.
So,
oh
of
course,
making
me
look
bad.
A
A
A
So
what
we're
doing
for
this
is
we're
overlaying,
something
that
I'm
not
showing
you
yet
because
it's
a
secret
and
we're
going
to
write
it
temper
early
and
then
we're
going
to
combine
it
with
cats
JPEG,
which
is
this
cute
picture
of
my
cats,
and
it
basically
draws
the
other
image
on
top
of
cats
and
then
puts
it
as
output,
which
is
not
present
right
now.
So
no
two
combined
Jas
yeah
it
works
and
then
we
come
back
here.
We
go
to
output
and
it
puts
really
cute
overlays
and
stuffing
for
my
cats.
A
A
A
Have
this
really
great
habit
of
not
giving
a
talk
unless
something
breaks,
so
I've
gotten
really
good
at
talking
when
things
go
wrong
instead
of
freaking
out,
because
you
really
can't
deal
with
it
when
this
stuff
happens
and
I
give
a
lot
of
hardware
demos,
so
those
never
work,
especially
when
you're
up
here.
So
of
course,
as
my
career
goes
on
and
I
transition
into
other
things,
nothing
goes
right,
so
I'm
gonna
show
you
how
everything
works
first
and
then
we'll
go
back
to
the
code.
Really
quick,
it's
magical
and
cute
image
bought.
A
Pre
booth
showed
you
some
code.
This
is
what
the
image
bought.
The
word
building
does
takes
a
picture
of
you
and
it
overlays
really
cute
noses
and
cheeks
and
ears,
and
cute
hearts
and
stars
and
stuff
on
it
and
the
way
that
it
works.
Is
you
tweet
an
image
at
it?
I
have
a
Twitter
stream
open
whenever
it
receives
an
image,
it
sends
it
to
Microsoft's,
face
API
I
get
those
XY
coordinates
back
and
then
I
over
lay
a
random
ear
or
nose
thing.
It
works
with.
A
Like
a
ton
of
you
can
put
it
up
to
I
think
64
people
in
a
picture,
though
sometimes
it
like
miss,
finds
your
facial
stuff
and
you'll,
get
like
a
big
nose
over
five
people,
which
is
fun
I,
think
I
launched
it
last
week.
It
was
really
awesome.
People
immediately
tried
to
break
it,
sometimes
it
glitched
out,
but
it
ended
up
being
super
cool.
This
is
it
it's
magical
and
cute.
A
A
I'll
make
this
bigger
and
it
runs
through
it
checks
for
a
bunch
of
conditions
like
if
you're
following
it.
If
you
added
an
image
and
then
if
you
meet
all
of
the
right
conditions,
it'll
take
it
over
to
face
J
S,
which
does
all
of
the
graphics
magic
compositing,
which
is
like
so
many
functions
and
callbacks
that
are
getting
the
coordinates
and
finding
the
right
position
on
the
image
and
then
smashing
it
all
together
until
it's
all
met.
A
A
No,
okay!
Well,
that's
it
I'm!
Just
gonna
leave
this,
so
you
can
see
it.
That's
the
end
at
one
minute.
I
did
such
a
good
job.
I,
don't
know
if
anybody
has
any
questions.
I
can
totally
answer
them.
I
know:
I
spoke
really
fast
and
had
some
technical
difficulties,
but
especially
like
in
regards
to
graphics
magic.
You
couldn't
find
my
slides
at
aka,
dot,
MS,
slash,
smarter
and
cuter.
All
of
the
code
is
on
my
github,
which
is
Rachel,
Nicole,
slash,
magical
and
cute
I'm
cool
dot
online
is
my
website.
A
C
B
A
A
There's
actually
there's
a
there's,
a
streaming
API
for
facial
recognition.
That
Microsoft
has
it's
pretty
neat
when
you
go
and
visit
the
main
campus
there's
a
demo
of
it,
so
people
are
walking
by
and
it's
finding
their
faces
and
like
telling
you
information
about
them,
and
you
can
also
use
it
to
overlay
stuff
in
a
real-time,
so
sort
of
like
how
snapshots
face
thing
is,
but
it
even
does
depth
a
little
bit
with
the
pitch
roll
and
yaw
attributes
that
you
back
with
the
face
landmarks.
So
that's
pretty
neat.
That's
actually
my
next
goal.