►
From YouTube: Build fault tolerant, distributed apps with Kubernetes and Temporal - Tihomir Surdilovic
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
使用 Kubernetes 和 Temporal 构建容错分布式应用程序 | Build fault tolerant, distributed apps with Kubernetes and Temporal - Tihomir Surdilovic, Temporal Technologies Inc.
为了满足不断增长的客户需求，公司必须在多个平台、基础设施、架构和编程语言上多样化其技术解决方案。采用分布式多语言体系结构使我们能够使用最合适的工具和技术为业务问题提供解决方案。它还允许公司从更广泛的潜在候选人中获得技术人才。在本课程中，我们将介绍一种开源的 Kubernetes 原生容错微服务编排平台 Temporal。我们将演示 Temporal 如何让您专注于业务需求 impl，而不考虑您的技术空间，同时解决诸如持久性、通信、错误传播、事务管理和跨 Kubernetes 上运行的多个微服务的可伸缩性等难题。该演示包括四个用 PHP、Go、NodeJS 和 Java 编写的微服务。
To keep up with the increasing customer demands, companies must diversify their tech solutions across multiple platforms, infrastructures, architectures, and programming languages. Adopting distributed polyglot architectures enables us to provide solutions to business problems using the the most appropriate tools and technologies. It also allows companies to acquire technical talent from a much wider pool of possible candidates. In this session we will introduce Temporal, an open-source, Kubernetes-native and fault-tolerant microservices orchestration platform. We will demonstrate how Temporal allows you to focus on business requirements impl, regardless of your tech space, while offloading hard problems such as persistence, communication, error propagation, transaction management, and scalability across multiple microservices running on the Kubernetes. The demo includes four microservices written in PHP, Go, NodeJS and Java.
A
Hello:
everybody
in
this
talk.
We
will
discuss
how
to
build
fault
or
tolerant,
distributed
applications
within
portal.
My
name
is
tichomir,
I'm
a
developer
advocate
at
temporal
and
I'm
also
the
project
lead
of
the
cncf
serverless
workflow
project,
so
table
of
contents.
First,
we
will
give
a
quick
introduction
to
temporal
we'll
discuss
how
you
use
temporal
to
build
resilient
micro
services.
We
will
also
take
a
look
at
the
polygon
aspect
of
building
microservices
using
temporal,
as
well
as
some
of
the
error
handling
features
that
it
includes.
A
A
So
to
give
an
introduction
to
temporal
temporal
is
composed
of
two
main
parts.
One
is
the
server
the
portal
server.
One
is
the
sdks
on
the
left
hand,
side
on
the
server,
it's
basically
a
golang
binary
that
can
be
deployed
in
many
different
ways,
for
example,
kubernetes,
docker
or
really
any
of
infrastructure
that
you
might
have
available.
A
You
can
scale
out
the
tempo
or
temporal
server
horizontally
but,
for
example,
deploying
multiple
servers
on
different
clusters,
but
each
service
individually
can
can
be
scaled
as
well
now,
just
to
take
a
little
quick
look
into
the
different
parts
of
the
temporal
server.
We
have
the
front-end
service,
so
all
the
communications
of
your
client
applications
and
your
services
can
communicate
to
the
temporal
server
via
grpc
calls
and
the
front-end
service
handles.
Basically,
all
the
inbound
calls
and
allows
for
things
like
multi-cluster
replication
and
things
like
that.
A
The
history
service
manages
workflow
state
transitions,
meaning
that
temporal
server
does
not
execute
your
code
directly.
However,
it
assists
during
this
execution
in
terms
of
storing
some
sort
of
important
events,
execution
events
that
then
later
on,
allow
you
to
either
resume
workflow
or
your
application
execution
and
deal
with
things
like
failures
and
stuff,
like
that,
the
matching
service
provides
host
task
queues,
which
are
basically
endpoints
dynamically
generated
through
which
you
can
host
multiple
types
of
workflows
or
type
of
services
at
some
particular
endpoints
and
in
the
end,
the
worker
service
is
a
background.
Processing.
A
Temporal
server
includes
a
bunch
of
different
background
system.
Workflows
and
replication
cues
and
things
like
that
in
order
to
achieve,
but
we
will
take
a
look
in
the
next
couple
slides
as
far
as
storage
goes
again,
temporal
does
not
store
anything
your
applications
code
or
some
sort
of
serialized
application
code.
No,
all
your
code
runs
actually
on
your
applications
as
we'll
see
soon,
but
the
server
storage
still
stores
some
information,
like
events
in
the
history
of
the
execution
of
your
different
services
and
applications.
A
A
I'm
sorry
observability
is
very
important
when
we
start
writing
any
sort
of
microservice
or
any
polyglot
type
of
service
or
distributed
service
stuff.
So
both
the
server
and
the
temporal
sd
case
provide
out-of-the-box,
provide
metrics,
and
these
metrics
can
then
be
consumed
by
things
like
prometheus
and
grafana,
and
you
can
build
your
dashboards
and
and
metrics
visualization
that
way.
In
addition
to
that,
the
temporal
sdks
also
provide
tracing
information.
A
So,
during
execution
of
your
application
and
services,
you
can
view
the
tracing
information,
for
example,
with
whatever
you're
tracing
type
of
software
is
that
you
want
to
use
now
I
like
to
kind
of
like
display
the
use
of
temporal
from
a
point
of
view
of
a
particular
user.
In
this
case,
let's
focus
on
a
developer
as
developers.
A
We
want
to
really
be
able
to
focus
on
our
business
logic
and
when
we're
dealing
with
writing
very
complex
applications,
especially
distributed
microservices,
we
have
to
deal
with
a
ton
of
different
things
with
temporal
as
a
developer,
you
really
can
focus
on
writing
your
own
code
and
the
temporal
server
on
the
right
hand.
Side
in
the
box
is
basically
you
can
think
of
it
as
a
black
box
that
provides
you
things
like
event,
handling,
durable,
timers,
durable
storage,
transaction
management,
queuing
analytics
and,
as
we've
shown
previously
metrics.
A
So
all
the
things
you
get
out
of
the
box
are
really
your
applications.
You
can
write
simple
code
and
you
have
all
those
benefits
that
you
can
utilize
as
far
as
what
the
temporal
server
provides
now
given,
of
course,
some
scaling
options
and
employments.
The
temporal
server
is
capable
of
executing
hundreds
of
millions
of
this
applications
that
we
call
workflows.
Concurrently,
as
we
said
in
portal,
server
does
not
directly
execute
your
code.
Your
code
still
executes
on
your
premises
and
your
deployment.
A
Now,
as
far
as
again
looking
at
from
developer
perspective,
we
can
utilize
the
temporal
sdks
to
write
our
applications
in
different
types
of
languages,
as
we
said,
go
php,
node
and
java,
and
each
one
of
these
sdks
provide
apis
for
use
that
we
can
utilize
during
development
such
as
develop.
You
know:
workflow
development,
apis
testing,
apis
and
also
client
apis,
because
temporal
again
it
takes
a
programmatic
approach
rather
than
some
sort
of
dsl
or
high
level
type
of
workflow
language.
A
We
write
our
code
as
developers
still
in
our
favorite
ide,
so
we
don't
have
to
get
our
environment.
We
don't
have
to
change
our
programming
language
use.
We
can
stay
within
the
same
type
of
environment
that
we're
used
to.
So
what
are
we
kind
of
targeting?
You
know
what
are
we
as
developers?
What
we
have
to
write-
and
he
has
three
things
really
the
first
one
is
workflows,
so
workflows
are
implementation
of
business
logic.
So
this
is
just
code.
You
know,
with
some
restrictions
that
we
will
go
over,
that
you
have.
A
You
can
write
in
order
to
execute
your
business
logic,
most
likely
orchestrate
some
third-party
services
or
things
like
that.
The
workflow
code
that
we
write
is
becomes
fault
tolerant
because
of
temporal,
and
there
is
many
different
things
like
configuration
based,
retries,
timeouts
compensation
and
all
those
things
that
in
portal,
through
sdks
and
with
addition
to
the
server
that
kind
of
manages
these
things
allow
you
to
have
out
of
the
box
as
well.
The
second
thing
are
activities,
so
activities
are
basically
parts
of
your
code.
A
Where
you
can
use
you
know
any
sort
of
library
database
access
file
access,
you
can
do
pretty
much
anything
you
want
there.
So
activities
can
be,
of
course,
invoked
sync
or
async.
Without
you
really
having
to
specify
any
of
that
information
in
your
code
or
use
some
sort
of
third-party
libraries
for
that
it
can
be
rate
limited
and,
of
course,
you
know
with
temporal,
you
get
automatic
retries
without
having
anything
to
specify
in
your
application.
A
As
far
as
coding,
for
it
goes,
the
third
things
are
workers.
Workers
are
processes
that
host
your
workflows
and
activities,
and
workers
are
responsible
for
execution
and
and
progress
of
execution
of
both
your
workflows
and
activities.
Workers
then
communicate
with
the
temporal
server,
and
that
communication
is
important
in
order
to
to
to
run
invoke
in
running
and
continuing
and
resuming
your
workflow
execution
as
we'll
see
also
in
the
later
slides.
A
In
addition
to
the
sdks
and
the
programming
model,
temporal
also
provides
for
developers
a
web
ui
through
which
you
can
see.
You
know
you
know,
what's
going
on
what
workflows
are
running,
what
state
they're
in
their
execution?
History
stack
traces
and
stick
like
things
like
that,
but
in
addition
it
provides
a
cli
which
is
probably
even
more
powerful
where
you
can
use
to
do
a
lot
of
different
things
like
again
start
workflows
and
you
can
have
your
batch
executions
of
things
and
stuff
like
that.
A
So
that's
kind
of
like
an
important
thing
to
understand
that
you
can
really
test
any
sort
of
application
or
service
that
you
write
with
temporal
easily
as
far
as
activities
goes,
that
can
be
tested
and
and
debug,
independent
and
again
because
of
the
testing
framework
that
that
temporal
provides
for
most
sdks.
You
do
not
have
to
even
have
the
temporal
server
running
in
order
to
test
your
code
now,
from
kind
of
like
an
architectural
perspective
or
a
little
bit
higher
up
view.
A
We
have
to
ask
ourselves:
can
we
if
we
start
adopting
in
the
portal,
can
we
still
use
frameworks
that
we're
accustomed
to
and
that
we
currently
want
to
use?
And
the
answer
is
yes,
temporal
is
not
intrusive
in
this
way
whatsoever.
We
talked
about
this.
Can
I
use
already?
Can
I
use
my
current
programming
language
and
the
answer
is
yes?
What
about
the
dev
environment
and
again
we
kind
of
went
through
that
for
both
testing
and
debugging,
and
also
writing
your
code.
A
A
A
So
once
if
we
wanted
to
incorporate
them
portal
into
the
mix,
what
we
have
to
really
pick
and
choose
is
which
parts
of
our
code,
mostly
that
you
know
the
business
logic
to
the
core
business
logic,
code,
executions
or
the
orchestration
of,
for
example,
these
services
and
ui's.
You
want
to
turn
into
workflows
and
which
parts
of
our
code
currently
interact
with
different
file
system
database,
access,
invocation,
rest
or
ac.
You
know
async
sync
invocation
of
this
third-party
systems.
We
want
to
turn
into
activities.
A
So,
finally,
what
is
the
value
proposition
of
temporal?
The
value
position.
Is
that
no
matter
what
what
we
do
or
what
our
title
is
in
the
end,
we
build
services
or
microservices
in
general,
and
these
services
have
to
be
durable,
distributed,
scalable
and,
of
course,
polyglot
in
most
cases
and
it's
up
to
you,
but
temporal
provides
it
for
you
really
allows
you
to
focus
on
writing
your
business
logic
without
having
to
think
about
all
those
benefits
that
you
get
pretty
much
for
free.
A
So,
let's
take
a
look
at
a
quick
example.
This
is
a
java
example.
Again,
comportal
provides
us
the
case
in
in
different
languages,
but
in
the
left-hand
side,
let's
say
that
we
have
an
existing
class
called
my
customer
that
has
some
state.
For
example,
it
holds
a
customer
and
it
has
a
mind
method
called
update,
account
account
message
where
we
want
to
update
some
information
about
this
customer.
A
It
also
has
two
methods
on
the
bottom
side
called
get
customers
which
allows
us
to
receive
information
of
the
customer,
we're
currently
processing,
and
he
has
an
exit
method
where
basically,
that
method
allows
us
to
stop
processing
if
that
needs
to
be.
As
you
know,
we're
dealing
with
java,
we
have
some
sort
of
interface,
my
customer
interface,
that
are
my
our
my
customer
class
implements,
and
this
is
kind
of
like
the
blueprint
it
has.
The
three
main
methods
of
the
account
message
get
customer
and
exit.
A
So,
with
that
itself,
our
class
on
the
left
hand,
side
has
become
a
temporal
workflow
and
two
things
we
have
to
take
a
look
at.
How
do
we
actually
interact
now
with
our
workflow?
How
do
we
start
it?
Stop
it,
and
how
do
we
get
information
out
of
it
and
how
do
we
signal,
for
example,
send
signals
to
it
as
well
as
how
do
we
write
this
business
logic
that
we
want
to
actually
implement
this
class
with?
A
So,
let's
take
a
look
now
at
some
of
the
temporal
features
provided
by
the
temporal
sdk
and
and
their
apis.
The
first
thing
that
you
get
out
from
the
temporal
apis
is,
for
example,
you
can
start
workflow
execution
and
you
it
can
be
long
running.
So
on
the
left-hand
side,
for
example,
we
showed
that
hey.
We
want
to
start
a
workflow
execution
implementation
of
your
business
logic
and
we're
going
to
let
it
run.
You
know,
up
to
a
year,
workflow
has
stayed
in
the
state.
A
A
Your
workflow
code
can
be
fault.
Tolerance
from
the
left
hand,
side
see
that
we
can
have
automatic
retries
for
activities.
We
can
have
retries
for
your
workflows.
You
can
reset
your
workflows,
cancel,
terminate
things
like
that.
There's
apis
for
that,
and
also
because
this
is
programming
language
approach.
You
can,
you
know,
try
catch
and
you
can
catch
some
certain
exceptions
even
perform
things
like
compensation.
A
Before
ending
your
workflow
execution.
We
can
also
define,
through
the
apis,
that
we
want
periodic
execution,
via
some
chrome,
for
example,
and
we
can
invoke
workflows
and
as
well
also
as
activities
sync
or
async.
So
temporal
provides
full
support
and
in
some
programming
languages
it
is
the
default
approach
to
to
actually
invoke
things
async
itself.
A
Finally,
we
also
know
that,
especially
when
you
have
long
long-running
business
logic
or
workflows,
you
need
some
sort
of
versioning
changes
happen.
So
temporal
ips
provides
you
a
way
to
version
your
code,
even
while
it's
currently
even
running
and
deploy
a
new
version
and
and
deal
with
updates
automatically.
A
Of
course,
we
we
talked
about
testing
already.
So
again,
all
your
workforce
activities
or
any
code
that
you
write
using
the
temporal
apis
is
fully
testable.
Now,
let's
take
a
look
a
little
bit
how
we
interact
with
our
workflows,
so
the
left
hand
side.
Let's
say
we
have
some
sort
of
client
application.
A
A
client
actually
wants
to
invoke
an
instance
of
our
business
logic
or
our
workflow
workflows
that
we
develop
in
our
services,
so
the
client
api
can
send
commands
those
are
again
commands
they're
based
on
grpc,
and
we
send
them
to
the
temporal
server
to
its
front-end
service.
So
one
of
the
commands
that
we
can
send
is
start,
but
there
are
many
more.
We
can
signal.
Query
cancel
things
like
that.
So
in
the
left-hand
side,
let's
say
we
have
some
code
that
actually
uses
the
temporal
java
sdk
and
its
apis
to
call
a
workflow
client.start.
A
This
is
an
async
invocation
that
will
request
this
for
the
server
the
server
itself
at
this
point
does
not
really
know
what
service
or
our
service
is
actually
going
to
execute
or
or
pick
up
the
request
to
start
a
workflow
execution,
but
what
the
server
does
it's
going
to
put
a
message
into
a
particular
task
queue
now
our
application,
our
service
on
the
bottom
left
that
we're
writing.
That
includes
our
workflows
and
activities
and
our
workers
are.
A
We
tell
our
workers
to
listen
to
this
particular
last
task
queue
and
then,
when
a
message
arrives,
or
in
this
case
on
the
right
hand,
side
a
little
red
circle
that
includes
some
information
about
wanting
to
start
work
for
execution
arrives,
our
work
is
going
to
pick.
It
up
is
going
to
read
the
instructions
and
start
the
execution
of
our
new
workflow
instance.
So
this
is
a
fully
distributed
system.
A
Things
like
that
into
you,
know
your
whole
equation
without
really
having
to
write
their
code
yourself.
Now,
once
our
task
worker
has
picked
up
the
initial
test
to
start
the
workflow,
it
started
processing
to
some
point
where
it
needs
some
more
information
from
the
server,
for
example,
schedule
an
activity,
execution
or
start
creating
a
timer.
In
case
where
our
workload
says,
let's
say
sleep
for
10
minutes
or
10
days,
our
workflow
can
send
again
a
message
to
the
important
server
say
basically
saying
I
want
to
schedule
an
activity
execution.
A
Just
going
a
little
bit
into
this
more,
we
talked
that
temporal
is
a
very
resilient
system
and
once
the
messages
arise
from
the
task
queue
and
the
worker
picks
it
up
on
the
left-hand
side,
let's
say
we
have
a
workflow
code
that
we
want
to
execute
the
message
the
itself
or
the
workflow
task
includes
things
like
what
should
be
executed
next
and
all
the
history
and
information.
That's
happened
so
far
on
the
left
hand
side
once
this
task,
workflow
task
is
received.
A
A
Once
we
have
replayed
and
put
the
workflow
state
into
the
exact
position
where
we
need
it,
we
can
use
the
what's
next
part
in
order
to
continue
workflow
execution.
From
that
point
on
again,
if
this
workflow
history
can
place
the
state
in
the
same
state,
it
was
before
it
was:
it's
called
deterministic,
so
we
can
move
on
with
the
with
the
execution
if
the
event
history
does
not
match.
A
So
let's
take
a
look
now
at
service
orchestration.
This
is
kind
of
like
a
common
way
where
we
have,
let's
say
a
food
delivery
service
written
in
any
programming
language
for
which
temple
has
sdks,
where
we
want
to
basically
with
the
use
of
temporal
server,
orchestration
third-party
services
on
the
right-hand
side
like
dispatch
service,
restaurant
service,
payment
service,
and
things
like
that.
So
with
temporal,
you
can
deal
with
intermittent
failures,
meaning
that,
if
some
of
those
third
per
party
services
are
down
does
not
mean
that
we
have
to
fail.
A
If
that
happens,
or
doesn't
really
mean
that
we
have
to
write
any
code
in
order
to
deal
with
these
particular
errors
with
temporal
server.
And
it's
as
the
case,
you
can
deal
with
intermittent
failures
without
any
worry
and
you
can
actually
fix
those
errors
and
and,
for
example,
issue
retries
until
those
particular
services
come
back
up
and
are
available,
another
things
that
we
temporal
provides
this
dealing
with
continuing
failure.
Let's
say
in
this
case
our
payment
service
is
down,
and
it's
not
coming
back
up.
A
A
So
even
though,
in
this
case,
these
are
third
party,
and
we
don't
particularly
have
anything
to
do
with
the
code
or
the
services
itself
we're
just
using
them.
We
still
through
temporal
server,
can
rate
limit.
So,
for
example,
if
the
payment
service
has
a
cost
associated
with
it,
and
we
don't
want
to
go,
for
example,
over
100
invoking
100
times
per
second
or
per
day
or
whatever
the
time
is,
you
can
define
this
rate
limiting
in
your
application,
and
temporal
server
will
make
sure
that
the
rate
limit
operations
are
falling.
A
Are
talking
through
the
temporal
server
food
delivery
service
and,
let's
start
dispatch
service?
In
this
case,
we
can
rate
limit
our
applications
ourselves,
so
we
can
define
rate
limiting
in
our
own
service
code.
We
can
still
deal
with
error
and
propagation
across
the
services.
So
as
in
before,
where
we
had
third-party
services,
we
had
to
deal
with
things
like
http
errors
like
404,
without
really
having
any
ability
to
gain
some
specific
errors.
A
If
you
have
different
services
that
are
using
temporal,
you
can
have
very,
very
powerful
error,
handling
and
propagation
that
we
will
see
here
in
a
minute.
In
addition
to
that,
one
service
that
we
write
with
temporal
can
be
written
in,
let's
say,
go
and
the
other
service
that
we
write,
that
might
be
a
different
team,
could
be
right
in
written
in
node.js.
A
A
So
as
far
as
the
polyglot
aspects
go,
the
temporal
server
is
able
to
serialize
and
deserialize
the
data
and
also
workflow
information,
so
that
we
can
have
this
polyglot
type
of
infra
collaboration
and
communication,
including
error,
message
and
propagation,
and
things
like
that
across
distributed
services
in
different
programming
languages
as
well.
So
let's
take
a
look
at
this
for
a
minute
at
as
far
as
polyglot
goes
as
far
as
air
handling.
A
Let's
see,
on
the
left
hand,
side
we
have
our
food
service
in
our
workflow
and
activity
written
in
go
and
our
our
application
starts
by
making
instance
of
our
food
delivery.
Workflow.
This
workflow,
let's
say
calls
our
activity
and
then
this
activity,
through
the
temporal
server
and
in
the
sdk
apis
client
apis,
invokes
the
restaurant
service,
that's
written
in
node.js
and
the
same
thing
here
we
go
again
and
the
restaurant
service
activity
invokes
the
payment
service
activity,
which
is,
in
this
case,
let's
say
written
in
php
now
in
systems
or
distributed
systems.
A
Typically,
you
know
our
food
delivery
service
is
going
to
have
no
idea
what
actually
happened
and
will
not
be
able
to
receive
the
proper
information
of
what,
where
and
how
things
broke,
in
order
to
be
able
to,
let's
say,
compensate
for
it
or
fix
the
air
when
you're
using
temporal
the
payment
service.
Actually,
we
has
through
the
temporal
server
error
propagation.
A
So
when
this
error
propagates
back
all
the
way
to
where
we
started
our
our
execution,
this
area
is
going
to
be
able
to
have
all
the
details,
specifically
the
servers
that
have
failed
all
the
information
that
the
payments
is
added
to
the
exception,
as
well
as
the
original
exception,
and
all
these
things
that
do,
for
example,
the
restaurant
service
added
to
the
air
and
again
when
we
catch
this
error
and
get
it
back
in
our
food
delivery
service.
We
know
exactly
what
failed
and
how
to
deal
with
a
particular
air
all
right.
A
So
we
come
to
the
demo
part.
In
this
case,
I
wanted
to
demo
the
resilient
server
orchestration
by
showing
actually
a
temporal
workflow
that
invokes
similarly
to
we've
seen
in
the
slides
before
a
couple
of
our
services.
So
in
this
case
we
have
a
patient,
onboarding,
workflow
and
below
is
the
url
github
url,
where
you
can
clone
it
and
run
this
project
yourself,
all
right.
So,
let's
get
started
in
our
case.
What
we
have
is,
first,
our
services.
A
So
let's
go
ahead
and
show
you
that
we
have
a
patient
onboarding
service
which
has
three
endpoints,
assign
a
doctor,
assign
a
hospital
and
notify
a
patient.
So
basically,
with
the
service
that
we
want
to
write
or
the
core
business
logic
is
to
invoke
this
particular
or
orchestrating
locations
of
the
patient
onboarding
service
and
it
endpoints.
In
order
to
onboard
a
customer,
we
also
have
an
application
that
we're
running
that's
running
our
workflow.
So
let's
take
a
look
at
the
workflow
itself.
A
The
workflow
is
written
in
java,
it's
an
onboarding
implementation
and
it
basically
has
a
bunch
of
different
activities
that
executing
in
order
so
first
we're
going
to
assign
hospital
to
a
patient,
we're
going
to
assign
a
doctor
to
a
patient,
we're
going
to
notify
the
patient
that
you
know
we
have
assigned
them
to
a
doctor
and
the
hospital.
And
finally,
we
have
a
final
onboarding
step
that
we
want
to
perform.
So,
let's
take
a
look
at
our
application.
Also
has
a
ui,
so
our
ui
is
basically
you
know
very
simple.
A
We
have
some
patient
information,
so
let's
say
we
have
a
customer
called
a
patient
called
john.
Let's
see
he
has
a
zip
condition,
so
they
have
asthma,
john
at
com
and,
let's
say
555
and
prefer
contact
method
is
text.
So
once
we
actually
press
this
on
board
impatient
we're
going
to
communicate
with
the
temporal
server
to
ask
you
for
a
creation
of
a
workflow
instance.
A
A
So,
for
example,
we
are
signing
currently
hospital
to
john
we're
signing
a
doctor,
and
this
is
again
per
our
instructions
in
the
workflow,
we're
notifying
the
patient
and
then
we're
going
to
finalize
the
on
board
once
we're
done
with
that,
we
can
see
that
our
ui
updated
and
we
see
that
our
patient
john
was
assigned
a
hospital
and
a
doctor.
It
has
been
successfully
onboarded
if
we
look
at
the
temporal
web
ui.
A
We
also
see,
for
example,
that
we
have
a
particular
workflow
that
is
in
status,
complete
it's
our
onboarding
workflow
and
we
also
can
have
the
history
such
as
we
can
see
the
input,
which
is
the
form
input
that
we
typed
into
our
form
as
well
as
can
see
the
results
there.
Also
we
displayed
on
the
page.
In
addition,
as
we
talked
about,
this
is
the
actual
history
of
all
the
execution
information
that
was
stored
within
the
temporal
server
and
is
able
to
have
the
ability
to
actually
resume
workflows
from
a
particular
failure.
A
Let's
say
they
have
headaches,
and
I'm
just
gonna
type
in
something
here
now.
What
we
want
to
do
is
show
failure.
So,
let's
say,
for
example,
we
start
our
workflow
execution
and
we're
going
to
stop
or
fail
our
services
during
the
execution
or
workflow.
So
what
we
have
done,
the
services
their
workforce
has
to
communicate
to
have
an
intermittent
failure.
As
you
have
seen
here,
our
ui
has
stopped
showing
progress
and
we
don't
have
this
patient
on
board.
Now.
A
If
we
look
at
the
temporal
web
ui,
we
see
that
even
though
our
services
have
failed,
we
have
not
failed
our
workflow
execution
and
we
still
see
the
onboarding
workflow
in
this
case
the
new
one
is
running,
so
let's
go
ahead
and
bring
our
services
back
up.
Let's
say
our
failure
has
been
fixed
and,
let's
see
what
happens
and
again,
we
did
not
write
any
code
for
this.
We
did
not
have
to
specify
in
our
workflow
anything.
A
This
comes
by
default
by
using
temporal
and,
let's
see
if
temporal
is
able
to
deal
with
a
failure
see
now
our
workflow
has
resumed.
Once
our
services
came
back
up,
we
were
retrying.
Basically,
our
workflow
was
retrying
to
invoke
these
activities
as
soon
as
they're
back
up
the
retry
stopped
and
we
were
able
to
onboard
mary
into
our
system
and
again,
if
we
look
here,
we
can
see
now
that
our
workflow
has
completed
all
right,
that's
all
I
had
for
today.
I
hope
you
guys
enjoyed.