►
Description
At Aleph Alpha we strive to create a sovereign, European AI tech stack. We use Rust to build our API. In doing so we also stumble upon Problems outside of the Domain of AI and Large Language Models. One of these sparked the Open Source `async-events` crate, which might also be helpful in your Distributed System.
--
Markus Klein is the Engineering Lead at Aleph Alpha. He spend most of his career as a C++ Developer, working in the Domains of Distributed Systems and Data Engineering. He is using Rust as his go-to Language for private Projects for eight years and is the core maintainer of several crates, with his most meaningful contribution to the Rust ecosystem probably being `odbc-api` crate. He also finds it strange to write about himself in third person.
A
Yeah
so
hello,
I'm,
Marcus
I
also
like
to
do
a
lot
of
Open
Source
work,
so
my
GitHub
handle
is
Pacman
82..
Most
of
my
open
source
work
happens
in
in
Rust
and
I.
Think
my
most
meaningful
contribution
is
probably
the
odbc
API
bindings,
so
odbc
is
for
accessing
databases,
standard
and
I
maintain
the
bindings
for
us
for
it.
So
data
Engineers
might
also
know
my
odbc
to
parquet
tools
and
there's
a
few
other
crates,
but
I
won't
go
in
there.
A
So
what
do
I
want
to
talk
to
you
about
today,
for
once
I
want
to
talk
to
you
for
what
we
use,
rust,
add
alif,
Alpha
and
then
I
want
to
drill
a
deeper
into
two
specific
problems
which
also
spawned
the
creation
of
two
open
source.
Crates
and
I
want
to
share
the
problem
with
you
so
that
you
might
know
how
to
pick
them
up
or
improve
upon
the
solution.
A
A
A
However,
what
I've
prepared
here
is
a
different
kind
of
task,
not
an
embedding
task,
but
a
completion
task
which
what
you
see
here
is
a
slightly
younger
version
both
of
our
UI
and
myself,
and
this
is
a
completion
task
based
on
the
multi-model
prompt
multi-model,
because
there
are
two
modes
of
input
here:
one
is
text
and-
and
one
is
a
picture
and
the
model
is
trained
on
text
and
what
it
will
do.
It
will
complete
the
text.
So
it
will
just
continue.
A
Writing
the
text
and
the
text
is
phrased
as
question
q
and
then
there's
an
a
for
answer,
so
the
model
is
primed.
Okay.
Now,
probably
there
should
come
an
answer
to
that
question
and
before
the
question
there's
a
picture,
and
so
that
resides
in
the
completion,
a
man
with
classes
in
the
build,
which
is
a
reasonably
accurate
description,
and
this
I
I
will
use
to
explain
what
we
do
at
alif
Alpha.
A
So
how
does
Factor
rust
into
it?
So
at
alif
Alpha
we
are
fairly
young
company,
so
there
has
never
been
at
least
from
my
point
of
view
in
alif
Alpha
without
rust,
I
was
hired
via
Russ
newsletter.
That's
how
I
discovered
the
company,
and
so
I
cannot
tell
you
a
tale.
Oh,
this
is
all
the
production
errors
we
had
and
when
we
switched
to
Russ
Norman
fine
in
that
regard
it
was
always
fine.
What
I
can
tell
you,
though,
is
what
we
are
using
rust
four
and
what
we
are
not
using
it.
A
A
Also,
the
the
prototypes
quick
pocs
are
usually
also
done
with
fast
API
and
python.
What
we
do
do
in
rustvo
is
inference
pre-processing,
so
inference
is
the
process
of
actually
using
these
models
in
in
production
and
what
the
user
says.
It's
not
quite
what
goes
into
the
model.
So
always
text
has
to
be
converted
to
numbers
in
various
ways,
but
rust
is
very
well
suited
for
this
job.
A
Also,
data
pipelines,
a
large
these
models
are
trained
on
larger
amounts,
terabytes
actually
of
data,
and
there's
a
lot
of
pre-processing
and
command
line
tools
or
stuff
it
goes
into
which
does
it
and
it's
reasonably
fast
doing
it
in
Rust
and
yeah.
Basically,
if
it's
done
by
the
engineering
team
at
alif,
Alpha
and
there's
no
good
reason
to
use
another
language,
it's
usually
done
in
Rust
and
all
the
boring
stuff
we
do
in
Rust
and
the
nice
thing
about
it.
So
that's
not
as
negative
to
to
me
as
it
may
sound.
A
And
in
short,
I
could
summarize
it
that
we
use
Python
for
fast
idea,
iteration
and
reuse
rust
for
production,
so
somebody
wants,
you
were
iterating
over
ideas
fast.
He
doesn't
want
to
be
buffered
by
oh.
What
would
happen
if
that
would
go
wrong.
He
just
wants
to
move
on
to
his
next
plot
to
his
next
idea
and
in
Rust
for
production
code.
I
actually
want
to
be
buffered
with
everything
that
could
go
wrong
and
this
design
is
actually
not
quite
right.
A
A
Another
thing
that
I
found
interesting
was
hiring
rust
developers.
It
turned
out
well,
I
thought
this
would
be
hard
and
in
a
sense,
I
was
right,
but
the
last
three
developers
which
were
hired
they
had
didn't
use
rust
before
at
all,
but
still
in
their
second
day
after
wearing
companies
commits
remember
already
running
in
production.
Pair
programming,
of
course
helps
a
lot
with
this,
but
I
I
reasonably
cannot
see
anymore.
That
rust
is
hard
to
learn.
People
can
pick
it
up
really
really
quickly.
A
What
is
actually
hard
to
learn
is
maybe
ownership
model
or
software
design,
but
these
were
Concepts
that
existed
before,
right
and
and
a
C
plus
plus
programmer,
who
understood
ownership.
He
can
just
pick
up
rust,
really
quick
and
yeah.
So
for
me
I
would
say:
iron
grass
developer
is
hard,
but
rust
doesn't
Factor.
You
know
into
it
because
I
don't
hire
rust
developer.
We
just
hire
good
software
engineers,
and
that
was
always
been
hard,
but
previous
knowledge
of
rust
itself
is
is
not
required.
They
pick
it
up
all
fairly
quickly.
A
Okay,
moving
on
to
the
open
source
grades,
which
have
been
spawned
by
little
problems,
we
had
at
alif
Alpha,
but
one
honorary
mention
must
go
to
the
alif
alpha
rust,
client
right,
so
the
majority
of
our
features
is
actually
covered
in
in
the
python
client.
The
rest
client
is
more
like
a
Spare
Time
rainy
day
project
of
myself
everybody's
invited
to
contribute.
A
A
A
A
worker
might
ask
for
work.
This
is
just
one
example
of
how
it
could
happen
and
when
a
user
sends
sends
a
completion
task,
the
API
does
a
bit
of
work
and
answers.
The
worker
's
request
for
work,
because
now
finally,
where's
work
in
the
queue
and
then
the
worker
answers
for
completion,
and
now
we
wanna
send
for
completion
to
the
user.
Okay.
A
Why
is
this
a
problem?
What
what
could
be
a
bit
of
a
problem?
Let's
consider
a
code
sample
and
in
this
code,
sample
I
switched
to
architecture
a
bit
now.
I
just
have
a
worker
which
is
a
service
I,
sent
him
a
completion
task
and
I
evade
the
response,
and
now
I
can
simply
wait
for
whatever
result
I
want
to
have
in
in
the
Handler
of
my
service
and
all
is
good.
A
So
I
can
wake
up
here,
and
the
question
is
what
do
I
wait
here?
What
what
is
the
future
I'm
awaiting-
and
it
turned
out
to
me,
but
I
already
solved
these
problems
some
time
ago.
This
is
the
readme
of
a
throttle.
Semaphore
server
smf4
is
a
synchronization
primitive,
like
a
generalized
version
of
a
mutex.
A
And
this
is
a
throttle,
is
a
semaphore
service
for
a
distributed
system,
not
itself
a
distributed
system.
This
is
important
distinctions,
but
usually
that's
what
you
want.
People
always
ask
for
distributed
mutexes,
but
they
want
actually
a
mutex
for
a
distributed
system,
and
actually
it
has
a
deadlock.
Detention
by
enforcing
lock
hierarchies
one
of
the
touching
points
to
a
talk
earlier
and
let
if
we
look
what
throttle
does
actually
well,
it
has
to
have
at
least
two
clients,
so
it's
fun.
So
we
can
wait.
A
We
wait
on
each
other
and
then
let's
say:
okay,
client
B
starts
asks
for
a
mutex
and
acquires
a
lock
and
then
keep
specific
for
lock.
Meanwhile,
client
a
asks,
but
throttle
car
will
answer
him
right
away.
Okay,
you
don't
get
it
or
delay
the
answer
and
after
a
while
client
B
releases
the
lock
and
then
it
gives
you
a
quiet
lock
to
client
er
a
and
this
second
part
of
a
flow
diagram.
The
part
how
we
answer
and
what
we
would
wait
for
in
the
Handler.
A
That's
exactly
the
same,
and
so
I
extracted
that
part
out
of
throttle
into
its
own
Asic
events
rate.
So
the
name
actually
I,
don't
like
it,
which
is
probably
a
hint
I,
don't
fully
understand
what
I
did
to
you,
but
at
least
it
served
me
well
in
in
on
two
occasions,
so
I
thought
I
might
just
share
it
with
everybody,
so
it
could
be
useful
and
remarkably,
once
I
extracted
it
I
discovered.
A
Okay,
this
depends
on
nothing
right,
I
don't
depend
on
talk,
I
o
or
standard
essing
or
anything
it's
completely
independent
of
the
environment,
which
just
I
think
goes
to
show
how
how
smart,
actually
the
rust
asyncavate
system
is
designed
and
how
well
the
the
execution
is
separated
from
from
the
usage
I
I
was
really
remarkably
of
ferric
about
that.
Although
I
only
use
talk,
I
O
so
and
how
does
it
help
us
with
our
problem?
A
A
The
structure
is
generic,
so
I
could
basically
do
any
kind
of
result,
including
rust
results
to
transmit
errors
and
also
the
ID
type
I
want
to
use
to
identify
the
event
and
when
I
give
it
to
methods
so
on
on
the
completion
route,
which
is
basically
the
sending
work
route
I
get.
Of
course,
the
prompt,
which
is
the
input
I
generate
an
ID,
then
I
need
to
send
the
work
to
the
worker.
I
will
omit
how
we
do
that.
A
The
second
artifact,
which
is
also
is
called
log
hierarchy,
so
personally,
I'm
a
big
fan
of
vector
model
and
usually
the
fix
for
me-
is
actually
to
switching
to
the
actor
model
because
obvious
State
got
too
confusing.
But
sometimes
you
just
say:
okay,
we
don't
have
time
for
that.
Refactoring
or
sometimes
actually,
not.
A
But
what
I
decided
to
do
was
not
to
search
for
actually
detecting
Deadlocks
at
runtime
just
catch
it.
One
level
earlier
detect
any
violation
of
the
lock
hierarchy.
The
fun
thing
there
is:
it
becomes
quite
deterministic
and
I've,
even
a
little
demo
prepared
for
that.
So
in
that
case,
no
realistic
demo,
just
one
of
these
academic
examples,
probably
the
most
academic
examples
of
them
all
is
because
it's
about
philosophers
sitting
around
a
table.
A
A
Let's
create
a
new
shell
and
let's
go
to
the
philosopher's
example.
I
think
it's
from
some
older
version
of
a
Rust
book,
I
modified
it
slightly.
So
the
philosophers
will
start
the
eating
thing
over
and
over
again,
I
just
run
it,
and
we
see
here
some
output,
where
five
philosophers
sitting
around
the
table,
they
all
eat
and
at
one
point
it
just
stops
and
that's
due
to
a
Deadlock.