Rust Programming Language RustFest Global 2020, 23 Dec 2020

Previous Meeting

⏯

youtube image

►

From YouTube: κeen - 言語自作を通して学んだRust — RustFest Global 2020

Description

English title: Rust learned through self-made languages

本講演では言語自作を通して学んできたことをベースに、Rustに入門したての段階でのありがちなトラブルや意外と知られていない機能、Rustコンパイラの進化などを紹介します。Rustの入門体験記は色々ありますが、1つのテーマに沿って何度もプログラムを書いたことで徐々に問題へのアプローチが上手くなっていく点や、言語実装者が新たな言語を学んでいくという側面にも触れられたらなと思います。

More at https://rustfest.global/session/19-%E8%A8%80%E8%AA%9E%E8%87%AA%E4%BD%9C%E3%82%92%E9%80%9A%E3%81%97%E3%81%A6%E5%AD%A6%E3%82%93%E3%81%A0rust/

A

Let’S begin., the title of this presentation is ‘rust: learning through the language itself’.

A

My name is keen.. I have a twitter and github account and use rust at the company, idein.

A

The image on the right is my twitter and github icon.

A

I’M the author of ‘a practical introduction to rust’ published in japan.

A

I helped translate the initial rust programming language and their old official website.

A

Before my introduction to rust,, I created various toy languages.

A

For example,, I made a schema-implementation commit, created a scala-type lisp,.

A

And wrote a whitespace complier.

A

I like functional programming languages like ml and lisp.

A

This is because algebraic data types and pattern matching are suitable for writing. An interpreter.

A

I moderately like c,, since you can quickly use it as a beginner.

A

But functional programming, languages and c have pros and cons.

A

Functional languages have potential and are safe, since they are adt and pattern. Matching.

A

But they’re not good for quick interpreters due to its runtime.

A

Conversely, c is quick and has no runtime,, but it’s, not safe..

A

I was searching for an adt and pattern, matching language that was safe, quick, and had no runtime.

A

One day, I came across rust..

A

This was during rust, 0.12.0 just before 1.0 alpha was released.

A

Rust came with all the functions I listed,, so I could use it to create languages.

A

These are several of the languages I created with it: first,. I created a lisp dialect,.

A

Then, after a year,, a shell script,, a rust-type small lisp,, a sml complier,.

A

And an in-memory webassembly assembler.

A

Then, after a year,, I created an alt json schema.

A

There were others,, but we’ll leave this for now.

A

I wanted to talk about all of them,, but I only have time to introduce the key takeaways from three languages.

A

These are the three languages.. I learned the basics from klisp.

A

Then I learned maintenance through webml.

A

Finally, I learned how to make a product with chema.

A

I'd like to talk about these three cases.

A

Let’S start with klisp.. This was created to interpret skklisp.

A

Skk is a japanese-language input method,, but it’s irrelevant to this discussion.

A

This was my first project with rust.. Initially I was keen to create this in four days.

A

In japan there’s a four-day holiday in fall,, so I wanted to create this then.

A

I thought it’d just take 4 days because I had created many lisp-types before.

A

The code on this slide is klisp..

A

I called it with a skk gadget unit conversion function to do a calculation to convert miles to kilometers.

A

Writing an interpreter requires the basic functions of the host language.

A

For example,, the language function requires structure,, method,, enums, and pattern matches.

A

The standard library requires iterators and hashmap.

A

I wrote an interpreter while commanding these basic functions.

A

This ended in failure., it wasn’t finished in 4 days.. The reason for this failure was several challenges.

A

Specifically, they were garbage collection, lifetime or ownership, and error handling.

A

This is when I discovered rust is a difficult language.

A

This is what set me back with gc..

A

I expressed the lisp expression with the enum expr, using box in the pointer that appears here.

A

The code snippet below defines the enum expr. then,. The second variation holds two pointers with cons.

A

I used box in these pointers.

A

This didn't work because rust has no gc.

A

That slipped, my mind, since functional languages, have gc.

A

This failed because the value of box in rust was private,, but my implemented lisp had a sharable value.

A

Instead of box, rc needs to be used.

A

Reference counting (rc) will manage your data.

A

The code snippet below is almost the same as before,, but the pointer held by cons is replaced by rc.

A

This is the correct approach.

A

The next issue was lifetime or ownership.

A

This was a new concept for me, and maybe for you, too.

A

When defining the function,, writing is simplified by taking ownership with arguments,, but calling is harder.

A

Let’S look at the eval function on this slide.

A

The top eval has &mut, env, and &expr in its arguments.. The second argument is referenced.

A

The bottom eval has &mut and expr in its arguments.. The second argument is not referenced..

A

The top with a referenced argument requires a clone to define it, but is easily called with just the value.

A

The bottom is handy, requiring no clone to define it,, but calling the eval is complicated and requires a clone.

A

At that time, I wasn’t sure which configuration was right.

A

Eventually, I realized the overall configuration was important.

A

The data is registered in its runtime,, which is why it has to keep running until the execution is complete..

A

Thus,, you cannot take ownership of it.

A

At the same time,, the borrow checker is lexical.

A

I got an error when typing get and insert for the famous hashmap.

A

The third issue was error. Handling.. I had no clue what I should do.

A

The initial code used panic in various places.

A

The code snippet below defines the k_add_aux function.. The return value of the function is expr.

A

I used panic for unexpected data in the function itself, when executing a pattern match for the argument.

A

If that didn’t provide the correct value,, I started to use option.

A

The code snippet below defines the read_aux function.. The return type of the function is option.

A

Next, I learned how to use a result type for errors.. I tried it,, but all the errors had string in it.

A

The code snippet below defines the k_funcall function.. The return type is result.

A

I used err(format!()) for unexpected data in the function when executing a pattern match for the argument.

A

At last, I understood the correct method..

A

I read andrew gallant’s, blog, ‘error handling in rust’.

A

I memorized the definition of each error type through this blog.

A

The code snippet below defines the enum called error and expresses the error occurring in klisp with a variant.

A

The key takeaway from klisp was the basics of rust.. Then I learned it has several pointers.

A

I used references,, boxes, and rc with klisp.. I also learned about ownership, lifetime, and data configuration.

A

Finally, I learned about error handling.

A

A

Webml is a standard ml to webassembly. Complier. webassembler-rs was a side project for this.

A

This was my largest hobby project. it’s. Also my bonsai project.

A

This image is a bonsai tree,, which you take care of over a long time.

A

Just like a bonsai tree,, I kept working on this project, taking care of it. When I had some time.

A

The colossal size makes creating webml difficult.. The gigantic code base must be well-organized.

A

Also,, the syntax of sml is more complicated than lisp,. So, unlike lisp, you can’t handwrite a parser.

A

Finally, many types are quite similar to a complier.

A

A complier can be tricky in the first place,, but this talk will focus on just three points.

A

Sml has syntax like in the code written on this slide.

A

First, infix /::/ \::\ in the first line announces that it’s the infix operator.

A

This can create an infix operator within operator.. It defines the infix operator for \: as fun innit\:operator.

A

The formal argument is defined on the right and left.. This unusual syntax is sml.

A

Code organization is the timing to use submodules,, which is also called directory/mod.rs.

A

It was complicated.

A

I wanted to announce mod.a within b.rs, but couldn’t,. So I didn’t know where to put mod.rs.

A

I eventually figured this out through trial and error and compiled this in my blog.

A

Next let’s look at parsers.. The sml parser is not a handwritten one.

A

I used a parser combinator library. namely, nom.

A

At the time, it was version 2.0,, which was full of macros.

A

There was no impl trait then,, so its performance was at this level.

A

On this slide is the parser code using nom.

A

In the named! macro argument was do_parse!,, separated_list, and opt!.. There are too many macros.

A

The number of macros on the api document are impressive.

A

This is the nom api document., an impressive number of macros are defined in the macro column.

A

This isn’t a matter of 10 or 20. rather 50 to 100 macros are defined.

A

I write a parser using this,, but the error messages in the macros were difficult to understand or debug.

A

There’S a wide variety of types., many of the compliers are similar but have subtle. Differences.

A

For example, untypedexpr,, untypedcoreexpr, and typedcoreexpr.

A

All of these are rather big and have over 10 variants. at first,. I wrote these by hand.

A

Through trial and error,, I realized I could do something with a type alias.

A

The code inserted below is a bit of a mess, but it’s a solution using a type alias.

A

This prepares an explorer type to change its behavior with the type parameter.

A

Type parameters are named by their desired behavior., I.e. untyped. Expr allows empty in expr.

A

In coreexpr,, empty is allowed by untypedcoreexpr and type is allowed by typedcoreexpr.

A

The rust-type alias are flexible, so they can easily be defined.

A

For example, coreexpr is an expr alias as well as an alias for untypedcoreexpr and typedcoreexpr.

A

To define the method in both untypedcoreexpr and typedcoreexpr,, you just add coreexpr to impl.

A

Traits like display implemented in all expr should be given a type parameter in the generics.

A

The key takeaway from webml was that my experience with interpreters is helpful to create a complier.

A

Internal data types and lifetime configuration were straightforward.

A

As mentioned, the module system was complex and debugging. A code written with macros is tricky.

A

I learned you want to use impl trait asap. also,. I found the rust system is great.

A

This is a slight digression,, but when I switched to this job, where I used rust at work,, my skills skyrocketed.

A

This is when I moved onto chema.

A

Chema is pronounced, like schema without the ‘s’. it’s, an json schema complier from an original notation.

A

It can also be called an alt json schema.

A

This had to be useable, since it was for a company project.

A

Below is a description of chema.. You write a comment with /* ~ / and define the type with type name =.

A

First, chema requires you to create a complier. at that point,. This was not a problem for me.

A

I focused on elements that didn’t work on webml and then brought them back home to webml.

A

Also, I had to release a command line, interface and binary.

A

My colleagues would be collaboratively using this,, so it needed an enriched command line. Interface.

A

This means a command line argument parser is needed.. I adopted structopt. here is code using structopt.

A

You add #[derive(structopt)] to the structure, definition and command the #[structopt()] attribute in each field. Definition.

A

Long and help were typed in the attributes. long commands a long list of options.

A

This commands no-swagger,, so it commands an option written as “no-swagger”.

A

Help is the message that appears with “help”.. The help message is generated with structopt.

A

As explained, “no-swagger” and input file are generated.

A

Next is the binary release., creating distributions and uploading it to github is time. Consuming. cross-compiling is also an issue.

A

Ci alone won’t do,, so I incorporated trust.

A

Trust easily creates binary releases for complicated targets using a travis ci template configuration.

A

Let's look at the release of chema.

A

As this shows,, you can easily create releases, like this, on various platforms.

A

For example,, you can create the binaries aarch64 on linux, x8664, on linux, x86 on apple darwin, or x8664, on netbsd.

A

The key takeaway from chema is that some tweaking is required to create a product.

A

Also,, it comes with what you need to tweak it.

A

Let’S review a bit before concluding., I developed my skills as I wrote. Various languages.

A

At first,, I kept wondering how to write with rust.

A

Then I became more attentive to what the overall code should be and how to release it as an entire project.

A

Some, you may have noticed this,, but the rust update solved several of the obstacles I encountered.

A

The impl trait I set to memory were released with 1.26.0.

A

The complicated module system and inflexible lifetime was solved with the 2018 edition.

A

To summarize, you can discover most elements of rust by first trying to write a lisp interpreter.

A

You can slowly dig into these details from any approach by grappling with the same issue. Repeatedly.

A

Once you have the overview you can get into the specifics. finally, rust is a continually evolving language.

A