DEV Community: Bastian Gruber

Announcing my book "Rust Web Development"

Bastian Gruber — Tue, 09 Nov 2021 20:31:20 +0000

A pragmatic and "in-depth-enough" book about writing web services/applications in Rust

Hey folks! Long time since I have posted here! My original articles on this platform kicked it all off for me: Getting into Rust, trying to explain to myself how to think about Rust when coming from NodeJS etc.

One thing lead to another and now I am currently writing a book with Manning titled "Rust Web Development". I want to take you on a journey into Rust and how to think about Web Development when working with Rust.

Ready for prime time?

Many things are new, different and at first glance, not as easy when comparing it to Java or NodeJS. However, once you understood the moving pieces, Rust is a joy to work with, a super solid language and enough great tooling to use it, right now, in production.

It also has some tooling from the ecosystem which make it, in my view, a no-brainer to use for any future web service you are going to push to production.

So in this short post, I want to invite you to come along with me on this journey, and figure it out as we go along converting to Rust! The language itself will eventually be used in the Linux Kernel (fingers-crossed), replace legacy systems and will find many friends in the new world which is currently built right before our eyes (Facebook calls it the metaverse...).

See for yourself

There is a website to the book, where I hopefully soon publish more little articles about Rust and the web, and I will list all talks I gave and Podcasts I am on, so you can learn more about this exciting new way of building for the web.

And, if you made it this far, with this link you will get a discount purchasing the book.

It is currently in MEAP, but will be ready starting 2022.

I am looking forward to your feedback!

Getting paid in Bitcoin - Experiences and misconceptions

Bastian Gruber — Wed, 03 Jun 2020 11:20:38 +0000

For the last year or so, I worked as a self employed consultant for companies based in the US. Therefore, I was getting paid in a foreign currency.

Now, I never encountered a situation like this and quickly found out how expensive it is to move money across borders and banks. Now my wife is Canadian and we both live in Germany, so I experienced the hassle of getting money inside the family across the border from second hand.

Getting paid for a job is a little different though, because:

You have to pay taxes
You want it in a timely manner
You don’t want to pay a lot of fees

Luckily, I had the option between US Dollar and Bitcoin, so I tried both for a while to see what’s the cheapest and fastest way for me.

Getting paid in USD vs. BTC

A rough TL;DR looks like this:

If your company has to buy Bitcoin to be able to pay you in Bitcoin, it is more expensive then just accepting US Dollar
If your company already owns Bitcoin, it is way faster and cheaper to get paid in Bitcoin vs. in USD
If you have to transfer your Bitcoin to Euro (which you probably have to do to pay rent etc.), it is still cheaper to get Bitcoin, sell it on an exchange and get the money onto your bank account. To some degree it is as fast or even faster then USD->EUR
You have to use Transferwise to be able to get the cheapest and fastest way to get paid in USD and transfer it to EUR, everything else is way more expensive and slower then to use Bitcoin

I am not a big Bitcoin maximalist, but I like the idea of programmable money. Having worked for larger financial institutions, I am aware of the tech stack and slow processes banks have to fight against.

Numbers

Lets say you bill your client 10k USD. If you want to transfer this amount of money to EUR:

Transferwise: 51.18 USD.
Bitcoin: Around 30 Dollar (give or take) where the transport from one address to the other can cost you around 2 Dollar, and around 26 Dollar to sell the Bitcoin onto the market (0.26% at some exchanges).

Transferwise vs. Bitcoin

Why using Transferwise? Well with this service, after creating an account and verifying yourself (which can take up to two days), you can create US bank accounts and EUR bank accounts with a few clicks.

After you have a virtual US bank account with Transferwise, you can give these details to your company or person which pays you. After receiving the Dollars, you can then convert them to EUR (based on the conversation rate + fee on Transferwise), and then transfer the money to your own bank account in Germany.

Transferwise is cheaper because they basically trick the banking system. They won’t immediately send the money across borders. They store transactions and make larger bulk transfer after a certain timeframe (lets say after each month). Transferwise has partnered with banks around the globe, so they can internally book transactions, and then rebalance after each month.

These bulk transfers make it faster (because they just internally move money) and cheaper (because of the bulk transfers each month).

Now it is pretty mind boggling to argue against Bitcoin when you have to cheat your way through the banking system to be able to compete against a, by design, slower method of transfer.

Experiences using Transferwise

The service itself is great, cheap and fast. I used it a few times and my Dollars got converted and were on my bank account in Germany in Euro at the same day.

Getting paid in Bitcoin

Lets assume the company which pays you already has Bitcoin. Therefore the speed of transaction is around 2-15 Minutes based on my experiences. Once the company send the transaction, you can follow it “live” on the Blockchain and see when it should arrive. I never felt so much in control of my money then when receiving Bitcoin.

I used an address on an exchange, so I could, after receiving BTC, sell it and transfer the money to my German bank account. Now I would love the price of BTC to be stable and my landlord and other suppliers to accept it, but for this to be happen, the price of BTC has to rise a magnitude of what it is now.

The whole process of receiving BTC, selling it and getting the money to my bank account was in a range of 5-8 hours. The most time lag came from sending Euros from the exchange to my bank account.

What about taxes?

Now people assume you gamble when you are getting paid in Bitcoin. That’s not true. It’s just another currency. Instead of being backed by the largest military in the world, it is in the hands of everyone who want to participate in it.

The German government doesn't particularly care in what and where you are getting paid. Which means:

You have a contract with a company or you write a bill
After you receive the money somehow, you have to declare this as income in your taxes

So even if you get paid in apartments on a Greek island, you still have to pay taxes on the value of this payment in EUR, in Germany after the end of the year.

Now when you receive Bitcoin and you sell it, you can either make a loss or a profit (or you sell it for the exact same price you received it). If you make a profit off this transaction, you have to pay taxes on this profit as well. If you make a loss, you can even reduce your income and pay less income at the end of the year. That's it.

Summary

All in all, it's actually not very exciting. I chose Bitcoin because it was faster and I believe it's good to get familiar with the future of money sooner rather than later.

If you want to get paid in Bitcoin, make sure you choose an exchange you can trust, and don't let Bitcoins pile up at your exchange, but rather use a hardware wallet to store them.

So if your company owns Bitcoin, it is faster and cheaper to get paid then it is in a foreign currency. If your company operates in Europe and you could already choose EUR, you will pay a higher fee to get paid in BTC if you have to sell it (which you probably will).

Side note

Be sure you understand that Bitcoin might not compete with the European banking system, and there are cheaper and faster currencies out there. Bitcoin could be used as a transition currency for the old FIAT world to move over to the digital space, and to be established as a form of reserve currency. But, as of now, it is a nice and easy way to cross borders with it.

Experiences of working fully remote

Bastian Gruber — Wed, 29 Apr 2020 10:11:59 +0000

Info: This article was written in December 2019, before the pandemic hit the world.

My personal biggest factor being a software developer is to be able to work at the cutting edge of technology + industries and generally think about how life should look like in 10 or 20 years from now.

One of the biggest aspects, at least in my opinion, will be how we see companies, jobs and our current system. I am meeting more people who don't want to join a big brand (lets say Amazon or Google) if they have currently a better lifestyle/money ratio.

How I see it is, that we are fortunate enough to earn a decent living while being able to dictate (to some degree) how we want to earn it. This might not be forever, or it might be the future, we will see.

So, being a freelancer for a bunch of years now, I wanted to take the next step in my "dream working life" and trying remote work. There are actually not that many projects or companies who offer a complete remote setup. I was lucky enough to have found one, and I was working remotely in a complete remote company for more than 6 months now. Here are my experiences.

The "Get dressed and showered" myth

Many people told me to get dressed, walk around the block, have a routine to start the day. For me personally, this didn't hold true (maybe just yet?). Having a small daughter at home, the nights can be up and downs. I enjoy the most to hold her in the morning until my planned start of the day, without having any delay in between. So sometimes, I don't get showered or properly dressed until the evening.

If I feel a mental road block, I use the showering and eating time to unblock myself
I can work from 6am on until noon, go for a run, and shower and eat then
If I had a rough night, I need a shower and proper breakfast to start the day

Maybe I am still new to the game, but my days vary a lot and how to get started and inspired. Sometimes I work on super interesting tasks where a spark comes up in the morning and I don't want to waste time with other things. That's the beauty of having your work environment within a 1 minute reach.

I love text meetings

The biggest factor of working remotely is to ask people via a team communication tool (whatever this might be). I work with people in different time zones so to be able to just ask a question, and have an answer when I wake up is sometimes a perfect start to the day.

I can always scroll back days, weeks and even months to rehash a discussion. I was sick lately for a week and had to dive back into a complex tasks. Instead of having the office noise around me, I could take a coffee and re-read my conversation with one of my colleagues a week ago to get back into the mindset.

It's hard to unlearn "just being there"

On some days, nothing really flows. When you are in an office and get paid for sitting in chair no matter what, you just wait a few hours, try to get into the zone. But sometimes, you have meeting interruptions etc. So everyone knows: Nobody is 100% productive for 21 days a month. And, as a software developer and depending on personal profile, you can make up for 1 or 2 days of not-productive work in a few hours.

At home, that's different. When I hit a road block, I have to write down why I am blocked and make an active choice about it. Do I take a break, go for a run and try to solve the problem later ones it could sit a bit with me? Or do I have to contact colleagues to unblock me?

At the office, a colleague will say something or you chat in the coffee break and this sometimes help your mental flow. Not at home. Nobody is here to help you out. This is one of the reasons why companies might just want to hire more Senior Developers, because you have to tell from experience which type of block you are facing and take ownership of it.

Feeling the vibe of the team

It is really hard to see where the team is at. Like you can have a few bad days and really feel you are letting the team down? Are other people also having slower days or weeks? In an office, you see the mood of the team. Just before christmas when people are more talking about the christmas events then actually being productive all the time. It's part of the social office experience.

But at home, every day is the same. Even if it's not, you can't really tell what other people are going through. Reaching out, just checking in on people with a big disclaimer that they don't need to answer right away is a big help.

Block your most productive hours

That one is a biggie. If you don't have a family, you literally can set your own working times. Lets say you work best from 5am to noon? That's alredy 7 hours of work. Nobody distracts you when you sent your team communication tool on silent and close your E-Mail application.

Need to get some serious work done? Tell the team that for the next few days you are just available for emergencies.

You have to motivate yourself

You have to know yourself quite a lot. What gets you out of bed? What are the signs that you don't want to get out of bed? Are you in a rut, depressed etc? Untangling your emotional state, finding this intrinsic motivation can be a challenge sometimes.

Isolation

Many people I met still prefer going to the office. It creates routine, structure and social contact. All of this has to be done manually from you if you work remotely. Meeting friends, setting working hours for yourself, actually getting work done. It's sometimes not easy, especially if you have to follow up on defining work etc.

It can be lonely, and a week can pass by without you having realized you actually didn't meet anyone. So making sure to join movie or football groups, meeting up with friends, leaving the house at least once day etc. can help to keep you going.

You can't work remote in a semi-remote company

I worked "remotely" at my clients projects here and there, but this doesn't work. Why?

The team and company still have the "office chair == being paid" mentality. If they don't see you, they think they pay for something they don't get
You miss out on coffee break chatter and office mentality. You will be considered a bit of an outsider if nobody else is doing it
Meetings just don't work. People will chit-chat in between and you never know if this was important know or not (also sound and video quality!!)

Summary

There is a lot of talk around work, remote work and office culture in the last few years. We have to accept that not everyone sees work as their primary calling or target in their life. It's hard to do pure remote work if you are not motivated and need a combination of sort-of-meaningful work and social environment to function. Right now, people who work or want to work remotely have the following characteristics:

Don't need social games or hierarchies to function
Really love what they do, but need more flexibility to function properly
Don't agree with the open office culture
Don't see value in playing corporate games
Have a family or want to travel more
Don't want to move to big city to do interesting work

What you have to remember is: People might need a network (co-working spaces, MeetUp groups etc.) to do this type of work. Rent prices are increasing and not everyone has a spare room at home to build an office in.

Companies right now are still going to major tech hubs (San Francisco, Toronto, New York, Berlin and London in my experience) to build their company and hire talent. These cities however are getting so expensive that new talent just can't afford to move there.

Not to miss out on talent might be the biggest driver for companies to switch to a remote-first company. For everything else, time will tell. We are still social people and need connections. Work doesn't have the status for newer generations as it had for the Baby Boomer Generation. So if you like your colleagues, have a nice office and decent pay, an office job has many advantages.

If you already have a network or want to build your own one, want to join a company you can't reach in your current home town or advance your skills unavailable to location-wise, remote work might be a good fit.

Rust for the Brave and True

Bastian Gruber — Tue, 11 Jun 2019 13:12:29 +0000

A few years ago, I wanted to make the switch from Frontend to Backend. One day I fell down the YouTube rabbit hole and (partly) watched a few tech talks. I realised that many of the really good talks were held at Clojure conferences. I looked into it, and decided to learn Clojure.

The lust for learning a new programming language often times comes from the sheer boredom of your old work, so you also want to redo part of yourself. After a bit of research, I found the book called "Clojure for the Brave and True", and absolutely loved it.

After a few weeks however, I gave up. The book wants you to jump all in and learn emacs at the same time as well, since it provides, according to the book, the best environment. I really liked the idea, but abandoned the project also due to the lack of Clojure positions for Juniors in my area (Berlin, Germany).

Rust

A few years later, today, I started to learn Rust and got excited about the whole ecosystem, community and possibilities. I read, wrote and programmed a lot in Rust during the last few months. So much so that I got a Rust position at a company starting August (yay!).

Brave

But, as I digged deeper and deeper into the language, and watched tutorials, I figured: I can't do Systems Programming on a MacBook Pro running macOS. Don't get me wrong: Once you mastered a skill, it doesn't matter where you operate it on. Also a different machine doesn't make a huge difference.

Although, I might disagree a bit with that. Take loosing weight for example. Just cutting down a few kcal per day won't make a huge difference. Your fridge has to look completely different so you are less tempted to eat all the bad things and naturally will nurture your body in a more healthy way.

I remembered the "Clojure for the Brave an True" concept, and wanted to apply it to my Rust environment

This is exactly what I aimed for when I ordered a Thinkpad X1, with the intention to install Arch Linux on it. Instead of nurturing my body, I wanted to nurture my brain in the right way. If I want to learn about TCP/IP for example, I don't want to brew install the right package every now and then.

My reasons:

I want to spend almost all of my time on the command line
I realized that even macOS is constructed as a perfect source of leisure and wasting time: I wanted a pure work machine instead
My plan was to spend idle time not in front of a screen, but read a book or write
I wanted less distractions from my operating system
I wanted to feel closer to the machine

True

After receiving my Thinkpad, I removed Windows and installed Arch from a USB stick, following these steps. After 4 tries I finally succeeded. When I booted the first time, I installed i3 and alacritty as my terminal. Next to this, I installed Firefox for browsing. That's it. I need a terminal, a window manager and a browser. I barely touch my mouse any more.

What I learned from it

Using my browser to keep up with Slack and E-Mail, I finally reached my goal of just checking it twice a day (because it's just less convenient).
Just installing Arch made me touch so many aspects of an operating system (again), and it's not all that hard to understand certain areas
I can buy books like The Linux Programming Interface or TCP/IP Guide and dig deep into the system
I can read the 2600 magazine and follow along the tutorials
It made me less distracted during work, since I literally have no notifications or other distractions

Working under macOS, I constantly opened other programs while I didn't know how to proceed on a task. With Arch, it's much harder to do so (and less fun). I have a physical notebook instead next to me and I will, if I can't proceed on a problem, note down my thoughts in the notebook and stare out of the window to think again.

I am excited again to be closer to the hardware I operate on. Being able to master nvim and setup a proper tmux setup is so much faster then doing the same workflow inside Visual Code or other editors.

Rust Coding Environment

I am using:

With the following PlugIns for NeoVim:

That's it. You can easily setup a tmux session, split the window where you have your code inside nvim and on the other half of the window cargo run/watch.

Moving through code like that makes it really hard after a while to use a normal code editor again. Allthough you can install vim bindings to VS Code I guess.

2 Warnings/Hints

Always have a workable Live-USB stick with Arch installed with you in case of an emergency. I tinkered with the startup script and got myself into a loop. I had to boot from the stick, decrypt and mount my hard drive and then revert the changes to be able to boot again.
In addition to my Thinkpad, I operate a remote server (for around 10 Euros/month) for backups (via rsync) and other fun activities. It can help to have another machine for backups just in case you want to take a snapshot of finally perfect configured Arch installation.

Additions

In addition to the Thinkpad, I got a vertical mouse and a proper keyboard. Both of which even increase the fun of my setup. My monitor is a Dell 4k 27" which is hooked up on a monitor stand.

This is so nice to work with that I even don't miss the beautiful 5k LG UltraFine at all, which I previously had.

Summary

I provide the basic dotfiles and steps for my setup on my GitHub account. It's good to run contrary to certain environments. The iPad for example became so powerful that the casual user doesn't need a Notebook anymore to do every day tasks. That's a perfect time to reconsider your relationship with your tools you are using.

If you follow the steps, it's really simple to operate within Arch on a Thinkpad. You need to tinker with HDPI here and there, and adjust some hibernate functionality. But overall it's what I wanted: The bare minimum to get to a comfortable coding environment, with almost no distractions.

In the end, the Thinkpad also has a much better keyboard than the MacBook Pro. Also the form factor is just perfect. A bit bigger than the 13" and smaller than a 15" model.

Explained: Futures in Rust for Web Development

Bastian Gruber — Thu, 02 May 2019 09:44:50 +0000

As I explained in one of my last articles, Futures are one part of an asynchrounous environment. We need Syntax, Type and a Runtime to be able to run code in an non blocking way. Futures are the type part of the Rust async story.

Content

The two perspectives
What actually is a Future?
When to use Futures?
Why does using Futures save time?
How to use Futures?
What's so different or hard about Futures?
How to execute more than one Future?
What is the future of Futures?
Summary

The two perspectives

If you are coming from NodeJS, Futures in Rust don't make much sense. In NodeJS, everything happens asynchronously. Therefore for you to be able to say "Hey, I really need to wait for the answer for this GET HTTP call", you are putting.then() on a Promise, so you can make sure you just execute the code inside the .then() when the HTTP call is finished.

In Rust, everything is blocking and synchronous by default, so you might ask yourself: "Why bothering with the complexity, that's exactly what I wanted in the first place!"

Rust is a Systems Programming Language. So for you to craft applications in Rust, you always have to wear two hats. The "Systems Hat"(⛑) and the "Programmer Hat"(🎩). The Systems Hat(⛑) lets you consider what is actually best from a machine point of view, the Programmer Hat(🎩) takes care of the syntax and how a software is written.

If you are coming from NodeJS, the Systems Hat is taken care of from Googles V8 runtime, so you can focus on syntax. In Rust you get help from crates, although you need to make certain decisions yourself.

The Systems Hat is the reason why we want to use Futures. Therefore, you need to handle Futures as a type in your application, and afterwards make sure that you use a runtime to actually execute them. If you are consuming Futures (for example when a crate you are using is returning a Future), you have to understand where the data comes from. In this case, the Programmer Hat and the Systems Hat are needed.

What actually is a `Future`? 🎩

A Future in Rust is actually a trait, which if you want to implement it, looks like this:

trait Future {
    type Item;
    type Error;

    fn poll(&mut self) -> Poll<Self::Item, Self::Error>;
}

The poll method is important. This will get called from your runtime and will return Async::Ready or Async::NotReady.

If you want to fetch different magazine issues from a remote location or from the file system, you can create and return your own Future for that:

struct Magazine {
    issues: Vec<u8>
}

and impl the Future trait on it:

impl Future for Magazine {
    // here we return a single byte
    type Item = u8;
    type Error = io::Error;

    // this method is getting called from the runtime. Everytime we can read
    // a byte into the buffer, we return `Async::Ready`
    fn poll(&mut self) -> Poll<Self::Item, Self::Error> {
        let mut buffer = [0;1];
        match self.0.poll_read(&mut buf) {
            Ok(Async::Ready(_num_bytes_read)) => Ok(Async::Ready(buffer[0])),
            Ok(Async::NotReady) => Ok(Async::NotReady),
            Err(e) => Err(e)
        }
    }
}

The runtime will fill the buffer with the information you are asking for, and the Future will return Async::Ready once the buffer is full.

This is the basic essence of a Future in Rust.

When to use Futures? 🎩

Implementing the Future trait for a type is your way of telling the machine "Hey, this might take a while, consider this when executing this part of the code".

Rust is a synchronous language. Everything happens line by line, and waits until each result is processed. This is not a problem until you want to do tasks which take longer time to process.

If you create a micro service with Rust, and you need to fetch three other micro services for some data, and afterwards you merge the data and write it into a database, that's a use case where you want to consider using Futures.

Why does using Futures save time? ⛑

In our test scenario, each of these HTTP calls could take, lets say, 1 Minute. Instead of now waiting 3 Minutes for all of the calls to finish, you want to run them in parallel so you still just wait 1 Minute instead of 3.

Therefore you create three methods which return a Future each, collect them, and then pass them on to a runtime (to tokio via tokio::run for example).

How to use Futures? 🎩 + ⛑

If a crate returns a Future, you can use .and_then() to work with the result once the Future is ready. Using .map() on a Future lets you alter the type of the Future.

Inside the async Rust world, we have to take care of different types of data. Instead of using Strings and Numbers for example, we deal with streams of values. You will most likely deal with Streams.

A Stream is an extension of a Future. Where a Future is about producing a single value, a Stream is yielding values as long as they are present.

Stream: A Stream is, like a Future, a trait you can impl on a type. This lets you iterate over the return values (which are Some(_) or None).
Sink: For writing data continuously (in an asynchronous way) to either a socket or to a file system

So Streams are for reading data, Sinks are for writing them. We have two types of Streams in our web ecosystem:

Stream of Bytes (like HTTP bodies or TCP streams)
Stream of Messages (like WebSocket Frames or UDP Packets), where each message has a fixed size

Code example

Lets look at a use case where a crate you are using returns a Future. When doing HTTP calls, reqwest is a good example. On the Future returned from reqwest, you can use .and_then() to process the results (🎩):

// We have to use "r#" before "async" because "async" is a reserved keyword.
use reqwest::r#async::Client;

// The return type has to be `Future<Item=(), Error=()>` to be able
// to use `tokio::run`. 
// If it has a different type, you have to use `tokio::block_on`
fn fetch_data() -> impl Future<Item=(), Error=()> {
    Client::new()
        .get(url)
        .send()
        .and_then(|res| {
            res.into_body().concat2()
        })
        .map_err(|err| println!("request error: {}", err))
        .map(|body| {
            // here you can use the body to write it to a file
            // or return it via Ok()
            // If you return it via for example Ok(users)
            // then you need to adjust the return type in impl Future<Item=TYPE
            // Examples can be found here: 
            // https://github.com/gruberb/futures_playground

            // For now, lets just turn the body into a Vector
            let v = body.to_vec();
        })
}

Once you created the method which returns a Future (fetch_data()), you have to pass it on to a runtime like tokio (⛑):

tokio::run(fetch_data());

High Level Overview

You receive a Future from an external crate
A Future is likely to return a Stream of values, so you have to form this Stream into a type you can work with in a synchronous way (like a Vector or String)
You return the whole Future from a method via -> impl Future<Item=(), Error=()>, where the braces () are placeholders for the actual type you want to return
You pass the method on to a runtime like tokio via tokio::run(method())
The run call will start the runtime, set up the required resources and then put this future on a threadpool and start polling your future
It will then try to pass the work on to the operating system
Each time the runtime polls your Future, if the underlying I/O resource your Future is waiting on is not ready, it will return NotReady. The runtime sees this NotReady return value and puts your Future to sleep
Once an event from the underlying I/O resource comes, the runtime checks if this I/O resource is associated with your Future, and starts polling again. This time, your Future will be able to return a Ready with a value, since the underlying I/O resource has provided a value
The runtime will then set the status of the Future to ready and will process the .and_then() part of the code

The Future is getting, unlike in NodeJS, executed via tokio::run and not before. In Node, as soon as you write a Promise, the object is being returned immediately.

What's so different or hard about Futures? ⛑

Lets walk through our example above:

We create a new client with Client::new() and .send() our request
We will get a Response back:

pub struct Response {
    status: StatusCode,
    headers: HeaderMap,
    url: Box<Url>,
    body: Decoder,
    ...
}

The body itself is a Decoder, which can be turned into a Body via .into_body().
Body itself implements a Stream (as mentioned earlier).
Now we can look into the Futures API from Rust and find out: We can turn a Stream of Bytes into single item via .concat2()

...
        .and_then(|res| {
            res.into_body().concat2()
        })
...

We use this single item in the .map() part as body
With the help of your code editor, you will find out that this body is actually a Chunk, returned from the library Hyper
We can turn this Chunk now into a Vector

...
        .map(|body| {
            let v = body.to_vec();
            // do whatever with v
        })
...

From then on, we are back in "normal" Rust land and can forget what just happened 🙃.

You can find the full example in this GitHub repository. There I receive a JSON and write it to a file.

This is one of the reasons why dealing with Futures is so hard in the beginning. You have to think much lower level than for example in NodeJS. In addition, the async/await syntax is not final yet, which leads to much more boiler code.

These mental steps help you to not get lost when dealing with Futures:

What's the return type or value I am getting from this library?
How can I access the Stream of values on this response?
What will the library turn this Stream into when I collect all the values via .concat2()?
How can I turn this new type into a Vector or another Rust std format so I can pass it on to synchronous methods?

Basically, you always want to figure out how to access the stream of values, collect them, and then process the resulting object.

How to execute more than one `Future`? 🎩

Generally you want to collect your values as Streams, so that for each item you get over a Stream, you can spawn off a new Future to handle it.

The Rust Futures API has a method called FuturesUnordered which you can use to add more than one Future:

use futures::stream::futures_unordered::FuturesUnordered;
use hyper::{client::ResponseFuture, Client};

fn setup_requests() -> FuturesUnordered<ResponseFuture> {
    let mut list_of_futures = FuturesUnordered::new();

    let client = Client::new();

    let first = client.get(URL);
    list_of_futures.push(first);

    let second = client.get(URL);
    list_of_futures.push(second);

    list_of_futures
}

In this example we used hyper for our HTTP calls. You can find the rest of the code over here on Github.

The syntax will look slightly different if you are using reqwest. Here you .join() multiple requests and return it as "one Future".

fn fetch() -> impl Future<Item=(), Error=()> {
    let client = Client::new();

    let json = |mut res : Response | {
        res.json::<STRUCT_TYPE>()
    };

    let request1 =
        client
            .get(URL)
            .send()
            .and_then(json);

    let request2 =
        client
            .get(URL)
            .send()
            .and_then(json);

    request1.join(request2)
        .map(|(res1, res2)|{
            println!("{:?}", res1);
            println!("{:?}", res2);
        })
        .map_err(|err| {
            println!("stdout error: {}", err);
        })
}

The full code can also be found on GitHub.

What is the future of Futures? 🎩 + ⛑

Futures are getting into stable Rust at 1.37, so around June. There are also changes around the syntax and runtime, which will benefit the amount of code you have to write to get this stream of values out of a Future and into a synchronous Rust format.

You can also use the Runtime crate, which saves you almost all of the boilerplate code. Although going through the process above helps you understand Futures on a deeper level.

Summary

If you perform asynchronous operations, like fetching files from the operating system or making HTTP requests to a remote server, then Futures let you work with the return values in a non-blocking way.

If it would be synchronous, you would have to block a thread the operation is running on, and wait for the result until you continue. To do this in an asynchronous way, we have a runtime which creates threads themselves, and takes on Futures. It will fill the value inside the Future when the operating system returns a value to the runtime.

Once the Future is fulfilled, the runtime sets Async::Ready and the .and_then() part of the code will get executed.

Web Development with Rust — 03/x: Create a REST API

Bastian Gruber — Mon, 15 Apr 2019 08:52:41 +0000

Follow me on twitter to always get the latest information about web development in Rust. Also checkout the GitHub repository to this series.

Content

HTTP Requests
POST/PUT/PATCH/DELETE are special
The Job of a Framework
Creating an API spec
Crafting the API
Input Validation
Summary

APIs are the bread and butter of how a modern and fast-paced web environment. Frontend application, other web services and IoT devices need to be able to talk to your service. API endpoints are like doors to which you decide what comes in and in which format.

Since Rust is a static typed language with a strong compiler you won't face many of the common pitfalls about running a web service in production. Although there are still run time errors which you have to cover.

HTTP Requests

When we talk about creating an API we basically mean a web application which listens on certain paths and responds accordingly. But first things first. For two devices to be able to communicate with each other there has to be an established TCP connection.

TCP is a protocol which the two parties can use to establish a connection. After establishing this connection, you can receive and send messages to the other party. HTTP is another protocol, which is built on top of TCP, and it's defining the contents of the requests and responses.

So on the Rust side of things, TCP is implemented in the Rust core library, HTTP is not. Whatever framework you chose in the previous article they all implement HTTP and therefore are able to receive and send HTTP formatted messages.

An example GET requests for example looks like this:

GET / HTTP/1.1
Host: api.awesomerustwebapp.com
Accept-Language: en

It includes:

GET: the HTTP method
/: The path
HTTP/1.1: The version of the HTTP protocol
HOST: The host/domain of the server we want to request data from
Accept-Language: Which language we prefer and understand

The most common used HTTP methods are:

GET
POST
PUT
PATCH
DELETE

POST/PUT/PATCH/DELETE are special

We are using GET every time we browse the web. If we want to alter data however (like using POST to send data over to another server), we need to be more cautions and precise.

First, not everyone is allowed to just send a bunch of data to another server. Our API can for example say: "I just accept data from the server with the host name allowed.awesomerustapp.com.

Therefore, when you send a POST to another server, what actually happens is the CORS workflow:

We first ask the server what is allowed, where do you accept requests from and what are your accepted headers. If we fulfill all of these requirements, then we can send a POST.

The web framework actix for example has its own cors middleware.

Disclaimer: Not all frameworks (like rocket and tide) are implementing CORS in their core. However, in a professional environment, you handle CORS on the DevOps side of things and put it for example in your NGINX config.

The Job of a Framework

We use the hard work of other people to create web applications. Everything has to be implemented at some point, just not from you for most of the time. A framework covers the following concerns:

Start a web server and open a PORT
Listen to requests on this PORT
If a request comes in, look at the Path in the HTTP header
Route the request to the handler according to the Path
Help you extract the information out of the request
Pack the generated data and HTTP StatusCode (created from you) and form a response
Send the response back to the sender

The Rust web framework tide includes http-service, which provides the basic abstractions you need when working with HTTP calls. The crate http-service is built on top of hyper, which transforms TCP-Streams to valid HTTP requests and responses.

Your job is to create routes like /users/:id and add a route_handler which is a function to handle the requests on this particular path. The framework makes sure that it directs the incoming HTTP requests to this particular handler.

Creating an API spec

You have to define your resources first to get an idea what your application needs to handle and uncover relationships between them. So if you want to build a idea-up-voting site, you would have:

Users
Ideas
Votes

A simple spec for this scenario would look like this:

Users
- POST /users
- GET /users
- PUT /users/:user_id
- PATCH /users/:user_id
- DELETE /users/:user_id
- GET /users/:user_id

Ideas and Votes behave accordingly. A spec is helpful for two reasons:

It gives you guidelines not to forget a path
It helps to communicate to your API users what to expect

You can tools like swagger to write a full spec which also describes the structure of the data and the messages/responses for each path and route.

A more professional spec would include the return values for each route and the request and response bodies. However, the spec can be finalized once you know how your API should look like and behave. To get started, a simple list is enough.

Crafting the API

Depending on the framework you are using, your implementation will look different. You have to have the following features on your radar to look out for:

Creating routes for each method (like app.at("/users").post(post_users_handler))
Extracting information from the request (like headers, uri-params and JSON from the request body)
Creating responses with proper HTTP codes (200, 201, 400, 404 etc.)

I am using the latest version of tide for this web series. You can add it in your Cargo.toml file and use it for your web app:

[dependencies]
tide = "0.1.0"

Our first User implementation will look like this:

async fn handle_get_users(cx: Context<Database>) -> EndpointResult {
    Ok(response::json(cx.app_data().get_all()))
}

async fn handle_get_user(cx: Context<Database>) -> EndpointResult {
    let id = cx.param("id").client_err()?;
    if let Some(user) = cx.app_data().get(id) {
        Ok(response::json(user))
    } else {
        Err(StatusCode::NOT_FOUND)?
    }
}

async fn handle_update_user(mut cx: Context<Database>) -> EndpointResult<()> {
    let user = await!(cx.body_json()).client_err()?;
    let id = cx.param("id").client_err()?;

    if cx.app_data().set(id, user) {
        Ok(())
    } else {
        Err(StatusCode::NOT_FOUND)?
    }
}

async fn handle_create_user(mut cx: Context<Database>) -> EndpointResult<String> {
    let user = await!(cx.body_json()).client_err()?;
    Ok(cx.app_data().insert(user).to_string())
}

async fn handle_delete_user(cx: Context<Database>) -> EndpointResult<String> {
    let id = cx.param("id").client_err()?;
    Ok(cx.app_data().delete(id).to_string())
}

fn main() {
    // We create a new application with a basic, local database
    // You can use your own implementation, or none: App::new(())
    let mut app = App::new(Database::default());
    app.at("/users")
        .post(handle_create_user)
        .get(handle_get_users);
    app.at("/users/:id")
        .get(handle_get_user)
        .patch(handle_update_user)
        .delete(handle_delete_user);

    app.serve("127.0.0.1:8000").unwrap();
}

You can find the full implementation of the code in the GitHub repository to this series.

We see that we first have to create a new App

let mut app = App::new(())

add routes

app.at("/users")

and for each route add the HTTP requests we want to handle

app.at("/users").get(handle_get_users)

Each framework has a different method of extracting parameters and JSON bodies. Actix is using Extractors, rocket is using Query Guards.

With tide, you can access request parameters and bodies and database connections through Context. So when we want to update a User with a specific id, we send a PATCH to /users/:id. From there, we call the handle_update_user method.

Inside this method, we can access the id from the URI like this:

let id = cx.param("id").client_err()?;

Each framework is also handling its own way of sending responses back to the sender. Tide is using EndpointResult, rocket is using Response and actix HttpResponse.

Everything else is completely up to you. The framework might help you with session management and authentication, but you can also implement this yourself.

My suggestion is: Build the first skeleton of your app with the framework of your choice, figure out how to extract information out of requests and how to form responses. Once this is working, you can use your Rust skills to build small or big applications as you wish.

Input Validation

Your best friend in the Rust world will be serde. It will help you parse JSON and other formats, but will also allow you to serialize your data.

When we talk about input validation, we want to make sure the data we are getting has the right format. Lets say we are extracting the JSON body out of a request:

let user: User = serde_json::from_str(&request_body);

We are using serde_json here to transform a JSON-String into a Struct of our choice. So if we created this struct:

struct User {
    name: String,
    height: u32,
}

we want to make sure the sender is including name and height. If we just do serde_json::from_str, and the sender forgot to pass on the height, the app will panic and shut down, since we expect the response to be a user: let user: User.

We can improve the error handling like this:

let user: User = match serde_json::from_str(&request_body) {
    Ok(user) => user,
    Err(error) => handle_error_case(error),
};

We catch the error and call our handle_error_case method to handle it gracefully.

Summary

Pick a framework of your choice
- rocket is nightly
- actix is stable
- tide is fostered close to the Rust Core and also works on Rust nightly
Know that there is no common CORS handling (yet). Recommendation is to handle this on the DevOps side (NGINX for example)
After picking a framework, spec out your resources (/users: GET, POST etc.)
Figure out how the framework of your choice is handling extracting parameters and JSON from the request and how to form a response
Validate your input via match and serde_json

Explained: How does async work in Rust?

Bastian Gruber — Tue, 02 Apr 2019 10:26:08 +0000

This article will outline an overview of the why and how async exists in Rust. The differences between concurrency, parallelism and asynchronous code won't be covered.

Async Concept

Asynchronous programming is a concept which allows not blocking the program workflow when waiting for the results of certain actions. So you can open a large file or query a database, but your program will continue processing each line afterwards.

This concept was first needed on the kernel level, because you want to be able to listen to music while you type something on your keyboard. On a software level, this was achieved through multi-threading. On the CPU side, you can have multiple processes running on each core at the same time.

Later on, web servers came into play and needed to be able to hold millions of connections while performing I/O tasks. To be able to do this in a non-blocking way, we can either use threads on the kernel level, or implement our own way of handling threads and events.

What's needed and why

The kernel already has the concept implemented (through threads and other concepts), however they are quite "expensive", which means there is just a finite amount of resources available and dealing with this problem on OS level adds a whole new level of complexity.

Therefore it would be nice to handle our internal async flow on program level. We need a so called runtime, which can handle async code and is able to communicate to the kernel.

The general idea is:

Implement your own way of handling threads and queues on program level (green threads)
Add syntactic sugar to your language so the runtime/compiler can identify async parts of the code
Add async types so they can notify when they are "done"

Instead of dealing with Strings for example, an async type needs to have certain states (processing and done). The runtime can handle these types and set the state in them. Afterwards in your code you can access the value at a later point or wait for them to be done before you continue.

Workflow

You mark a method in your code as async, in this async method you can now use your async types. You can either wait for them to finish ("fetch data from GitHub...") or you "start" them, continue with your flow and later on check if they finished and use the value from them.

Once done writing the code, you need a runtime which can take this async part of your code and actually run it. The runtime needs also to take processes from the queue and hand it over to the operating system, since there is where the real work happens.

After the operating system is done with the processing, it will notify the runtime, which in return will set the state inside the async type and hand it back to the program workflow.

NodeJS vs. Go. vs. Rust

Lets look at how Node, Go and Rust are implementing the concepts we talked about, namely: Syntax, Type and Runtime.

NodeJS

In NodeJS you have the async/await syntax and Promises. You can await a Promise aka an action which might need more time to process.

const async_method = async () => {
    const dbResults = await dbQuery();
    const results = await serviceCall(dbResults);
    console.log(results);
}

Go

In Go, you start goroutines instead of Promises. And instead of async/await you simple write go method_name(). Instead of V8, Go ships with its own Go runtime.

f(greeting string) {
    fmt.Println(greeting, ", World!")
}

go f("Hello")

Rust

The Rust Async ecosystem is still in progress and not final yet. The proposal here is to also use async/await, instead of Promises and Goroutines you have Futures.

The Rust Language Team decided not to include any runtime. Rust wants to be as small as possible, and to be able to swap parts in and out as needed. Therefore you need to rely on crates to provide the appropiate runtime for you.

The most popular one is tokio, which uses mio internally as its event queue. Even other runtimes are using mio since it’s providing abstraction over kernel methods like epoll , kqueue and IOCP.

One special feature about Rust is also that you have to "start" a Future. So just declaring it like a Promise in NodeJS or writing go name_of_goroutine() doesn't trigger the Future to do something yet. So in case you are using tokio, you need to:

let response = client.get("http://httpbin.org")

let response_is_ok = response
    .and_then(|resp| {
        println!("Status: {}", resp.status());
        Ok(())

tokio::run(response_is_ok);

In the hopefully not so distant future, you can use async in Rust like this:

#[async]
fn async_function_name(...) -> Result<ReturnType, ErrorType> {
    let db_results = await!(query_database());
    let more_data = await!(fetch_another_service(db_resukts));
    process(more_data)
}

The async/await syntax is still in process and needs to be approved, merged and parts of the language adjusted to the new form.

Rust Async in Detail

Lets zoom in a bit on how a runtime works or can work:

Tokio is using internally the Reactor-Executor pattern.

What tokio and other runtimes want to achieve is a highly scalable server for high raw data throughput. They don't want to block when doing I/O operations. We have basically two options here: Thread-Based or Event-Driven Architecture. To make it short: Thread-Based is limiting because of the limited physical resources.

So Event-Driven is best in our case. It is registering incoming Future requests and saves a pointer to the async function handler. It then triggers an event in the kernel. Once the I/O operation is done, we call the pointer and execute the async method with the results from the I/O (kernel).

For this, we need a Reactor, which notifies if data is coming over the network or a file writing operation is in progress, and an executor which takes this data and executes the async function (Future) with it.

In addition, each runtime needs to understand kernel methods (like epoll) for starting I/O operations. For Rust there is a crate called mio which implements these kernel methods. Tokio is using mio internally.

Is it usable?

There is a lot happening at the moment in async Rust world. It will take a bit of time to have a final version out there which is easy to use and understand. Until then you can use your web frameworks like you are used to, since they already ship with a runtime.

Dropbox for example is using Futures in combination with tokio in production to serve data from the disk on Dropbox's servers. Futures is in Rust stable in version 0.1, and in Rust nightly in version 0.3. The runtime tokio is relying on Rust stable, so it is using Futures 0.1.

You can transform 0.3 into 0.1 Futures and vice-versa via the compat module.

Rust needs a few more months to get ready for its easy to use and powerful Futures. They are less expensive as in other languages, and you can have a thin or thick runtime, it's totally up to you.

Get started

As mentioned, tokio is one of the runtimes you can use. Another one is a combination of Romio and Juliex.

If you are building web applications, there is a crate called hyper, which already includes tokio. So here you can use Futures 0.1 in you application.

Keep up to date

You can check out the website areweasyncyet to follow the progress on async Rust. Similarly, arewewebyet is tracking the progress of frameworks and tools around building web applications.

Take a Business Approach for Learning a Programming Language

Bastian Gruber — Wed, 27 Mar 2019 19:48:00 +0000

The one most overlooked aspect of every field is to merge ideas from other areas of life. A business approach runs mostly contrary to programming. Business is outcome focused, programming is, for most of the time, done out of joy and the input (for the compiler) you are creating.

However, every field runs and behaves like any other. Teaching has mostly the same problems as programming, which has has the same complications as investing. You just have to dig deep enough, cut out the complex and look at it from different angles.

All fields share humans, and humans share psychology. What we also share is to improve, create and change. We want things faster and cheaper. This will always be true, no matter where you go. We also share our appetite for convenience and pleasure.

Birds eye view

In economics, macro is always deciding micro. Which means: How a country governs, the age and distribution of a population decides how successful a company operating within this border will be. On city level, being for example in San Francisco at the right time has a bigger impact on a companies success then having a superior product in a poor country with no access.

So what makes a company successful? You could say that they have the right market fit. They brought out a product at a time when people are ready for it. When the underlying infrastructure is there. Smartphones and mobile banking are two examples. Once there, not successful, but the idea came back and now everywhere.

If Rust would be a company, would you invest in it?

When investing in companies, investors generally ask themselves:

Will this company play a bigger role in 5 years then it does now?

If this answer is yes, then you are on the right track. So first we have to ask ourselves: Will Rust play a bigger role in 5 years then it does now?

Lets make a few assumptions:

IoT will play a bigger role. Factories start to get into IoT, they want to improve processes. The amount of money saved is not comparable to customer applications. Speeding up production line by 1 hour each day has a massive impact. Much more then speeding up a web page by a few seconds.
The last 10 years were focused on e-commerce and the creation of basic CRUD applications. The next 10 years will focus on moving assets, getting real time information and processing a massive amount of information.
At the same time, services will be more centralized and more de-centralized at the same time. Look at Googles Stadia gaming service. And think about Blockchain applications. I don't talk about money, but some form of Blockchain environment will be out there.
Applications will run almost exclusively in the browser, and the underlying operating system will play a smaller role.

For the past 10 years, NodeJS and JavaScript enabled a massive push towards the web. Picking up Node and creating a web service is super easy. And there will still be companies out there who will use NodeJS. There are still companies out there who need COBOL, PHP and C++ developers. But the push happened, e-commerce happens in the browser. The next move however will be to digitize assets, to put sensors on machines and humans, calculate the data and give feedback.

Two movements in the current programming environment

There are two movements happening right now:

Rust
WASM and WASI

Instead of just being a new programming language, Rust wants to be safe and fast. If you process billions of MQTT messages per hour, you want to rely on a fast and secure application stack. With more complex applications, you want to reduce server costs to a minimum, since you will host more on servers then in the last 10 years.

Rust can be compiled to a single binary, with statically linked C libraries.

In combination with WASM, which follows the trend that more and more applications will run in the browser, you can enable the feeling of using a native application. You could say that the browser is our new operating system. If there is no browser installed, how do you get your work done?

Just look at Atlassian, Google Docs and others. WASM will enable many languages to compile nativaley to WASM, which will then execute the bytecode in the browser. The web as we know it is long dead. There is just a limited space for "Inspect Source". Which doesn't need to be bad, there will be tons of other tools out there which can teach you to code and dig into things. People generally just have a hard time saying goodbye to once loved tools and environments.

WASI takes this idea even one step further.

What is our Macro in Programming?

Software is not really depended on countries (at least to some degree). So instead of countries, our macro are corporations. So if corporations push towards IoT, streaming games and processing much more data to be able to target customers, the company which does that in the most reliable way will succeed.

Lets look why Mozilla (and others) want WebAssembly:

The wasm format makes it possible to support graphics-heavy games in a browser without plug-ins. It can be used to port scientific simulation and other compute-intensive applications to the web platform. It also has non-web applications such as the Internet of Things, mobile apps and JavaScript virtual machines. (source)

So our Macro is companies who want to charge you a monthly fee by running their applications. And what is easier then distributing a product and charging for it through the browser? You also have less gatekeepers since the mobile platform is pretty much owned by Apple and Google.

Many languages will compile to WASM, so you could say it will help other languages stay alive as well. The trend of processing billions of messages, reducing server costs and having one language for the whole stack (embedded, server, frontend) brings Rust into a leading position.

Steve Jobs goal with NeXT was to to bring powerful computing into education. Well, maybe we are there now: You can spend 200 Dollar on a Computer but run a high-tech simulation program within your browser. Streamed from more powerful servers, rented for a monthly fee.

Which languages will thrive in this new macro environment?

Certainly Go has a big share. Why? Because companies use it to develop deployment pipelines and tooling for operations. Just have a look at DigitalOcean. Deploying software will play a bigger part in the future then it does now. So where Rust doesn't thrive, Go will certainly stay and take a piece of the pie.

As we said before, the next generation of companies who were at the sidelines are moving into the market as well. Manufacturers.

For these companies, one minute of downtime or production halts can cost millions. Creating reliable software, creating reliable data stores is worth a fortune. Rust with its low level, safety and speed focus will thrive in this environment. In addition it will soon be able to run on embedded devices as well.

Will one language rule them all?

No. The reason is that it's too expensive to switch every stack in the world. An example:

The Deutsche Bahn in Germany, which runs the rail roads, discovered, that if they increase the width between the tracks just by a few centimeters, the train would run much smoother. Easy solution right? Not so fast. The cost of this improvement outweight any gain. So it wasn't done.

The same happens to programming languages. Once a product earns money and supports workers, it will run until the company dies. There will be a fair share of NodeJS, Python, Ruby and other languages out there until you retire.

But, the question is: What would you teach your children? Which language (if they would like to become programmers) would you advice them to learn? Under this assumptions from before, I assume it would be Rust.

Which languages are a safe bet?

Under my current assumptions, languages with the biggest leverage will be Go and Rust. Go is straight forward, is reducing discussion time with developers and with its type system is easier to handle then JavaScript.

Rust will thrive due to its focus on safety and speed. Both of these things will be even more needed in the future then they are today. Just think about running async web servers. NodeJS was so much faster and easier to use then anything before. It focused on the right metrics. Rust does the same for the future metrics.

TypeScript, Python and Java will still be there. It doesn't mean you won't find jobs, but under the new circumstances, Go and Rust outshine the rest. I wouldn't advice Junior Developers or colleague graduates to focus on Java or TypeScript at all.

Summary

Take some time every now and then and take a strategic approach. Reading The Economist or Financial Times can be as valuable as reading a programming blog. Companies will decide in which environment you are operating in. The once who can leverage technology to fit the current needs will be the once who strive. But never forget:

Tools don't solve problems, people solve problems. You use the tools to solve problems.

What are your biggest obstacles/questions for moving from NodeJS to Rust?

Bastian Gruber — Sat, 23 Mar 2019 15:32:41 +0000

NodeJS is the perfect entry for building web applications. I am getting many messages lately that people want to give Rust a try and implement their first web service in it. I would love to hear from the dev.to community what your biggest hurdles are or which questions need to be answered before you feel comfortable jumping in!

How to combat JavaScript fatigue with Rust

Bastian Gruber — Wed, 20 Mar 2019 19:05:24 +0000

Do you find yourself in any of those thoughts when creating applications with NodeJS?

"I for sure need to pick the right npm module to solve this problem, but which one fits best and is getting supported well?"
"Uh, my app is growing so big, I have no idea what the difference between user and customer is anymore and which properties they have."
"TypeScript will definitely solve my problems!"
"Hm, I don't know if this is an error with TypeScript or in the transpiled JavaScript? Or did I just disable the wrong ESLint option?"
"Is this all what I will do for the rest of my career?"

Creating JavaScript applications is easy at first but it's getting harder and harder once the application grows. You can wait and hope that any of the big tech companies will solve your problems through TypeScript and other tools. A few years go by and you ask yourself: Is this all? Do I create CRUD application for the next 20 years?

It is sometimes harder to keep up with the latest hype train then actually learning fundamentals.

NodeJS became a victim of its own success. The market is flooded with developers, every company is using Node in some part of their tech stack. You can say that companies don 't pick it because of it's underlying technology but because they can get cheap labor and feel safe because all the other companies are also using Node, so it must be a good choice.

Meet Rust

You probably know the saying that you should learn Lisp, even if you won't use it, just because the learned mindset will make you a better developer? Instead of learning Lisp to become a better developer, you can learn a language which is actually in use and will improve you as good as Lisp does. Meet Rust. Rust is not the latest hype, but has fundamentals which offer you a wide range of opportunities, even in the long run.

Rust is a systems language, but it's ergonomic syntax empowers the developer after the first struggles.

From a NodeJS perspective, Rust can be used to create web server applications and services. You can write Lambda functions or full fledged applications. Instead of using TypeScript, Rust has a strong build-in Type system. Instead of using ESLint, it ships with the pickiest compiler out there.

The pain you feel in the beginning is different. I call it good pain. Good pain lets you grow as a person (and developer), bad pain just hurts. I would categorize many of the TypeScript and NodeJS pains as bad pain. Not because the tools are bad, but a language which was created in 10 days just gets you so far.

At some point you need to make a personal decision of how your life should look like. Your day to day has a huge impact on your happiness and the well-being of your surroundings. Rust can help you rediscover the joy of programming again.

After you learned Rust, you may think you just replaced tool X with tool Y and end up in the same place. But, there is a difference. Once learned, you can use Rust to create:

Games
CLI tools
Operating Systems
Web Services
Browser Applications
Applications on Embedded Devices

One would say you can cover the whole stack. Rust creates a self feeding system:

Developers from C, C++, Java, Kotlin, Python, Ruby and JavaScript flock to the language
Rust and its features are getting discussed not only in a web context but in a wide range of scenarios
Different areas, perspectives and use cases are getting fed back into the language in an open way

Lets picture a perfect world 5 years from now

You are writing your web service in Rust. Instead of installing five different npm modules which help you cover your mistakes, you are using build-in tools to harden your code base. After the compiler gives you a green light you are 98% sure your application will run safe and fast in the years to come.

But, hold on, you are still sometimes frustrated by the repetitive work environment. But now you mastered a language which you can use to build your own Operating System on the side, you can implement the TCP protocol, you can create Games and join a local game community. You can even create applications for the Browser which are compiled to WASM. If this is not enough, you can get yourself a piece of hardware and run Rust on it.

Node is not gone

NodeJS won't be completely gone, and your skills neither. However you found a great addition and you might just focus on Rust more in the future. Remember, no language survived forever. NodeJS could end up with the same faith as PHP and others. Picking a systems language like Rust gives you strength and opportunities for many years to come.

Should you learn Rust?

From a developers perspective: Yes, definitely. Rust has many unique features which will be copied also to other programming languages (like Ownership in Swift). Learning Rust sharpens your thinking and will make you more considered in your day-to-day language of choice.

From a safety and correctness perspective: You should aim to replace part of your web service with Rust. Start small and see where it leads you. Other companies like NPM, Dropbox and Yelp are doing it and had big success.

Rust for the web is not completely there, but once fully landed, it can not only convert your web services into safe bets. Once mastered, Rust lets you jump up and down the stack, explore Computer Science concepts first hand and build tools in your free time other then the next CRUD application.

So, take the chance and make 2019 the year you learn Rust. Have something in production by Christmas and try to replace part of your services at work in Rust and see if it's successful. Rust will not only let you sleep calmer at weekends, knowing that a rock solid binary is running in production, but the skills you gained let you explore a vast universe of possibilities around technology.

I will help you with that. Follow my series on "Practical Web Development with Rust" and build with me a shiny web application in the next few months.

Web Development with Rust— 02/x: Deploy your first App

Bastian Gruber — Tue, 19 Mar 2019 09:12:53 +0000

You can find the Introduction to web programming in Rust over here. Follow me on twitter to always get the latest information about web development in Rust. Also checkout the GitHub repository to his series.

Update 08.07.2019: tide 0.2.0

This series has 3 goals

Show the obstacles new developers will face when programming in Rust but also its advantages when it comes to the specific topic (this time: Deployments).
Show different options for web development in Rust.
Always have an updated application in production afterwards.

Number three is super important for me. That’s the reason why we start part 02/x with a few ideas on what you can build, and why this tutorial will look like as it does. Whenever you learn something new, take this mental model:

Never do things for their own sake

Which translate to: Never learn Rust just because you want to learn Rust. This is the biggest factor why you will fail to learn a new language or basically anything in life. You have to have a goal in mind, a reason to do something.

“So how should I learn Rust then?”

Have an app or idea in mind you want to see in production. This can be a service to calculate prime numbers, a web app to track your programming goals, a service which fetches your latest likes on GitHub and tracks their activities etc. You decide what you want to build.
Hold yourself accountable to it. Tell your friends, work colleagues or your partner that you promise them that in 6 months time, they will be able to use this service. Every few days or weeks, you will keep them updated about your current status.

It doesn’t need to be a polished idea or a service to compete with other apps. It has to be something you would like to have. It will help you staying through the hardships in the beginning, and through the phase when the honey moon is over and you see that Rust can be hard at times.

I will make mine public as well:

At the end of the next 6 months, I have a running web service with frontend for my MeetUp “Rust and Tell Berlin” up an running so speakers can submit proposals and slides and videos can be watched from talks which were held the previous events.

I will hold myself accountable through this tutorial series.

Lets move on. In this part of the series, we want to deploy our first application. If you come from NodeJS, the deployment life cycle looks like this:

With NodeJS, you can push any code to a production server. You have to have good tests, ESLint and other tools to catch undefined and Type errors.

In an ideal world, we have a development cycle which looks like this:

So we want to break things as early and close to the code (your local machine) as possible. Once we figured out a working code base, we would like to bring exactly this working solution onto a server. Because of Rusts Type System and strong compiler, we would be able to pack a working binary and move it to production. Tests would cover the rest of the errors.

Rust moves possible errors closer to the coding environment

a) The Rust Compiler will catch a lot of problems, almost all of them.

b) You can catch the rest with good tests (in our case: Error handling when receiving the wrong parameters).

c) After you can compile your Rust Code, you have a binary which can be shipped in many different ways.

Difference between local and production-ready code

When we talk about deploying, we have to make sure that our code is able to:

randomly assign a PORT based on the environment it is running
handle errors gracefully
respond to not expected input with proper return codes and messages
fail early in the deployment pipeline with a proper test setup
log events so errors can be traced

In this article we will cover the first must-have (randomly assigning a PORT). Each article in the series will cover the rest of the requirements.

Four different deployment options

We generally have different deployment and hosting options. Some are more suited for large scale application and some are better for private projects and to get a project off the ground without too much complexity. Our options are:

Managed Deployments / Hosting (Heroku)
Self managed via Docker and a Docker registry
Self managed via Docker and a Git registry
Managed Serverless Lambda functions (AWS Lambda, ZEIT now)

We will cover each of these options in this article and see advantages, disadvantages and how to prepare your Rust Code so it can be deployed (in the best possible way).

Building the first version of your app

As we said in the beginning, we need an idea and what we want to build. Even if we map out a bigger picture of the application in the next article (03/x), we can get started and choose a framework we want to build it with:

rocket
actix
gotham
tide (work in progress)

As seen in the first article, you can go lower level if you want:

We will pick one framework for the written version of this article. I will pick tide, since I am planning to contribute to it more in the future. I will map out solutions for rocket and actix in the GitHub repository for this series.

Set up our app

We want to make sure to use asynchronous code, which is not in Rust stable yet. Therefore we need to install and set the nightly version of Rust:

$ rustup install nightly-2019-02-25
$ rustup default nightly

This will generate our first folder structure. The bare bones of a running web app with tide look like this:

Cargo.toml

[package]
name = "my-cool-web-app"
version = "0.1.0"
authors = ["YOUR NAME + EMAIL"]
edition = "2018"

[dependencies]
tide = "0.2.0"

main.rs

#![feature(async_await)]

fn main() {
    let mut app = tide::App::new(());
    app.at("/").get(async move |_| "Hello, world!");

    app.serve();
}

As we said earlier, we need to give the hosting environment the chance to assign a PORT to our application.

Our main.rs has to accompany these requirements:

#![feature(async_await)]

extern crate tide;

use tide::App;
use std::{env, net::SocketAddr};


fn main() {
    let mut app = App::new(());
    let address = SocketAddr::from(([127, 0, 0, 1], get_server_port()));

    app.at("/").get(async move |_| "hello world");
    app.serve(address).expect("Start server");
}

fn get_server_port() -> u16 {
    env::var("PORT")
        .ok()
        .and_then(|port| port.parse().ok())
        .unwrap_or_else(|| 8186)
}

With this setup ready, we can go over each deployment option.

Managed Deployments via Heroku

Managed environments are for the most part just an abstraction. They internally do the same as you would with your own pipeline: Push code to a git repository. A “hook” is watching this repository and on changes will start to compile the latest version and run it. For you however, it’s just a git push heroku master.

To get started, you need a Heroku account (free). Login with your new account and create a new app:

After clicking “Create app”, Heroku explains under the “Deploy” tab how to push your code to their servers:

Prepare your code

First, we need to be able to push our code base to the remote location (Heroku). Therefore please install the Heroku tool chain. Afterwards we can add the remote location to our GIT repository:

$ cd my-cool-web-app
$ heroku login
$ heroku git:remote -a my-cool-web-app

Next, we need to tell Heroku how to run our application after it is build. Heroku expects a file with the name Procfile, which has the start command in it:

$ touch Procfile

And put the following line it it:

web ./target/release/my-cool-web-app

We also have to tell Heroku which version of Rust we are using. Since we want to use nightly, we create a file called RustConfig in the root directory:

$ touch RustConfig

with the following line:

VERSION=nightly

Caveat

Rust is so new that Heroku doesn’t support it out of the box. We need to install and activate a “buildpack” for Rust. So inside the root directory of your application, execute the following commands:

$ heroku create --buildpack emk/rust
$ heroku buildbpacks:set emk/rust

This will activate the language support for Rust.

Now we can:

$ git add .
$ git commit -m "Init"
$ git push heroku master

When succeeded, we go back to the Heroku dashboard in the browser and click on the the generated domain (under “Settings”). A browser windiw should open and display “Hello, World!”.

Summary

Heroku makes it easy to deploy your application
In less then 5 minutes you have a running version of your app live
You can assign your own domain and activate HTTPS (if you pay for it)
Heroku ist the best option when it comes to this tutorial and starting side projects: Cheap, easy to use and removes the overhead of deplyoments especially in the beginning

Docker

Using Docker has the huge advantage of being free in choosing your pipelines and environments. You can either build the image locally and push it as-is to a Docker registry. From there a server can take(download) and execute (docker run) it. Or you create a blueprint (Dockerfile) which other service can use to build on their servers.

If you are using Docker for your deployments, you have two options. The first one is to push your code (with a Dockerfile) to a Git registry (like GitHub or Bitbucket) and then have a configured deployment server which listens to changes, SSHs into the Git registry, takes the code, deploys and runs it.

Your second option is to use a Docker registry. There you have the advantage to pre build your container and ship it as-it-is. This makes it sometimes faster to run deployments and you have to ship less code (especially in case of Rust).

We can use Rusts feature of being able to be compiled to a binary. We can even go one step further and compile a static Rust binary with no external dependencies. What we would need for this, is:

Build a Rust binary
Statically linked the needed C libraries to it so it can run on it’s own

The result would be to have a binary which doesn’t even need Rust to run. Thanks to the Open Source community and Erik Kidd, there is already a solution out there which helps us with that.

The result is a super small Docker image with no external dependencies. Meet rust-musl-builder. It is a Docker image which helps you build static Rust binaries. It will download the whole image just after the first execution.

Everything we type and create happens from the root directory of our application.

$ cd my-cool-web-app

Before we create our Dockerfile, lets see what we actually trying to do. We are using the rust-musl-builder to statically link the musl-libc library into our binary.

$ docker run --rm -it -v "$(pwd)":/home/rust/src ekidd/rust-musl-builder cargo build --release

This will create our super small binary. You can inspect it like that:

$ ls -lh target/x86_64-unknown-linux-musl/release/my-cool-web-app

It is just a few MB small (in my example: 4,4MB). To be able to recreate this procedure over and over again, and not just on our local machine but also in a deployment pipeline on different servers, we create a multi-stage Dockerfile.

FROM ekidd/rust-musl-builder:nightly AS build
COPY . ./
RUN sudo chown -R rust:rust .
RUN cargo build --release

FROM scratch
COPY --from=build /home/rust/src/target/x86_64-unknown-linux-musl/release/my-cool-web-app /
ENV PORT 8181
EXPOSE ${PORT}
CMD ["/my-cool-web-app"]

You can build the image now via:

$ docker build -t my-cool-web-app:latest .

And run it with:

$ docker run -d --rm -P --name heroku heroku:latest

Now you can open your browser (in macOS) via:

$ open http://$(docker container port my-cool-web-app 8181)

We just created a super minimal Docker image which contains our binary with no external dependencies. You can inspect your just created image via:

$ docker image ls my-cool-web-app

Summary

Docker is a beast, but when used wisely can be quite helpful
Especially with Rust: You can create statically linked binaries which are super small and don’t even need a Rust environment to run in
You also have much more options to host and run your application when choosing Docker
However, managed hosting environments like Heroku don’t allow pushing Docker images to their environment

Serverless runtimes — ZEIT/now

Serverless is a different mindset then the first two options. Serverless also means stateless, so you are not building web applications but functions. Instead of having API endpoints build into your app, you basically just have those API endpoints (in serverless terms: handlers). Our web frameworks like rocket and actix might be an overkill here. Right now, ZEIT is not supporting Rust nightly builds in their new serverless environment.

So instead of creating a binary (with cargo new web-app), we create a library:

$ cargo new now-service --lib
$ cd now-service

Here we have to create a file called now.json

{
  "name": "now-service",
  "version": 2,
  "builds": [
    {
      "src": "src/index.rs",
      "use": "@now/rust"
    }
  ]
}

And our src/lib.rs example looks like this:

use http::{Request, Response, StatusCode, header};

fn handler(request: Request<()>) -> http::Result<Response<String>> {
    let response = Response::builder()
        .status(StatusCode::OK)
        .header(header::CONTENT_TYPE, "text/html")
        .body("<!doctype html><html><head><title>A simple deployment with Now!</title></head><body><h1>Welcome to Rust on Now</h1></body></html>".to_string())
        .expect("failed to render response");

    Ok(response)
}

As with Heroku, you need to install the ZEIT toolchain, which is called “now”. There are several options. If you are on macOS, you can do it via:

$ brew cask install now

Which installs the Now application. Find it in your /Applications folder and open it. You can finish the installation by typing in your email address. This will also install the command line tool chain.

That’s basically it. You can type:

$ now

and hit Enter. This will start the upload of your application. Login to your ZEIT dashboard and click on the provided link.

Summary

Serverless lets you save costs: The service is just running when requested
This ends up in higher boot times which need to be considered
The mindset of serverless lets you rethink state and if you really need a fully fledged web application for some use cases
The deployment can take a bit longer when using AWS Lambda or ZEITs now

Follow me on twitter to always get the latest information about web development in Rust. Also checkout the GitHub repository to his series.

Intro to Web Development with Rust for NodeJS Developers

Bastian Gruber — Mon, 18 Mar 2019 21:37:37 +0000

You can find the second article in this series ("deploy your first Rust app") over here.

Rust is different. You can pick up Python or Ruby over the weekend, create a first CRUD application and be happy with the results.

With Rust… with Rust you will struggle to pass a String to a different method, change and return it. You then will order the Rust book, see its size, sigh and get started.

After a few weeks of fighting through the book after work, you give up and wait until someone else creates an easy-to-follow tutorial.

Here is your “easy” tutorial

I struggled with the same problems. Life circumstances however gave me a few months time on my hands to really focus on Rust.

What follows is a first overview, concept, and paths to follow. In the coming weeks and months, I’ll publish a series of articles to help you to get from concept to product.

NodeJS vs Rust

After installing them (I chose brew for macOS in this example, the method doesn’t matter), the underlying stack looks different. NodeJS needs V8, the runtime engine from Google, and bindings to the JavaScript library to run JavaScript code.

Rust depends almost completely on Rust itself. Just the compiler is using llvm libraries, which are written in C and C++.

How much "web" is in Rust?

It was and is a design decision not to include a standard http library in Rust. The OSI layer is therefore covered differently:

Node covers the whole stack, and offers with Koa and Express, two well-known and “rock-solid” web frameworks which help you to build applications on top of HTTP.

On the Rust side of things, just TCP is implemented in the Rust Core. The current web frameworks (actix and rocket) are implementing everything up until HTTP though. So you don’t need to care where this is coming from.

If you want to use pure HTTP calls without any larger framework, you can install “crates” (equivalent to npm packages in the Node world) which implement the HTTP protocol (like hyper and tiny_http).

npm vs cargo

Node is using npm for its package management:

npm install is installing dependencies
npm run xyz is executing scripts inside the package.json

On the Rust side, cargo is handling everything related to your project:

cargo new NAME --bin is creating an application
cargo new NAME --lib to create a library
cargo run is executing the code
cargo build is creating an executable
cargo test is running all tests inside the project

There is an open PR to add cargo add to install dependencies. Right now you have to add them by hand to your Cargo.toml file. As you see, you don’t need to include scripts in a package.json to run tests or build and test your application.

Mindset change: Cargo is fetching the packages after cargo run, and just if the version changed. So the first time it will fetch all packages, the second time just when a change in the version number happened. Unlike npm i which fetches the packages right away, and will add it to the package.json with the save notation.

Ecosystem

Node is not successful for no reason. The ecosystem is rich and flourishing. Rust is still developing, but has already many great “crates”. The website arewewebyet.org is tracking the progress and showing you interesting packages in the Rust world.

There is also an attempt to create an official Rust Web Framework, called Tide. It is already pretty mature and can be used for side projects. Feel free to contribute and help craft a great environment for web development in Rust.

Asnyc programming aka Promises aka Futures

Nodes killer feature are Promises. Although not always easy to understand and handle, Promises and the event loop are what makes Node so lucrative.

Rust is also implementing an asynchronous mechanism, which are not yet in the final version. They are called Futures. A library called Tokio is already offering an asynchronous run time. You can track the progress on asynchronous programming in Rust over at areweasyncyet.

How to get started?

Install Rust: curl https://sh.rustup.rs -sSf | sh
Create a new project: cargo new web-app --bin
cd web-app

Now you can choose your web framework of choice. You can either start with rocket or actix. You can follow the tutorials on the website to get a first web application running.

Heads up: undefined, borrowing and types

To not to get frustrated until my next post, here are the main four things about Rust you will have to get used to (which are quite awesome after a while).

There is no undefined. Rust has no real null value. This is a feature. Rust has a type called Option, which encapsulates either the return value or None. In this case, we use the Result type to return a value. You could, instead of returning a String, return an Option which might have a String value, or None if the website we are fetching from doesn’t contain any text. An easy solution, which you should not use in production, is to .unwrap() results to get the String out of the encapsulation. Homework: Try to check if the GET request errored and return an Error in this case instead of Ok().
You must have heard about borrowing. In short: Every assignment (=) to a non trivial type (everything which doesn’t have a fixed size) moves the ownership over. The method fetch_text() is not taking ownership over the url but just using a reference to it (via the & ). Homework: Figure out why this code here fails and how to solve it.
You always have to know which type is returned from a method. In this example, our HTTP crate reqwest is returning a Response struct (type) which implements certain methods. The documentation is, as with so many crates, excellent. So have a look here.
You don’t have to type return to return a value from a method. Just don’t put a ; at the end of a statement and this becomes your return value.

Use this example to play around, see where you can get errors and learn to fix them.

Is this all?

Two opinions:

“Sadly no!”
“No, but that’s a good thing!”

You actually have to learn a decent amount of Rust to get started. This is what I am here for. In the next few days, weeks and months I will cover the basics up until creating a solid web application.

Up until then, I can recommend the Rust track on Exercism.io and the Rust Book which you can find also in a paper version at your local book store or at Amazon.

As mentioned in my first article, Rust is making you a better developer, so the road will be long at times, but always worth it. Follow me on twitter, Medium or on dev.to to stay up to date!

DEV Community: Bastian Gruber

Announcing my book "Rust Web Development"

A pragmatic and "in-depth-enough" book about writing web services/applications in Rust

Ready for prime time?

See for yourself

Getting paid in Bitcoin - Experiences and misconceptions

Getting paid in USD vs. BTC

Numbers

Transferwise vs. Bitcoin

Experiences using Transferwise

Getting paid in Bitcoin

What about taxes?

Summary

Side note

Experiences of working fully remote

The "Get dressed and showered" myth

I love text meetings

It's hard to unlearn "just being there"

Feeling the vibe of the team

Block your most productive hours

You have to motivate yourself

Isolation

You can't work remote in a semi-remote company

Summary

Rust for the Brave and True

Rust

Brave

My reasons:

True

What I learned from it

Rust Coding Environment

2 Warnings/Hints

Additions

Summary

Explained: Futures in Rust for Web Development

Content

The two perspectives

What actually is a Future? 🎩

When to use Futures? 🎩

Why does using Futures save time? ⛑

How to use Futures? 🎩 + ⛑

Code example

High Level Overview

What's so different or hard about Futures? ⛑

How to execute more than one Future? 🎩

What is the future of Futures? 🎩 + ⛑

Summary

Web Development with Rust — 03/x: Create a REST API

Content

HTTP Requests

POST/PUT/PATCH/DELETE are special

The Job of a Framework

Creating an API spec

Crafting the API

Input Validation

Summary

Explained: How does async work in Rust?

Async Concept

What's needed and why

Workflow

NodeJS vs. Go. vs. Rust

NodeJS

Go

Rust

Rust Async in Detail

Is it usable?

Get started

Keep up to date

Further reading

Take a Business Approach for Learning a Programming Language

Birds eye view

If Rust would be a company, would you invest in it?

Two movements in the current programming environment

What is our Macro in Programming?

Which languages will thrive in this new macro environment?

Will one language rule them all?

Which languages are a safe bet?

Summary

What are your biggest obstacles/questions for moving from NodeJS to Rust?

How to combat JavaScript fatigue with Rust

What actually is a `Future`? 🎩

How to execute more than one `Future`? 🎩