DEV Community: Philipp Renoth

Consider Rust for cloud computing.

Philipp Renoth — Fri, 17 Mar 2023 03:24:05 +0000

The costs from the big public cloud providers went down every year till today, but it looks like 2023 can be the year of the so called cloud-repatriation. To be honest I did not invent that term, but my gut feeling always was, that there will be huge downsides in the future and your company is better prepared when operational costs skyrocket.

Forbidden fruits in the cloud

Companies talked about success stories, when they "lift and shift" everything to the cloud. No datacenter hassle, no hardware orders. Just a few bucks a day and of course high-availability and infinite resources.

The psychology of subscription based payments plays well together with the idea of on-demand cloud resource provisioning and the short-term thinking of companies. Putting a foot into the door of cloud computing is not hard nor expensive, but it felt like that honeypot makes companies easily forget to continuously track objectives and thus may experience their homemade pay-day in the near future.

While the gates of heaven into cloud are open, their providers did not rest, but did a lot of optimizations to reduce costs and give the overall cloud costs for customers a decreasing trend. This may come to an end now as crises are piling up. Those are in particular the aftermath of the pandemic, like global semiconductor shortages, as well as the global energy crisis which is also favored by war.

The long forgotten term of "resource efficiency"

The computer you're sitting in front of might have been one of the fastest ten years ago and will maybe just be scrap in ten years, while today you're using it for some very simple development tasks where setting up a local environment allocates half of your memory and the CPU fans goes up all the time. In my free time I enjoy doing some game development and it's completely insane what a computer can do at 60 frames per second, while on the other hand messed up with some poor implemented CRUD applications.

Such invisible but ubiquitous slowdowns distort any sense about performance and makes us think that hundred concurrent users is a thing for a server. But it's actually not and never was in the last decade, but we got used to technologies with lot's of friction and started investigating how to fix symptoms and not the root cause, because it was cheaper.

One of the large milestones in web development was node.js, which proved that running a script with one thread is enough to serve hundreds of concurrent users, with the ease of JavaScript and even without compilation. A bit later Go came up and showed the same for a native compiled language without decreasing developer experience.

Modern technologies understand that IO should not be thread-bound. Still a very small minority of servers really use all resources efficiently. In common almost all of them use a virtual machine (VM) indirection with a garbage collection (GC), which means memory and performance overhead. Due to optimizations performance can be similar to native speed, but memory is far away from raw native data and now let's go back to the cloud: servers are not utilizing their resources in an efficient way. It's daily business that companies rather pay more for resources and the topic of "paying more than used" didn't carry weight in the past.

The mindset of excessively throwing around resources, because they are cheap, shall end this year. The golden age for resource friendly technologies has started and it's the best time to rethink the past.

Technology stacks at companies

Comparing software technologies regarding all aspects is nearly impossible. There are so many parameters starting with the knowledge and experience of the team(s), going over to operational costs and the calculation for future risks, for example if hiring new employees may be a problem. For good reasons companies tend to not reinvent the wheel every year and do not start a technology stack shift.

Another thing is that even if operational costs decrease, software has to be developed and is very costly. To put everything together your company may optimize costs by using the best technology to get up and running in a short time, so a technology which is good for prototyping and the team is used to.

If you ask experienced developers, a lot of that discussion is just like a religious question and mostly of course the majority of the developers stick to the same technology, because they were hired exactly for that position and so they all pull together.

From a company's perspective having an experienced team working well together is an important asset, but in the long-run dev teams should also consider new paths. It's very important that CTOs do a good job and consider new technologies. Computer science is still a new field and there is so much to learn from the past, that more and more things will show up in the future.

Consider Rust

Finally, for a cloud technology I'd like to list some of the invincible advantages of Rust and mitigate some of the disadvantages:

Pro

Rust is small, fast and secure: Most of the technologies we have in the cloud are lacking at least one of those characteristics. The reason why "security" is mentioned is that C/C++ is also small and fast, but in terms of native technology it's not possible to write secure apps by default, like being memory-safe. Rust has it all and running 15MB alpine linux images (or even from scratch) as containers which e.g. take 10MB idle memory and go up to 50MB memory for peak while utilizing all computation power are normal metrics for services written in Rust. Other technologies may take about factor 10-100 more memory which of course has a bad impact to the running costs. Also any "serverless" idea or "scale to zero" plays well with Rust as service startup is blazing fast.
Rust uses an explicit control flow for error handling as well as concurrency: Making a mess is possible by opting-in with writing code that you're not going to handle the error case or you're not interested at all, but code reviews will show up fishy code. There are no hidden control flow channels, like throwing exceptions and that's for a good reason. Also asynchronous code needs to fit all thread-safety constraints and the so called "async-runtimes" will distinguish between computation-heavy thread based tasks and simple asynchronously glued code. So you can consider Rust code being very expressive all the time, which is good for maintenance.
Software architecture by language design: Reference counting and garbage collection is also a curse. Together with concurrency or parallelism there is no language that can express the way components are allowed to be used in a given multi-threaded context. Rust is very strict about ownership and usage e.g. immutable or mutable as well as being used from a different thread. That seems to be a huge constraint, but it's an overall positive game-changer for software design and architecture. The way you have to manage resources feels very natural after getting used to it and then unrestricted, shared components seems like fishy design.
Rust has a modern tooling: So far all main platforms and many, many more are supported. Rust projects are built according to conventions and the common build tool for many years is cargo and the community loves it.

Contra

Rust has a steep learning curve: Just no! You may have seen code where lifetimes and generics are used, but using Rust from userland you're not supposed to write such code and probably not having to use e.g. explicit lifetimes at all. Coming from a garbage-collected (GC) language, you may argue that in Rust things are not possible anymore. That's true, but simply don't try to solve it the same way with a GC language. The solution with your Rust framework is much easier, faster and less error-prone, I promise.
Rust builds take long: First of all, this is known to the community and folks work on it, but it's the current, actual price to pay for other benefits, like e.g. borrow-checking is a coherent part of Rust, as well as of none other system language. Second, incremental builds are typically fast enough (1-5s) for development cycles and can be optimized for large-scale projects, like using unoptimized dynamic linking for debug and optimized static linking for production.
Rust's ecosystem is still immature: I cannot deny this, but the overall code quality in general is quite encouraging as we have compiler guarantees to exclude categories of CVEs like overflow or memory corruption is not possible by default and as the language is very expressive and strict, it's harder to make bugs. One of the main problems is that we still need more companies to use Rust from userland and that's probably where you can jump in, by evaluating and suggesting Rust as a new technology.

Conclusion

Once companies read about success stories of other companies and Rust developers are not rare anymore, it soon gets a piece of cake to onboard others. Of course the first show cases were in the embedded world, as well as C/C++ projects calling for an alternative like Linux kernel or Android. Now we should recap and understand that the power of system languages can still be utilized for almost everything, but in the meantime we got smarter about handling the category of memory related issues, which was the show-stopper for system languages in all areas where we had better alternatives.

What do you think about the future?

10 Rust technology essentials

Philipp Renoth — Tue, 26 Jul 2022 19:09:05 +0000

This article will cover a bunch of examples what really makes Rust so interesting for developers. It’s not just about system programming anymore. Rust moved on to boost different areas where classical system programming languages haven’t really taken hold. The formula for success is a sophisticated combination of new and carefully selected concepts from other languages that form Rust and developers already love Rust.

Of course the language itself is no reason to blindly adopt it. There are so many different aspects starting with a module eco-system and ending with the question about hiring new developers, but that’s another thing that is always hard for new technologies. Even if you can’t or don’t plan to use Rust it might be a huge asset for your daily work to have some Rust understandings that makes you a better developer.

Essentials

When we compare Rust to other system programming languages it’s clear, why memory-safety is such an important thing, while VM-based technologies don’t have to deal a lot with those problems. The other way round for asynchronous programming there is less often a need for native local programs, but as an IO-intensive application running in the cloud it’s a huge benefit to use less resources more efficiently.

Read about 10 Rust essentials at my company's blog.

Good bye ReactNative, hello Ionic.

Philipp Renoth — Sun, 10 Jul 2022 17:01:30 +0000

Developing a ReactNative (RN) app for Android and iOS for the last 2 years I have more and more doubts that ReactNative is still a good choice for mobile apps these days.

Another alternative would be to write native Java for Android and objective-C / Swift for iOS and that can also be a good or bad choice depending on what you plan to do with your app. The more sophisticated your app will be regarding UI/UX and performance the more likely this can be a good choice to have more or less two apps with different development teams and maybe some shared roles, but in the end two products with the same name, but for different targets.

On the other side, the more you're focused on functionality over UI/UX and other non-functional criteria, the more likely is that a hybrid or cross-platform app is a good idea, because you only have to maintain one application. So to put that benefit into numbers, it's possible that more than 50% of development resources can be saved and that's a lot.

Before we continue I'd like to clarify the difference between hybrid and cross-platform technologies. Articles often mix them up or use them as synonyms but actually hybrid apps have some embedded browser backend like Ionic/capacitor where you can also write native plugins and utilize native APIs, which is impossible in progressive web apps (PWAs) that run in a browser sandbox.
Cross-platform apps on the other hand side use a native backend controlled by a unified runtime. E.g. ReactNative looks like the usual JSX output we have with React in the browser, but it's forwarded to an engine that mounts native UI components.

A bit of history

Mobile development really had hard times when there was more than Android and iOS and the browser wasn't well integrated. That changed over time. Actually you "only" have to support Android and iOS today and the browsers have so much power with new APIs like PWAs that you should of course even consider to use the plain browser as a suitable frontend.

Because of the reason that you have limitations in your browser sandbox it can also be difficult to make a good decision for now and the future, but good news is that around 2015 when browsers started PWA support and becoming more and more an app look and feel, there already was PhoneGap (known as Cordova framework nowadays) and Ionic/capacitor and Facebook announced ReactNative as a new cross platform mobile technology.

Today we still have PWAs in a browser sandbox with known limitations we cannot circumvent, PhoneGap/Cordova end of life was in 2020 so not a choice anymore, and we have ReactNative and Ionic/capacitor.

There is more like e.g. Flutter or Xamarin backed by Google and Microsoft, but it seems like their communities don't really grow fast enough to compete with the huge JavaScript/TypeScript community which is literally running the web front- and backend as well as being used in ReactNative or any hybrid app technologies like Ionic/capacitor. So I allow myself to question if Flutter or Xamarin should be recommended as mobile development technology today, which may of course change in the future, but for me today it's hard to believe.

That's the reason why I only see real native development for Android/iOS and JS/TS based hybrid- or cross-platform app technologies in the near and far future. With my ReactNative experiences the last 2 years and proof of concepts with Ionic I did for the current app I finally have an opinion for the technology I would use for my next mobile app. Let's start with some of the problems we faced using ReactNative.

Problems with ReactNative

Disclaimer: Some of the problems might not occur anymore or are due to reasons we caused. In the end it took a lot of time for investigation and I can't remember I had to deal with such partly nasty problems in other front-ends.

iOS HTTP2 nginx compatibility issue

The cool thing about HTTP2 is that actually the networking layer of any technology can upgrade to it at runtime if the server supports it without having to change app code so it's transparent for the app.

With a specific iOS version 14.x and higher we got strange HTTP2-protocol errors. Debugging down into ReactNative networking and reading native error logs we finally found out, that it seems like there is a problem when we send requests in parallel. Searching for it on the internet takes us to different threads where someone said it should be fixed on iOS while others say it should be fixed for nginx or forcing HTTP1. Only latter would be possible as a quick fix, but somehow experimental and as nginx is quite system critical component we decided to make iOS requests in serial for the affected versions and hopefully can remove that workaround after an official nginx version that fixes it or the iOS versions we support have it fixed.

I can't tell if that was a ReactNative issue or not, but I can't remember having such problems in the browser and the nginx version is an official AWS EKS ingress nginx.

RN updates

One of the golden rules of mobile development is that you should not wait too long for making updates. It's a ticking time bomb because in my opinion iOS and Android API development is still on a journey where they do not focus on API stabilization a lot, but make frequent breaking changes to new APIs while deprecating old API versions. That kick-off has an impact to all dependencies which also impacts your application.

That means ReactNative updates happen frequently and many dependencies especially new ones may not support yesterday's RN versions. At some point we had everything we need to implement new features without having to add new dependencies and then you should better have a different concept of keeping an eye on updates.

In our case we ran into the next nasty conceptual problem that forced us to add a new dependency which was not compatible to our current RN-version. Because there were no needs for our customer to update RN for about 20 months our update recommendations were not prioritized high enough and it was postponed to the future. But then we had to update RN and here comes the next negative criticism about RN.

We actually knew that it will not be just a small task for the afternoon, but the ReactNative update helper will run into patch problems and some manual effort is needed. Not talking about source code, asset- or dependency metadata like lock files or auto generated json assets we have:

iOS: 1600 lines of config
Android: 500 lines of config

And we also don't use manual linking which would blow up configuration so this is actually the raw configuration setup checked in to the code repository. Whenever there is new RN version, they provide a new empty project template and the update helper tries to patch your project. The larger the diff is through version updates, the likely it is that the patch fails. And that's what happened when we tried to jump from 0.63.3 to 0.66.3.

The diff was meaningless and so we compared the files by hand. Until that day we haven't had that much experience with iOS and Android setups. It took a lot of time, especially the iOS setup, because you have to modify and run the installation until it somehow worked. At some point one or another dependency could not build and you had to investigate why, until you have the next problem in the setup where the error message doesn't tell you anything about the actual problem, but like "cannot read /some/xconfig.rb" which looks odd as it's an absolute path, but having a look at the custom script reveals that some environment variable is not set for "${SOME_PATH}/some/xconfig.rb" and that's just one example out of many.

The good thing is, that it's more or less iterative, but we needed many, many iterations and I think almost two or three weeks to get everything fixed. Of course we also had to update npm packages because finally when we could deploy the application to our devices, it didn't start there because of an error at runtime.

In short, the iOS setup is mess of xcode project configuration no human should actually read and with custom inline bash scripts running Ruby or node.js scripts setting hidden environment variables and all in all really hard to debug. Android setup was much more easier, but also took some time.

RN "core" packages issues

The ReactNative runtime provides some of the APIs you also find in the browser and a react setup, that transforms JSX into native views, but it's lacking some of the core functionality you need in mobile apps starting with native screens and e.g. navigation gestures and ending at filesystem access. RN community is somehow fine to move that responsibility to npm packages that brings such fundamental features to the mobile app. Referring to the last section about RN updates those "core" packages are very tightly coupled to the runtime and the operating system's API that we of course also had to update them. And then one package broke our debugging setup.

It's important to be able to debug the application, but we don't need it every day. After the update journey we were so happy to consider it finally as resolved, but we were so wrong as debugging now freezes the application and it's not possible anymore. The solution was to change the underlying JavaScriptCore (JSC) engine to Hermes and use Flipper as tool.

Although it's feels like the change of the JS engine is the next upcoming nightmare it was really just a one line configuration change. Introducing Flipper was more to do, but in contrast to the RN update it was okay.

Debugging

Since ReactNative is not a hybrid app, although it feels like we have JSX and CSS, we can't use well-known debugging tools like the dev tools in the browser. It's not quite correct, because indeed the React-Native-Debugger is an Electron application simulating the runtime on the local machine and has dev-tools where you can debug network requests like in the browser, but not everything works as expected (like file uploads).

Same for debugging the UI where you can update CSS classes or styles in the browser dev-tools, but with ReactNative you only have some smaller debugging features. Either use them or default hot module reload and hack in your IDE and check your device which also sometimes doesn't work for styles. If you know what browsers can do for debugging you feel a bit far away or even lost in ReactNative, although it seems to be getting better over time like Flipper is much more better integrated for debugging.

What else if not ReactNative?

As I've mentioned before this is very opinionated, but the remaining technologies for mobile app for me is still native development or Ionic / capacitor. The decision to use one or another shouldn't be too difficult, as you can have the same technology for iOS, Android and browser apps with Ionic / capacitor, while you may want to stick back to the roots of native mobile development for maybe serious UI performance reasons or you already have native development teams with good performance.

Ionic / capacitor

Now I owe you an explanation why I always wrote "Ionic / capacitor". So capacitor is the technology that brings the browser runtime to the device, while Ionic is a set of UI components that should look like being native. So it's also possible to drop capacitor into your web app and make it a mobile app, if it has a responsive UI.

Comparing it with some disadvantages of ReactNative

Debugging on the local machine in any browser with all browser tools is always possible and you can also use Safari for debugging your iOS application or Chrome for Android. Anyway from our ReactNative experience most of the debug sessions were not device related and within a browser runtime it's more likely not a device issue
"core" packages are maintained by the Ionic community and backed by the company itself
full-featured browser runtime on mobile devices using native web-view and the browser itself for desktops allows the use of the browser APIs, like fetch-API, web-worker or IndexedDB should be possible everywhere, while WebSQL is only supported on chrome based front-ends, like Chrome or Brave for desktops or the chrome-based web-view on Android, but not Firefox, Safari or iOS at all using some Safari-based web-view.
like with ReactNative, we don't have to make a full build every time because hot module reload can update the JS source, but on the other hand a full capacitor build takes a few seconds, while our current ReactNative application needs about 2-3min for a full build and sometimes like when you have to update everything a faster build is better

That comparison is maybe a bit unfair, because it's a full-blown RN app vs. an almost empty Ionic app and of course build times are better and whatnot, but I assume it will not have the problems we had with ReactNative:

The application is using fundamental browser APIs in the first place. That is primarily a browser DOM out of HTML, CSS and JS for the UI, which obviously never breaks. Then the fetch-API or IndexedDB which is provided by the web-view, but not implemented by capacitor also have very stable APIs.
Gestures, animations and navigation is maybe not 100% accurate to their native UX, but Ionic is again using browser APIs and we can use well-known libraries like react-router for navigation. It's much more easier to think in URLs than in screen names and parameters.
Finally, it's also not possible to break debugging :).

Conclusion

It's very important that as a developer we are not too far away from the code we write. That means we should be able to frequently change and verify it with almost no costs and mature technologies like the browser have so many introspection features for apps, that we may only use a few, but we're at least standing on the shoulders of giants.

I also don't see a problem, that a web application looks and feels like a mobile application and I think that "one UI for everything" is mostly a good idea and saves money. If you want to change some UI components for the desktop then simply go left or right, but not for the whole application because it's mostly not worth the time, I think.

Ionic is a very good example how to achieve that and from what I read and tested within the proof of concept, I've already recommended to switch to Ionic away from ReactNative.

Vulkan - some introductory words

Philipp Renoth — Thu, 07 Oct 2021 22:17:53 +0000

A new graphics API was born

There are a few popular "low-level" graphic APIs available like OpenGL, DirectX, Metal and Vulkan. Depending on your requirements you had to pick one or use some higher-level graphic abstractions to be able to develop for multiple platforms with a single code base.

For a long time, before Vulkan, only OpenGL was "cross-platform". OpenGL wasn't famous for its performance and therefore often not supported by AAA-games. Most of them use DirectX on Windows.

Now it's 202x. It feels a bit like waste of time, that we need to use different graphic APIs on different operating systems using exactly the same hardware under the hood. Only with OpenGL it was possible, but the amount of resource overhead needed to for example run a game with OpenGL compared to the same settings on Windows with DirectX seems to be a dealbreaker for most game companies.

The next milestone towards cross-platform graphic APIs was set in 2016 with Vulkan. Behind its development is none other than the Khronos Group, well-known for OpenGL and many other open standards. In 2018 commercial Apple drivers (MoltenVK) for Vulkan got open-sourced and since then free to use.

Vulkan is also well suited for mobile devices. We were talking about resources and performance on desktop machines. Well, mobile device app developers should also care a lot about performance and Vulkan can help, especially superseding OpenGL ES.

How to start writing a Vulkan application?

I think most of us using a new API just read through some specs and finally start. That's possible, but without a good background you will not understand it at all. The next thing is, that a simple "hello world" triangle in a Vulkan C++ application has about 600 lines of code. Changing its color per frame, double/triple buffering or input vertex buffers easily takes you beyond a thousand lines of code. And I think that's also the moment, when developers suddenly start feeling fine with OpenGL. But don't despair, it's not too hard.

1. Compare some Vulkan and OpenGL code

Comparing 600 lines of C++ code using Vulkan to 100 lines for OpenGL should maybe make us think what OpenGL is doing behind the scenes. One of the big changes coming from OpenGL is that state changes are now not only expensive in performance, but also in writing additional code. In Vulkan we kinda pre-bake graphic pipelines and run pre-recorded command buffers so e.g. changing to wireframe rendering would be maybe another graphic pipeline with an almost identical setup. At least it's not just one gl... call in the render loop.

The reason why everything seems to get more complicated is because we give more detailed information to the Vulkan driver, so optimizations are better. In OpenGL everything can happen at any time and the driver needs to be very smart. The good thing on the other hand side is that the render loop is much more compact.

2. Understand why some Vulkan boiler-plate is necessary to get the most out of the hardware

A good starting point without copying code from tutorials not knowing what it does is to understand how today's hardware is working. E.g. before Vulkan I haven't heard of tiling. I could not understand why linear reading from an image buffer storing it like a screenshot is maybe not possible. But after reading about hardware and caching it absolutely makes sense.

Kudos to Samsung's Galaxy GameDev pages. You can find some good resources about how graphic hardware works and how Vulkan is working.

3. Getting comfortable with the base Vulkan architecture and the first steps

Although there are a lot of good resources, too, I'll try to give a much more compact overview.

Vulkan C API

Like OpenGL, Vulkan is a dynamic API with spec versions. Depending on the version you should be able to dynamically load function pointers
There are 3 API-tiers. Every program starts with discovering the hardware setup top-down and finally picks one ore more suitable devices for rendering or computation:
- entry-point functions
- instance functions
- device functions
Function return value is a Vulkan result code or void. All other return values are out parameters. Mostly functions with void return value are somehow validated at a subsequent API call.
Functions may return bad result codes for expected errors or simply crash for unexpected input or state. The driver is sometimes not even wasting time for null-checks of pointers just crashing, while there is a so called optional validation layer for debugging and tracking bad usage, like a bad order of destroying handles.
There is no thread context at all. Handles can be passed to other threads as well. Multi-threading is possible and good for performance, primarily for command buffer operations.

Vulkan architecture overview

On top we have so called Vulkan instances, like a context boundary.
Inside an instance we have physical devices.
Physical devices can be used to discover hardware features and limits to determine a suitable target device to use
Out of a physical device, we can create a logical device handle which will be the mostly used handle in the application.
(Logical) devices have so called (command-) queues to push working items. There are queues for graphic- and presentation operations. It can also be that there is only one queue doing both. Queues are CPU-concurrently overlapping processed and each queue in parallel. We may need synchronization:
- host sync(hronization) via fences => CPU thread blocking
- intra-queue sync via semaphores => disallow overlapping
- inter-queue sync via events => sync queues
There are a lot of other objects, like a pipeline, buffers, memory, descriptors and so on, but they are just building blocks for recording command buffers, which are finally submitted one or multiple times.

Vulkan pseudo code

setup()                     // A
loop {
  i = wait_for_next_image() // B
  update(i)                 // C
  draw_and_present(i)       // D
}

wait_idle()                 // E
teardown()                  // F

A: setup device, queues, buffers, memory etc.
- e.g. if we calculate frames ahead, we may need multiple buffers/memory allocations to allow render overlapping with writing buffers
- static immutable buffers can be shared
B: as long as the swapchain doesn't have some already presented and not queued image, we stop here
C: we update whatever we need for index i
D: we draw and present i-th image
E: before we destroy everything, we need to wait until it's not used anymore
F: destroy in LIFO-fashion

4. Go through some tutorials

Vulkan Tutorial eBook + GitHub repo
- step by step source files and shaders of that tutorial
SaschaWillems Vulkan example triangle
Vulkan Guide for more

5. Do your thing

Well, this is where I stop for now. I think I'm somewhere between tutorials and guides and still cannot yet "do my thing", but the more time I spend reading, understanding code and iterating, the less I have to copy code, because "it makes sense".

Conclusion

Vulkan API is not a "natural" thing like a web shop API. A lot of internals are exposed and it feels like it's overall boiler-plate code, but it's actually about performance, a good memory design, smart and small updates and clear picture about staging work that hardware is never getting idle. All in all progress is sometimes not that easy, but I think for future development it's a good choice to adopt Vulkan - also called glNext as the successor for OpenGL.

There are a lot of beginner tutorials and expert stuff like ray-tracing and I see a gap in the middle, so I'm planing to share some experience and some of my best practices for intermediate Vulkan content.

Understanding Rust as a C++ developer

Philipp Renoth — Wed, 05 May 2021 20:04:02 +0000

Rust has a steep learning curve and I would also say, not even for professional C++ developers it's an easy switch, although it's said that when you have some C++ background, Rust and you are getting best friends, soon. Well, I really love Rust, also having a C++ background, but the first steps were hard.

One thing you can do is reading "The Rust Programming Language" book for free. It's well-written, but as a C++ developer, we already have some common understandings which would really makes it easier to dive into Rust with the C++ wording. I hope this article may close this gap to get you on board as a C++ developer.

As a C++ developer, why should I learn Rust?

Check out some of latest news:

Rust is to be introduced as the second language in the Linux kernel.
Microsoft with a large C++ codebase uses Rust for new modules. According to their CVE analysis, 70% of all CVEs are memory related bugs and Rust is kinda famous to not allow memory bugs.
Google is adopting Rust for Android.
Rust is "the language" for WebAssembly, which is also getting more and more import for the cloud native world.

From my experience with Rust:

I have much more less bugs (no random segfaults or undefined behavior) my components are better designed
I can write code way faster
I can use Rust everywhere: embedded, native, web/browser

To use Rust is not an either or choice, so you don't have to change everything, but you can simply start adopting Rust for some non-critical components and get a feeling and I can almost promise that you'll love it.
That's also where we start. Calling into Rust or C++ with FFI.

Foreign function interface (FFI)

Probably, C++ projects will not be entirely written in Rust from one day to another, just because Rust has many advantages. One thing that is happening at some companies is, that they start to adopt Rust and keep their C++ codebase while developing small meaningful components with Rust and include them.

In order to allow calling from C++ into Rust and vice versa, Rust has to follow calling conventions and other interface specifications. For optimization reasons the Rust compiler mangles everything: function names, attribute and parameter order and so on. Also the types have to match, like long from C++ is the same like a i64 in Rust, but only for 64bit arch, so we should use isize for that case. Rust is also not compatible to C++ char[] for string types. Rust is using UTF-8 encoding with explicit length, unlike C++ has null-byte terminated char buffers. std::ffi module has some convenient helpers bringing those worlds together.

Rust also supports pointer-types like we have in C++, but you will not likely have to use them, unless you do high sophisticated Rust development or more likely doing FFI. To use references in Rust, we don't do that with pointers, but so called borrows. I also spend a whole section for that topic.

Primitive data-types

Let's start with the real basic types:

// comments are C++

// void u; doesn't work
let u = ();

// bool b = true;
let b = true;

// char c = 'c';
let c: i8 = 99;
// unsigned char uc = 'u';
let uc    = b'c';

// short s = -8;
let s = -8i16;
// unsigned short us = 8;
let us = 8u16;

// int i = -8;
let i = -8i32;
// unsigned int i = 8;
let ui = 8u32;

// long long ll = -8;
let ll = -8i64;
// unsigned long long ull = -8;
let ull = 8u64;

// float f = 8.f;
let f = 8f32;
// double d = 8.0;
let d = 8f64;

// long i = -8;
let l = -8isize;
// unsigned long i = 8;
let ul = 8usize;

// unsigned char buf[1024];
let buf = [u8; 1024];

let t = "hi"; // type "&str" (utf8 u8 buffer)

So the naming convention of the primitive types in Rust is quite simple. A special case is the type &str . That's a reference to a byte buffer, but it's representation is UTF-8 so you cannot have random access. What you can do is to make a so called slice (like an array) out of it and then you have random access, but of course, not to the characters itself, but the underlying bytes. We will cover more details in the "Strings" section.

Arrays and slices

Arrays

// C++
long a[] = {1, 2, 3, 4};
std::array<long, 8> b{1, 2, 3, 4};

// Rust
let a = [1, 2, 3, 4]; // type is [i32; _]

Arrays in C++ and Rust is contiguous data, where every item has the same size. For FFI if you need a pointer to an array, it's safe to use my_array.as_ptr() (also same for Rust slices).

Slices

It's hard to compare std::slice with Rust slices, because Rust references data, not only index, size and stride information, like std::slice does. Rust slices store a pointer and the length. The stride is known from the type itself:

// Rust
let a = [1, 2, 3, 4];
let b = &a;             // type &[i32; _] => array borrow
let c = &a[..];         // type &[i32]    => slice
let d  = &a[1..3];      // type &[i32]    => slice
println!("{:?}", &a);   // [1, 2, 3, 4]
println!("{:?}", b);    // [1, 2, 3, 4]
println!("{:?}", c);    // [1, 2, 3, 4]
println!("{:?}", d);    // [2, 3]

Initialization

We don’t have uninitialized data in Rust. When we declare a variable, we also have to initialize it with value.

// Rust: not working
let i;
let a = i + 4;

Although it’s valid Rust to separate declaration and definition it’s quite unlikely to see this:

// Rust
let i;
// ...
i = 5;

Mostly we do this:

// Rust
let i = ...;

To say that there is no uninitialized in Rust to not quite correct. Well, there is, but without unsafe Rust we can't access it and no data structure should expose uninitialized data. For example Rust also has a vector called Vec and we can create one with a given capacity, so the needed size is allocated and the memory is uninitialized (because it's faster than zeroed memory). Vec doesn't let us access uninitialized elements, so we are safe.

Type inference

In Rust we don’t have to write types on the left-hand side of an assignment statement, when it can be inferred from the right-hand side. In C++ we have auto, but a type that really doesn't do unnecessary copies whenever we use it on the left-hand side, is the universal type auto&&.

// C++
auto&& text = "some text";
auto&& i = 8; // int
auto&& j = i; // &int

// Rust
let text = "some text";    // vs. let text: &str = "some text";
let i = 8u32;              // vs. let i: u32 = 8;
let j = &i;                // &u32

There are also generic types, which can be inferred, even for return values, so we either pass the generic parameter type to the function or we simply give the left-hand side a type, so the generic parameter on the right can be inferred from the left.

Functions

fn get_five() -> u32 {
    5                            // no semi-colon
}

fn add_five(value: u32) -> u32 {
    value + 5                    // no semi-colon
}

fn main() {
     let five = get_five();
     let ten = add_five(five);
}

A function starts with the keyword fn and has the return type at the end like -> u32. If the function doesn't return anything which is the unit type (), then you can omit it, like in the main function. Rust also has the return keyword, but it's mostly used for "early return", because in Rust the last expression without an appended semi-colon is the return value.

Meta programming

Rust provides an advanced macro system in contrast to C/C++ "replace" macros. In short, at first it's possible to do the same like in C/C++, but you can also be more specific about the macro parameter types and second, you can use macros as a preceding phase where Rust code can create other Rust code to be compiled. This allows us to do incredible things, like e.g. embedding preprocessed resources or do compile-time integrity checks of data we embed. Calling Rust macros is like with functions, but with ! at the end.

let name = format!("{} {}", get_first_name(), get_last_name());

What look’s like a waste of resources in Rust with string formatting with format! is the way to go when you want to concatenate strings. Everything is done at compile time and it's safe, not like printf in C with runtime cost. The string is parsed at compile time. For all curly braces there has to be a parameter. If not, it will not compile.

Embedding a file is also very easy, so that the file's content is in the binary itself.

let image_data = include_bytes!("./image.png");

Strings

C++ has std::string and char * (and some other flavors, like wide strings). Rust has byte slices [u8] or [i8] as a byte representation and &str and String as UTF-8 string.

// C++
const char* t = "some text";
std::string s(t);
std::string a = s + " in C++";

// Rust
let t = "some text";
let s = t.to_owned();
let a = format!("{} in C++", &s);

In Rust every value or variable has a type and all types have some functions to be called, even numbers. &str implements .to_owned() which makes a String out of it. The same is happening as in C++: heap is allocated and data is copied, so the string is now owned.

Optional values and exceptions vs. results

One of the core types in Rust (beside the primitive ones) is Option and Result. Rust doesn't have exceptions, so without that side-channel our return value needs to wrap information about it. Those two types are also part of the prelude, so we don't have to "import" and just use them.

`Option<T>`

The fact that a value exists or not can be represented by an Option in Rust and like std::optional in C++. Let's say we build an CLI argument parser.

fn is_dry_run() -> Option<bool> {
    Some(true)
}

fn get_host() -> Option<String> {
    None
}

fn main() {
    let dry_run = is_dry_run().unwrap_or(false);
    let host = get_host().unwrap_or("localhost".to_owned());
}

Option has a lot of convenience methods like unwrap_or which is like std::optional::value_or.

`Result<T, E>`

I think C++ doesn't have such a wrapper type in std, but one can simply return the success value or throw an error. Result in Rust has two generic parameters: the success value type and the error type.
Let's say we want to get the current user name form a DB or return the error code.

fn fetch_current_user() -> Result<String, u32> {
    Ok("robert_rust".to_owned())
    // ...or
    // Err(34)
}

That way the result can have two different values and it's not possible to access let's say the success value, if the return value is Err(..), because it will immediately "crash" (we say "panic" in Rust and we'll cover that in the next session as well).

`?`-operator

Don't confuse it with the ternary conditional operator in C++. It's some syntactic sugar. So we don't have exceptions on the one hand, but writing error handling boilerplate on the other hand is also not fun. This is why we have a new nifty operator for "early return".

For the next example we have some functions for a DB and of course, they may fail see we use Result.

fn connect_db() -> Result<(), u32> {
    Ok(())
}

fn fetch_current_user() -> Result<String, u32> {
    Err(34)
}

Let's glue it together with all boilerplate code. To be honest there are really better ways without ?-operator, like with match and we will cover them soon, but let's make it real bad first.

// not so good Rust :(
fn fetch_users_from_db() -> Result<String, u32> {
    let connect = connect_db();
    if connect.is_err() {
        return Err(connect.unwrap_err());
    }
    let current_user = fetch_current_user();
    if current_user.is_err() {
        return Err(current_user.unwrap_err());
    }
    Ok(current_user.unwrap())
}

We store the result in a variable, we check if it has an error and then we unwrap the error and wrap it in another Err which is a Result and we have to do that for all checks. That's not so readable. Let's use the ?-operator.

fn fetch_users_from_db() -> Result<String, u32> {
    connect_db()?;
    let current_user = fetch_current_user()?;
    current_user
}

Or even shorter, because the result of fetch_current_user is of the same return type like fetch_user_from_db().

fn fetch_users_from_db() -> Result<String, u32> {
    connect_db()?;
    fetch_current_user() // omit semi-colon
}

That even works for Option when None values should be "early returned". When you have different error types, you need to map them to the the same type to be allowed to use the ?-operator, but that's more advanced Rust, we skip for now.

`std::terminate()` vs. `panic!()`

No exceptions in Rust, but when there is something going so terribly wrong that there is no chance to recover, we can panic the current thread. Doing that on the main thread means our application will crash. An uncaught exception in C++ has the same behavior.

Rust allows us to write robust and safe code, but you can make wrong assumptions, that will panic at runtime.

let name: Option<String> = None;
println!("{}", name.unwrap());    // means "it MUST be Some(..)"

For example all .unwrap...() methods of Option or Result will panic if it can't unwrap. Another example are "index out of bounds" panics. Consider panics as a last resort for a Rust program to terminate, before more bad things may happen. Panics are always unexpected errors. E.g. a connection to a database may break, so it's an error but not unexpected.

Control flow — conditions and loop

// comments are C++

// if (i == 4) {
// } else {
// }

if i == 4 {
} else {
}

// switch (i) {                      
// case 3:                               
// case 4:                               
// default:                              
// }

match i { // no fall through!
    3 => {}
    4 => {}
    _ => {}
}

// for (int i = 0; i < 10; i++) {
// }

for i in 0..10 {
}

// while (i < 10) {                  
// }

while i < 10 {
}

// do {                              
// } while (i < 10);

loop {
    if i < 10 {
        break;
    }
}

There is more. if, match and loop are also expressions in Rust, so they can return a value which is quite handy for many use-cases.

// C++
int v = i < 10 ? 3 : 5;

// Rust
let v = if i < 10 { 3 } else { 5 };

The C++ code may look more pleasant, but let's do more in the conditional blocks.

let meal = if customer.is_very_hungry() {
    let burger = new_burger();
    for i in 0..3 {
        burger.add_paddy();
    }
    new_plate(burger)
} else {
    new_plate(new_cheeseburger())
};

The elegant part of that code is, that the outcome of if is stored in meal, so it can also be guaranteed that it is initialized. In C++ it might be better to put everything in another function or an immediately invoked lambda.
We can do the same with match and even loop.

let age = 24u32;
let score = match age {
    0..=18 => 0,
    19..=35 => 1,
    _ => 2
};

Another very useful thing about match is that all patterns from all arms have to be exhaustive regarding the input type. Code will not compile if we omit the _-placeholder which matches "the rest" of it all. That's very helpful for enums, because let's say an enum has 3 possible values and you all handle them, and then someone adds a forth one, then you have to handle that case, too, because it's a compile error to be non-exhaustive.

Enums

I'd say enums is Rust are a combination of C++ enum, union or std::variant. It's not only the case, that we define enumerators, but they can hold different types each and that's a Rust killer feature. We can even match the type and unwrap its inner value together in one line.

enum Error {                 // an app error type
    DbQuery(u32),            // db has error codes
    Unexpected(String),      // everything else should be a string
}

fn handle_error(err: Error) {
    match err {
        Error::DbQuery(code) => {},    // code is the u32 db code
        Error::Unexpected(msg) => {},  // msg is a String
    }
}

We've already used the most popular enums in Rust: Option and Result.

OOP

Most of the things in C++ can also be done in Rust and for a few things we need change our mindset. It may look like a limitation, but Rust community didn't introduce every known OOP feature from all languages. Even such basic OOP features like inheritance is nowadays often considered harmful, because it introduces complexity or even a bit phoniness, a straight specialized design may not have.

struct/class vs. struct/enum

In C++ we have struct and class and they can have attributes and methods. In Rust there is struct and enum with the attributes and impl with its methods.

struct Person {
    first_name: String,
    last_name: String,
}

impl Person {
    fn say_hello(&self) {
        println!("Hello, I'm {} {}", &self.first_name, &self.last_name);
    }
}

enum Value {
    High,
    Low,
}

impl Value {
    fn is_high(&self) -> bool {
        match self {
            Value::High => true,
            Value::Low => false,
        }
    }
}

What looks a bit strange at first, is the &self parameter. That parameter has an implicit type, namely the struct itself, so it's like the this from C++. Given &self says, that it is an object method. Omit it and it's a static method.

impl Person {
    fn name_placeholder() -> String {
        "Please enter your name".to_owned()
    }
}

fn main() {
    let placeholder = Person::name_placeholder();
}

Inheritance vs. traits

There is no inheritance (or overloading) in Rust, but it’s possible to extend the implementation with a so called trait and they also allow to have abstract interfaces. Rust is also quite explicit about dynamic dispatch, as it adds additional runtime costs (same in C++) and most of the things that are not obvious for free, have some code hints in Rust, like dyn for dynamic dispatching.

trait HasName {
    fn get_name(&self) -> String;
}

impl HasName for Person {
    fn get_name(&self) -> String {
        format!("{} {}", &self.first_name, &self.last_name)
    }
}

fn good_bye(sth: &dyn HasName) { // dyn for dynamic dispatch
    println!("Good bye {}", sth.get_name());
}

Let's simply assume & is like a reference in C++. There will be a section about references, borrows and so on, but for now we are okay to know that this is a reference to some HasName. Because HasName is a trait, the call has to go through a vtable. The same happens in C++ with virtual methods, but we don't see it there. Here, Rust clearly says, that we have a variable where all calls are dynamic dispatched, because the type is a trait. The additional costs are not that much, but we may don't want it, because we are in a tight loop and performance matters and with dynamic dispatch the compiler cannot make further optimizations like inline the invoked method, because at compile time Rust doesn't know which specialized function will be invoked at runtime. In "Templates vs. generic code" there will be the same example, but with monomorphization to make it static dispatched.

con-/destructor vs. constructor/`drop()`

Now what about a constructor and destructor in Rust? While a constructor in C++ is a special method, it's simply a static method in Rust, like a factory method. Per convention its name is new, but one can use any name, e.g.

let s1 = String::new();               // -> String
let s2 = String::with_capacity(1024); // -> String

Let's also give Person a constructor.

impl Person {
    fn new(first_name: String, last_name: String) -> Person {
        Person {
            first_name, // shorthand for "first_name: first_name"
            last_name,
        }
    }
}

There is also a destructor in Rust and it's quite interesting how it's used. There is a special trait Drop with one method, which is called from the Rust compiler, when an object goes out of scope.

impl Drop for Person {
    fn drop(&mut self) { // we will handle "mut" later
        // ...
    }
}

That should be enough for OOP. There is a section about "Const-correctness vs. mutability" which will dive into OOP again. Then we clarify what mut is.

Templates vs. generic code

Generic Rust code is similar to C++ templates. It's compile-time generated code for all different use-cases in the code, so we write it once and the compiler inserts it multiple times for every different generic parameter set.

The big difference is, that C++ templates are not checked by the compiler, until they are used and the generated is compiled, more or less like macros. It's possible to put constraints on the generic parameters to only allow what really works for the implementation, but at least everything is allowed as long as it finally compiles.

// C++
#include <iostream>
template<class T> // any T allowed here
void good_bye(T& sth) {
    std::cout << "Good bye " << sth.GetName();
}

struct Person {
    std::string name;

    const std::string& GetName() const {
        return name;
    }
};

int main()
{
    Person p {name: "Charlie Cohan"};
    good_bye(p); // okay
    good_bye("bad value"); // error: request for member ‘GetName’ in ‘sth’, which is of non-class type ‘const char [10]’
    return 0;
}

Rust is different as we have to write generic code which is valid for all possible values. Going back to our trait HasName:

// Rust
fn good_bye<T: HasName>(sth: &T) {
    println!("Good bye {}", sth.get_name());
}

// or different (just cosmetics)
fn good_bye<T>(sth: &T)
where
    T: HasName,
{
    println!("Good bye {}", sth.get_name());
}

// , but not
fn good_bye<T>(sth: &T) { // notice missing type constraint here
    println!("Good bye {}", sth.get_name()); // err: no method named `get_name` found for reference `&T` in the current scope
method not found in `&T`
}

I remember situations, when I had no clue what the C++ compiler wanted to tell me, like generic code errors can get very confusing. Rust will tell if you wrote incorrect generic code. If you really (really) need fuzzy templates in Rust you can you use macros.

There is another point worth mentioning here, which also applies to C++. What we've done here is called monomorphization. We've already had the good_bye function with dynamic dispatching. The versions we create with the generic code is for each individual type when we use the function, so the type of the borrow is not &dyn HasName, but the specialized type e.g. &Person which allows static dispatching with all compiler optimizations. It's a trade-off between performance and larger binaries, but I think larger binaries are mostly okay.

Namespaces vs. modules

Like C++ namespaces, modules in Rust tie together code and hide implementation details. There are strict rules about the module hierarchy and the source file locations in Rust, while in C++ each file is considered without any context like file location. Because Rust's module system is quite different (at least to the ones I know from other languages), I spend another article for that topic:

Rust module system

Philipp Renoth ・ May 5 '21

#rust

`const` vs. `mut`

In C++ everything can be mutated if not const. In Rust it's the other way round: everything is immutable if you don't make it mut (for "mutable").

// C++
const int j = 4;
int i = 4;
i = 8;

// Rust
let j = 4;
let mut i = 4; // mandatory "mut"
i = 8;

Rust has a const keyword, but we don't use it for making a variable const as this is the default. There are other use-cases for const in Rust:

for global values e.g. const SOME_VALUE: u64 = 8;
for FFI compatibility e.g. *const u32

Be prepared, for Rust mut is more than just a keyword whether we change something or not. There is a section "Const-correctness vs. mutability" where we will talk about the difference.

Passing Values — reference, borrows, ownership

In C++ there are maybe 6 or more value types, like "lvalue", "rvalue", "xvalue" and so on. In short: forget them for Rust.

Rust has 2 value types:

borrow: like a C++ reference
ownership takeover: like C++ "move", but a bit different so I also spend a section for move semantics

// Rust
fn foo(s: &S) {}
fn bar(s: S) {}

fn main() {
  let s = S {};
  foo(&s); // borrow, see the "&"
  bar(s);  // ownership takeover
  // don't use `s` here anymore (or it will not compile)
}

struct S {}

One of Rust's core concepts is "ownership". If you want to know more about it have a look at this article:

Rust ownership and borrows - Fighting the borrow-checker

Philipp Renoth ・ May 5 '21

#rust

So there are only 2 options in Rust I'll show with some C++ code:

// C++
struct S {};
void foo(S& s) {} // reference
void bar(S s) {}  // by value:
                  // copy-constructed with lvalue (not in Rust)
                  // move-constructed with rvalue

int main() {
  S s = {};
  foo(s);            // reference, but we don't see it here
  bar(std::move(s)); // make rvalue of s, so it's move-constructed
}

The first foo function is taking a reference, which is quite the same in Rust, beside the borrow rules we cover a few sections later. The second bar function is getting more interesting. It's a by value parameter, so according to C++ OOP it's possible to create copy-constructed or move-constructed S. The first one is not possible in Rust as we can only pass a reference or an rvalue in Rust (but it's of course possible to pass a "copy" see the next section):

reference means: you can use it, but you just borrowed it. It's not yours.
ownership takeover: it's like passing a rvalue in C++ where you are not allowed to use the old lvalue anymore. If you use it, it will not compile anymore.

Understanding move semantics in C++ can become quite heavy. That's really gotten easy in Rust. It's not possible to implement that on your own, so the compiler is in full charge to do a good job for us.

Lifetimes

Both C++ and Rust have lifetimes for data, but the difference is, that C++ is very permissive according their checks and the language C++ doesn't support any lifetime annotations so we have conventions, good and bad practices, like the compiler allows us to do that with a warning:

// bad C++
int& get_int() {
    int i = 4;
    return i;
}

In that case the compiler has some checks which say, that returning locals as reference may be a bug. But let's have a look at this horrible piece of C++ code which should hopefully not compile, but it does.

// very bad C++
#include <iostream>
struct Person {
    std::string name;
};

int main()
{
    std::string* name;

    {
        Person p {name: "Charlie Cohan"};
        name = &p.name;
        std::cout << "Name: " << *name << std::endl; // ok
    }
    std::cout << "Name: " << *name << std::endl; // bad!
    return 0;
}

Now let's try to crash a Rust program.

fn get_int() -> &u32 { // err: missing lifetime specifier
    let i = 8;
    &i
}

The first error is, that when we return a borrow, we need to be explicit about a lifetime. Lifetimes in Rust look like 'a or 'b or 'whatever_you_like. Then there is another special lifetime 'static which means "it lives for the entire runtime of the application". For example let t = "text"; is of the type &'static str, because "text" is from the data segment and valid through the whole program.

Now, let's fix that and face the next issue.

fn get_int<'a>() -> &'a u32 {
    let i = 8;
    &i // err: returns a reference to data owned by the current function
}

So what is a warning in C++ is an error in Rust and that's good. Let's try the other example:

// Rust
fn main() {
    let name: &String;
    {
        let p = Person { name: "Robert Rust".to_owned() };
        name = &p.name; // err: `p.name` does not live long enough

        println!("Name: {}", name); // ok
    }
    println!("Name: {}", name); // err: borrow later used here
}

struct Person {
    name: String,
}

Rust compiler says: "I will not allow you to use name after the lifetime of p and p is dropped inside the block, like in C++. So the Rust compiler also terminates the lifetime of name inside the block to fix the constraint that the lifetime of p is shorter.

This is another reason why Rust is safe. We are not allowed to violate those constraints and the borrow-checker ensures that.

`Copy` vs. `Clone`

Rust is very restrictive and there is a difference in cloning or copying data. You're not supposed to "duplicate" data only because you want to. The only data you're allowed to do that are scalar primitive data, like u32. No struct has a default implementation for copying or cloning data and that's for a reason, because for example, how does Rust know that you're not storing resource handles as u64 and duplicating the struct means double freeing the resource when it drops?

`Clone`

What's called copy-constructor in C++ is cloning in Rust and the convention is different, because we have a simple object function implemented called .clone() and return the clone. To be more precise the trait Clone is implemented. Rust will never ever make a clone for you, unless you call it. So in other words, if someone wants to takeover ownership, but you don't want to give away ownership, pass a clone (if possible).

fn main() {
  let s = "text".to_owned();
  foo(s.clone());
  println!("{}", &s);
}

fn foo(s: String) {} // wants ownership

A clone can be expensive during runtime, like a vector is doing heap allocation. That's why we have to call it and we will read it in the code. In C++ it's possible that we're copying data around, where there is no need to, because like e.g. we're passing lvalues to pass by values parameters, although we actually want to move it.

`Copy`

Simple data structures can be marked as Copy as long as their attributes also implement it, which means that the compiler is allowed to make a bit-wise copy and the structure itself has to be Clone, because a Copy struct should always also be Clone.

fn main() {
    let mut f = Fraction{ nom: 3, denom: 10 };
    let c = f; // can't be moved as we want to borrow "f" later,
               // but a copy fixes it
    f.nom = 4;
}

#[derive(Copy, Clone)] // Rust magically implements everything
struct Fraction {
    nom: u32,
    denom: u32,
}

Tuples

Since C++11 we have e.g. std::tuple<int, long>. In Rust that's simply the type (i32, isize). You can't have multiple return values in Rust, but you can return a tuple, which is the same. There is a special tuple called "unit" in Rust which is () and is what void is in C++, so nothing.

// C++
std::tuple<int, long> values {1, 2};
int i  = std::get<0>(values);
long l = std::get<1>(values);

// Rust
let values = (1i32, 2isize);
let i = values.0;
let l = values.1;

Memory management

Like C++, Rust also ...

doesn't have a garbage collector
has two memory types for dynamic data: stack and heap. Everything with fixed size (not too big) can be allocated on the stack, while dynamic sized data has to live on the heap.
has no runtime per default (no overhead, just a main thread and a main() entry point)

Unlike C++, Rust ...

also has move semantics, but implements it differently, so I spend a complete section for it
has a default safe mode, which should not allow undefined behavior, e.g. memory corruption, dangling pointers, double free, used after free and so on.

Runtimes

There is also no runtime default in Rust. We have a main() entry point and the main thread running it. Especially IO-heavy Rust applications, like many web applications are, often stick to so called "async-runtimes" like tokio being an executor for Rust's async/.await syntax for non-linear async code. Just mentioned it, we will not cover that. Another problem solved are transitions from computation-heavy to IO-heavy code and back or using legacy/blocking APIs. Those runtimes solve similar problems like ReactiveX (Rx) or boost async package does.

Move semantics

C++ and Rust have move semantics, but they are very different. I think it's important to understand the difference so let's recap move semantics in C++, first.

C++ move

The main use-case in C++ is instead of copying data around, we move internal data, that e.g. strings or vectors can be moved, by simply transferring the heap pointer from our stack-object to another object and leave the old object in a valid state. One should actually not use them anymore, after they have been moved and at least they are destructed, so it has to be valid, that e.g. the memory is not double-freed. In C++ having classes without explicit dynamic memory allocation (new, malloc) we (mostly) also don't need to implement the move semantics on our own, as the default is what we want. I said "mostly" because there might be some edge-cases where a move is more complicated and need to be implemented manually or someone holding a pointer to an object (for whatever reason) and moving that object then is not allowed.

To make one's life easier, we can use smart pointers in our classes, so there is no dynamic memory allocation done by the class itself and default implementation should be fine. If one of the attributes doesn't implement "moving" or "copying" nor either does the class, e.g. copying unique_ptr will not work.

Rust move

Good news is, that we have another concept in Rust called "ownership". I'd call it a good practice in C++ to have every piece of data being owned by another component, but even the definition of "ownership" in C++ can be vague. Modern C++ with unique_ptr obviously expresses who owns that data, but what about a function returning a pointer? Should the caller free it, is it somehow garbage-collected or what's happening? In that case a function should have documentation about ownership.

We always have two objects involved in C++, where one moves into the other and obviously the first one going out of scope and getting destructed. In Rust "moving" means, that object data is copied bit-wise and the source is not "destructed". That boilerplate is implemented by compiler and he's even allowed to not really "move" but just reference the data and shorten the scope. From our perspective it is moved, but compiler is in charge to optimize it. So whatever data structures we create in Rust, they are all "move"-ready! Isn't that great?

To be more precise they are all "move"-ready, but there is a concept called "pinning" which suppresses moving data for a good reason coming from the async/.await world of Rust, but we don't cover that here.

Const-correctness vs. mutability

For scalar values to be changed, they need to be mut in Rust, like they are not allowed to be const in C++. There is this thing called "const-correctness" in C++. Almost no case needs us to const_cast except for a few edge cases, like "caching", where the outer view is to just read values, but internally we also cache the value. We have the same view in Rust, plus another "core rule" for mutability.

The rule in Rust says, that we can either have multiple immutable borrows of something or exclusively one mutable borrow. That way for example it's not possible to iterate over a collection while modifying it directly. But there are also other cases, where we want multiple mutable owners, like shared state in multithreaded applications. If the threads only want to read data, they are allowed to do this in parallel, but if someone wants to mutate the state, too, it needs to be wrapped in a Mutex or RwLock to ensure the rule and that wrapper pretend immutable access to mutable data, only because they can make sure that only one thread can hold a lock at any time.

Smart pointer vs. `Box`/`Rc`/`Arc`

`std::unique_ptr<T>` vs. `Box<T>`

Let's start with the std::unique_ptr in C++ which can't be copied, but moved and it's holds a pointer to something living on the heap. In Rust it's called Box and the same applies, so it's not Clone.

// Rust
fn main() {
    let b = Box::new(Person {
        name: "Robert Rust".to_owned(),
    });
    b.say_hello(); // via `Deref`
}

struct Person {
    name: String,
}

impl Person {
    fn say_hello(&self) {
        println!("Hi, I'm {}", &self.name);
    }
}

Interesting points are

Rust always creates a Box from moved data, so we create a Person on the stack, where the owned String also has data on the heap. Then the person is bit-wise moved to the heap, so name itself is moved, but a String consists of a heap pointer and a length, so a few bytes (I think 4+4 Bytes) no matter how long the string is. This is safe, because Person is destroyed, when Box is and we're not allowed to use moved-out values in Rust.
For convenience reasons we have an operator-> in C++ so that we can access the underlying pointer with an arrow. We have the same in Rust, but it's called Deref which is a trait, but access with a simple dot. So we can call all Box and Person methods.

`std::shared_ptr<T>` vs. `Rc<T>`/`Arc<T>`

std::shared_ptr<T> in C++ is what Arc<T> (Atomic Reference Counted) is in Rust. It's not possible in C++ to have an instance with a dangling pointer, and it's not double freed, so the reference counter is atomic and it can be used from multiple threads, but the internal data structure has to be synchronized for multiple threads.

In Rust Arc<T> has the same properties, but the internal data structure can only be accessed as immutable. That way it's safe in Rust to use any T for an Arc, because we are not allowed to mutate it and multiple reads are fine. In the next section we'll talk about how to anyway share mutable state with Arc<T>.

The fact that Arc<T> is using atomic counters is unnecessary overhead in a single-threaded context. Rust always addresses such concerns so we can stick to the little brother Rc<T>. Again the internal value is immutable. To break that restriction, we can use Rc<RefCell<T>> to enable interior mutability, but to also ensure exclusive mutable access, RefCell<T> adds runtime checks for borrows. Of course accessing the value won't block the thread as it would be an immediate deadlock, but it panics if the the borrow rules are violated.

Lambdas vs. closures

Let's check out the next example. C++ lambdas and Rust closures are also quite similar.

// C++
#include <iostream>
#include <vector>
#include <algorithm>

bool has_number(int i, const std::vector<int>& v) {
    return v.end() !=
      std::find_if(v.begin(), v.end(), [i](auto& n) {
        return n == i;
      });
}

int main() {
    std::vector<int> v {1, 2, 3, 4, 5};
    std::cout << (has_number(4, v) ? "yes" : "no") << std::endl;
    return 0;
}

The lambda is capturing i as copy, but that's fine, because it just a scalar value. Let's have a look at Rust closures.

// Rust
fn main() {
    let v = vec![1, 2, 3, 4, 5];
    println!("{}", if has_number(4, &v) { "yes" } else { "no" });
}

fn has_number(i: i32, v: &Vec<i32>) -> bool {
    v.iter().find(|n| {
        **n == i
    }).is_some()
}

So what's happening inside has_number? v is our borrow to Vec and it has a method .iter() which creates an iterator. In Rust collections only have their container specific methods, like .push() or .pop() if possible, but e.g. no .map(..). For iteration they all provide .iter() which creates an iterator where we can find like .map(..) or .find(..). Here we use .find(..) which wants a predicate closure, taking a borrow of the item type. That means .iter() creates an iterator over &i32 because the items still live in the Vec, not moved into the iterator. And .find(..) wants a closure taking &&i32, which is borrow of a borrow of i32, like double pointers in C++. That's why we need to deref it twice (**n) and then compare. The return value of .find(..) is an Option so we check with .is_some().
Rust has 3 closure types (Fn, FnMut, FnOnce) for expressing different capture types of the closure context. Closures have an anonymous type as well as a lifetime depending on the capture type. In the last section we have another interesting example about closures and lifetimes.

Multithreading

In C++ we're allowed to share everything between threads. It's our duty to make sure that data is synchronized. That can be, everyone is just reading or only one thread is writing exclusively. It's easy to violate that and immediately crash your application. But to have an application running correctly for hours until it segfaults and crashes or even worser doing undefined things, like corrupting memory, can drive you nuts.

`Send` and `Sync`

Rust has two traits Send and Sync to manage correct data access in multi-threaded applications. They are called auto traits as they are automatically implemented for all types where appropriate. The traits are marker traits without any implementation, just marking the type. Send means that the type can be safely transferred between threads and Sync means that access from multiple threads is allowed. So let’s do some examples:

// Rust
fn main() {
    let p = Person { age: 24 };
    std::thread::spawn(|| { // err: closure may outlive the current function, but it borrows `p`, which is owned by the current function
may outlive borrowed value `p`
        println!("I'm {}", p.age);
    });
}

struct Person {
    age: u32,
}

It sounds intuitive to not make this compile, because what's happening here is, that p may have already been dropped when the thread starts computation and using a borrow of p, it seems correct to not allow this. Let's have a look at the definition of spawn to clarify why Rust doesn't let us create such silly bugs:

pub fn spawn<F, T>(f: F) -> JoinHandle<T>
where
    F: FnOnce<(), Output = T> + Send + 'static,
    T: Send + 'static

closure is called once (FnOnce) and needs to be Send and 'static
the return value of the closure needs to be Send and 'static

With the borrow of p we are shortening the lifetime of the closure to the lifetime of p so it cannot outlive it. That violates the first rule that it has to be 'static. In order to make it work, we can either move the Person into the closure or use the age value itself. Let's move p:

let p = Person { age: 24 };
std::thread::spawn(move || { // move keyword
    println!("I'm {}", p.age);
});

So p ownership is taken over by the closure, so it stays 'static and it is Send because Person is auto Send because all attributes are Send. For the sake of completeness, also the struct itself is not explicitly marked as !Send, which is possible to manually disable Send. For example it may contain some resource that can only be handled from that thread, like some OpenGL handle which are only numbers, but useless in other threads.

let p = Person { age: 24 };
let age = p.age;
std::thread::spawn(move || {
    println!("I'm {}", age);
});

The same rules apply for age here and we're not moving the whole Person.

Shared state

There is so much more about concurrency in Rust, like channels, but let's go back to an easy example of how we can share a mutable state in Rust between threads. I'm not saying that this is a good idea or it will scale with all threads, but at least it will work and there are no race conditions possible.

// Rust
use std::{
    sync::{Arc, Mutex},
    thread::spawn,
};

fn main() {
    let s = Arc::new(Mutex::new(0));
    let mut handles = Vec::new();
    for _ in 0..10 {
        let thread_state = s.clone();
        handles.push(spawn(move || {
            let mut number = thread_state.lock().unwrap();
            *number += 1;
        }));
    }
    for handle in handles {
        handle.join().unwrap();
    }
    let state = s.lock().unwrap();
    println!("n = {}", *state); // n = 10
}

A common pattern is to put a Mutex or RwLock into an Arc, because:

Arc<T> is Clone, but it will not make a deep clone, just allow sharing the internal value. It will also not allow to access &mut T, so from that point cloning is safe.
Arc<T> is Send if T is Sync and Send, which means
1. Send takes care that the value "is useful/allowed" on another thread, like the example of OpenGL handles are not Send at all and
2. Sync means, that &T can be shared between threads, or in other other words &T is Send which works for a Mutex, because access is synchronized.
Mutex<T> is Send if T is Send, which again makes sense, because of the same reason of the OpenGL handle example.
Mutex<T> is also Sync if T is Send and the reason is, only one thread can access the value at one time. T doesn't need to be Sync, so we can put like HashMap or Vec in it and because of the mutual exclusive access, it's safe.

Crashing a Rust program is possible when doing something wrong, but we will never ever run into undefined behavior, because it's simply not possible (without unsafe Rust).

Conclusion

So again, Rust is not easy, but the way the language compels me to write code and I'd really say better and safer code, all in all less boilerplate code, but also more explicit and I'm fully aware about what my code does, is a price worth to pay, I think.

If Rust has no chance to be adopted for your work, nevertheless, I would also recommend that you should give it a try. Definitely, you will write better code next time you use C++. It's not all about the hard stuff, like multithreading, but also the easy parts where we tend to write C++ code a Rust compiler would not compile for good reasons. We can also take those mutability and lifetime concepts into account for C++, but I think it's better to have a compiler doing that job, so I prefer Rust — change my mind :).

Rust ownership and borrows - Fighting the borrow-checker

Philipp Renoth — Wed, 05 May 2021 19:55:14 +0000

A thing that is often quite time-consuming writing Rust code is the fight with the borrow-checker. This post will give a small overview of the problem and how to fix them or better, how to prevent them.

Having some native development background makes it a bit easier to jump into Rust, but there is also a need to change the mindset. Coming from any other language it’s very likely that the way one wrote code will simply not work with Rust, although being a “good developer”. At least I learned that the hard way.

One of Rust’s core concepts is:

ownership: data has exactly one owner
exclusive mutable or shared immutable borrows: borrowed data is exclusive mutable or shared immutable

Ownership

Languages without garbage collector like C++ provide concepts and conventions for ownership. The problem is that the language can’t enforce them. Thus static code analysis tools may help you, but at some point even they can’t.
Rust on the other hand has a strict ownership concept. That also means reading any code will clearly show what is happening regarding ownership.

pub fn main() {
    let cat = Cat {};

    feed(&cat); // borrow

    // we are still owner of `cat`

    sell(cat);  // pass ownership

    // we are not allowed to use `cat` anymore

    feed(&cat); // err: cat moved
}

struct Cat {}

fn feed(_cat: &Cat) {}
fn sell(_cat: Cat) {}

Obviously, ownership will not be a big problem as it feels kinda natural to have this concept. It’s getting more interesting when borrowing is involved.

Exclusive mutable or shared immutable borrows

While data is exclusively owned by other data or a local variable, it’s possible to borrow data during its lifetime and for example pass it as parameter to a function. There are two different borrow types:

immutable borrows: &T
mutable borrows: &mut T

Depending on whether you want to mutate data or not, you need to pass the borrow. Technically it’s just a reference, but Rust has strict rules about borrowing:

either have multiple immutable (&T) borrows
OR exclusively one mutable (&mut T) borrow

Those rules may sound reasonable at first place. But let’s look at the next example which will not compile.

pub fn main() {
    let mut cat = Cat::new();
    let checker = FeedChecker { cat: &cat }; // immutable borrow

    cat.feed(); // ERR: mutable borrow

    if checker.is_fed() {
        println!("cat is ok");
    } else {
        println!("cat is hungry");
    }
}

struct Cat {
    fed: bool,
}

impl Cat {
    fn new() -> Self {
        Self { fed: false }
    }

    fn feed(&mut self) {
        self.fed = true;
    }
}

struct FeedChecker<'a> {
    cat: &'a Cat,
}

impl<'a> FeedChecker<'a> {
    fn is_fed(&self) -> bool {
        self.cat.fed
    }
}

We pass an immutable borrow to the FeedChecker that it can check the cat later and we then also mutate the cat, because you can read cat.feed() same like Cat::feed(&mut cat), because feed takes a &mut self, so it creates a mutable borrow of cat. Having the immutable borrow living in the FeedChecker will overlap the mutable borrow for feeding the cat, which then break the second borrow rule.

Borrow-checker issues — Code smell or bug?

Now, why on earth should the last example be a problem? You may have heard the term “multiple readers or one writer” in a multi-threaded context. That is not an issue here, because Rust has other concepts for data access with multiple threads, but it is going in the same direction.

Let’s assume is_fed is a very expensive operation and in Rust we have the second rule, that when I have a borrow (no matter if mutable or immutable) it’s not possible for any other component to have a mutable borrow, so reading is_fed once could be cached for the whole lifetime of the FeedChecker.

Think of &mut T means “exclusive (mutable) access” and &T means “no one has mutable access”.

What helped me understand the second rule and “mutable borrows” is to better consider them as application wide exclusive access, so the other way round: whatever you may mutate, you can assume to not break any reader’s discovery for that data.

Mutating a vector while someone has an iterator into it is not possible in Rust and that’s a very good reason to have the rule of exclusive mutability.

Still the fact that the simple example with FeedChecker is not compiling is a bit annoying and there are quick (but somehow also more dirtier) ways to fix that. Some of them (like RefCell) also come with a runtime cost, which is rarely what we want. The overall problem is actually simpler than that. A good question is: Why should FeedChecker need a reference to a cat before and after is_fed() calls?

What about that our FeedChecker only encapsulates the is_fed implementation and we borrow the cat for the check? Yes, we may also go further and ask why at all do we need a struct FeedChecker when it’s empty? Sometimes all what we need is a simple pure function.

The following example shows, what we at least have to do to make it compile, but we can also omit the whole struct FeedChecker and simply implement one function.

pub fn main() {
    let mut cat = Cat::new();
    let checker = FeedChecker {};

    cat.feed();

    if checker.is_fed(&cat) {
        println!("cat is ok");
    } else {
        println!("cat is hungry");
    }
}

struct Cat {
    fed: bool,
}

impl Cat {
    fn new() -> Self {
        Self { fed: false }
    }

    fn feed(&mut self) {
        self.fed = true;
    }
}

struct FeedChecker {}

impl FeedChecker {
    fn is_fed(&self, cat: &Cat) -> bool {
        cat.fed
    }
}

How to get rid of (most) borrow-checker issues?

My personal #1 experience to most of the borrow-checker issues I ran into were excessive borrow lifetimes, which then overlap. That means, for example, there is mostly no reason to store borrows in structs (e.g. via constructor), only to have them when any of the struct methods are called. That “pattern” is tempting to reduce boilerplate code, but soon or later may break the second borrow rule.

Reducing borrow lifetimes to the needed minimum may fix borrow-checker issues.

Creating a struct I always try to have only owned data, no borrows, which on the other hand is also simpler code without lifetimes. I think there is no rule how to write Rust code to always avoid borrow issues. They may happen at a time, telling you either you have some code smell or they even save you from bugs, like mutating a vector while having an iterator, which e.g. leads to nasty undefined behavior in C++.

Conclusion

The borrow-checker doesn’t only guarantee, that data access is legit. The additional positive result is, that we should start to rethink our code design. It’s likely that simpler code designs may fix borrow-checker issues.

A borrow-checker issue may be code smell or a real bug. Rust doesn’t allow either, respectively doesn’t know the kind. It doesn’t compile.
It’s mostly possible to fix borrow issues throwing mutability wrapper on them, but you should rather investigate on fixing it with a better design, because it may bite you again or worst case: panic at runtime.
“Code smell” borrow-checker issues are fixed by reducing borrow lifetimes so that they don’t overlap. Passing borrows to store them (e.g. via constructor) for later use may be one use-case when a borrow lifetime is rather excessive than necessary.
Try to design structs owning all their data and pass short-living borrows as parameters for their methods. That may increase boilerplate code but elegant design simplicity will shine through. Those “backdoor borrows” may feel like hidden side-channels and confuse the reader.

Rust module system

Philipp Renoth — Wed, 05 May 2021 19:38:48 +0000

The module system of Rust is very powerful and strict. It’s powered by cargo, the de facto build tool for Rust projects. Following the idiom “convention over configuration” there is only a few configuration to be done.

Before we dive in, let’s make sure we have the same understanding for modules, packages and crates:

module: Rust language term for code tied together having different visibilities from outside.
package / crate: cargo ecosystem term for a Rust project ( Cargo.toml/.lock + src + …).

Build Targets — bin vs. lib

Before we can create a package, we need to decide whether we’re going to have a binary executable build (bin) or a library build (lib).

cargo init --bin / cargo init (default): binary executable target
cargo init --lib: library target

If we don’t know yet, just start with a bin package, because it’s almost no work to change it and if we just wanna hack around without writing tests, we can’t have a main entry-point in our lib package, so it’s also easier to start with the default bin. Well, it’s possible to have a package being both, but we will not handle that case here. Let’s rather have a look what those types do.

`bin` build target

input: fn main() entry-point in src/main.rs
output: executable binary in target/debug/your-project for cargo build (will change if we change the build target, e.g. arm, x86 or release)

`lib` build target

input: all pub components from src/lib.rs
output: Rust library intermediate target/debug/your-project.rlib. You will probably don’t use it directly, but have a dependency to the lib package itself, so we can forget about it.

Now let’s have a look at the module system itself.

Module system

We’ve talked about bin and lib packages and that they do different things. Fortunately they are both based on the same Rust module system and to understand what the Rust compiler is doing, we can treat them equal.

Modules everywhere

While other languages try to push developers to have one language component per file, it’s bad practice to do so in Rust, because Rust has the term of modules. That also leads to larger source files, so your IDE better should support Rust “Outlines” (e.g. Visual Studio Code + rust-analyzer is great), to jump to your components quickly.

Every Rust file (.rs) is a module, so the intention is not to map every component like a struct to your filesystem, but to give it a home and make it part of a module. That being said it would be possible to have everything in a new file, but modules have to be interconnected and in a few minutes you’ll see why this is not a good idea.

Let’s make some examples as an abstract definition is more confusing. Here we have a package created by cargo init. We say that it has a module main and per convention it should also contain the fn main() entry-point, so a bin target package should at least have a main module.

That was simple, right? Now let’s talk about a more interesting case. Let’s say we’re developing a web server with a database connection. At first we put everything in main.rs and then talk about how to split it.

pub fn main() {
    let config = read_configuration();
    let db = connect_db(&config.db_host);

    let server = Server::new(&config, db);
    server.run();
}

// server

struct Server {
    port: u16,
    db: DbConnection,
}

impl Server {
    fn new(config: &Configuration, db: DbConnection) -> Self {
        Self {
            port: config.server_port,
            db,
        }
    }
    fn run(&self) {}
}

// db

fn connect_db(host: &String) -> DbConnection {
    DbConnection {}
}

// configuration

fn read_configuration() -> Configuration {
    // read env-vars, whatever
    Configuration {
        db_host: "my-db".to_owned(),
        server_port: 8080,
    }
}

struct Configuration {
    db_host: String,
    server_port: u16,
}

// 3rd party library
// Only there that it compiles

struct DbConnection {}

To be honest this is way too less code to really think of splitting it, but let’s assume it’s more and of course our package is getting larger over time. For the sake of simplicity this example does nothing, but it should help to understand how we can use modules in Rust. So we have 5 sections here:

fn main(): where everything starts
server: our server implementation
db: our connect implementation. As we likely use a third-party library for database connection, we only have to write some glue code here.
configuration: it’s mostly a good idea to read the configuration into a struct, e.g. from env variables or cli parameters.
3rd party library: ignore that code, it’s only there that it compiles

The fn main() stays in main.rs, but we can think of the other sections to be modules, because they are responsible for a specific thing and they should hide details. Let’s assume that we want to split main.rs and add 3 other modules. We have two options for our sub-modules:

embedded modules in the main.rs file
file-based modules in the /src folder

Embedded Modules

You’ll probably see them less often, but they are easier to understand for now. In order to understand what needs to be changed to put our code in modules, please read the comments in the code.

// main.rs
// This module is called `main`.



// We put everything into modules, so we don't see it anymore.
// We have to make them visible with `use`.
// It's not needed to make everything we "use" visible, but
// the code we write can only contain visible things, like
// we call `read_configuration`, so we need to make it visible.
// On the other hand we're using the `Configuration` but we don't
// need to make it visible, as `fn main()` doesn't contain it.
// It's only the inferred type for `config`.
use config::read_configuration;
use db::connect_db;
use server::Server;

pub fn main() {
    // We could also write this and forgo the use statement on top.
    // `let config = config::read_configuration();`
    let config = read_configuration();
    let db = connect_db(&config.db_host);

    let server = Server::new(&config, db);
    server.run();
}

mod server {
    // Same here. We only see what we define in our module, so we need
    // to make something from another module visible.
    // We can't jump into that directly, but have to traverse to it
    // - either with `super::...` which is the parent module
    // - or with `crate::...` which is the root module of the package
    use super::{config::Configuration, db::DbConnection};

    // struct is used outside, so needs to be `pub`
    pub struct Server {
        // Those attributes are not `pub` as we have `pub fn new()`.
        // From inside the module, everyone can access them.
        port: u16,
        db: DbConnection,
    }

    impl Server {
        // we also want to create a Server from outside
        pub fn new(config: &Configuration, db: DbConnection) -> Self {
            // creating a server with `Server {...}` from outside is not possible
            // as its attributes are not `pub`
            Self {
                port: config.server_port,
                db,
            }
        }

        // also has to be `pub`
        pub fn run(&self) {}
    }
}

mod db {

    // also has to be `pub`
    pub fn connect_db(host: &String) -> DbConnection {
        DbConnection {}
    }

    // 3rd party library
    pub struct DbConnection {}
}

mod config {

    // also has to be `pub`
    pub fn read_configuration() -> Configuration {
        Configuration {
            db_host: "my-db".to_owned(),
            server_port: 8080,
        }
    }

    // also has to be `pub`
    pub struct Configuration {
        // As we see this struct more like a data container
        // it's not bad practise in Rust to simply make those attributes `pub`.
        // In this case it really makes sense to not implement a `fn new()` with
        // a confusing parameter list, just `Configuration { ... }`.
        // Other structs like `Vec` don't have `pub` attributes, as
        // they contain sensitive data.
        pub db_host: String,
        pub server_port: u16,
    }
}

Let’s summarize what happened to the code to work with modules:

The mod { ... } keyword + block in Rust starts a new submodule in the current module.
The file itself already is a module so we don’t need mod for it.
Rust doesn’t have the term “import” or “include”. It’s all about visibility.

We’ll also shortly stick to the visibility rules of Rust, but let’s first have a look at the file-based modules. We already have one (main.rs), but let’s put all mod {...} sections into different files.

File-based modules

What we have now are 4 files: main.rs, config.rs, db.rs and server.rs. At first we put the content of the mod blocks into a new file. We don’t have to change visibility or use statements, simply copy and paste. Per convention module names are lower snake case and so do the files.

// config.rs

pub fn read_configuration() -> Configuration {
    Configuration {
        db_host: "my-db".to_owned(),
        server_port: 8080,
    }
}

pub struct Configuration {
    pub db_host: String,
    pub server_port: u16,
}

// db.rs

pub fn connect_db(host: &String) -> DbConnection {
    DbConnection {}
}

// 3rd party library
pub struct DbConnection {}

// server.rs

use super::{config::Configuration, db::DbConnection};

pub struct Server {
    port: u16,
    db: DbConnection,
}

impl Server {
    pub fn new(config: &Configuration, db: DbConnection) -> Self {
        Self {
            port: config.server_port,
            db,
        }
    }

    pub fn run(&self) {}
}

The interesting part is the main.rs where we remove the mod blocks and replaced them by simple mod statements (without block). For the compiler mod server; means “please add server module via server.rs and make it visible“. So use server; doesn’t make sense at all, as we already added it. To make our Server visible to be used in the code we say use server::Server;. Really mod is just for our submodules. All of our dependencies are already visible to all modules.

// main.rs

use config::read_configuration;
use db::connect_db;
use server::Server;

mod config;
mod db;
mod server;

pub fn main() {
    let config = read_configuration();
    let db = connect_db(&config.db_host);

    let server = Server::new(&config, db);
    server.run();
}

Module vs. filesystem hierarchy

The module structure of our example is quite flat. But what about nested embedded modules or directories with modules? Wait a second, we should clarify one thing first: filesystem and module tree do not match?

Filesystem         Modules
- main.rs          - main.rs
- config.rs        |-- config.rs
- db.rs            |-- db.rs
- server.rs        |-- server.rs

That’s correct. So for example the config.rs module is sitting next to the main.rs module on the filesystem, but in Rust it’s a submodule and that looks weird at first. We have some files in a directory and one of them is the super module and the others are submodules? Now, do we have to read the code to find the super module? No — fortunately we don’t have to. In this “flat” case it’s clear, that main.rs has to be the super module and all other files are submodules. With that in mind, we can now have a look at nested modules.

Nested modules

Let’s say our server.rs module is growing and we actually want to have more modules. Large files can become a problem for reading, but again, in Rust hundreds or thousand lines of code in a file is actually fine and there is another reason I’ll try to explain in the next sections.

From what we know, what options do we have?

We can embed submodules into server.rs via e.g. mod routes { ... } block.
We could add another file, but that is then a submodule of main, so that doesn’t work for us.

The first option works, but what about rather spending a new file? Of course it’s possible, let’s check out the next example.

We move server.rs to server/mod.rs
We create a submodule routes at server/routes.rs
Add mod routes; to server/mod.rs
The rest of the package doesn’t have to be touched

// /src/server/mod.rs

// we add a submodule here
mod routes;

use super::{config::Configuration, db::DbConnection};

pub struct Server {
    port: u16,
    db: DbConnection,
}

impl Server {
    pub fn new(config: &Configuration, db: DbConnection) -> Self {
        Self {
            port: config.server_port,
            db,
        }
    }

    pub fn run(&self) {
        // Call something from `routes`.
        // Instead of `use` the function we can also call it like this:
        routes::handle_request();
    }
}

// /src/server/routes.rs

pub fn handle_request() {}

That’s it, so the file structure should look like this now.

To correct a previous statement: mod server; means “please add server module via server.rs or server/mod.rs and make it visible“. That’s how we can nest modules and we’re almost done understanding how modules work in Rust, but what about accessing nested modules from outside? Can I really hide intrinsic code when I want nobody to use it from outside? Let’s have a look at the visibility rules in order to figure out how we can access or really hide something.

Visibility in Rust

Think of: In Rust everything “is there”, but you may not see it from your current view. You can’t use what you cannot see. What we see without use:

what the module itself defines: pub or not, everything
parent modules via super keyword (can be concatenated like we have relative file paths, e.g.super::super::somemodule )
root module via crate keyword (like a Linux filesystem root / )
all dependency packages from everywhere in our package
(prelude stuff like String , Box and so on)

But that’s not all. There is more, because they all can expose things to the outside. You can’t see them directly, but you can access them either via it’s relative or absolute path or make them visible via use keyword. The path separator is a double colon (::), e.g. routes::handle_request();.

Modules and `pub`

For submodules, to “expose something” means, that the module has to make it pub. The other direction, a submodule has full access to its parents, but only to them. If you as a module wants to access a sibling module, you have the same access constraints like your parent, because you have to access it via super or crate path.

Now we should talk about how to access submodules of a submodule. So from outside a module, we can only see what is pub inside the module without exception. Let’s say we need pub const PREFIX: &str = “/app”; from server/routes.rs in our main.rs module. In order to expose something from another submodule, we have two options to consider for server/mod.rs as server/routes.rs is a submodule of it:

“re-export” it e.g. like pub use routes::PREFIX;: in main.rs we can access server::PREFIX.
“re-export” submodule e.g. like pub mod routes;: in main.rs we can access server::routes::PREFIX, but we can then also access server::routes::handle_request() and how does that make sense?

In that case it would be better to only expose PREFIX, but there are cases, when your module has so many things in it, that you want put it in submodules and also represent them to the outside world like this. The bigger the packages are, the more likely they do not only expose one module with everything in it, but a bunch of submodules.

When to split modules into submodules?

At least, there is nothing stopping you from throwing a module at every little component, but the main goal of modules is different: hide details, tie code together and design a clear minimal interface to the outside.

If you’re not sure whether to split or not, simply don’t split. It’s more likely that to make things more difficult at first than it helps. I mostly have a main.rs with hundreds of lines of code, before I create the first the submodule, but when I create it, I have a good overview over all components and how they work together.

Depending on what you do you should take that more or less seriously. For bin packages it shouldn’t be a big problem to fix visibility issues as you can see if it still compiles or not and also fix that. As a lib package maintainer you should better check what you’re going to share. Narrowing visibility breaks the public API so it’s going to be an mature version release for “a small patch”, because maybe someone is using it. You can also live with it as technical debt and fix it one day, when there are a few big changes, but better take care before.

When to use embedded or file modules?

To be honest there is no good rule of thumb known to me for this. I mostly see and use file modules. At least I know two cases where we often have embedded modules.

At first, it’s convention to embed a tests module at the bottom for unit tests.

// routes.rs

fn handle_request() {}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_handle_request() {
        handle_request();
    }
}

The module is only compiled for cfg(test), it has full access to its parent (we could also test private state, not saying you should) and it belongs together, so we don’t even have to switch files for writing tests.

Second, some embedded modules are the result of code refactorings, like moving code into submodules need “re-exports” which is not worth a new file.

Putting together packages and modules

A bin package simply starts at main.rs as root module and it has to contain the fn main() entry-point. main.rs doesn’t need to expose anything.

A lib package starts at lib.rs as root module and dependants only see what lib.rs exposes.

So we have 3 special files of the Rust module system: src/main.rs, src/lib.rs and any mod.rs. Think of what the mod.rs file is to its module, main.rs is to a bin package and lib.rs is to a lib package, because the same rules apply. You can choose any other file name just as you like.

Well, and that’s what it’s all about, although we skipped some details like there are multiple different pub visibility modifiers like pub (crate) or pub (super). Just have a look at the visibility and privacy section of the Rust reference and you’ll understand them in the context of modules.

Summary

A Rust module represents code that belongs together because it may share details, the outside world should not be interested in or use.
A clear and smart interface to a parent module makes it possible to change code without breaking the interface. Especially lib maintainers should keep an eye on that, to not break the API.
Visibility in Rust is not absolute. What you see really depends on where you are, like submodules can see everything from the parents, but a parent does not see private things in a submodule.
In Rust a submodule can’t make something visible to the parent’s parent. For this case, the parent needs to “re-export”, so it’s like an “export-chain” where every link has to “re-export”.
Sibling files are part of the same module, except mod.rs, src/bin.rs and src/lib.rs files which are the “module barrels” itself.
Finally: don’t overdo modules. It’s fine to start in main.rs or lib.rs and rethink your modules later. You should better double-check what you’re going to make pub to not unnecessarily expose details from your lib package, because other mistakes can be fixed quite easily.

DEV Community: Philipp Renoth

Consider Rust for cloud computing.

Forbidden fruits in the cloud

The long forgotten term of "resource efficiency"

Technology stacks at companies

Consider Rust

Pro

Contra

Conclusion

10 Rust technology essentials

Essentials

Good bye ReactNative, hello Ionic.

A bit of history

Problems with ReactNative

What else if not ReactNative?

Ionic / capacitor

Conclusion

Vulkan - some introductory words

A new graphics API was born

How to start writing a Vulkan application?

1. Compare some Vulkan and OpenGL code

2. Understand why some Vulkan boiler-plate is necessary to get the most out of the hardware

3. Getting comfortable with the base Vulkan architecture and the first steps

Vulkan C API

Vulkan architecture overview

Vulkan pseudo code

4. Go through some tutorials

5. Do your thing

Conclusion

Understanding Rust as a C++ developer

As a C++ developer, why should I learn Rust?

Foreign function interface (FFI)

Primitive data-types

Arrays and slices

Arrays

Slices

Initialization

Type inference

Functions

Meta programming

Strings

Optional values and exceptions vs. results

Option<T>

Result<T, E>

?-operator

std::terminate() vs. panic!()

Control flow — conditions and loop

Enums

OOP

struct/class vs. struct/enum

Inheritance vs. traits

con-/destructor vs. constructor/drop()

Templates vs. generic code

Namespaces vs. modules

Rust module system

Philipp Renoth ・ May 5 '21

const vs. mut

Passing Values — reference, borrows, ownership

Rust ownership and borrows - Fighting the borrow-checker

Philipp Renoth ・ May 5 '21

Lifetimes

Copy vs. Clone

Clone

Copy

Tuples

Memory management

Runtimes

Move semantics

C++ move

Rust move

Const-correctness vs. mutability

Smart pointer vs. Box/Rc/Arc

std::unique_ptr<T> vs. Box<T>

std::shared_ptr<T> vs. Rc<T>/Arc<T>

Lambdas vs. closures

Multithreading

Send and Sync

Shared state

Conclusion

Rust ownership and borrows - Fighting the borrow-checker

`Option<T>`

`Result<T, E>`

`?`-operator

`std::terminate()` vs. `panic!()`

con-/destructor vs. constructor/`drop()`

`const` vs. `mut`

`Copy` vs. `Clone`

`Clone`

`Copy`

Smart pointer vs. `Box`/`Rc`/`Arc`

`std::unique_ptr<T>` vs. `Box<T>`

`std::shared_ptr<T>` vs. `Rc<T>`/`Arc<T>`

`Send` and `Sync`

`bin` build target

`lib` build target

Modules and `pub`