Viddy v1.0.0: Reimplementing in Rust from Go

Introduction

In this article, I would like to share my experiences and insights gained during the reimplementation of Viddy, a TUI tool I have been developing, from Go to Rust for the v1.0.0 release. Viddy was originally developed as a modern version of the watch command, but this time, I took on the challenge of reimplementing it in Rust. I hope this article serves as a useful reference for those interested in developing with Rust.

About Viddy

https://github.com/sachaos/viddy

Viddy was developed as a modern alternative to the watch command found in Unix-like operating systems. In addition to the basic functionality of the watch command, Viddy offers the following key features, which are better illustrated in the demo mentioned later:

• Pager functionality: Allows you to scroll through the output of commands.
• Time machine mode: Enables you to review past outputs of commands.
• Vim-like keybindings

Originally, I aimed to implement Viddy in Rust, but due to technical challenges, I decided to prioritize the release by using Go, a language I was more familiar with. This time, I was able to overcome those challenges and finally realize my initial goal, making this release particularly meaningful to me.

Demo

Motivation for the Rewrite

It's important to note that I had no dissatisfaction with the Go language itself. However, since the original implementation was more of a Proof of Concept (PoC), there were many areas that, upon review, I wanted to improve. These areas had become obstacles to fixing bugs and extending functionality. This growing desire to rebuild the project from scratch was a significant motivator.

Additionally, I had a strong interest in Rust and, as I progressed in learning the language, I wanted to apply my knowledge to a real project. Although I had studied Rust through books, I found it challenging to truly grasp the language's unique features and gain a sense of mastery without hands-on experience.

Insights Gained from the Rewrite

Prioritize Release Over Perfect Implementation

The primary focus during the reimplementation was to prioritize the release. Rather than getting caught up in achieving the most optimal implementation, I decided to defer optimizations like memory usage and code conciseness and aimed to get a release out as quickly as possible. While this approach may not be something to boast about, it allowed me to push through the rewrite in an unfamiliar language without getting discouraged.

For instance, at this stage, I implemented the code using frequent cloning without fully considering ownership. There is plenty of room for optimization, so the project has lots of potential for improvement!

Additionally, there are many parts where I could have written more elegantly using method chains. I believe that using method chains could have reduced the use of if and for statements, making the code more declarative. However, my limited Rust vocabulary, combined with my reluctance to do more research, led me to implement many parts in a straightforward manner for now.

Once this release is out, I plan to revisit ownership, perform optimizations, and refactor the code to address these concerns. If you happen to review the code and notice any areas that could be improved, I would greatly appreciate it if you could open an issue or submit a PR to share your insights!

Pros and Cons of Rewriting in Rust

In the process of migrating to Rust, I’ve noted some pros and cons compared to Go. These are just my impressions, and since I’m still a beginner with Rust, I might have some misunderstandings. If you spot any mistakes or misconceptions, I would appreciate your feedback!

👍 Propagating Errors

In Rust, propagating errors allows you to write concise code that returns early when an error occurs. In Go, a function that can return an error is defined like this:

func run() error {
    // cool code
}

And when you call this function, you handle the error like this. For example, if an error occurs, you might return the error early to the caller:

func caller() error {
    err := run()
    if err != nil {
        return err
    }

    fmt.Println("Success")
    return nil
}

In Rust, a function that can return an error is written like this:

use anyhow::Result;

fn run() -> Result<()> {
    // cool code
}

And if you want to return the error early in the calling function, you can write it concisely using the ? operator:

fn caller() -> Result<()> {
    run()?;
    println!("Success");
    return Ok(());
}

At first, I was a bit confused by this syntax, but once I got used to it, I found it incredibly concise and convenient.

👍 Option Type

In Go, it's common to use pointer types to represent nullable values. However, this approach is not always safe. I often encountered runtime errors when trying to access nil elements. In Rust, the Option type allows for safe handling of nullable values. For example:

fn main() {
    // Define a variable of Option type
    let age: Option<u32> = Some(33);

    // Use match to handle the Option type
    match age {
        Some(value) => println!("The user's age is {}.", value),
        None => println!("The age is not set."),
    }

    // Use if let for concise handling
    if let Some(value) = age {
        println!("Using if let, the user's age is {}.", value);
    } else {
        println!("Using if let, the age is not set.");
    }

    // Set age to 20 if it's not defined
    let age = age.unwrap_or(20);
}

As shown in the final example, the Option type comes with various useful methods. Using these methods allows for concise code without needing to rely heavily on if or match statements, which I find to be a significant advantage.

👍 The Joy of Writing Clean Code

It's satisfying to write clean and concise code using pattern matching, method chaining, and the mechanisms mentioned earlier. It reminds me of the puzzle-like joy that programming can bring.

For example, the following function in Viddy parses a string passed as a flag to determine the command execution interval and returns a Duration.

By using the humantime crate, the function can parse time intervals specified in formats like 1s or 5m. If parsing fails, it assumes the input is in seconds and tries to parse it accordingly.

// https://github.com/sachaos/viddy/blob/4dd222edf739a672d4ca4bdd33036f524856722c/src/cli.rs#L96-L105
fn parse_duration_from_str(s: &str) -> Result<Duration> {
    match humantime::parse_duration(s) {
        Ok(d) => Ok(Duration::from_std(d)?),
        Err(_) => {
            // If the input is only a number, we assume it's in seconds
            let n = s.parse::<f64>()?;
            Ok(Duration::milliseconds((n * 1000.0) as i64))
        }
    }
}

I find it satisfying when I can use match to write code in a more declarative way. However, as I will mention later, this code can still be shortened and made even more declarative.

👍 Fewer Runtime Errors

Thanks to features like the Option type, which ensure a certain level of safety at compile time, I found that there were fewer runtime errors during development. The fact that if the code compiles, it almost always runs without issues is something I truly appreciate.

👍 Helpful Compiler

For example, let's change the argument of the function that parses a time interval string from &str to str:

fn parse_duration_from_str(s: str /* Before: &str */) -> Result<Duration> {
    match humantime::parse_duration(s) {
        Ok(d) => Ok(Duration::from_std(d)?),
        Err(_) => {
            // If the input is only a number, we assume it's in seconds
            let n = s.parse::<f64>()?;
            Ok(Duration::milliseconds((n * 1000.0) as i64))
        }
    }
}

When you try to compile this, you get the following error:

error[E0308]: mismatched types
   --> src/cli.rs:97:37
    |
97  |     match humantime::parse_duration(s) {
    |           ------------------------- ^ expected `&str`, found `str`
    |           |
    |           arguments to this function are incorrect
    |
note: function defined here
   --> /Users/tsakao/.cargo/registry/src/index.crates.io-6f17d22bba15001f/humantime-2.1.0/src/duration.rs:230:8
    |
230 | pub fn parse_duration(s: &str) -> Result<Duration, Error> {
    |        ^^^^^^^^^^^^^^
help: consider borrowing here
    |
97  |     match humantime::parse_duration(&s) {
    |                                     +

As you can see from the error message, it suggests that changing the s argument in the humantime::parse_duration function to &s might fix the issue. I found the compiler’s error messages to be incredibly detailed and helpful, which is a great feature.

🤔 The Stress of Thinking "Could This Be Written More Elegantly?"

Now, let's move on to some aspects that I found a bit challenging.

This point is closely related to the satisfaction of writing clean code, but because Rust is so expressive and offers many ways to write code, I sometimes felt stressed thinking, "Could I write this more elegantly?" In Go, I often wrote straightforward code without overthinking it, which allowed me to focus more on the business logic rather than the specific implementation details. Personally, I saw this as a positive aspect. However, with Rust, the potential to write cleaner code often led me to spend more mental energy searching for better ways to express the logic.

For example, when I asked GitHub Copilot about the parse_duration_from_str function mentioned earlier, it suggested that it could be shortened like this:

fn parse_duration_from_str(s: &str) -> Result<Duration> {
    humantime::parse_duration(s)
        .map(Duration::from_std)
        .or_else(|_| s.parse::<f64>().map(|secs| Duration::milliseconds((secs * 1000.0) as i64)))
}

The match expression is gone, and the code looks much cleaner—it's cool. But because Rust allows for such clean code, as a beginner still building my Rust vocabulary, I sometimes felt stressed, thinking I could probably make my code even more elegant.

Additionally, preferences for how clean or "cool" code should be can vary from person to person. I found myself a bit unsure of how far to take this approach. However, this might just be a matter of experience and the overall proficiency of the team.

🤔 Smaller Standard Library Compared to Go

As I’ll mention in a later section, I found that Rust’s standard library feels smaller compared to Go’s. In Go, the standard library is extensive and often covers most needs, making it a reliable choice. In contrast, with Rust, I often had to rely on third-party libraries.

While using third-party libraries introduces some risks, I’ve come to accept that this is just part of working with Rust.

I believe this difference may stem from the distinct use cases for Rust and Go. This is just a rough impression, but it seems that Go primarily covers web and middleware applications, while Rust spans a broader range, including web, middleware, low-level programming, systems programming, and embedded systems. Developing a standard library that covers all these areas would likely be quite costly. Additionally, since Rust’s compiler is truly outstanding, I suspect that a significant amount of development resources have been focused there.

🤔 Things I Don’t Understand or Find Difficult

Honestly, I do find Rust difficult at times, and I realize I need to study more. Here are some areas in Viddy that I’m using but haven’t fully grasped yet:

• Concurrent programming and asynchronous runtimes
• How to do Dependency Injection
• The "magic" of macros

Additionally, since the language is so rich in features, I feel there’s a lot I don’t even know that I don’t know. As I continue to maintain Viddy, I plan to experiment and study more to deepen my understanding.

Rust vs. Go by the Numbers

While it’s not entirely fair to compare the two languages, since the features provided aren’t exactly the same, I thought it might be interesting to compare the number of lines of source code, build times, and the number of dependencies between Rust and Go. To minimize functional differences, I measured using the RC version of Viddy (v1.0.0-rc.1), which does not include the feature that uses SQLite. For Go, I used the latest Go implementation release of Viddy (v0.4.0) for the measurements.

Lines of Source Code

As I’ll mention later, the Rust implementation uses a template from the Ratatui crate, which is designed for TUI development. This template contributed to a significant amount of generated code. Additionally, some features have been added, which likely resulted in the higher line count. Generally, I found that Rust allows for more expressive code with fewer lines compared to Go.

	Lines of Code
Go	1987
Rust	4622

Go

❯ tokei
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
 Go                      8         1987         1579           43          365
 Makefile                1           23           18            0            5
-------------------------------------------------------------------------------
(omitted)
===============================================================================
 Total                  10         2148         1597          139          412

Rust

❯ tokei
===============================================================================
 Language            Files        Lines         Code     Comments       Blanks
===============================================================================
(omitted)
-------------------------------------------------------------------------------
 Rust                   30         4622         4069           30          523
 |- Markdown             2           81            0           48           33
 (Total)                           4703         4069           78          556
===============================================================================
 Total                  34         4827         4132          124          571
===============================================================================

Build Time Comparison

The Rust implementation includes additional features and more lines of code, so it’s not a completely fair comparison. However, even considering these factors, it’s clear that Rust builds are slower than Go builds. That said, as mentioned earlier, Rust’s compiler is extremely powerful, providing clear guidance on how to fix issues, so this slower build time is somewhat understandable.

	Go	Rust
Initial Build	10.362s	52.461s
No Changes Build	0.674s	0.506s
Build After Changing Code	1.302s	6.766s

Go

# After running go clean -cache
❯ time go build -ldflags="-s -w" -trimpath
go build -ldflags="-s -w" -trimpath  40.23s user 11.83s system 502% cpu 10.362 total

# Subsequent builds
❯ time go build -ldflags="-s -w" -trimpath
go build -ldflags="-s -w" -trimpath  0.54s user 0.83s system 203% cpu 0.674 total

# After modifying main.go
❯ time go build -ldflags="-s -w" -trimpath
go build -ldflags="-s -w" -trimpath  1.07s user 0.95s system 155% cpu 1.302 total

Rust

# After running cargo clean
❯ time cargo build --release
...(omitted)
    Finished `release` profile [optimized] target(s) in 52.36s
cargo build --release  627.85s user 45.07s system 1282% cpu 52.461 total

# Subsequent builds
❯ time cargo build --release
    Finished `release` profile [optimized] target(s) in 0.40s
cargo build --release  0.21s user 0.23s system 87% cpu 0.506 total

# After modifying main.rs
❯ time cargo build --release
   Compiling viddy v1.0.0-rc.0
    Finished `release` profile [optimized] target(s) in 6.67s
cargo build --release  41.01s user 1.13s system 622% cpu 6.766 total

Comparison of Non-Standard Library Dependencies

In Go, I tried to rely on the standard library as much as possible. However, as mentioned earlier, Rust's standard library (crates) is smaller compared to Go's, leading to greater reliance on external crates. When we look at the number of libraries Viddy directly depends on, the difference is quite noticeable:

	Number of Dependencies
Go	13
Rust	38

For example, in Go, JSON serialization and deserialization are supported by the standard library, but in Rust, you need to use third-party crates like serde and serde_json. Additionally, there are various options for asynchronous runtimes, and you need to select and integrate them yourself. While there are libraries that can be considered de facto standards, the heavy reliance on third-party libraries raises concerns about increased maintenance costs.

That said, in Rust, it seems wise to adjust your mindset and be more open to depending on external crates.

Other Topics

Ratatui Template is Convenient

For this project, I used a crate called Ratatui to build the TUI application in Rust. Ratatui offers templates that I found extremely useful, so I’d like to introduce them here.

Similar to GUI applications, TUI applications are event-driven. For example, when a key is pressed, an event is triggered, and some action is performed. Ratatui provides the functionality to render TUI blocks on the terminal, but it doesn’t handle events by itself. Therefore, you need to create your own mechanism for receiving and handling events.

The templates provided by Ratatui include this kind of structure from the start, allowing you to quickly build an application. Additionally, the templates come with CI/CD setups using GitHub Actions, key mapping, and style configurations that can be customized by reading from files.

If you’re planning to create a TUI in Rust, I highly recommend considering the use of these templates.

Calling for RC Testing in the Community and on Reddit

To let the community know that Viddy v1.0.0 is the version reimplemented in Rust, I announced it via a GitHub Issue and on Reddit. Fortunately, this resulted in various feedback and bug reports, and some contributors even found issues on their own and submitted PRs. Without this community support, I might have released the version with many bugs still present.

This experience reminded me of the joys of open-source development. It boosted my motivation, and I am truly grateful for the community's help.

New Features in Viddy

For some time, Viddy users have requested a feature that would allow them to save the history of command outputs and review them later. In response, we’ve implemented a "lookback" feature in this release that saves the execution results in SQLite, allowing you to relaunch Viddy after the command has finished and review the results. This feature makes it easier to share the change history of command outputs with others.

By the way, the name "Viddy" itself is a nod to cinema, and I plan to continue incorporating movie-related themes into the project. I’m particularly fond of the name "lookback" for this new feature, as it aligns with this theme. Also, the Japanese animation movie Look Back was absolutely fantastic.

Demo

About the Icon

Currently, Viddy uses a Gopher icon, but since the implementation language has switched to Rust, this might cause some confusion. However, the icon is fantastic, so I plan to keep it as it is. 😄

The phrase "Viddy well, Gopher, viddy well" might have taken on a slightly different meaning now, too.

Conclusion

Through the challenge of rewriting Viddy from Go to Rust, I was able to deeply explore the differences and characteristics of each language. Features like Rust’s error propagation and the Option type proved to be extremely useful for writing safer and more concise code. On the other hand, the expressive power of Rust sometimes became a source of stress, especially when I felt compelled to write the most elegant code possible. Additionally, the smaller standard library in Rust was recognized as a new challenge.

Despite these challenges, prioritizing the release and focusing on getting something functional out there allowed the rewrite to progress. The support from the community in testing and improving the RC version was also a significant motivator.

Moving forward, I plan to continue developing and maintaining Viddy in Rust to further improve my skills with the language. I hope this article serves as a helpful reference for those considering taking on Rust. Finally, if you see any areas for improvement in Viddy’s code, I would greatly appreciate your feedback or PRs!

https://github.com/sachaos/viddy