I tried building a shell in Rust

Inspiration

Sometime ago, I decided to embark on a new endeavor, I was curious about how shell works, and came across the the codecrafters.io build your own shell challenge, and thought this is a good opportunity for me to learn shell, as well as sharpen my skills in rust.

The endless rewrite

So I start working on the project right away, started with the REPL, the exit, echo, discovered that there are built in commands, which run differently that actual programs, so I restructured the code a bit to fit this shift. Then I proceeded to read the PATH variable, and find binaries, feed them the current ENV variables, and then came the part that got me stuck the most, parsing the arguments.
Now parsing thing is nothing new to me I though, I had built one when I was working on jlox following crafting interpreter book, but somehow, the shell parser turns out to have very weird parsing rules. I first tried splitting things by but that broke once quotation was added, so then I handled the spacing correctly, but an argument does not end when a quote start. Take for example

echo hello"world"

The whole helloworld string is one argument not two.
So that was awkward to handle. but then you have to also handle variables inside double quoted string, which do not necessarily create a new argument, as they can be inside the double quotes, which can be concatenated with a single quote or no quote string, and these variable will need to be tokenized so that later on in the execution the can actually be resolved into their own values. So something like this

echo hello"world"what'one'$USER"user"

and that was a few more complete code refactors.
Once I got this thing working, I just could not really handle how messy my approach looked, and the project flame kind of faded away as other commitments came.

Conclusion

This is not really a problem with the codecrafters platform, as they provide very robust testing, which allowed me to actual find a lot of edge cases in my approaches, and it gave me a road-map and a scope of what needs to be built. But the main issues was me jumping in without really understand the global vision, and how the internals of the system interact with each other.
Learning some theory about how a shell parser works would have saved me a lot of rewrites and rewrites. as I am writing this, I was thinking that maybe the approach of the tokenization might now have been the right one, and maybe something more straight forward would have given me the properties I was looking for without having to mold it too much.
Another thing that was blocking for me, was that I was very much new to rust, and not familiar with the patterns and the actual way of structuring and writing rust code, so part of the rewrites was also me just exploring the rust language as well. This would be fine and all, but it came in the way of my actually learning how shell works.