DEV Community: Syed Faraaz Ahmad

How GraalVM improves Ruby

Syed Faraaz Ahmad — Tue, 20 Dec 2022 13:41:16 +0000

Recently, there was a Java meetup held at work (Deepsource) where I gave my first ever talk, "How GraalVM improves Ruby".

In this I talk about how the Java ecosystem has become so much larger than just Java the language, that it's optimizations get carried over to other languages (in this case, Ruby) which ends up improving them.

I hope you find this talk useful!

web3 does a lot more than you've been told

Syed Faraaz Ahmad — Fri, 31 Dec 2021 18:43:56 +0000

Look, I get it.

The past few years have been full of cryptocurrency scams, thinly veiled Ponzi schemes, widespread art fraud for NFTs, and all the other stuff that didn't make the news. Blockchains have been the underlying tech powering all of that, so you have no reason to be be lending any of your trust to them.

Yes, blockchains are overhyped, and yet they do a lot of good work that you haven't heard of. Just because there are people being stabbed with knives doesn't mean someone somewhere isn't doing a masterful job cutting onions! (weird analogy but bear with me for a moment)

So where are all the onion masters?

All right, lets assume blockchains actually do some good stuff. Then why haven't I heard about them? Where is that big app that will make me believe in blockchains?

Unfortunately, the answer is not very simple.

No one is going to use something just because its built on blockchains. It has to be good enough to be the best option to choose at the time, never mind what tech is used to build it. Let me explain a bit why a blockchain could be needed.

I'll raise you a scenario -- You're happily (or begrudgingly) working along on that task that you need to finish by tonight. Maybe it is so boring and soul-crushing that you're starting to contemplate leaving your job. But that doesn't matter right now, you're already 2 weeks late and you've run out of excuses. So your only option is to grind through, pull up some NPM packages and finally finish that damn thing.

But wait here's the fun part, NPM is down

But that wont stop you today! You know that package's Github repo. So you go there, download and store it in your node_modules. You hack along undeterred, convinced that you're going to finish it by tonight. Finally its done, reviewed and merged into the dev branch. But wait! Why's it not showing up on the site? The deploy seems to be failing. You google around, ask your team members who are going through their own personal hell, and then finally in your late night, coffee fueled frenzy open up twitter to check what the heck's going on.

// Detect dark theme var iframe = document.getElementById('tweet-1471148104057458691-319'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1471148104057458691&theme=dark" }

Great. AWS is down again. So you just sit there like...

If you look deeper than the surface level, these are problems that arise due to single points of failure. Its these problems that a decentralised network aims to fix. What's "decentralised"? well, it just means that no single entity is in control of the whole thing. If one node goes down, there's lots of other ones to keep it afloat. When you have your app hosted on a fairly large decentralised network (like ethereum), you essentially have a failsafe for the moment of a node failure. For when one node goes down, the app will just be moved to another node.

Now you certainly don't need a blockchain for everything, especially if you have control of all the hardware on the network. But in blockchains when anyone can join the network from anywhere in the world, how do you know which node is a bad actor and which isn't. To put it differently, who do you trust?

Well, you cant trust anyone over the internet. So you simply take trust out of the equation, that's why you'll often hear the word "trustless" in the context of blockchains. What enables these trustless networks are "consensus algorithms". Simply put, they make sure all the nodes are doing what they're supposed to. They reward the nodes that do good and in a way punish those who don't.

Now on to the onion masters.

Hopefully my rambling has made at least some sense to you and you see value in the existence of the blockchain. Now lets introduce you to some of the big (and small) names taking the industry forward:

Ethereum (Website link)

Unlike Bitcoin, which was made to be a currency, Ethereum lets you write something called "Smart Contracts" which basically enable you to run your app on the blockchain. This way your app is up 24/7 and you wont have to deal with us-east-1 going down yet again. Although, the main Ethereum network has become quite slow and has a very high transaction fees, there are Level 2 chains that are built on top of Ethereum and are faster and cheaper (but that's a topic for another day).

Filecoin (Website link)

What if you have a lot of important files stored on Dropbox and it goes down just as you want to access one of them? Yes, we're back to that idea of the centralised service that can fail anytime. Filecoin is another blockchain that stores files on a decentralised network. Anyone who stores the files reliably on the network, is paid "Filecoin" (its the name of the organisation AND the cryptocurrency) and whenever someone wants to retrieve those files, they need to pay Filecoin. Think Dropbox, but where all the servers that store data are owned by unrelated individuals around the world who're paid to reliably store data. There's a very good introduction about Filecoin over at their website, I highly encourage you to check it out.

Fleek (Website link)

Remember how I told you your app can go offline anytime the centralised cloud you're hosting it on experiences an outage? Here's the alternative. Fleek is a "cloud provider" that lets you host apps on Ethereum and files on Filecoin to get rid of the single point of failure problem. It's as easy to use as any cloud provider (probably easier). This whole goes back to the concept of being the best tool to use irrespective of the underlying tech.

ThirdWeb (Website link)

Like any new technology, building for blockchains is initially hard and cumbersome. It's not as bad as writing machine code by hand, but its certainly not as easy as scaffolding an app with Ruby on Rails (if you know you know). You need to write smart contracts in another language called Solidity, or in the case of Solana, you need to write it in Rust. And deploying your app to the chain is certainly not as easy as a git pushing your Node.js app. But all is not lost, ThirdWeb has recently come up with SDKs in languages like Javascript, Python and Go which makes astonishingly easy to write and deploy "web3" code.

Take back control

If you've made it this far into the post you probably see a pattern in all these services. These systems are set up in a way that takes power away from a centralised entity and distributes it to the individual nodes. Ideally these individual nodes are run by one person or a small group of people (which is where a potential problem lies).

It's not all sunshine and rainbows

If everything goes well, blockchain tech will take away or at least reduce the power held by big tech companies (Meta, Amazon, Apple, Google, Microsoft, etc.). What's bad about them having all this power? Well, if they one day decide to be more greedy, there won't be a viable option elsewhere and people will just have to follow suit regardless. Sure, legal courts can intervene. But that process starts when the damage has already been done, and to be fair legal proceedings can take a fairly long time.

But there's a loophole in blockchains too. It's fine if each person runs one node, then the system is truly decentralised and each person gets their own say. But nothing's stopping you from pooling resources with people and start a crypto mining farm that out competes most single person nodes, and essentially centralises power again (we're already seeing this happen extensively). And remember big tech and its prophecised demise? Well, nothing's stopping them to start their own mining farm and earn through that.

The mere existence of blockchain does not guarantee a better future for tech. It has to be actively worked on by enthusiastic developers in every corner to make it work. Or else, it will be just another piece of tech that gets dominated by soulless large corporations whose only aim is to make profits. Which is why you will see supporters of blockchain or "web3" hype it up so much. It is to get more individuals get involved and have a majority stake in it, so that when the big corporations want to have a piece of it, they can only be a minority.

So will you help bring power back to the individual or will you let big tech have it all? Will you take the blue pill or the red pill?

You take the blue pill, the story ends. You wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes

NOTE: I am in no way affiliated to or supported by any of the above services. I don't even hold any Ethereum (!)

What frustrates you most about developers' hiring process?

Syed Faraaz Ahmad — Thu, 20 May 2021 14:24:35 +0000

While hiring developers for your organisation, what are the biggest challenges you have come across? If you could change some things in the hiring process, as a recruiter or a developer what would they be?

I would love to chat.

I was bored, so I built a programming language

Syed Faraaz Ahmad — Wed, 24 Mar 2021 19:49:37 +0000

One fine evening in the first lockdown of 2020, there I was, sitting idly, not knowing what to do.

Calmly freaking out.

You see, I really needed something to do. I had been doing a few web related projects on the side and that was something I didn't want to do any more, at least for a while. So I looked into doing something "closer to the metal", something much lower level than sending requests back and forth to a web server. So I quickly fired up Learn X by doing Y and searched for something interesting, eventually ending up on Building your own Lisp (We all have a Lisp phase, it was just my turn).

For the uninitiated, Lisp is (simply put) a family of programming languages whose syntax looks more or less like this (You can quickly see where the parentheses jokes stem from).

(defun area-circle(rad)
   "Calculates area of a circle with given radius"
   (terpri)
   (format t "Radius: ~5f" rad)
   (format t "~%Area: ~10f" (* 3.141592 rad rad)))
(area-circle 10)

But I didn't just want to create any Lisp, I wanted to make some modifications as per my preference. So there was only 1 way to go about doing this.

Coming up with a syntax

What got me into lisps and writing Lisp was (suprisingly) the Emacs text editor (or Emacs OS for the hardcore users). What was I doing going anywhere even remotely near Emacs? I don't know, I'll tell you some other day. You see, writing Emacs Lisp is the recommended way of configuring Emacs. Even though I didn't write a lot of Emacs Lisp, a few of the keywords I saw there didn't feel very natural to me (especially coming from using Ruby). So one of the major goals would be to have more "natural" sounding keywords.

You can't talk about Lisp without talking about the parentheses. A big advantage in readability for other languages is that everything isn't inside a parenthesis. Square brackets usually denote an array, braces usually denote code blocks, while parenthesis are used for function calls. It leads to easier visual grepping of code, not so much in Lisp, especially as a beginner. So I need to reduce the usage of parentheses, but I cant just change all of its syntax because then it might not stay a Lisp anymore (Ship of Theseus, anyone?). I think Clojure strikes a good balance between staying a Lisp and improving visual grepping. So another goal would be to reduce the reliance on parentheses so that the code can be more readable.

'(1 2 3)     ; list
[1 2 3]      ; vector
#{1 2 3}     ; set
{:a 1, :b 2} ; map

(Example from Clojure syntax guide)

So it began

After setting all these requirements for the language, I set out to finally start building it. I chose Rust to build it because it seemed like the right tool to do so because of it's speed and memory safety. I was building my own programming language, I felt like a "real programmer" (no such thing btw) so naturally I started tweeting about it.

// Detect dark theme var iframe = document.getElementById('tweet-1329145543314980865-337'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1329145543314980865&theme=dark" }

Starting simple

I wanted to start off with something as simple as possible, like adding a few numbers. It felt like the perfect starting point. I'll be evaluating a simple arithmetic expression written in Lisp, for example, 2 * 3 + 1 - 4, which would be written as (- (+ 1 (* 2 3)) 4) in Lisp. To the unaware, this is in the form of Polish notation, just think of it as a simple function with multiple arguments. (+ 1 2 3) is simply the sum of 1, 2 & 3. Its result can then be used as an argument to a parent function, and then that gets evaluated. Similarly the whole expression can be solved using the method below.

(Source: Crafting Interpreters)

After building the initial prototype I tested it with (+ 2 (- 4 6) 8). Doing it manually, you can tell it's 4 - 6 + 2 + 8 = 8. And if you see the result in the bottom red rectangle, the result is indeed 8.

Laying the building blocks

Now that we've had something to play around with, it's time to grow up and lay the foundations for a proper compiler. The first step for any compiler is to read the source file and turn it into a stream machine readable objects called tokens. This step is called lexical analysis, or lexing. It simply maps all possible characters in the programming language to a Token object. Like so:

Character	Token
(	Token { kind: OpenParen, value: None }
let	Token { kind: Identifier, value: Some("let") }
-?([0-9])+	Token { kind: Number, value: Some(getVal()) }

In this way you can build a mapping for token and their various types, so that its easier to build your syntax tree in the next step.

Cool, now that I can generate tokens, it was time to build on top of this abstraction and give this stream of tokens a proper structure so that it's easier to work with. We're going to go examine every token and make sense of it all by generating a syntax tree. This step is called parsing and at its end, we going to have something like this:

(Source: Abstract Syntax tree:Wikipedia)

So I go about trying to build a syntax tree, starting with defining a basic grammar in the form of an enum which looked something like this

pub enum Expr {
    Constant(Value),
    Builtin(Ident), 
    Call(Box<Expr>, Vec<Expr>),
    While {
        condition: Box<Expr>,
        body: Vec<Expr>,
    },
}

Reading the source code of the Rust programming language was very helpful and I might have stolen some concepts from it. Let me just quickly define what these types are:

Constant: Any literal of type Value (number, string, boolean)
Builtin: Any builtin token like an operator (+,-,*,/) or a keyword (let, print, if)
Call: this is simple, any function call (truth be told, everything should just be a function call since Lisp is a functional language)
While: A while loop (duh)

(If you're wondering, Box is Rust's built-in type for pointers, broadly speaking.)

This is the gist of it, rest of the process was just finding ways to build this syntax tree using the token stream I had. Slowly but surely, I was making progress.

// Detect dark theme var iframe = document.getElementById('tweet-1330497714107404291-774'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1330497714107404291&theme=dark" }

At this point, the syntax tree felt robust enough for what I wanted the language to do for now. It was time to move on to bigger and better things.

LLVM

Low..Level..Virtual Machine? good guess, but not anymore. What is LLVM? let's first take a step back and see why it's needed in the first place. You see, a compiler has 2 sides - a frontend and a backend - the former containing lexing and parsing, while the latter has intermediate code generation, optimisation and target code generation. What I'd built until now was just the frontend, and I had the internal representation that can be used by the backend.

To build all the backend from scratch, here's what I would have to do. Firstly, limit myself to an architecture since supporting multiple processor architectures was not an option, simply because of the intricacies of the way processors work. After selecting the most popular processor architecture I'd need to get a good grasp of its assembly language. You see, because of the different ways microprocessors are designed and made, they all have different instruction sets. To support an instruction set, you need to take the code the user has written in your language and translate it in the form of instructions of that specific set. Processors can work in weird ways, sometimes for compatibility reasons and sometimes just because they can. So when you optimise your generated code, you have to take all that into account. Let's say you do manage to support one architecture successfully but now there's (somehow) popular demand for your language on some other architecture, you will have to repeat this process all over again, and then again for another architecture, and so on. Don't even get me started on how you would debug errors that are specific to only some architectures. It would be a huge mess.

This is where LLVM comes in, it provides a convenient abstraction layer over all the instruction sets that you can target. You only need to worry about translating to its own internal representation (LLVM IR). LLVM will then handle all the code optimisation to its IR, and then for the instruction set you're targetting. But it's still a good idea to first optimise your code and then give it to LLVM, instead of just expecting LLVM to do it all.

So to use LLVM, all I needed to do was to take my AST and convert it into LLVM IR. I did it using Inkwell the LLVM API in Rust. It is a wrapper around the C API that LLVM has built, and on top of that provides the memory safety guaranteed by Rust. Oh, and this is what LLVM IR looks like in text format (Don't try to understand it, feel it):

; ModuleID = 'example'
source_filename = "/home/faraaz/hello.tp"

@format_string = private unnamed_addr constant [4 x i8] c"%f \00", align 1

define i32 @main(i32 %0) {
entry:
  %num = alloca double
  store double 4.200000e+02, double* %num
  %num1 = load double, double* %num
  %printf = call double (i8*, ...) @printf(i8* getelementptr inbounds ([4 x i8], [4 x i8]* @format_string, i32 0, i32 0), double %num1)
  ret i32 0
}

declare double @printf(i8*, ...)

Code in LLVM IR that sets a variable to 420 and prints it

Making it all legit

Until now, executing the compiler's binary did only one thing and it wasn't very configurable or helpful for someone using it for the first time. I wanted it to have a proper Command Line Interface (CLI), something like this:

So I used the Clap crate (aka package) to build the CLI and I think it turned out all right.

Bringing it all together

Now that I have the compiler set up, and the language has features like storing variables, performing arithmetic, looping (nested loops don't work yet) and printing to screen, it was time to make a small program with it. Here's me writing a program to compute the first 5 Fibonacci numbers in Tisp and then compile and run it:

Here's the project if you want to check it out yourself.

faraazahmad / tisp

A toy programming language based on Lisp and built in Rust & LLVM

Tisp (Toy Lisp)

A Lisp-like programming language that is typed and compiled. It aims to support multiple processor architectures by being built upon LLVM. It takes inspiration from programming languages like Rust, Lisp and Elixir.

Current working example

A program to compute first 5 fibonacci numbers:

(let first 0)
(let second 1)
(let fib)
(let n 0)

(while (< n 5)
    (let fib (+ first second))
    (let second first)
    (let first fib)

    (let n (+ n 1))
    (print fib)
)

Features to build

Convert raw code into token stream
Convert token stream into Expression tree
Handle multiple levels of nested expressions
Have multiple (independent) expressions per file
Generate LLVM IR for the currently supported features
add CLI flag to emit llvm
add while loop
Declare variables
add…

View on GitHub

Here's what I learned

Programming languages are hard. Its' hard to read code in the first place. To understand and learn one is way harder. One can only imagine how hard it is to build one, let alone build one that is widely used. This project was mostly me trying to answer the question "How hard can it really be?", I guess I have my answer now. But it's also kinda easy low-key. Once you dip your toes in the hard parts, the next time you do it, it feels a little easier. Every day you do something hard, it gets a little easier, but you gotta do it everyday, that's the hard part.

Can "good code" be measured?

Syed Faraaz Ahmad — Tue, 23 Feb 2021 07:51:10 +0000

I was recently reading Code Complete, a really good book on writing better software. Naturally, I had a lot of questions while reading it, but the biggest one that kept coming up was, "Can I measure this?" or "Can I quantify this?"

I'm aware of static code analysers, and the vast improvements they bring, but I don't think they can do most of what the book teaches.

So can we write software that "measures" the quality of our software?

Discussion: It should be hard to write bad code

Syed Faraaz Ahmad — Thu, 21 Jan 2021 19:01:02 +0000

So I recently came across a huge React codebase that was hard to understand. There was no proper documentation, it wasn't clear how data is passed around and there are no automated unit tests. Now this is just the tip of the iceberg and I don't want to waste your time with a long rant. I would rather spend time discussing and coming up with solutions.

I think people write bad code (myself included) not because they are bad people but because it is easy to do so, unrealistic deadlines make it even worse. I think if it were hard to write bad code, there would be less of it around.

It is also not just a technical problem, people have to enforce it too. Have a well defined spec on how your codebase should be structured, documented and tested. Then don't approve that PR unless it passes those specs.

I could go on about this, I want to know how you feel on this topic and how you go about fixing it.

I'm building a programming language: Creating tokens

Syed Faraaz Ahmad — Sat, 19 Dec 2020 16:14:33 +0000

Last time around I told you why I'm making this programming language, and how I'm going to go about doing it. Here, we're going to read the code file and start working with it.

When evaluating code of a programming language, you can't just work with raw text all of the time.You see, working with text is expensive, as in, it is (relatively) slow to work with. Not to forget, cumbersome, let me show you how.

Let's say you have a module (a collection of functions) in your code:

module  my_module {

    function say_hello() {
        let name = my_module.get_name()
        print("Hello, ${name}!")
    }

    function get_name() {
        print("What is your name? ")
        let name = get_input()
        return name
    }

}

Then lets say you called one function in the module, like so:

my_module.say_hello()

If you're working off of just text, you'll have to search for a my_module module in the file, then search for a function say_hello in there. Great you got the function. But here's the thing, this function has a call to my_module.get_name inside it. So you got to find the module in the file again, then in that module you need to find the func...

You get the point, its a lot of repetitive process and doing with raw text, makes it even slower. Sure, you could argue that its only 2 functions, how long could it even take on modern hardware. And you're right, it doesn't take long in this example. But you also know that code files are usually fairly long and doing this over and over in big files would add up to a really long time.

This is why we don't want to work with text directly all the time. We read the text once, build some in-memory abstractions for them in our code, then build abstractions upon abstractions to make it easier to create our programming language. The first step of that is called Lexical Analysis (or Lexing).

In lexing, we go through our raw code in text format, and convert that into tokens, the smallest level of abstraction we can create (as far as I know).

Let me show you how a simple expression would be tokenised. Consider a simple expression:

(+ 2 5)

It should converted into a token stream as follows:

[
  Token { kind: OpenParen, value: None },
  Token { kind: Plus, value: None },
  Token { kind: Literal(Number), value: 2.0 },
  Token { kind: Literal(Number), value: 5.0 },
  Token { kind: CloseParen, value: None }
]

I looked at a few approaches to do this step. One was to write a separate Lexer, and the other was to write a parser-combinator (a method that combines lexing and the next step, i.e. parsing). While parser-combinators are amazing, it's not the route I felt like I wanted to take at the time. Also, a I wasn't too excited about scanning the source file character by character myself, so I started looking into some libraries that would ease up that work for me. That's when I stumbled upon Logos.

maciejhirsz / logos

Create ridiculously fast Lexers

Logos

Create ridiculously fast Lexers.

Logos has two goals:

To make it easy to create a Lexer, so you can focus on more complex problems.
To make the generated Lexer faster than anything you'd write by hand.

To achieve those, Logos:

Combines all token definitions into a single deterministic state machine.
Optimizes branches into lookup tables or jump tables.
Prevents backtracking inside token definitions.
Unwinds loops, and batches reads to minimize bounds checking.
Does all of that heavy lifting at compile time.

Example

 use logos::Logos;
 #[derive(Logos, Debug, PartialEq)]
 #[logos(skip r"[ \t\n\f]+")] // Ignore this regex pattern between tokens
 enum Token {
     // Tokens can be literal strings, of any length.
     #[token("fast")]
     Fast,

     #[token(".")]
     Period,

     //

…

View on GitHub

It is a library that generates optimised, fast lexers and that are extremely easy to use. Since they claim to generate lexers that are faster than anything I could write by hand, and is used by lots of people, I guess I gotta take their word for it. They also promised that it will be extremely easy to use, and what do you know, it really is.

It is as easy as mapping an enum value to a string or regex expression, and Logos does everything for you.

#[derive(Debug, PartialEq, Logos)]
pub enum TokenKind<'a> {
    #[token("(")]
    OpenParen,

    #[token(")")]
    CloseParen,

    #[token("+")]
    Plus,

    #[token("-")]
    Minus,

    #[regex("-?([0-9])+", |lex| lex.slice().parse())]
    Number(f64),

    #[regex(r"[ \t\n\f]+", logos::skip)]
    Whitespace,

    #[error]
    Error,
}

Yes, its a very simple lexer. But that's what I want right now. I want it to be able to parse a small, simple (but valid) expression. Like this:

(+ 2 (- 4 6) 8)

And it does do that, as you can see below.

Note that the lexer code you just read doesn't automatically produce the list of Tokens below. Token is an abstraction similar to what I saw in the Rust codebase. It reduces the work that the parser would need to do when consuming this token stream.

[
  Token { kind: OpenParen, value: None }, 
  Token { kind: Plus, value: None }, 
  Token { kind: Literal(Number), value: Some(Value::Number(2.0)) }, 
  Token { kind: OpenParen, value: None }, 
  Token { kind: Minus, value: None }, 
  Token { kind: Literal(Number), value: Some(Value::Number(4.0)) }, 
  Token { kind: Literal(Number), value: Some(Value::Number(6.0)) }, 
  Token { kind: CloseParen, value: None }, 
  Token { kind: Literal(Number), value: Some(Value::Number(8.0)) }, 
  Token { kind: CloseParen, value: None }, 
]

The Somes you see in this token stream is an Option type in Rust, since some tokens don't have any use for the value field

Remember how I wanted each step in the development process to be "able to take me from source code to executable binary in some form or another"? Well, Now I want to evaluate this simple expression and show me the result.

I will use a simple stack based approach for this. Essentially, push all tokens to the stack, and when you get a CloseParen, pop the values for that expression and push the result of the expression back into the stack.

And I ran the code:

It worked! As you can see, the result is a token with a value of 8.

I guess that's it for now. See you in the next post.

Note (this is the last one I promise): In order to keep the quality of my posts to an acceptable level, the rate at which I submit posts is much slower to the rate at which I'm adding features to the language.

If you would like to see the codebase and maybe even contribute, you can head over here:

faraazahmad / tisp

A toy programming language based on Lisp and built in Rust & LLVM

Tisp (Toy Lisp)

Current working example

A program to compute first 5 fibonacci numbers:

(let first 0)
(let second 1)
(let fib)
(let n 0)

(while (< n 5)
    (let fib (+ first second))
    (let second first)
    (let first fib)

    (let n (+ n 1))
    (print fib)
)

Features to build

Convert raw code into token stream
Convert token stream into Expression tree
Handle multiple levels of nested expressions
Have multiple (independent) expressions per file
Generate LLVM IR for the currently supported features
add CLI flag to emit llvm
add while loop
Declare variables
add…

View on GitHub

I'm building a programming language

Syed Faraaz Ahmad — Mon, 14 Dec 2020 15:00:35 +0000

A lot of people that talk about building their own programming language, have a lot of lofty goals for them. They want them to have this feature, that feature, to conquer the world and back.

Me? not so much. I'm building one just to learn how to do it "for real". Sure, I have some opinions about what features I would like more. I would use them as goals for the whole project, not just as design goals for the language.

So what are my goals, well...

I want to build it using Rust: I've been trying to learn Rust for a long time now, and while I've learnt a lot, built a few side projects here and there, there's still a lot I don't know. So I thought, "Why not just go for it and throw myself in the deep end", and that's what I'll be doing. Using Rust to build the language and learn Rust as I move along.
Truly experience how to build a compiler: Sure I've learnt how to do so at university, wrote some code for building automata etc. But nothing beats the experience of actually building it all on your own.
Have a Lisp like syntax: This is really not a big deal. I've recently been diving into Lisp dialects and I like some concepts and some syntax in it, so I'll try adding some of those in my language.
Be fast and support multiple platforms: I don't think there's much benefit building a programming language that's painfully slow to use. To make it fast I want it to be a compiled language (as opposed to an interpreted one), Rust also helps in this area as it is pretty fast. Since it's also going to be multi-platform, I will be using LLVM because I don't want to be dealing with any assembly code and optimisations yet.
Be usable at every step of development: I will be gradually adding features, but I want every feature made to be able to show me some visible output.

Like the image above, I want each step in the development process be able to take me from source code to program in some way or another. Instead of building perfect separate components like the perfect parser and perfect lexer that on their own cannot (obviously) get me from source to program. I want to build a lexer than kinda works and a parser that kinda works (and so on) which combined can get me from source code to executable program.

I hope I come out the other side, having a lot more knowledge in this area than I do now (which is really tiny).

See you around!

APIs to help build your next side project

Syed Faraaz Ahmad — Sat, 29 Aug 2020 20:14:51 +0000

Say you have an idea for a new side project, but dont know what topic to base it on (for example: Bears, Beets, Battlestar Galactica). You really don't wanna be building another to-do list.

You might want to test only a small idea you have, and dont want to build a full fledged app. Here's where you can use APIs as substitutes for parts of the app you're not focusing on.

I made a project using Vue that lists all publicly available APIs for you to use.

As you can see there are a lot of APIs to choose from (643 to be accurate), going through all of them to select one is counter productive. So I have provided a way to filter them by category, and am working on adding more filtering and search options.

It's open source so you can help me improve this project here.

Github Link: https://github.com/faraazahmad/apiland
Live Demo: https://faraazahmad.github.io/apiland

How do you manage your reading list(s)?

Syed Faraaz Ahmad — Fri, 14 Aug 2020 09:55:04 +0000

As programmers, we require lots of reading on a daily basis. Be it code, documentation or any article that interests us. However there isn't enough time to read all the interesting articles you come across, so you save them for later (in a reading list)

Over time, reading lists get larger and larger if not dealt with properly, I want to know how you manage this problem

Do you use an app (ex. Pocket, Instapaper, etc.) to manage tour reading lists?
Do you have your personal routines to deal with this problem? (ex. getting through maximum possible articles on weekends, deleting old articles, etc.)
What would be your ideal app to help fix this problem for you?

I'm asking this because I've had this problem for a long time, and I really want to do something to fix it. Thanks

I had nothing to do, so I built a text editor.

Syed Faraaz Ahmad — Tue, 07 Apr 2020 20:30:37 +0000

What do you do when you have nothing else left to do? I freak out.

So there I was, furiously looking for something to do. Maybe I could start a new open source project (because obviously I'm not going to complete my half finished ones), maybe learn a new programming language, maybe I could finally learn Svelte.

Needless to say, I had analysis paralysis.

But then it hit me, I was recently having problems sticking to a single text editor. I liked Sublime because of its speed but couldn't use it for much because of a lack of plugins support. I mostly use Visual Studio Code because its extensive plugins support but sometimes it can get really slow (thanks electron), its not that big a deal but if you encounter something slightly annoying way too many times, it will get on your nerves. I also tried Vim/Neovim and Emacs and my likeness for them (or lack thereof) is pretty much equal. So go away with your Vim vs Emacs arguments :)

So yeah, I started building my own, this is what it looks like right now.

I first thought of using Electron to build it, since it would be easy (yeah, right) building in technologies I already know. But wouldn't that be counter-productive in the end? The whole vscode community is doing the best job they possibly can and there's no way I can make a faster text editor in electron than they do.

So after a lot of irrelevant steps in my thinking process, I went with creating a text editor in the terminal with Rust (because I wanted to! and I liked it!). Now I'm not some genius who conceptualised all of this out of thin air, I followed
a tutorial by Philipp Flenker which is (thankfully) based on a tutorial I'd previously tried but didn't complete because it was in C.

Building a text editor for the terminal is super challenging as you have to build a lot of UI stuff from scratch and I will never take GUIs for granted ever again.

How'd I do it?

To begin with, you need to clear the terminal window. You can do that you need to print the escape sequence \x1b[2J.

Escape sequences instruct the terminal to do various text formatting tasks, such as coloring text, moving the cursor around, and clearing parts of the screen.

Reading Key presses

In a terminal, all you have are key presses. You need to bind those key presses and their combinations to specific functions you want them to perform. Each key on your keyboard refers to a different byte. You can check them out by running a loop and printing all the input. In C, it is done like this:

#include<stdio.h>

int main() {
  char c;
  while (1) {
    scanf(&c);
    printf(c);
  }
  return 0;
}

When you press the arrow keys you can see how pressing letter keys and character keys produces different output bytes. From here on out, handling key presses is just matching the input against the bytes and performing the required actions.

I'm using a (great) package called termion that abstracts away these little details and makes my life easier. Here's some actual Rust code from the editor:

fn process_keypress(&mut self) -> Result<(), std::io::Error> {
        let pressed_key = Terminal::read_key()?;
        match pressed_key {
            Key::Ctrl('q') => self.quit(),
            },
            Key::Ctrl('f') => self.search(),
            Key::Ctrl('s') => self.save(),
            Key::Ctrl('h') => self.show_help(),
            Key::Char(c) => {
                // don't move cursor to the right if enter is pressed
                self.move_cursor(Key::Right);
                self.document.insert(&self.cursor_position, c);
            },
            Key::Delete => self.document.delete(&self.cursor_position),
            Key::Backspace => {
                // Backspace = going left and perform delete
                if self.cursor_position.x > 0 || self.cursor_position.y > 0 {
                    self.move_cursor(Key::Left);
                    self.document.delete(&self.cursor_position);
                }
            }
            Key::Up
            | Key::Down
            | Key::Left
            | Key::Right
            | Key::PageUp
            | Key::PageDown
            | Key::Home
            | Key::End => self.move_cursor(pressed_key),
            _ => (),
        }

        // ...

        Ok(())
    }

NOTE: Some code has been omitted to make it simple and readable for this post

You can see how the package provides abstractions for the Keys that eliminates my need to work with bytes.

Displaying a status bar

I think a text editor is not a serious text editor until it has a status bar. I kid of course, but it does provide a good user experience by displaying some important info while editing files. The message bar below it can show useful info like help messages and a text input while searching in file.

Roadmap

Although I've managed to build some basic functionality into the editor, there are a lot of features I'd like it to have:

Syntax highlighting
A gutter that displays line numbers
Multiple tabs support
Autocomplete
User specific preferences

Help me out

If this project interests you and you would like to help out or if you just want to take a look at the code, you can visit the Github repo here.

Why do I call it Loop? Well I've been watching Tales from The Loop and couldn't think of another name.

Do you like interactive tutorials?

Syed Faraaz Ahmad — Sun, 15 Mar 2020 19:08:28 +0000

By interactive tutorial I mean, tutorials that contain unit tests for the tasks that you are supposed to achieve in that tutorial. One example is LearnYouNode. It is a tutorial series containing 13 problems that teach you Node.js

Each problems has fixed task(s) that you need to complete and there are unit tests in place to check whether you've completed the tutorial properly, there is no guessing work.

So, do you like such tutorials? Have you ever seen similar ones? I'd really like to know.