Posted on Oct 11, 2023

Build your own Shell : PART 1 👨🏾‍💻

#c #codenewbie #beginners #programming

A shell is a program that provides a user interface to the operating system. It allows users to interact with the system by typing commands. The shell interprets these commands and executes them on behalf of the user.
When a user logs into a system, the shell is typically the first program that is started. The shell then presents the user with a prompt, which is a symbol or sequence of symbols that indicates that the shell is ready to accept a command.

The user can then type a command and press Enter. The shell will then interpret the command and execute it. If the command is successful, the shell will display the output of the command to the user. If the command is not successful, the shell will display an error message.

In this article, we are going to build our own Shell.

Pre-requisite

What is Shell
Basic understanding of C programming
System Calls

I would also encourage you to familiarize yourself to use the man pages to read the implementation of any command you wouldn't understand. I would provide the links at various points in this article.

Getting Started 🔥

Now let's get to the core of building the shell. As we go along, the various system calls I implement would be explained.

Basic Functionality

Let's look at what happens when you enter a command in the shell. Eg.

ls -l

STEP 1. User enters command ls -l.
STEP 2. The shell creates a new process(child process) using the fork() function. This creates a copy of the shell process, with its own memory space and program counter.
STEP 3. This child process, executes the ls command with another system call (execve). This replaces the child process's image with the image of the ls command.
STEP 4. The shell waits for the child process to terminate using the wait() function. This ensures that the shell does not continue to the next command until the ls is done executing.
STEP 5. The child process then parses the command line using the strtok() function. This breaks the command line into individual words and arguments. In our case ls, -l.
STEP 6. The child process executes the ls, with the -l argument. This displays a long listing of the content of the current directory.

As shown in the picture below

STEP 7. The child process terminates.
STEP 8. The shell continues to wait for the next command from the user.
STEP 9. If the user enters "exit" or Ctrl+D, the shell exits. This means the EOF (end-of-file) has been reached. Read more on EOF here

From the above steps we now understand what happens anytime we interact with the shell.

Let's start by creating the files we need as seen.
First create a directory named my_shell like so mkdir my_shell, then add these two files touch main.c and touch main.h.

Let's add some codes to both files.
Add this to the main.h file

#ifndef MAIN_H
#define MAIN_H

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>

#endif /* MAIN_H */

And to the main.c file

#include "main.h"

/**
  * main - Main entry point for our program
  * @argc: Argument count to the main
  * @argv: Pointer to array of argument values
  *
  * Return: O Always success
  */

int main(int argc, char *argv[])
{
        (void)argc, (void)argv;
        write(STDOUT_FILENO, "MyShell$ ", 9);
        return (0);
}

Now we would execute our program using this command.

gcc -Wall -Wextra -pedantic *.c -o shell && ./shell

This of course would print a simple "#myShell$ " to the screen. Feel free to rename it as you please.

Following the steps I listed above, we need accept input from the user to be executed. This can be achieved with the function getline().

Getline()

man getline

Synopsis:

 #include <stdio.h>

 ssize_t getline(char **lineptr, size_t *n, FILE *stream);

Description: The getline() function reads a line of text from a stream. The stream can be a file, a pipe or the standard input (meaning reading what the user enters to the shell like in our case).
Return Value: On success, the number of characters is returned, otherwise -1 on failure.

Let's use this to read input from the user. In the main.c file, update the codes to look like so.

int main(int argc, char **argv)
{
        (void)argc, (void)argv;
        char *buf = NULL;
        size_t count = 0;
        ssize_t nread;

        write(STDOUT_FILENO, "MyShell$ ", 9);
        nread = getline(&buf, &count, stdin);

        if (nread ==  -1)
        {
                 perror("Exiting shell");
                 exit(1);
        }
        printf("%s", buf);
        free(buf);
        return (0);
}

Explanation

The argc and argv arguments of the main are typecast to void since they are not being used currently.
We initialize the various variables with their datatypes.
We first print "MyShell$ " to the screen and wait for the user input.
The getline function is used to get the user input and it stores it in the buf variable.
- buf - Holds the entire user input
- buf_size - This is the number of bytes in the buf
- stdin - This allows us to receive input from the user
We then store the return value of the getline in nread.
The condition then checks if getline failed to read (ie. if it returns -1) as we saw in the return statement, and then prints a system error message using the perror function (perror is used to print system error messages).
However if getline is successful, we want to print the user input.
Finally, getline dynamically allocates memory for the buf, hence we need to free up that memory and then return 0 to the main.

When this code is compiled and run we should get the following output.

But wait ✋🏾, our program just exited. That's not how the shell behaves right? We want the user to manually exit the shell by typing "exit", or a combination of "Ctrl + D" which signifies the end-of-file.

We can solve this adding an infinite while loop to our program as shown below

int main(int argc, char **argv)
{
        (void)argc, (void)argv;
        char *buf = NULL;
        size_t count = 0;
        ssize_t nread;

        while (1)
        {
                write(STDOUT_FILENO, "MyShell$ ", 9);

                nread = getline(&buf, &count, stdin);

                if (nread ==  -1)
                {
                        perror("Exiting shell");
                        exit(1);
                }
                printf("%s", buf);
        }
        free(buf);
        return (0);
}

When this is compiled and executed we should get as shown in the images below.

So far, our program is reading our input and returning it correctly as supposed.

Now we would want to execute our program so that when the user inputs a command, a result would be given. As stated in the steps earlier on, our shell would need to create a child process for executing every command.

NOTE: Every process that runs in the shell has a unique identifier known as process ID (PID). Each process on a system has a unique PID. The PID is used by the operating system to track and manage processes. A process (parent process) can create another process (child process).

Example
When a user inputs /bin/ls, the following happens:

The shell forks a child process.
The child process executes the /bin/ls command.
The child process terminates.

With this analogy in place we would go ahead and create a child process which would execute the user input. The function fork() would be used.

Fork()

man 2 fork

Synopsis:

#include <sys/types.h>
#include <unistd.h>

pid_t fork(void);

Description: The fork() function simply creates a child process by duplicating the calling process.
Return Value: On success, the PID (process ID) of the child process is returned in the parent and 0 is returned in the child. Otherwise, -1 is returned to the parent and no child is created.

Once we are able to create the child process, we need to make the parent process wait for the child process to execute. Remember that the child process was created to execute the "/bin/ls" command from the user. To make the parent process wait, we use the function wait();

Wait()

man 2 wait

Synopsis:

#include <sys/types.h>
#include <sys/wait.h>

pid_t wait(int *wstatus);

Description: The wait() function causes the current process to wait until one of its child processes terminates. When a child process terminates, the wait() function returns the child process's exit status.

The wait() function is typically used to ensure that all of a process's child processes have terminated before the process itself terminates.

Return Value: On success, return PID of the terminated child otherwise -1 on error.

Let's edit the main.h file to include the header files which would enable us use the functions we have discussed

#ifndef MAIN_H
#define MAIN_H

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>

#endif /* MAIN_H */

Now let's implement the fork() and wait() in our program.
Update the main.c file with this code


int main(int argc, char **argv)
{
        (void)argc, (void)argv;
        char *buf = NULL;
        size_t count = 0;
        ssize_t nread;
        pid_t child_pid;
        int status;

        while (1)
        {
                write(STDOUT_FILENO, "MyShell$ ", 9);

                nread = getline(&buf, &count, stdin);

                if (nread ==  -1)
                {
                        perror("Exiting shell");
                        exit(1);
                }

                child_pid = fork();

                if (child_pid == -1)
                {
                        perror("Failed to create.");
                        exit (41);
                }

                if (child_pid == 0)
                {
                        /* The creation was successful and we can execute the user input */
                        printf("The creation was successful\n");
                }
                else
                {
                        /* Wait for the child process to execute before terminating the parent process */
                        wait(&status);
                }
        }
        free(buf);
        return (0);
}

If the fork() returns a 0 in the child, the process has been created and hence we can execute the command given by the user. To achieve this, we use the execve() function.

Execve()

man 2 execve

Synopsis:

#include <unistd.h>

int execve(const char *pathname, char *const argv[], 
          char *const envp[]);

Description: The execve() function is typically used to execute a new program referred to by the pathname variable as seen above. *The pathname is an executable file which is found on your system.
- Argv is a an array of strings
- Envp is also an array of strings (more on that later)
- Both Argv and Envp should be terminated by a NULL pointer.

Let me explain with an Example.
When the user inputs ls -l to the shell, the fork() creates the child process to execute that command. When the child process is successfully created (child_pid == 0), the execve function looks for a NULL terminated array format like so argv = {"ls", "-l", NULL}. The first argument of the execve (pathname) is replaced by ls which is argv[0] or the first element in the array argv.

NOTE: This ls is an executable file which is found in the /bin folder. So when you input ls it would give you the same output as /bin/ls in the shell (we would implement that later in our shell). Our execve function currently accepts only the full path of the executable and therefore ls wouldn't work; however /bin/ls which is the path of the full executable would work.

Return Value: On success the execve doesn't return, however it returns -1 on error.

Phew! That was a lot 😀. Let's implement it in our main.c file

/* Add this code to the condition when child_pid == 0 */
 if (execve(array[0], array, NULL) == -1)
           {
                   perror("Couldn't execute");
                   exit(7);
           }

So far we have seen that the execve accepts an array of strings which is terminated by the NULL pointer. The question now is, how do we get that array of string inputs. Remember that, the user only inputs /bin/ls -l and our shell should put this in an array format for the execve function to use. To accomplish this, we use a function called strtok() to split the inputs of the user.

Strtok()

man 3 strtok

Synopsis:

#include <string.h>

char *strtok(char *str, const char *delim);

Description: The strtok() function breaks a string into a sequence of tokens. A token is a sequence of characters that is separated from other tokens by a delimiter character (delim). The strtok() function returns a pointer to the first token in the string.

The strtok() function is typically used to parse strings into individual words or arguments.

In the case of the /bin/ls -l command, the *str becomes "/bin/ls" and our delim becomes a space (" "). This means we are separating the string inputs by the space.

The shell uses the strtok() function to parse the command line into the following words and arguments:
/bin/ls (command name)
-l (argument)
The shell then passes these words and arguments to the execve() function, which executes the /bin/ls command with the -l argument.

Return Value: On success, the PID (process ID) of the child process is returned in the parent and 0 is returned in the child. Otherwise, -1 is returned to the parent and no child is created.

Let's implement the strtok in our main.c file.

/* Add these initializations just below the main function */
char *token;
char **array;
int i;

/* Add these lines in the while loop after the getline condition */
token = strtok(buf, " \n");

array = malloc(sizeof(char *) * 1024);
i = 0;
while (token)
{
      array[i] = token;
      token = strtok(NULL, " \n");
      i++;
}
array[i] = NULL;

Don't forget to update your main.h file with this

/* Add this line to the include statements */
#include <string.h>

Explanation of the code

The strtok takes in the input of the user which has been stored in the buf and separates it by a space or the newline "\n"(because the user would press ENTER for the command to execute).
We create an array so we can store the tokens (this would be used by the execve).
Since an array is created, we enter a loop where we assign each token to a space in the array.
We update the token and set it's first argument to NULL, and then assign the delimiter appropriately. This is found in the usage of strtok() function.
If there are no more tokens (or strings to be parsed), we need to add the NULL pointer to the array per the demands of the execve function.

Now that we have implemented all these functions, this is how your main.h file should look like.

#ifndef MAIN_H
#define MAIN_H

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <string.h>


#endif /* MAIN_H */

This is the final code in the main.c file

#include "main.h"

/**
  * main - Getline function
  * @argc: Argument count
  * @argv: Array of argument values
  *
  * Return: 0 on success
  */

int main(int argc, char **argv)
{
        (void)argc, (void)argv;
        char *buf = NULL, *token;
        size_t count = 0;
        ssize_t nread;
        pid_t child_pid;
        int i, status;
        char **array;

        while (1)
        {
                write(STDOUT_FILENO, "MyShell$ ", 9);

                nread = getline(&buf, &count, stdin);

                if (nread ==  -1)
                {
                        perror("Exiting shell");
                        exit(1);
                }

                token = strtok(buf, " \n");

                array = malloc(sizeof(char*) * 1024);
                i = 0;

                while (token)
                {
                        array[i] = token;
                        token = strtok(NULL, " \n");
                        i++;
                }

                array[i] = NULL;

                child_pid = fork();

                if (child_pid == -1)
                {
                        perror("Failed to create.");
                        exit (41);
                }

                if (child_pid == 0)
                {
                        if (execve(array[0], array, NULL) == -1)
                        {
                                perror("Failed to execute");
                                exit(97);
                        }
                }
                else
                {
                        wait(&status);
                }
        }
        free(buf);
        return (0);
}

When we compile and execute with

gcc -Wall -Wextra -pedantic *.c -o shell && ./shell

We should get as seen in the images

You can test your shell with these commands to make sure it's working
/bin/ls, /bin/ls -l, /bin/ls -la, /bin/pwd, /bin/echo "Hello World", /bin/cat main.c.

Remember that this is the absolute path of the commands ls, pwd, echo, cat etc. We would add functionalities so that these could give us the results their absolute path gives us.

Congratulations 👏🏾, you have built a simple shell with basic functionalities.