I Built a Unix Shell from Scratch in C. Here's How It Works.

As a second-year CS student, I recently gave in to the urge to build a Unix shell from scratch. You know, that black screen you type ls into fifty times a day without ever thinking about what is actually happening underneath. Honestly, this is the perfect chance for me to start writing about technical stuff. Since you are already here, how about joining me? Come on, grab a coffee or some chai, I will be waiting :D

Ok. Here is how we are doing this. We are going to take a look at how a terminal actually works under the hood. We will start with the main loop, see how raw text turns into running programs, and replace the mystery of the terminal with our own C code. While doing that, we will talk about some OS theory so everything makes sense.

This whole thing started as a CodeCrafters challenge. What was supposed to be a quick weekend project turned into me staring at dup2() man pages at 2 AM, wondering why my pipes were freezing the whole terminal. Good times.

Oh, and BTW, we are using C (please do not kill me in my sleep, I know pointers are scary, I promise it will be fun though!). If you want to keep the repo open in another tab while reading, the code is at github.com/shiva-kar/SXSHELL-unix-like-shell.

Concepts Covered

• The REPL (Read, Eval, Print Loop)
• String parsing and POSIX quoting rules
• Creating processes (fork, execv, waitpid)
• Environment variables ($PATH searching)
• File descriptors and I/O Redirection (dup, dup2)
• Pipes (|)

Assumptions

• Two-command pipes only: The shell handles ls | grep txt, not massive 5-stage pipelines.
• Output redirection only: We handle >, >>, 2>, and 2>>. No < input redirection yet.
• No signal handling: Pressing Ctrl+C will kill the whole shell, not just the command running inside it.
• Fixed argument limit: MAX_ARGS is set to 64. Nobody is typing 65 arguments by hand anyway.
• No scripting: No if/else blocks, no background & jobs, and no $() subshells.

Sections

• What even happens when you type a command?
• The project layout and the REPL (main.c)
• Parsing: harder than it looks (parser.c)
• Finding programs in $PATH (executable.c)
• Running programs with fork and exec (process.c)
• Builtins (builtins.c)
• Redirection: swapping file descriptors (redirection.c)
• Pipes (pipeline.c)
• The Finale!

What even happens when you type a command?

We use the terminal every day, right? So how does typing ls -la and hitting Enter actually put text on your screen? Let's take a quick tangent to talk about the command lifecycle.

1. The shell reads the string ls -la from your keyboard (stdin).
2. It splits that string into separate words (tokens): ls and -la.
3. It checks if ls is a "builtin" command that the shell handles by itself. (Nope.)
4. It searches through the folders listed in your $PATH environment variable until it finds the actual binary file at /usr/bin/ls.
5. It calls fork() to clone itself into a child process.
6. The child calls execv() to replace its own memory with the /usr/bin/ls program.
7. The parent shell waits for the child to finish using waitpid(), and then prints the $ prompt again.

Side note: Why can't cd just be a normal program inside /usr/bin like ls? Think about it for a second. If the shell forked a child process and ran cd /tmp inside the child, the child would change its directory and immediately exit. Meanwhile, your main parent shell would still be sitting in the exact same folder you started in. Useless, right? That is why cd, exit, and pwd have to run directly inside the shell process itself. We call these builtins.

The project layout and the REPL (main.c)

To make sure we know where everything lives, here is what the folder structure looks like:

src/ main.c # The main loop (REPL) parser.c # Splits input into tokens, handles quotes builtins.c # echo, cd, pwd, type, exit executable.c # Searches $PATH for programs process.c # fork() + execv() + waitpid() redirection.c # The > >> 2> 2>> stuff pipeline.c # Pipes (the | stuff) 

Ok, we can officially start with main.c (please take a quick look at the code :3 )

char *buffer = NULL; size_t size = 0; ssize_t len; setbuf(stdout, NULL); // do not remove this, trust me while (true) { printf("$ "); if ((len = getline(&buffer, &size, stdin)) != -1) { buffer[strcspn(buffer, "\n")] = '\0'; char **argv = parse_arguments(buffer); int saved_stdout = -1; int saved_stderr = -1; hijack_redirections(argv, &saved_stdout, &saved_stderr); if (argv != NULL && argv[0] != NULL) { if (has_pipe(argv)) { execute_pipeline(argv); } else { int status = handle_builtin(argv); if (status == -1) { free_arguments(argv); break; } else if (status == 0) { char *exec_path = find_executable(argv[0]); if (exec_path != NULL) { execute_program(exec_path, argv); free(exec_path); } else { printf("%s: command not found\n", argv[0]); } } } } free_arguments(argv); restore_redirections(saved_stdout, saved_stderr); } } free(buffer); 

I think you may already have an idea about what that code is doing. Print a prompt, read a line, run it, repeat. Still, a couple of lines in there gave me a headache when I first wrote them:

• setbuf(stdout, NULL): In C, stdout usually waits until it sees a newline \n before it actually prints anything to your screen. Because our prompt is just "$ " with no newline, getline() would pause and wait for keyboard input before the $ even showed up on the terminal! Turning off buffering with setbuf fixes this. I wasted way too much time figuring that out.
• getline(): This function is great because it allocates and resizes the memory buffer for you automatically. You just have to free(buffer) when the loop ends.
• strcspn(buffer, "\n"): When you press Enter, getline() leaves the \n character at the end of your string. This line finds that \n and overwrites it with \0 so our command names do not have newlines attached to them.
• handle_builtin() return values: -1 means the user typed exit (so we break out of the loop), 1 means the builtin ran fine, and 0 means "this is not a builtin, go look in $PATH."

Parsing: harder than it looks (parser.c)

At first I thought, "I will just split the input on spaces using strtok(), easy."

Well, try running this in a terminal:

$ echo "hello world" 

If you split on every space, you get three pieces: echo, "hello, and world". That is totally wrong. You are supposed to get two arguments: echo and hello world with the three spaces kept intact and the quotes removed.

So strtok() had to go. Instead, parse_arguments() loops through the string one character at a time:

char **parse_arguments(char *buffer) { char **args = malloc(sizeof(char *) * MAX_ARGS); int args_count = 0; bool is_single_quotes = false; bool is_double_quotes = false; char *current_word = malloc(strlen(buffer) + 1); int word_len = 0; for (int i = 0; buffer[i] != '\0'; i++) { char c = buffer[i]; // backslashes outside quotes: next char is literal if (c == '\\' && !is_single_quotes && !is_double_quotes) { char next_char = buffer[i + 1]; if (next_char != '\0') { current_word[word_len++] = next_char; i++; } continue; } // backslashes inside double quotes: only escape specific chars if (c == '\\' && is_double_quotes) { char next_char = buffer[i + 1]; if (next_char == '"' || next_char == '\\' || next_char == '$' || next_char == '\n') { current_word[word_len++] = next_char; i++; continue; } } // toggle quotes on and off if (c == '"' && !is_single_quotes) { is_double_quotes = !is_double_quotes; continue; } if (c == '\'' && !is_double_quotes) { is_single_quotes = !is_single_quotes; continue; } // space means split, ONLY if we are not inside quotes if (c == ' ' && !is_single_quotes && !is_double_quotes) { if (word_len > 0) { current_word[word_len] = '\0'; args[args_count++] = strdup(current_word); word_len = 0; } continue; } current_word[word_len++] = c; } if (word_len > 0) { current_word[word_len] = '\0'; args[args_count++] = strdup(current_word); } free(current_word); args[args_count] = NULL; return args; } 

parse_arguments() is our protagonist right now:

• We keep two booleans: is_single_quotes and is_double_quotes. Whenever we hit a quote character, we flip the matching boolean.
• If we see a space ' ', we check those two booleans. If both are false, we finish the current word, copy it into args using strdup(), and reset word_len to 0. If either boolean is true, we are inside quotes, so we just add the space to current_word like any normal letter.
• For backslashes \, POSIX rules are a little weird. Outside quotes, \ makes the very next character literal (echo hello\ world becomes hello world). Inside single quotes, backslashes do nothing at all. Inside double quotes, \ only escapes ", \, $, and \n. For anything else like echo "\a", the backslash stays in the string.

Note: Because we malloc the args array AND strdup() each individual string inside it, freeing args alone will leak memory. In free_arguments(), we have to loop through the array and free(args[i]) for every word before calling free(args).

Finding programs in $PATH (executable.c)

Alright, the user typed ls. We parsed it, checked our builtins, and ls is not in there. Now we have to find where the ls binary actually lives on the computer.

If you run echo $PATH in your terminal, you will see a list of folders separated by colons, like /usr/local/bin:/usr/bin:/bin. Our function find_executable() checks each of those folders one by one:

char *find_executable(const char *command) { char *path_env = getenv("PATH"); if (!path_env) return NULL; char *path_copy = strdup(path_env); if (!path_copy) return NULL; const char *delimiter = ":"; char *dir = strtok(path_copy, delimiter); char full_path[1024]; while (dir != NULL) { snprintf(full_path, sizeof(full_path), "%s/%s", dir, command); if (access(full_path, X_OK) == 0) { char *result = strdup(full_path); free(path_copy); return result; } dir = strtok(NULL, delimiter); } free(path_copy); return NULL; } 

Look closely at char *path_copy = strdup(path_env). Why did we make a copy of path_env before giving it to strtok()?

Because strtok() modifies the string you pass into it by replacing the : characters with \0 null bytes. And getenv("PATH") gives you a direct pointer to your program's actual environment variables! If you run strtok() directly on getenv("PATH"), you will corrupt your own $PATH variable after running one command. Making a copy with strdup() saves us from breaking our own environment.

Inside the loop, snprintf glues the folder path and the command name together (like /usr/bin/ls), and access(full_path, X_OK) checks if the file exists and has execute permissions. If it returns 0, we found our binary.

Running programs with fork and exec (process.c)

Now that we found /usr/bin/ls, how do we run it without closing our shell? This happens in process.c:

int execute_program(const char *exec_path, char *const argv[]) { pid_t pid = fork(); if (pid < 0) { perror("Error: fork() failed"); return -1; } if (pid == 0) { if (execv(exec_path, argv) == -1) { perror("Error: execv() failed"); exit(EXIT_FAILURE); } } if (pid > 0) { int status; waitpid(pid, &status, 0); } return 0; } 

Here is what is happening step by step:

• fork() clones the current process. After this line runs, there are literally two copies of our shell running at the same time. In the newly created child process, pid is 0. In the original parent process, pid is the process ID number of the child.
• In the child (pid == 0), we call execv(exec_path, argv). This tells the OS to wipe the child's memory and load /usr/bin/ls into it instead. If execv() works, it never returns because the child process has turned into ls. If it fails and returns -1, we call exit(EXIT_FAILURE) right away so we do not end up with two copies of our shell running at once.
• In the parent (pid > 0), we call waitpid(). This pauses the parent until the child finishes running ls. Without waitpid(), the parent would print the next $ prompt immediately while ls is still printing filenames, messing up the terminal output.

Why are fork() and execv() two separate steps instead of one run_program() function? Because that pause between fork() and execv() lets us change the child's input and output files before the new program starts. That is the exact trick we use for redirection and pipes.

Builtins (builtins.c)

As we discussed earlier, commands like cd, exit, and type have to run inside the shell process itself.

static char *builtins[] = {"type", "quit", "exit", "echo", "pwd", "cd"}; 

Here is how type works inside handle_builtin(). When you run type echo or type ls, it checks the builtins list first, and if it is not there, it uses our find_executable() function from earlier:

char *target = argv[1]; bool is_found = false; for (int i = 0; i < LENGTH(builtins); i++) { if (strcmp(target, builtins[i]) == 0) { fprintf(stdout, "%s is a shell builtin\n", target); is_found = true; break; } } if (!is_found) { char *exec_path = find_executable(target); if (exec_path != NULL) { fprintf(stdout, "%s is %s\n", target, exec_path); free(exec_path); is_found = true; } } if (!is_found) { fprintf(stdout, "%s: not found\n", target); } 

And here is cd. It just grabs the path (or looks up HOME if you typed cd ~ or just cd) and passes it to the chdir() system call:

if (argv[1] != NULL) { const char *path = argv[1]; if (strcmp(path, "~") == 0) { path = getenv("HOME"); if (path == NULL) { fprintf(stderr, "cd: HOME not set\n"); return 1; } } if (chdir(path) != 0) { fprintf(stdout, "cd: %s: No such file or directory\n", argv[1]); } } else { const char *home = getenv("HOME"); if (home != NULL) { chdir(home); } } 

Redirection: swapping file descriptors (redirection.c)

Ok, this is where things get really interesting. When you type ls > output.txt, ls has no clue that output.txt exists. ls just writes its output to File Descriptor 1 (STDOUT_FILENO), which normally goes to your terminal screen.

To make ls write into a file instead, we swap what File Descriptor 1 points to before running the command.

1. Save a backup of the real terminal output using dup(STDOUT_FILENO).
2. Open output.txt using open().
3. Use dup2(fd, STDOUT_FILENO) to point File Descriptor 1 at output.txt.
4. Close the extra file descriptor from step 2.

Here is the code in redirection.c:

void hijack_redirections(char **argv, int *stdout, int *stderr) { char *output_file = NULL; int cut_index = -1; for (int i = 0; argv[i] != NULL; i++) { // stdout overwrite: > or 1> if (strcmp(argv[i], ">") == 0 || strcmp(argv[i], "1>") == 0) { output_file = argv[i + 1]; if (cut_index == -1) cut_index = i; if (output_file != NULL) { if (*stdout == -1) *stdout = dup(STDOUT_FILENO); int fd = open(output_file, O_WRONLY | O_CREAT | O_TRUNC, 0644); dup2(fd, STDOUT_FILENO); close(fd); } } // 2>, >>, and 2>> follow the same pattern using STDERR_FILENO and O_APPEND // ... } // hide the > and filename from the command! if (cut_index != -1) { argv[cut_index] = NULL; } } 

And once the command finishes, restore_redirections() puts your terminal screen back to normal using the backup we saved:

void restore_redirections(int stdout, int stderr) { if (stdout != -1) { dup2(stdout, STDOUT_FILENO); close(stdout); } if (stderr != -1) { dup2(stderr, STDERR_FILENO); close(stderr); } } 

Two important details here:

• If you forget to restore stdout after the command runs, every future command you type will also write into output.txt, and your terminal will look completely frozen.
• Notice argv[cut_index] = NULL at the bottom of hijack_redirections(). If you run ls -la > output.txt, we have to hide the > and output.txt from ls. Otherwise, ls will think you are asking it to list a folder literally named >! Putting NULL at cut_index cuts the array short so ls only sees ["ls", "-la", NULL].

Pipes (pipeline.c)

Pipes are where I spent the most time debugging at night. When you run ls | grep txt, two programs run at the same time: ls writes data into a pipe, and grep reads from the other end of that pipe.

The pipe(pipefd) system call gives us two file descriptors in an array:

• pipefd[0] is the read end.
• pipefd[1] is the write end.

Let's look at execute_pipeline():

void execute_pipeline(char **argv) { // find the | symbol and split the array into left and right commands int pipe_index = 0; for (int i = 0; argv[i] != NULL; i++) { if (strcmp(argv[i], "|") ==...