This is the first of two articles about Linux terminals. By the end of the two articles, we should be able to:
describe the main components in th...
For further actions, you may consider blocking this person and/or reporting abuse
In the old days it was too expensive for every character to be read and interpreted by the underlying program (i.e. a shell) because the UNIX RAM was small (at most 64Kwords) and with 20 people typing 60wpm the system would need 100 program context switches + swaps from disk per second. In this situation 100% of cpu time would be spent handling keypresses - no time would be left over to run programs!
The line discipline was actually a programmable middleman kernel module that could buffer from all 20 ttys until each user had successfully finished a command, at which point the middleman passed the text on to the underlying program, i.e the shell or a line-editor like ed / xed. Only one line discipline needed to be in memory for all 20 users. If command entry time is 30 secs you would now have a program wakeup every 1.5 seconds - much better than 100 times per second!
The line discipline is sort of like emacs, which has a function table of size=127 (one for each key) and does a table lookup and invokes the associated function the most common being "buffer-me" but important ones being "newline" (which sends the buffered text to the shell), "erase", "word-erase", "line-erase" (used to be bound to "@" not ^U and would erase your line buffer, displaying @ CR LF and leaving the discarded text on the screen, one line above.) Only the truly important keypresses like , ^C ^Z ^S ^P ^O ^Z ^Y required interaction with the underlying program, i.e. a shell or editor or command pipeline.
You could put the terminal in "raw mode" which is also known as "no line discipline" and the function table would be filled with 127 copies of "send-char-to-program" function, immediately producing a task wakeup. That began happening for interactive games such as rogue and later, editors like vi / vim / emacs which were viable on faster CPUs (like the 3 mips PDP-11/70). Most people are too young to know that the vi / vim editor at first used line-discipline (=cooked, the opposite of raw) mode. When you inserted characters into the middle of a line (by typing e.g. "ixyzpdq") the screen would be messed up and xyzpdq would overwrite later characters and the screen wouldn't get fixed until you hit escape. This fixup policy made vi / vim efficient on slower machines. It runs in full raw mode today but originally it used the line discipline to avoid swaps and process wakeups.
Today computers are 3000x faster with 1,000,000x more memory and have only 1 user, so the feature is an unnecessary artifact of history.
Hi Donald! Thank you for sharing this!
When I started reading about the line discipline I was hoping to find its history. I intuitively thought the reasons were hardware limitations but I did not find much on the topic. Thanks to you I now have a clear picture of the context and the constraints that led to the line discipline.
Since I wrote the article, I do not think I'll offend anyone by saying that your comment is (even?) more valuable then the article itself - I know it is for me. In the second part of the series I briefly describe the line discipline, do you think I could include your comment in the article? Of course, I'll give full credit to you.
Thank you :)
No worries Nicola you helped me immensely with your kubernetes internals article, thanks! I started with UNIX v7 in 1977 and worked with or befriended some of the original UNIX inventors and developers - that wealth can now be shared!
Thanks!
Thanks Donald!
( I logged in my dev account that not in use for a long time just for click a "like" on this excellent comment.
Hi there,
I have a question kinda related to this topic :)
How can I "mirror" everything that's happening on one terminal to another?
Backstory:
Sometime ago, when updating my rPI over ssh (no keyboard/mouse directly connected, only things connected are display over mHDMI, and power plug) I wanted to see on /dev/tty1 (the one that is displayed by physical monitor) whats happening, ie progress of update.
After some adjusting I managed to do this (Arch) like this:
pacman -Syyu | tee /dev/tty1It worked, but...
How to do that from outside, as an owner of the system.
Fictional case:
I have found out that there is suspicious ssh connection made to my PC, happening on /dev/pts/1. I, as a root, want to hook into it, to see whats happening in it, without disturbing any communication in it.
I've tried this:
cat /dev/pts/1 > /dev/tty1, but that is not exactly working as I thought (characters are lost in pts1, and system lags hard)Any idea? :)
Hi,
I think you could use tmux. When you ssh to the machine, create a tmux session.
Then from the second PC, ssh to the machine and attach to the tmux session you have created.
Of course, this works if you ssh both times with the same user.
A more hackish way would be to redirect standard out and error of bash also to a file.
Then from the second terminal you could tail the fail:
Terminal 1
Terminal 2
I am not sure how reliable it is, bit it seems to work:
I would stick with tmux though :)
Your solution of redirecting the tty does not work because /dev/tty1 is a special file:
The c at the beginning means is a character device. Although these files have the same primitives of regular files (open, read, write, etc.), they are not a representation of data on the disk - that is, they might be weird.
Learn something new every day ;)
Thanks for your answer, but it's still not exactly what drills my mind.
This requires the user (first terminal) to do something special before hand, to allow root from second terminal to "spy" on it.
There has to be another way.
Hi, sure, no problem.
Not sure what you mean by something special.
You can set up tmux to start automatically when ssh ing to the box, for example see this stack overflow answer: stackoverflow.com/a/40192494
The fist user does need to do anything :)
That's one way to tell ssh what to do when you log into the box. I believe you could also change the ssh agent config to achieve the same goal.
Will look into it soon'ish,
Thanks! :)
It also looks like you can spoof the tty output by using eBPF, but it basically requires running the program in the kernel. Again, it's not as easy as using tmux XD
github.com/iovisor/bcc/blob/master...
Hi,
Please, help me to clarify sequence of events:
question 1: why does xterm need echo from line discipline if in step 1 it already writes to pty master?
question 2: what echo operation really does? If it simply writes to pty master, why no recursion occur (pty master <-> line discipline)?
Back in the day, terminal were essentially just a peripheral. They were only able to send and receive data to the mainframe. Those terminals were not smart enough (not enough memory or compute) to be able to process the data typed by the user. The mainframe had to do everything for them (including echoing back characters).
A lot changed since then. We have all have plenty of computing power in our laptops and do not need to connect to mainframes anymore. What did not change is the architecture. When laptop started to become common, smart folks decided to reuse the same architecture by replicating in software (in the OS) what went on between terminals and mainframes.
I'm also confused about both questions.
"The std input, output and error of the bash will be set to be the pty slave."
What do you mean standard streams of bash will be set to be the pty slave? How are they set, is pty slave just another program and we use pipes to pass the data from pty slave to bash?
Hi! I'll try to unpack your question:
The std input, output and error of bash will be set to be the pty slave means:
the fd0, fd1 and fd2 of bash all have the same value, which is the file descriptor of the pty slave device file.
Hi Nicola,
First of all, let me thank you for your article, its was very useful and it guided me into other topics of study that clarified many doubts I had. Keep the good teaching!
Secondly, I still have two doubts.
You explained that when Bash opens a process (command), that process also sets its files descriptors to the same PTY slave file as Bash, and from there the TTY driver reads the output and sends it back to the terminal through the PTY Master.
I have two questions about this mechanism:
If a command (process) initiated by Bash, has its output set to the same PTY Slave as Bash, wouldn't Bash read the command's output as an input?
My other doubt is about a command whose output is piped as the input of a second command?
I'm guessing that in such cases, the stdout of the first command is set to a different intermediate file, which would be read as the stdin of the second command, then the second command writes its output to the same PTY slave file used by Bash. Am I close?
Hello Maximiliano, thanks!
Regarding the first question - no the process standard input, out and error will be connected to the PTY slave. This seems to be one of the most tricky things to get, my guess is because the concept of a kernel module (like the pty module) might be confusing. I was thinking to write a follow up article on that.
About the second question - I do not think Bash needs to create temp files. I have always assumed that Bash duplicates the file descriptor so that the output of one program can be passed as standard input to next one in the pipe. This seems to be confirmed by the Bash source code: github.com/bminor/bash/blob/master...
That being said, pipes are a different beast. I do have an article which touches on that, but I haven't dived deep on they work behind the scene.
Oh, this is brilliant! A great explanation - well done.
(very much looking forward to the next bit)
I am glad you liked it.
Thank you David :)
Great explanation! Thanks for sharing~
Thanks!
Hi, Nicola
Amazing article! However I have one question regarding the "What's a shell' section:
You wrote:
However, in the 2nd part of the article where you're describing the "Line Discipline", you've said that when we type CTRL +C or CTRL +Z, the "Line Discipline" sees that and sends proper signal to the process.
So who is responsible for this job control? Is it a shell or line discipline?
I was searching for something like this which could help differentiate the gui terminals and the pure cli terminals.
Thanks for your great article!
I have a question about your article.
You said PTY is "Teletype emulated by a computer program running in the user land".
However, other airticle says "the pseudoterminal lives in the OS kernel.", so PTY doesn't seem to run in user land.
ishuah.com/2021/03/10/build-a-term...
Is PTY the terminal emurator which runs in user land? Or does PTY mean PTY master and slave, which are running in the OS kernel?
I am really confused with the difference between pty and tty....