Clipboards, Terminals, and Linux

I've recently switched to Neovim, and with it begun using the terminal mouse support. But, this has the side-effect that I can't just click-and-drag to select text in the terminal anymore -- Neovim controls that as well.

Which leads me to clipboards. Linux has two of them! Adding to the interest, I typically use Neovim remotely, via an SSH connection to a Tmux session. And on my Linux system, I use urxvt as my terminal program. All of these are very UNIX-y tools, and somehow they all need to play nicely together.

With this sort of challenge, typically the best approach is to learn as much as possible about the different parts, do some experimentation to see how they work in practice, and then try to stitch together a working solution. Sometimes Reddit posts or other information you come across can be useful, but as often as not it's wrong or outdated, so using it without understanding it might just make things more confusing.

In the interest of sharing, here's some of what I've learned.

Security

A brief word about security: clipboards often have passwords in them, especially if you use an external password manager like LastPass. So it's worth thinking carefully about what un-trusted software might be able to read from your clipboard.

Depending on what you do with the information in this post, you might enable any binary you run to access your clipboard from any host you SSH to. Maybe that's OK in your situation, but it is worth thinking about.

X Clipboards

X Windows has lots of "selections", but the two we'll be concerned with are PRIMARY and CLIPBOARD. Most times you select some text, that's automatically moved to the PRIMARY selection. When you specifically request copying a value (i.e., control-C), the value is put into the CLIPBOARD selection. Note that this means selecting some text and pressing Control-C will usually put that text in both selections!

The common xclip utility can be used to get or set clipboards. And, there's a tiny little utility script called clipnotify that I found really useful for debugging this. I cloned and built it, then ran it in a loop:

./clipnotify -l | while read x; do
  clear
  for s in primary clipboard; do
    echo == $s
    xclip -o -selection $s
    echo
  done
done

I ran this in a dedicated terminal so I could easily see what was in the X selections at all times.

OSC-52

OSC-52 is how information about clipboards is transmitted over a terminal. This is an ANSI terminal escape, so a bunch of bytes beginning with ESC that mean something to a terminal emulator. There are a bunch of OSC codes, but 52 is the one that handles clipboards. Documentation is a little hard to find, so I'll summarize how it works here.

Set Clipboard

Sending the sequence <ESC>]52;<board>;<content><BEL> to the terminal sets clipboard <board> to <content>. Here <ESC> is 0x1b and <BEL> is 0x07, sometimes written \e and \a, respectively. The <content> is base64-encoded. The <board> can be c (clipboard) or p (primary) on Linux, but things like MacOS only support c.

You can experiment with this with printf in the shell:

printf $'\e]52;c;%s\a' "$(echo foobar | base64)"

This will put "foobar" in your clipboard.

Get Clipboard

Sending the sequence <ESC>52;<board>;?<BEL> to the terminal will "query" the value of <board>. The terminal responds with a sequence like the "set clipboard' above, containing the content of the requested board.

⸩ printf $'\e]52;p;?\a' ; sleep 1; echo
^[]52;;Zm9vYmFyCg==^G

This makes a bit of a mess of the terminal, but you can see the response, and that base64 decodes to the value in the PRIMARY selection.

Advertising Support for OSC-52

Terminals advertise their support for functionality via some really arcane symbols. The particular piece that indicates clipboards are supported is Ms=\E]52%p1%s;%p2%s\007. This is typically found in the "terminfo" database, keyed by the terminal name, which an application finds in $TERM:

⸩ echo $TERM
rxvt-unicode-256color

Bracketed Paste Mode

While in principle "paste" into a terminal application can just pretend the pasted data was typed on a keyboard, in practice this can go very badly. For example, if pasting into Vim when it's not in insert mode, the result is usually a mess. Even in insert mode, if autoindent is enabled when you are pasting code, the indentation is not what's expected. Vim has a paste option for this, but in fact there's a better way: bracketed paste mode.

An application using the terminal "sets" this mode, after which the terminal will prefix pasted content with <ESC>[200~ and suffix it with <ESC>]200~.

You can see this with a quick little Python program:

import sys

sys.stdout.write('\x1b[?2004h') # set bracketed-paste mode
sys.stdout.flush()
try:
    while True:
        c = sys.stdin.read(1)
        if c:
            print(repr(c))
        else:
            break
finally:
    sys.stdout.write('\x1b[?2004l') # reset the mode

urxvt

The urxvt terminal emulator doesn't support OSC-52 out of the box. Honestly, it does very little out of the box! But there's a tiny script, 52-osc that adds this support. This is actually nice, because you can see what it does:

sub on_osc_seq {
    my ($term, $op, $args) = @_;
    return () unless $op eq 52;

    my ($clip, $data) = split ';', $args, 2;
    if ($data eq '?') {
        my $data_free = $term->selection();
        Encode::_utf8_off($data_free); # XXX
        $term->tt_write("\e]52;$clip;".encode_base64($data_free, '')."\a");
    }
    else {
        my $data_decoded = decode_base64($data);
        Encode::_utf8_on($data_decoded); # XXX
        $term->selection($data_decoded, $clip =~ /c/);
        $term->selection_grab(urxvt::CurrentTime, $clip =~ /c/);
    }

    ()
}

Breaking that down, it's hooking into the OSC sequences, and specifically number 52. If it gets a query ('?'), it gets the current selection from $term->selection() and sends that back to the app running in the terminal in an OSC-52 escape, using $term->tt_write(). Otherwise, it decodes the data and sets the X selection with $term->selection(data, clipboard). The /c/ is a regular expression matching c.

Putting the printf and clipnotify bits from above together shows that when <board> is c then this plugin updates the CLIPBOARD selection, otherwise PRIMARY. This still works over an SSH connection to a remote host.

Unfortunately, because this is accomplished with a plugin, the terminal info for urxvt doesn't advertise this support.

Copy and Paste

Urxvt has a built-in plugin, selection-to-clipboard, to copy every selection to the clipboard. In fact, it populates both the PRIMARY and CLIPBOARD selections.

Right-click will paste from the PRIMARY selection. With the following in .Xresources, shift-ctrl-V will paste from the CLIPBOARD selection.

Rxvt.keysym.Shift-Control-V: eval:paste_clipboard

Either paste option supports bracketed-paste mode.

tmux

I typically run tmux on remote systems, so that I can leave everything running while my laptop is asleep, or if I lose my network connection. Tmux is basically a terminal emulator that runs in a terminal: you can run other things in a tmux window, and tmux lets you switch between those windows, show them on different systems, etc. But, that means that tmux is intercepting terminal escape codes, whether they're for cursor positioning or clipboard management. It is then using (possibly different) escape codes to draw the tmux UI on your terminal.

In its default settings, the printf above won't do anything in tmux when running inside urxvt, because urxvt doesn't advertise support for it. That can be fixed in .tmux.conf:

set-option -ga terminal-override ',rxvt-uni*:XT:Ms=\E]52;%p1%s;%p2%s\007'

Note that this "fix" only works for things running in tmux. I do everything in tmux (even locally), so it's fine for me.

Buffers

Tmux has a concept of buffers which are named bits of text that can be injected into an application as if they were keyboard input. Tmux also has a "copy mode" where keyboard navigation can be used to select text that will be put into a buffer. The concept is pretty general, although I don't know what would be built from it.

Tmux is designed around the idea that you would only use buffers for copy/paste. That isn't especially practical, since for example web browsers tend not to be terminal applications!

Clipboard Integration

Tmux has a wiki page about clipboards. The main integration is set-clipboard. If this is "external", then tmux will issue an OSC-52 sequence to update the system clipboard whenever a buffer is set. If this is on, then tmux will additionally accept OSC-52 sequences from applications running inside it.

set-option -g set-clipboard on

With both of these options set, repeating the same experiment within a tmux session reveals that any OSC-52 sequence now updates only the PRIMARY selection. That is, the implementation of set-clipboard only updates the PRIMARY selection.

Furthermore, running the get-clipboard printf from above confirms that tmux only returns its own paste buffer -- it never queries the external terminal emulator for the value of the system clipboard.

Even tmux's support for external commands only supports copy operations. There's no way to feed data on the system clipboard into tmux.

Bracketed Paste

When an application in a tmux pane has bracketed paste enabled, tmux will enable it in the parent terminal. It will also "pass through" the brackets from that parent terminal when a paste is performed. Basically, it just works.

Tmux also has a -p option to its paste-buffer command to bracket the buffer contents.

Neovim

Neovim's :help clipboard lays out the situation pretty clearly, and it works quite nicely out of the box. It uses xsel to interact directly with X selections when $DISPLAY is set, and it uses tmux commands when running in tmux. Its checkhealth command makes it easy to see what it's chosen:

• vim run locally, outside of tmux: xsel
• vim run locally, in tmux: xsel
• vim run via SSH, outside of tmux: no support
• vim run via SSH, in tmux: tmux

Here, I'm not forwarding X11 via SSH. If I do so, and install xsel remotely, then Neovim will use xsel in all four of the above situations.

Tmux does enable bracketed paste mode.

Summary

All in all, this is not pretty!

Two clipboards (selections), handled slightly differently by each application -- so maybe copy in one app doesn't use the same selection that paste does in another. That's certainly more than ctrl-c/ctrl-v muscle memory can deal with, especially for those who also use more .. ahem .. user-friendly systems on a daily basis.

Most people think of having one clipboard.

Tmux is a pretty substantial impediment here, too, basically rendering tmux's buffers useless, since they can't mirror the "one clipboard".

At any rate, this continues my habit of writing posts as a way to learn about a new topic. I hope this information is useful to others as well!

Comments

[djmitche]

Slight amendment here: tmux can query the enclosing terminal via OSC-52 query, using the -l argument to refresh-client.