DEV Community: Christian Parpart

A Look into a terminal emulator's text stack

Christian Parpart — Tue, 07 Sep 2021 19:02:43 +0000

TL;DR

I am going to describe how I implement rendering text in my terminal emulator so that we get programming ligatures, emoji, variation selectors, and yes, even ZWJ emoji.

Introduction

Text rendering in a virtual terminal emulator can be as simple as just iterating over each grid cell's character, mapping it to a Font's bitmap glyph, and rendering it to the target surface at the appropriate position. But it can also be as complex as a web browser's text stack[1] if one may want to do it right.

In contrast to web browsers (or word processors),
terminal emulators are still rendering text the way they did render text 50 years ago - plus some non-standard extensions that did arise over the decades.

Also, terminal screens weren't made with Unicode in mind, Unicode did not even exist back then, so there were a few workarounds and non-standardized ideas implemented in order to display complex Unicode text and symbols in terminals without a common formal ground that terminal application developers can rely on.

Text rendering in a terminal emulator puts some additional constraints on how to render, mostly because character placement is decided before text shaping is performed and must align to a fixed-size grid, which makes it almost impossible to properly render traditional Japanese text into the terminal, or Hewbrew right-to-left text (though, there is a handful of virtual terminal emulators that specialize
on the latter and an informal agreement on how to deal with wide characters on the terminal screen).

Not every character, or to be displayed symbol (such as Emoji) is as wide as exactly one grid cell's width, so additional measurements have to be taken into account for dealing with these characters as well.

Terminals and terminal applications until now did depend on a standard library API call wcwidth() that does ideally return the number of grid cells that an input character will occupy. This however became useless as soon as multi codepoint grapheme clusters have been introduced that still form a single user perceived character. wcswidh() should return the total number of grid cells of a sequence of characters, but these do not handle grapheme clusters nor variation selectors either.

Unicode - a very quick rundown in the context of terminals

Unicode is aiming to have one huge universal space where every imaginable "user perceived character" can be represented. A "user perceived character" is what the user that looks at that character thinks of as one unit. This is in direct contrast to what a character is in computer science. A "user perceived character" can be as simple as one single codepoint (32-bit value representing that
character) and as complex as an ordered sequence of 7 unbreakable codepoints to compose one single "user perceived character".

This places additional requirements to a virtual terminal emulator where each grid cell SHOULD contain exactly one "user perceived character" (also known as grapheme cluster), that is, an unbreakable codepoint sequence of one or more codepoints that must not be broken up into multiple grid cells before the actual text shaping or screen rendering has been performed.

Also, some grapheme clusters take up more than one grid cell in terms of display width, such as Emoji usually take two grid cells in width in order to merely match the Unicode (TR 51, section 2.2) specification's wording that the best practice to display Emoji is to render them in a square block to match the behavior of the old japanese phone companies that that first introduced Emoji.

Rendering text - a top level view

A terminal emulator's screen is divided into fixed width and (not necessarily equal) fixed height grid cells.
When rendering this grid, it is sufficient to iterate over each line and column and render each grid cell individually, at least when doing basic rendering.

Now, when non-trivial user perceived characters need to be supported, the rendering cannot just render each character individually, but must be first grouped into smaller chunks of text with common shared properties, across the grid cell boundaries.

Here we enter the world text shaping.

Text Shaping

Simply put, text shaping is the process of translating a sequence of codepoints into glyphs and their glyph positions. This differs from normal text processors and web browsers in a way because glyph placement in virtual terminal emulators are constrained.

When shaping text of a single grid line, the line is split into words, delimited by spaces, gaps and SGR attributes, that is, each word must represent the same SGR attributes for every "user perceived character" - for example the text's font style (such as bold or italic) or background color must be equal for each position in this sequence, from now on called "word").

The word can be used as a cacheable unit, in order to significantly speed up rendering for future renders.
The cache key is composed of the codepoint sequence of that word, as well as, the common shared SGR attributes.

This cacheable word is further segmented into sub runs by a series categories before text shaping can occur, that is, by Unicode script attribute (such as Latin or Hangul) and by symbol presentation (such as Emoji text presentation or Emoji presentation). This is important because one cannot just pass a string of text to the underlying text shaping engine with mixed properties, such as Hebrew text along with some Latin and Kanji or Emoji in between or a font style change for obvious reasons.
Each segment (usually called run) must be shaped individually with its own set of fallback fonts. Emoji are using a different font and font fallback list than regular text which uses a different font and font fallback list then bold, italic, or bold italic fonts.
Emoji also have two different presentation styles, the one that everybody expects and is named Emoji Emoji presentation (double-width colored Emoji) and the other one is named Emoji text presentation, which renders Emoji in non-colored text-alike pictograms.

The result of all sub runs is composing the sequence of glyph indices and glyph positions and is used as the cache value for the cacheable word to be passed to the next stage, the text renderer.

Text Rendering

The text renderer receives an already pre-shaped string of glyphs and glyph positions relative to screen coordinates of the first glyph to be rendered onto the screen.

In order to lower the pressure on the GPU and reduce synchronization times between CPU and GPU, all glyph bitmaps are stored into a texture atlas on the GPU, such that the text rendering (when everything has been already uploaded once) just needs to deal with indices to those glyph bitmaps into the appropriate texture atlas as well as screen coordinates where to render those glyphs on the target surface.

There is one texture atlas for gray scaled glyphs (this is usually standard text) as well as one texture atlas for colored glyphs (usually colored Emoji). Additionally there can be a third type of texture atlas for storing LCD anti-aliased bitmap glyphs.

Now, when rendering a string of glyphs and glyph positions, each glyph's texture atlas ID and atlas texture coordinate is appended into an atlas coordinate array along with each glyph's absolute screen coordinate and color attributes into a vertex buffer to be uploaded to the GPU.

When iterating over the whole screen buffer has finished, the atlas texture and vertex buffer are filled with all glyphs and related information that are required for rendering one frame. These buffers are then uploaded to the GPU to be drawn in a single GPU render command (such as glDrawArrays, or glDrawElements).

Other Terminal Emulator related Challenges

Most terminal applications use wcwidth() to detect the width of a potential "wide character". This is broken by design and a terminal emulator has to deal with such broken client applications. Some however use utf8proc's utf8proc_charwidth, another library to deal with Unicode,
and maybe some are using wcswidth(), which doesn't handle grapheme clusters nor variation selectors either.

The suggested way for future applications (emulator and client) would be to introduce feature detection and mode switching on how to process grapheme clusters and their width, if legacy apps are of concern.

Just looking at the algorithm, implementing grapheme cluster segmentation isn't too hard but in execution very expensive. Also grapheme cluster width computation is expensive. But luckily, in the context of terminal emulators, both can be optimized for the general case in terminal emulators, which is mostly US-ASCII, and yields almost no penalty with optimizations or a ~60% performance penalty when naively implemented.

Also, implementing proper text shaping into a fixed-grid terminal wasn't really the easiest when there is no other project or text to look at. I used "Blink's text stack" documentation and the Blink's source code (the Web renderer of Google Chrome) as bases and source of truth to understand this complex topic and then mapped my findings to the terminal world.

Since text shaping IS expensive, this cannot be done without caching without severely hurting user experience.

After investigating into the above optimization possibilities however, I do not see why a terminal emulator should not do provide support for complex Unicode, as the performance I have achieved so far is above average at least, and therefore should be sufficient for everyday use.

Bidirectional text was not addressed in this document nor in the implementation in the Contour terminal yet, as this imposes a new set of challenges that have to be dealt with separately. If that is of interest, there are a few terminals (such as mlterm) that have put great effort in getting such scripts into the terminal.
Hopefully this will be eventually added (or contributed) to my project at any time in the future, too and if so, I'll update this document accordingly.

Conclusion

If one went through all the pain on how Unicode, text segmentation, and text shaping works, you will be rewarded with a terminal emulator that is capable of rendering complex Unicode. At least as much as most of us desire - being able to use (power user/) programming ligatures and composed Emoji.

Some terminal emulators do partly support ligatures or rendering trivial single codepoint Emoji or a few of the composed Emoji codepoint sequences, but sadly, most seem to get Emoji wrong. While this is a great start, I'd say we can deliver better and more.

Final notes

I'd like to see the whole virtual terminal emulator world to join forces and agree on how to properly deal with complex text in a somewhat future-proof way.

And while we would be in such an ideal world, we could even throw away all the other legacies that are inevitably inherited from the ancient VT standards that are partly even older than I am. What would we be without dreams. ;-)

Some other terminal emulator developers and I have
started to address at least some of the many Unicode problems that are up until now undefined behavior by creating a formal specification on how a terminal emulator should behave in a backward and forward compatible way so that app developers and users will benefit.

References

Blink's text stack
UTS 11 - character width
UTS 24 - script property
UTS 29 - text segmentation (grapheme cluster, word boundary)
UTS 51 - Emoji

Builtin Solidity Language Server

Christian Parpart — Tue, 06 Apr 2021 12:23:57 +0000

Hey fellows!

At Ethereum I am currently part time developing a language server mode as part of the solidity compiler.
While this is still in early development stage, the 3 features I am currently concentrating on are already working:

✅ compiler diagnostics
✅ goto definition (also works across files)
✅ find all references (does not yet work across files)
✅ semantic highlighting

While this is not much, it is a good starting point to not reinvent the wheel but reuse the existing compiler infrastructure to implement a language server on top of it.

For those of you being curious, I would now like to show you how to use it.

Vim/NeoVim with coc.nvim

This is actually what I am using and your coc-settings.json would be extended like this:

{
    // ...
    "languageserver": {
        // ...
        "solidity": {
            "command": "/path/to/solc",
            "args": [ "--lsp" ],
            "trace.server": "verbose",
            "rootPatterns": [".git/"],
            "filetypes": ["solidity"]
        }
    }
}

So assuming you already have https://github.com/neoclide/coc.nvim installed and working that is all you need to add to it. Now of course you need to have a solc binary available that does indeed support the new command line parameter --lsp.

Download a `solc` with LSP support

I've uploaded a prototype version here:

https://github.com/christianparpart/solidity-language-client-vscode/releases/download/TP_1/solc-linux64

Visual Studio Code!

When you are on a 64bit Linux, you are lucky. The VScode extension does provide a builtin LSP server that you can use. Other platforms (such as OS/X or Windows) I will provide some precompiled LSP-compatible solc executables and even some syntax highlighting, but give me some time on that.

So for VScode, I have just written the glue code extension last week and you can download here:

https://github.com/christianparpart/solidity-language-client-vscode/releases/tag/TP_1

Fetch the .vsix file and then run the following command in your virtual terminal emulator:

code --install-extension solidity-0.0.1.vsix

Start (or restart) your VScode and you should be all set. If you now want to use your own solc binary (which should more frequently update), then you can configure that in your local settings.json as follows:

{
    // ...
    "Solidity.solc.path": "/path/to/solc", // or null if builtin should be used
    "Solidity.trace.server": "messages"
}

Bare in mind, if you already had another solidity related extension loaded in VScode you may want to see if you should disable that to not cause any conflicts or confusions. :)

Other Editors

The solc's --lsp mode does use standard Input/Output streams (stdio) for communicating with the client. This is fairly standard. In QtCreator for example you can just go to the IDE's config page and put in the path to the solc executable, and configure on what file extensions to use that LSP for and you're done with it. I want to try out Neovim's native language client interface at some point, but I am certain it'll just work like any other in this sense. :-)

Future Outlook

🔨 improving goto definition / find all references / semantic highlighting
🔨 show type signature on hover
🔨 show natspec documentation on hover
🔨 show IR / EVM byte code / gas costs on hover
🔨 code completion at least on .
🔨 publish the VScode extension to the VScode marketplace
🔨 configurable list of warning/error codes to suppress?
🔨 and maybe we can integrate that in Remix, too?

Final Words

I really hope you enjoy doing some early testing on this Solidity language server. Please keep in mind that the features are indeed minimal but slowly I would like to improve on that. Tell me what you feel about it, what you would like to have the most and/or what should be different.

If you are interested in how to build Solidity compiler (including the language server) from source, please drop me a comment and I can do that in a separate post then.

Have fun and take care!
Christian.

Functional C++

Christian Parpart — Thu, 03 Oct 2019 12:30:17 +0000

Functional Programming in C++

When I learned about C++, around 28 years ago, it's been just labeled as an object oriented programming language, or C with classes and a stronger type system than C.

Now, with the rise of functional programming outside universities since quite some years, the hype has also landed in C++, starting at C++11 and the introduction of lambdas.

One may very well code purely procedural in C++, or purely object oriented. It's just a matter of style. C++ is a very complex programming language, sadly, but that doesn't mean you have to use it all, just because your tool provides this - a common beginners mistake.

However, there are some advantages in using a functional programming style, and what I like most about, are algebraic data types (ADT), that are most sexy declared in first-class functional programming languages (such as F# and alike), but also possible in C++ (one may want to read a future post on this subject right here).

So what's so bad about the imperative style?

for (int i = 0; i < array.size(); i++)
    doSomething(array[i]);

Having a look at this example, the first thing one may note is the index variable being of type int. However, the comparison operator < on its right side has an operand of type size_t (unsigned, most importantly). You can't always compare this, and nowadays some compilers warn (some not), which can render into a bug. Not to mention, that sometimes the loop body may become bigger, and then it's not entirely clear that the index variable i is only and really only modified in the loop header, which isn't guaranteed because it is a mutable variable.
Also, i is only used for referencing the elemeent, nothing else, so we would ideally not reserve another variable just for this - can we avoid that?

So in functional programming style, you'd write that as:

// using namespace std;
// using std::placeholders::_1;

for_each(begin(array), end(array), bind(&doSomething, _1));

This limits the class of errors and uncertainties I mentioned above.

So how would one now sum up an array or list of elements?

auto constexpr numbers = array{1, 2, 3, 4, 5};
auto const y = accumulate(begin(numbers), end(numbers), 0,
                          [](auto a, auto b) { return a + b; });
cout << "sum of elements: " << y << endl;

The function accumulate is from the numeric header, check out cppreference.org.

So what is so good about it?

It not potentially reduces the lines of code by reusing existing STL functions (such as std::accumulate or std::for_each) but also eliminates mutability, which greatly helps code maintenance due to lesser questions to care about when reading foreign code, but also helps the compiler generating more efficient code.

By using algorithms that have been implemented by others, and used by many, such as the C++ standard library, the chances that those code parts are buggier is also lesser likely than doing it all by your own.

This way of thinking, even in C++, helps you writing more clean code.

Conclusion

In the end, it's always a personal matter of taste. Do you like imperative, procedural, object oriented, or functional programming paradigms? It's at least good to know, that with modern C++ (my code snippets require C++17 above) you have the choice.

A Sneak Peak

I'd also like to talk about algebraic data types in C++17, all the good parts that make C++17 fantastic, and all the amazing improvements about to come in C++20.

DEV Community: Christian Parpart

A Look into a terminal emulator's text stack

TL;DR

Introduction

Unicode - a very quick rundown in the context of terminals

Rendering text - a top level view

Text Shaping

Text Rendering

Other Terminal Emulator related Challenges

Conclusion

Final notes

References

Builtin Solidity Language Server

Vim/NeoVim with coc.nvim

Download a solc with LSP support

Visual Studio Code!

Other Editors

Future Outlook

Final Words

Functional C++

Functional Programming in C++

Conclusion

A Sneak Peak

Download a `solc` with LSP support