DEV Community: Anatoly Scherbakov

documented: make docstrings in your exceptions work

Anatoly Scherbakov — Fri, 06 Oct 2023 17:49:17 +0000

During my time in writing Python code, I have gone through a certain evolution; one of the aspects of it is how I create and manage exceptions.

Level 0

raise ValueError("I'm sorry Dave. I'm afraid I can't do that.")

Here, to signify an error, we just use a standard built-in error class (such as Exception or ValueError). The exception message will be printed to the log and/or provided to the user.

If we would like to catch the exception and to base our future behavior on the particular error that we had caught then we will be in trouble; likely we'll have to manually parse the exception message. That's dirty.

Level 1

class PodBayDoorsStillClosed(ValueError):
    pass

# …

raise PodBayDoorsStillClosed("I'm sorry Dave. I'm afraid I can't do that.")

Now, I can catch PodBayDoorsStillClosed in some other place of the application, and modify my behavior based on that. For instance, I can

try:
    ...
except PodBayDoorsStillClosed:
    retry()

It is unlikely that HAL complies but at least the code will be arguably a bit more expressive.

Level 2

At some point, I realize that the text of HAL's message belongs to the class of the exception; there is no reason to keep them apart.

class PodBayDoorsStillClosed(ValueError):
    def __str__(self):
        return "I'm sorry Dave. I'm afraid I can't do that."

# …

raise PodBayDoorsStillClosed()

This is better, but… is it cool to write __str__ all the time, for each exception message, separately? — No, decidedly not.

Level ∞

Some time ago I wrote a very small Python library by the name of documented. Which makes it look like this:

@dataclass
class PodBayDoorsStillClosed(DocumentedError):
    """
    I’m sorry, {self.user_name}.

    I’m afraid I can’t do that.
    """

    user_name: str

# …

raise PodBayDoorsStillClosed('Dave')

That will print:

PodBayDoorsStillClosed: I’m sorry, Dave.

I’m afraid I can’t do that.

I am now using it to format both internal and external exceptions for web and console applications. Flight condition nominal, so far. Perhaps this library will be useful for you too; you can easily grab it via

pip install documented

Reflections on Literate Programming

Anatoly Scherbakov — Fri, 31 Jul 2020 16:25:02 +0000

Introduction

Documenting and commenting your program is an activity many programmers despise. Documentation either is never written or gets abandoned and rots. Documentation becomes irrelevant and depressing.

However, a program which is totally undocumented is hard to understand and maintain, up to the point of inability to support and use it. Which is a risk.

Literate Programming

Literate Programming (LP) as a term was coined by Donald E. Knuth in his article from 1984. Quote from https://literateprogramming.com goes:

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

Rather than embedding documentation comments into code, a literate programmer embeds pieces of code into the essay (article, book) which tells the reader about what these pieces of code are doing, and why.

Example

Knuth has been using literate programming to write TeX and many other programs over the years. Here is one of them -- a port of an old Adventure game to Knuth's literate programming tool called CWEB. Check out this PDF.

This document is one of the two outputs of the CWEB system -- this is a reader friendly, readable book (107 pages long) which fully describes the game, including its annotated source code.

The other output would be the code itself in C language, ready for compilation.

When writing the literate source, you can interleave and link the blocks of code and the blocks of text. Thus, the document can be written in a way that's the easiest for a human to read, but the code will be rearranged in the specific order which the compiler prefers.

Reasoning

In a talk named Literate Programming in the Large Timothy Daly, long term developer of Axiom computer algebra system, urges us - fellow developers - to change the situation, suggesting that LP is the only (and best) way to build maintainable software systems. Why so?

Change of perspective. From LP point of view, the essence of a computer program should be communicating its purpose and logic to a human reader. Only incidentally it so happens that this program also can be executed by a computer.

The Why. The human reader wants to understand why the program is doing what it does. This ensures that the reader will be able to change the program with full understanding of the implications.

This means the reader will be able to maintain the program even if the original author is no longer available.

Adoption

Literate Programming - Tools page provides a few of the available tools which support the Literate Programming technique. But the overall adoption in the industry seems, at least to me, rather limited.

Knuth himself states that, since the evolvement of the LP technique, he found himself more productive and the quality of his programs improved. Indeed, TeX is probably one of the best-quality pieces of software of all time. The question is though: can the positive impact of LP that Knuth observed apply to majority of programs and teams?

We don't have statistical evidence to support or deny that; thus, I am going to list a few of very opinionated and subjective points on the matter.

Expressivity of programming language

Daly is providing an example from the history of Axiom. He was once looking at a piece of its code and he did not understand why that piece was there.

The code did obvious things with some bytes in memory, and Daly knew exactly what the code was doing (he was likely the author) - but he couldn't ascertain why it was doing that. That's why an essay-like description would have helped.

But, I would say the code that controls logic of the application should not manipulate any bytes in memory directly; it should be written in a much more abstract way. Like this, for example:

Warehouse().find_package(package_id).location

TeX was written originally in Pascal and then rewritten in C. Both are quite ascetic by modern standards - in terms of expressivity, available data structures, type systems, libraries, and possible syntax extensions.

lambda anonymous functions make the code more concise where previously you would have had to create a separate named function;
map filter reduce and friends alleviate cycles;
Garbage collector or boundary checker permits to skip all the details of memory management;
Type system helps to express concepts, not byte sequences, in code.

All of this reduces the need for a lot of extra explanation. In the example of Adventure game above, there would be no need to say

#include <stdio.h> /∗ basic input/output routines: fgets , printf ∗/

instead (this is pseudocode, I have no idea whether Rust has an stdio crate):

use stdio::{fgets, printf};

Need for explanations alleviated. Code speaks for itself.

Power of Notation

In the Middle Ages, European mathematicians had to communicate their ideas in natural language. Proofs were incredibly hard to understand and reason about when written in lengthy prose. Or, sometimes, in rhymes:

When the cube and its things near
Add to a new number, discrete
Determine two new numbers different
By that one; this feat
Will be kept as a rule
Their product always equal, the same
To the cube of a third
Of the number of things named.
Then, generally speaking
The remaining amount
Of the cube roots subtracted
Will be our desired count.

(Source: Tartaglia's Poem - ProofWiki)

This is the description of the formula for roots of cubic equation by Niccolo Tartaglia.

Invention of algrebraic notation was a great step forward which enabled us to reason about much more complex concepts - and still not blow our heads.

Today, mathematics is using a lot of special symbolic languages to manage extremely complex and abstract concepts. They are expected to be concise, strict, unambiguous, and versatile.

Computer programming language is actually even stricter than the most abstract mathematical paper. Vladimir Arnold, a renowned mathematician, says in one of his lectures that Nikola Burbaki in their series of books on mathematics foundations aspire to be as strict as possible in the first books, but their grip kind of losens from one book to the next. That will not work with computers, though.

Alternatives or complements to LP

Being on the side of Literate Programming, it looks like that we have no hope for the programming language to convey the concepts of the domain. Its expressiveness is poor, and we resort to the natural language with all of its limits - ambiguousness, verbosity, uncertain and vague nature - to do the job.

Maybe we should do something different?

Yes, I think we should.

Take inspiration from mathematics

Programming is, essentially, a specialized branch of mathematics (at least that's what Alexander Stepanov, author of C++ STL, says in his talk at Yandex). Mathematics is the art of abstraction in its purest form. We struggle with abstraction in programming. That's why I almost irrationally believe that mathematics should have methods and ways to help us make our programs better.

Anyway, if you are familiar with

mathematical logic,
graph theory,
finite automata theory,
category theory,
type theory

...and/or any special areas which may be relevant to your sphere of interests - this won't hurt you.

Functional programming ideas which we've briefly touched above are now practically mainstream. They were completely derived from mathematics and only in time found their way to the minds of the majority of software developers.

Domain Specific Languages

DSL = Domain Specific Language is a formal language designed specifically for certain domain: computer graphics, warehouse management, flight path computation. SQL is one good example of a DSL. DSLs can be internal or external.

JetBrains coined the term Language Oriented Programming to describe a paradigm where you build a special language for every special job.

Alan Kay was working for a while on STEPS operating system which was built upon dozens of languages, each designed for its particular purpose.

IDEs

Among the things Daly mentions as benefits of Literate Programming are:

Indexing and reverse links from a piece of code to other pieces which use it
Tables of contents
References

However, these things are not surprising for anyone who has been using any modern IDE. To the contrast: using a literate programming technique will likely make it impossible for us to use a normal IDE. We're very much accustomed to refactorings, search in code, snippets, integration with linters etc, - and that will more likely obscure our development experience rather than improve it.

Linting

With linters and static code analysis, we can find particular errors in our code which are easy to recognize. Daly provides an example. Someone introduces a number in code which is 153. What does that number mean? Why 153 and not 654?

Daly continues that, because it happened in a literate program, the author was able to write something like:

I chose this number for no particular reason. Choice of the number does not change anything.

If a linter would find such a number it would say this is a magic constant which should be given an explanatory name.

But the linter can't do anything if you forget to describe something in the literate source: this is natural language, no hope.

Notebooks

Well, when I was saying that LP did not get any traction above I was not entirely honest. One particular form of Literate Programming is quite widespread. It is called Notebooks.

Under a notebook, we mean a document which consists of a series of blocks; every block can be either code or free-form text; and people use these to perform data analysis, conduct experiments, write one-off scripts, or do some Reproducible Research. Examples:

Conclusion

Should you use Literate Programming in your project or not? It depends.

After watching Daly's emotional talk, I was for a while fascinated by the idea of converting my ysv project into a literate program. However, I decided not to.

Rust is a very expressive language. If I can master it well enough, I can introduce powerful abstractions which will permit me to reduce the syntactic clutter, and to make the code as transparent and concise as humanly possible.
I will however use rustdoc as the recommended way of documenting your source code. I will comment difficult parts of the code to describe what they are doing, and I will use standard Rust tools to generate documentation from this source.
And, if I want to invite other people and communicate with them, I'd better use the language many people speak instead of creating my own obscure toolchain to support my overly literate project.
And finally, I was thinking lately about the meaning of focus in life. Adding literate programming as another dimension of ysv project would mean I am continuing to scatter my attention. That will not help me to succeed and get this project done.

(Cover source.)

Running Ubuntu on Lenovo Yoga 530 Transformer Laptop

Anatoly Scherbakov — Thu, 25 Jun 2020 18:54:00 +0000

Lenovo Yoga 530 is a rather inexpensive laptop computer. It transforms into a tablet, has a touchscreen and a stylus. I have its 14ARR variety, which runs on AMD CPU.

This article is a compilation of tips & tricks I found useful when running this device on Ubuntu.

Hardware Upgrade

(This section is not OS related just yet.)

Stock configuration came to me with 8G RAM, which is insufficient
According to a number of sources,
- Lenovo website
- Reddit thread
the upgrade is only possible to 16G RAM total (not to 32G, which I would love to have).

The laptop comes with two 4G sticks in it, so you'll have to replace them both with 8G sticks to have 16G total.

WiFi

Out of the box, Ubuntu won't see the built in WiFi adapter (which happens to be RealTek RTL8821CE).

BIOS

Go to bios and turn off SecureBoot. It tends to turn it on again after BIOS upgrades.

Blacklist

Popular solution is to blacklist ideapad_laptop kernel module (like mentioned at this Lenovo forum thread), like this:

echo "blacklist ideapad_laptop" >> /etc/modprobe.d/blacklist_ideapad.conf

However it did not enable WiFi for me. (Though I am not certain it was not helpful together with the next point.)

What helped was this:

echo "blacklist 8821ce" >> /etc/modprobe.id/blacklist-ideapad.conf
sudo apt-get install rtl8821ce-dkms

WiFi connection turned on immediately.

Onscreen keyboard nastily appearing every time you touch the screen

This is enormously annoying, trust me. But:

sudo apt-get install chrome-gnome-shell

and this Gnome extension — voila, problem solved.

Some generic stuff not related to this laptop but I will write it anyway

Docker won't run without sudo

sudo usermod -aG docker $USER
reboot

ElasticSearch won't run under Docker

...saying:

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

fix:

echo 'echo "vm.max_map_count=262144" >> /etc/sysctl.d/local.conf' | sudo bash
sudo service procps restart

As per this https://github.com/docker-library/elasticsearch/issues/111#issuecomment-268511769.

Issues not yet resolved

When configured Caps Lock for switching keyboard layouts, the light indicator on the button itself won't work
Keyboard is not disabled when laptop is laying on it

Planning Poker app for Humans

Anatoly Scherbakov — Thu, 25 Jun 2020 18:28:33 +0000

In Scrum, you are usually expected to estimate your tasks — preferably relying on estimation points, a transcendental measurement unit that no living human I have met can completely understand.

Process of estimation is often called poker.

How is it done?

Here is how one Agile Poker app, which we happen to use in our company, does this.

This article might start sounding like a piece of stupid marketing. Except it's not, - because the user experience of this app is awful.

Every member of the team clicks the little dropdown to choose the preferred estimation number;
When the one who leads the meeting decides that sufficient count of estimates has been gathered - he or she clicks the "Show Estimates" button, and the estimates are revealed.
Now the attendees discuss the estimates and agree on one final number;
The lead uses the same nasty dropdown to choose that number.

This dropdown is enormously annoying. It forces you to do too much work:

click to open it,
and then click on the desired number.

You have also to scroll the issues list and click on the issue itself if you'd like to find out additional information from the issue...

This world is full of pain.

Did I say this was a quick and dirty article?

Well, it is. And that's why I did not spend too much time looking for alternatives. I've come up with the following instead.

Drawing this, I tried to rely upon the following principles.

When using the tool, you can keep your hands on the keyboard. No mouse needed.
The tool should be independent of the task management system you use. It should integrate with Jira, Notion card boards, Trello, ... - whatever you wish.
I would love it to be open source.
The screen displays the summary of the task, maybe a piece of its description and probably some basic properties like rank, etc.
The estimation figures are displayed in a large font.
To specify your estimate, just enter the number and hit Enter. It will be submitted.
Or, you can use arrows to move from one estimate to another. System will show you which other tasks (preferably those of yours) were previously marked with the given story points value.

I want to specifically highlight the diagram which explains story point values. I got it from a colleague of mine, who says she found it on some video and made a screenshot, but she can't remember what video that was and who was the author. So, if anyone can help here, I'd be grateful - since I like this diagram a lot.

So what?

I will be glad to hear any opinions. I do know there are planning poker apps on the web, but I couldn't find quite the one matching my description - open source, integrated with many task management systems, and easy to use. Maybe such a marvel does exist and I don't know?

But if you are an aspiring web developer and want to do something interesting - planning.casino seems to be available yet :-)