DEV Community: gmantelet

Lyth: Better be safe than sorry

gmantelet — Mon, 17 Feb 2020 21:19:16 +0000

The most immediate mistake young and unskilled coyotes make, at the very beginning of each episode, is to believe they can match the Road Runner. They start running after them, get ready for the fray too early, and after a while, they run out of gas, stunned.

First failure

It seems that even hobby projects do require a bit of QA. Never mind what test cases provide! I first thought. As an enthusiastic runner I swooped hungrily on my project and started implementing the wonderful series on how to implement a basic interpreter with Python.

Then came the bug... The lexer never caught properly the following expression:

let:
  a := 1 + 2

Was it the analyzer? Did it come from the tokens? or even the scanner? Damn! Why the indent was not retrieved properly? After many clutches and hacks, I had to come to terms with this: The project stopped, and that Road Runner escaped, sticking out defiantly its tongue. Meep! Meep!

What is the most frustrating part in this story is to consider how many times the Coyote fails because it did not test its tools first...

A Python package tool

Ok, ok so my silly project needs some harness. And now that I am unpacking my recent acquisition comprising a linter and a test framework, why not doing things right, and follow the instructions on ACME's blueprint by Ionel Cristian Mărieș?

To get an out of the box python package cookie cutter comes to the rescue. Package templates are available here.. Please do not ask why, I chose this one, because well, it comes from Ionel's blog.

Packaging Lyth

The instructions are clear and pretty straightforward. We first need cookiecutter installed on the machine, we provide it the target github template we desire, we answer a couple of questions. I chose the following set of tools:

pytest
Travic-CI
Coveralls
flake8

Executing tox went smoothly. I only had to create a new empty project. By empty, I guess, do not check the README option, no .gitignore as well, and no license.

And then, the steps were quite similar to what was instructed:

git init .
git add .
git commit -m "Add initial project skeleton"
git remote add origin https://github.com/$USERNAME/$PROJECTNAME.git
git push -u origin master

Polishing badges

At that point, we get man shiny badges, most of them shaming you, the build fails, there is no coverage at all etc. etc. So, to get this:

I had to register everywhere and link my repository to my accounts:

Travis CI
Coveralls
Read the docs
Pypi to publish my work!
And even here to write my reports.
If you need a doi, and want your work to be tracked, you could even use Zenodo. I am too shy for this.

So much fun for today, but is my project a burden on Mother Nature? I mean, any time I git push I have now a bunch of small workers contributing to global warming, basically testing, keeping track of code coverage, building documentation, publishing and so on... Sorry polar cubs...

All set up?

So, definitely, this will not guarantee you will catch the Road Runner, but it will raise your confidence next time you ride a rocket, you will not end up in some cactus... Outch!

A couple of caveats

Unicode issue in .readthedocs.yml

I got an error with .readthedocs.yml, and had to remove the last lines, path: .. It did not bother tox on my local machine, nor it hurt Read the docs from building the documentation.

Added .readthedocs.yml to MANIFEST.in

I got issues with tox -e docs and thought adding .readthedocs.yml could ease the problem:

include tox.ini .travis.yml .appveyor.yml .readthedocs.yml

NoInterpreterFound for docs

tox -e docs kept failing, and I found that issue tricky, it seems, that in tox.ini, adding the missing docs helps it find the right Python interpreter to complete (successfully) its objective:

[testenv]
basepython =
    py37: {env:TOXPYTHON:python3.7}
    {bootstrap,clean,check,docs,report,codecov,coveralls}: {env:TOXPYTHON:python3}

Do not forget to have docs included.

Improving coverage in template main.py

When running a test for pytest, it is practically almost impossible entering in:

if __name__ == '__main__':

of __main__.py file. Consequently, coverage reports that the corresponding couple of lines are not covered. I added into my .coveragerc file exclude_lines:

[report]
show_missing = true
precision = 2
omit = *migrations*
exclude_lines =
    if __name__ == .__main__.:

And made the promise I will NEVER EVER write any code in that section.

setup.py

I removed the versions of Python I did not want to use. As I use a lot of f-strings, Travis CI will report failure with Python < 3.7.
I also removed the unwanted python versions in tox.ini.

Read the docs

It is correctly said in Ionel's cookie cutter template, but it was very important to do these extra steps. Go to read the docs, open your repository, and go to admin:

You need to check the box regarding virtualenv via setup.py install, specify the docs/requirements.txtto make the build run smoothly

and excuse my French.

bumpversion and Windows

Are not good friends... I needed to use bump2version instead.

So keep in mind that now changing version of project requires this small 2, otherwise, bumpversion will scr*** your project on Windows. Oh, and do not forget to commit first, otherwise it will complain the repository is messy

bump2version patch to increase version from 1.0.0 to 1.0.1.
bump2version minor to increase version from 1.0.0 to 1.1.0.
bump2version major to increase version from 1.0.0 to 2.0.0.

Publishing

I dump here the set of commands I used to publish as I guess I will have to write a script later to automate this process:

rm -rf build
rm -rf src/*.egg-info
tox -e check
tox
python setup.py clean --all sdist bdist_wheel
twine upload --skip-existing dist/*.whl dist/*.gz

Tada! That makes this, an empty project that burnt a lot of CO2...

Pending issues

I am still unsatisfied with the recipe, but using bump2version and pushing to repository may lead to temporary broken badges, because github cannot compare the new version (as it expects it to be a branch) to master.

So the thing I did so far, and that may be dead wrong, is to go to github, create a branch named the new version generated by bump2version, that is present in setup.py and that is on Pypi, and then push to master. There is nothing to compare indeed, but read the docs does not complain about a broken link, and I guess it is the essentials.

Lyth: A Naive Attempt at Catching the Road Runner

gmantelet — Mon, 17 Feb 2020 02:09:25 +0000

I got a "Wile E. Coyote" obsession running through my mind for days now, trying desperately to catch the idée fixe that I should definitely drop my rock into this reg known as yet another language, and build yet another assembler for an embedded project.

I take as fact that Carnivorous Vulgaris, especially those who are not born yesterday, will likely - and with reason! - roll their eyes, sigh and ask that same old question, the one they keep repeating with a sickened tone... Why would you do that? Nothing today can beat the available tools. So this is waste of time. Well... A starving Coyote would do anything and everything to eat the bird, and this desert is so boring anyway.

There are (however) benefits in this endless task, all brilliantly described by Ruslan Spivak, but I guess the claim that it would wake us as better software developers makes a valuable starting point...

However, designing a potentially new domain-specific language for a target embedded platform, taming intricacies of compilers and the mechanics of lexers, assemblers and linkers... is not it too overcomplicated a task for a rookie? In other word, why would an inept yet ambitious Coyote, an aspiring candidate for disaster, and a proud neurotic show here the perfect example of the Dunning-Kruger effect?

Well... Good question, my fellow reader... This is where I wiggle my eyebrows with overconfidence... Let us explore the couple of potentially wrong statements and misled (if not knowingly biased) opinions that lead me to a potential plan to catch the Road Runner:

Industry seems to grin and bear it: The floodgates are wide open with more and more Famishus Famishus stampeding through the desert, knives and forks in hand, but sometimes... ill-prepared.
The no premature optimization idea can lead to cost-effective code development. This and the fact that some embedded platforms now boast multi CPU SBCs and gazillion bytes of RAM, this is a giant chinese buffet cheering up Coyotes they can now chase after Road Runners, and start using scripting and high-level programming languages for their embedded projects.
As a consequence, is the (millenial) Coyote of 2020 craving for immediate results rather than displaying art and patience?
Many Coyotes leapt into the very overcomplicated pitfalls (precipices) left wide open by cross-compilation tools for target platforms (I throw simply dreaded keywords here: host compilers, target compilers, staging, toolchain, broken dependencies and so on...), or worse, the tools sponsored by (ACME) hardware vendors often turns traitorously against them...
Will there be a lack of expertise among the Coyotes regarding linkers and assemblers? Or simply put, when our forefathers will kick the bucket, shall we blindfully use these tools without any knowledge of their mechanics?

No premature optimization could be applied to this kind of project for the sake of education, to reach out the vast assembly of hobbyists and software developers who already use Python, for example to show the different stages at stake such as:

Interpretation of allegedly elegant syntax and semantics of popular and modern languages.
Transparent demonstration of the translation into assembly and bytecode, leaving the control freak all the room he or she wants to optimize code.
An extra Tetris-like tool letting the said control freak put an address for the variable he or she wants to assign, thus showing what happens under the hood.
And many extra stuff dealing with determinism, FPGA (???), and kind of what you get is what you meant... Shush... I will not let the cat out of the bag now.

Would this be an interesting project to build a blinking LED app, involving all these steps without the widely known tools for the sake of self-education to complement existing initiatives?

This would definitely not reach performance the existing assembler, or catch the Road Runner. But better use cunning than speed, and lure our target: What about another blinking LED project in a portable language that eliminates the use of a host compiler, and that makes assembly great again? If the steps involved expose everything so that future generation of hungry snot nosed Coyotes like me can pretend beating state of the art (just a bit) and try making an explosion of assemblers, and linkers, for the sake of nothing, that day would not be lost totally...