DEV Community: Andres 🐍 in 🇨🇦

Don't do unit test in frontend, Getting a Code Coverage report With Selenium

Andres 🐍 in 🇨🇦 — Tue, 18 Jul 2023 18:57:55 +0000

SOURCE: https://opensrc.mx/posts/getting-a-code-coverage-report-with-selenium/ and better formatted

Introduction

The code coverage always comes with unit-test because it is something you put after doing it.

You can get a percentage of the code what was tested. If all the conditionals paths included in the code was cover.

I was viewing youtube when this video showed.

https://www.youtube.com/watch?v=C8g5X4vCZJA

HolyMolly, from a e2e test you can check the coverage part of the frontend. I know doing unit-test from front-end it's hard because you have to cover every component. But with e2e you can get the same result of the unit-test and get a coverage.

It's like the missing part from a integration testing in the frontend.

This is amazing.. But does it work for only cypress ? What about Selenium?

I'm a advocate of selenium, because my first experiments with it was spamming Facebook groups, with a selenium script.

At the end I get the way to implement code coverage on e2e testing with Selenium. Here the recipe

1.- Instrument the javascript code and get the `coverage` object

If you have written unittest with javascript you know the tool called
Istanbul it is the tool what do the
magic behind the coverage. The way that it works is creating a wrapper
of all code and register the execution path of
javascript.

When your js code is wrapped is called Instrumented Code.

After the execution of the Instrumented code, you will get a __coverage__
javacript object with all the executed path and percentage of coverage.

Therefore if you have a nextjs project or vanilla javascript(a simple js) you
have to instrument the files to get the __coverage__ variable.

There isn't a simple way to do this, could be variable according of the situation.

If you use Nextjs projects SWC-coverage-instrument works good. For getting it working, depends of nextjs version, the package needs to be downgraded.

Let me know by twtter @zodman if you like a little tutorial how implement this.

2.- With the code instrumented execute the e2e test.

Here comes the good part where you have fired the e2e script with selenium. I'm a python dev,
I used unittest.TestCase to implement my selenium testcases.

After finishing the tests you will get window.__coverage__ variable with all the code executed.

    def tearDown(self) -> None:
        r  = self.driver.execute_script("return window.__coverage__");
        with open(f"coverage_{__name__}.json","w") as f:
            f.write(json.dumps(r))

The reason because I saved this object like a json. It is because that variable contains all the coverage code executed.

I need recollect all coverages like pokemon, to be analyzed by istanbul later.

3.- Analyze coverage json files and get the report.

Now with files like:

coverage_test_todo.json
coverage_test_addtodo.json

I move all the files where the frontend project was instrumented because there is the source code is located and link with the instrumented code, and put behind .nyc_output directory


mkdir -p .nyc_output
mv *.json .nyc_output
npx -y nyc report

--------------|---------|----------|---------|---------|------------------------
File          | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s       
--------------|---------|----------|---------|---------|------------------------
All files     |   72.91 |    69.23 |   63.26 |   72.53 |                         
 app.jsx      |      80 |    76.47 |    64.7 |      80 | 58-59,67-84,95-97       
 footer.jsx   |     100 |       75 |     100 |     100 | 6                       
 todoItem.jsx |   34.61 |    41.17 |   55.55 |   34.61 | 16-41,72-74             
 todoModel.js |   66.66 |       50 |      50 |   65.71 | 46-50,64-68,72-76,80-84 
 utils.js     |     100 |    90.47 |     100 |     100 | 34-42                   
--------------|---------|----------|---------|---------|------------------------

PUM! 💥💥💥💥💥💥

Code Repository: https://github.com/zodman/code-coverage-with-selenium

more details here or extend the article: https://opensrc.mx/posts/getting-a-code-coverage-report-with-selenium/

anybar - check if command has finished, succeed or failed

Andres 🐍 in 🇨🇦 — Mon, 31 Oct 2022 02:47:18 +0000

Original Article: https://opensrc.mx/posts/anybar/

Hey fox, with this post I will start writing in plain English.

Anybar it's a desktop-app what I have in my arsenal and I have used day by day.

There are versions for linux, windows and Mac.

It's like a semaphore that you can control, only sending colours to a port.

# Change the colour of an AnyBar
# Usage:
#   anybar <colour> <n>
#     Change the colour of the n'th AnyBar
#   anybar <colour>
#     Change the colour of this tab's AnyBar
#   anybar
#     Change the colour of this tab's AnyBar to white (resetting it)
function anybar() {
  local COLOUR=${1-white}
  local OFFSET=${2:-$(_anybar_iterm_offset)}
  local ANYBAR_PORT=$((1738 + $OFFSET))
  echo -n $COLOUR | nc -4u -w0 127.0.0.1 ${ANYBAR_PORT}
}

I used to manage the long execution commands and saw how long it takes:


# Monitor a long-running command using AnyBar
# This sets AnyBar to orange whilst the command is running,
# then to green or red depending on whether the command succeeded or not
#
# Usage:
#   alias m=anybar_monitor
#   s <some command>
function anybar_monitor() {
  if [ $# -eq 0 ]; then
    anybar white
  else
    anybar orange
    $@

    local EXIT_STATUS=$?
    if [ $EXIT_STATUS -ne 0 ]; then
      anybar red
    else
      anybar green
    fi
    return $EXIT_STATUS
  fi
}

With this small snippet I can execute a command and see the status of the command
from another desktop screen.

I like the minimalist approach of the command. Because with simple communication
you can create an excellent development approach like this:

After a successful command you can play a sound with voice: "master your command was successful"
Monitor your local server for development:

    while true
    do
        (echo >/dev/tcp/localhost/"$1") &>/dev/null && anybar blue  || anybar exclamation
        sleep 1
    done

In my case, if the server stop listening it's because an exception was executed. I
need to check the output to see the problem.

Or you can go further:

You can create your anybar strip and control the lights from commands.

https://github.com/htruong/stripbar

and here My guahiro proyect:

https://www.instructables.com/Wireless-Arduino-Traffic-Light/

Who doesn't want this kind of semaphore when your CI didn't pass ?

How to use JDownloader in headless mode and in remote server

Andres 🐍 in 🇨🇦 — Wed, 07 Sep 2022 04:41:27 +0000

JDownloader it's a downloder files supported Dozen of file hosting websites like mega.nz fileupload.com etc etc ...

if you a old pirate of internet you should know it. Because it's the only descent java application desktop i really know! And deserve spending my memory ram on it.

Like a desktop app only can run locally using your actual speed. In countries of Latin America we can support medium Internet speed.

The jDownloader have a web-application in my.jdownloader.org that use <my-ipaddress>.mydns.jdownloader.org:33561, very rare because a lot of software in that kind, have integrated a daemon what serve the web application like qBitorrent or Transmission.

Let's do the trick... For this I will need a cheap vps with java/openjdk installed and a ssh client.

you should follow the tutorial showed here: https://support.jdownloader.org/Knowledgebase/Article/View/install-jdownloader-on-nas-and-embedded-devices



$ wget http://installer.jdownloader.org/JDownloader.jar
$ java -jar JDownloader.jar -norestart
....
|---------------------------Headless Information-------------------------------
|       MyJDownloader
|       Los datos de la cuenta de 'My.JDownloader' no son correctos.
|       Compruebe su Nombre de Usuario/Correo Electrónico y Contraseña!
|       Enter y -> Enter Logins
|       Enter n -> Exit JDownloader
y
|       Please Enter your MyJDownloader Email:
|>
|       Please Enter your MyJDownloader Password(not visible):
|>

|------------------------------------------------------------------------------

As you can see, the jdownloader will prompt a login, now it's when you go to https://my.jdownloader.org and register an account with a email.

introduce the same login in both side, on the website and the terminal.

After doing that we will use a little trick called ssh tunnel. where you can redirect all your local connections to the server like simulating are running locally executing:

ssh ubuntu@myvps -L 33561:localhost:33561

Doing that you have creating a connection from ipLocal:33561 to your ipaddressVps:33561 when you reload the https://my.jdownloader.org will connect to <my-ipaddress>.mydns.jdownloader.org:33561 doing the same and PUM! You have your jdownloader using remotely.

The same trick works for windows using putty, only use 33561 instead of 8080.

** WARNING:

take it very seriously this. Executing jdownloader remotely expose the port 33561 to internet what it mean anybody might connect to you without authentication and use jdownloader for download something malicious.



tcp        0      0 0.0.0.0:33561           0.0.0.0:*               LISTEN      3007624/java         off (0.00/0/0)

Watch mega.nz videos with mpv

Andres 🐍 in 🇨🇦 — Fri, 26 Aug 2022 23:00:52 +0000

yeap!, As you read videos hosted in mega.nz!

I like watch anime videos and there are tons website sharing anime videos in mega.nz well, here a trick!

with this command when megatools its downloading the file it passing to mpv very similar like streaming.

megatools dl "https://mega.nz/#hashvideo" --path - | mpv -

the requirements are megatools plus mpv

[Ask] For Server linux security is this enough ?

Andres 🐍 in 🇨🇦 — Tue, 16 Aug 2022 21:44:08 +0000

Checking for

lastlog
last
/etc/passwd
chkrootkit
rkhunter

It's enough to check the server security ?

Get always the last version of the github/gitlab software.

Andres 🐍 in 🇨🇦 — Thu, 28 Jul 2022 02:52:37 +0000

Do you use a of software from github ? that not have a packaging system ?

then lastversion its for you!

lastversion apache/incubator-pagespeed-ngx 
#> 1.13.35.2

lastversion download apache/incubator-pagespeed-ngx 
#> downloaded incubator-pagespeed-ngx-v1.13.35.2-stable.tar.gz

lastversion download apache/incubator-pagespeed-ngx -o pagespeed.tar.gz 
#> downloads with chosen filename

lastversion https://transmissionbt.com/
#> 3.0

lastversion format "mysqld  Ver 5.6.51-91.0 for Linux"
#> 5.6.51

If you follow the awesome cli seria I have wrotten with this software you can get the last version of all
:>

dvershinin / lastversion

Find the latest release version of an arbitrary project

lastversion

English | 简体中文

A tiny command-line utility that helps to answer a simple question:

What is the latest stable version for a project?

... and, optionally, download/install it.

lastversion allows finding well-formatted, the latest release version of a project from these supported locations:

GitHub
GitLab
BitBucket
PyPI
Mercurial
SourceForge
Wikipedia
WordPress plugin directory
Arbitrary software sites which publish releases in RSS/ATOM feeds

Why you need `lastversion`

In general, quite many project authors complicate finding the latest version by:

Creating a formal release that is clearly a Release Candidate (rc in tag), but forgetting to mark it as a pre-release
Putting extraneous text in release tag e.g. release-1.2.3 or name-1.2.3-2019 anything fancy like that
Putting or not putting the v prefix inside release tags. Today yes, tomorrow not. I'm not consistent about it myself :)
Switching from one version format to another, e.g. v20150121 to v2.0.1

There is no…

View on GitHub

I don't use Makefile anymore! instead I use bake-cli

Andres 🐍 in 🇨🇦 — Sun, 17 Jul 2022 02:16:17 +0000

before know about bake-cli I'm do a lot of makefiles.. Because it's oldschool and because you could define task to execute easily!

And its great, then I find out fabric and invoke, OH my good! I felt in :love: I start a writing task in python for my local development. But I miss the a lot the simpliest of the makefiles.

But the makefiles isn't create to execute tasks, it was created and designed to compile code.

Then I ever miss in makefiles the possibility of run bash scripting inside.

The same feeling kennethreitz had.

The problem with doing this is that you can't use familiar bash–isms when doing so, as GNU Make doesn't use the familiar Bash syntax, nor does it allow for simple ad–hoc use of arbitrary scripting languages (e.g. Python).

Therefore bake-cli born.

You create a Bakefile and put the task inside with bash power!

$ cat Bakefile                                                $ bake install
install: install/node install/python                            + Executing install/node:
    echo 'All ready!'                                           |  yarn install v1.17.3
install/full: install/system install                            |  info No lockfile found.
install/python: @skip:key=Pipfile.lock                          |  [1/4] Resolving packages...
    pipenv install                                              |  [2/4] Fetching packages...
install/node: @skip:key=yarn.lock                               |  [3/4] Linking dependencies...
    yarn install                                                |  [4/4] Building fresh packages...
install/system: @confirm                                        |  success Saved lockfile.
    brew install pipenv yarn                                    |  Done in 0.05s.
                                                                + Executing install/python:
python/format:                                                  |  Installing dependencies from …
    black .                                                     + Executing install:
                                                                |  All ready!
utils/argv:                                                     + Done.
    set -u && echo "$HELLO: $@"                                 
                                                                Rinse and repeat…
utils/deploy: @confirm:secure                 
    exit 0

Scrollme right

You will speedout your development with this! replace the makefiles

kennethreitz / bake

Bake — the strangely familiar workflow utility.

$ bake — a s☿rangely familiar workflow utility.

~ under development ~

$ cat Bakefile $ bake install install: install/node install/python + Executing install/node echo 'All ready!' | yarn install v1.17.3 install/full: install/system install | info No lockfile found install/python: @skip:key=Pipfile.lock | [1/4] Resolving packages... pipenv install | [2/4] Fetching packages... install/node: @skip:key=yarn.lock | [3/4] Linking dependencies... yarn install | [4/4] Building fresh packages... install/system: @confirm | success Saved lockfile. brew install pipenv yarn | Done in 0.05s. + Executing install/python: python/format: | Installing dependencies from … black . + Executing install: | All ready! utils/argv: + Done. set -u && echo "$HELLO: $@" Rinse and repeat… utils/deploy: @confirm:secure exit 0

~ see bake's own Bakefile ~

What's in the oven?

A Bakefile, which looks and feels like the good parts of a Makefile.
Except…

View on GitHub

Its a nodejs version too of it.

awesome cli that you didn't know, replacement of dbeaver

Andres 🐍 in 🇨🇦 — Fri, 15 Jul 2022 03:39:59 +0000

Always I'm looking to replace my desktop apps with a terminal application. Now take time to search some sql clients to connect the most famous databases: mysql , postgresql and sqlite.

If you are tired about use the dbeaver, well I will showup 3 tools can be a replacement of database explorer.

The history

It's very interesting the history of this 3 tools:

Earlier this year I wrote pgcli to scratch my own itch. I realized that most of the work that went into that project was not specific to Postgres database. I received a few requests from MySQL users requesting a MySQL equivalent of pgcli. I kept procrastinating for a few months and finally decided to launch a kickstarter campaign to fund the development time. That's this project came into existence. But if you'd like to know why I bothered to write a auto-completing repl with syntax highlighting, here's the reason:
I'm a huge fan of CLIs. Especially REPLs. When I first encountered BPython, I fell in love with it. BPython is a Python REPL that has auto-completion, syntax-highlighting and displays the docstring of functions as you type. Once you get used to that level of pampering, all other REPLs start to look like Dumb Terminals.
When I started using psql for interacting with my Postgres database, I yearned for a BPython equivalent for psql.
One fine day I got a twitter DM from Jonathan Slenders asking me if I'd be interested in trying a new Python library that he's working on called Python-Prompt-Toolkit. This library provided a way to write CLIs without having to deal with ncurses and had the requisite tools for doing auto-completion and syntax-highlighting. So I jumped at the opportunity to try it out. After I got myself familiarized, I started writing pgcli to scratch my own itch.
So this app exists because Jonathan was kind enough to request my feedback.

Feature:

AutoCompletation

highlight

Install

pip install mycli pgcli litecli

mycli

MyCLI is a command line interface for MySQL, MariaDB, and Percona with auto-completion and syntax highlighting.

pgcli

Pgcli is a command line interface for Postgres with auto-completion and syntax highlighting.

litecli

A command line interface for SQLite with auto-completion and syntax highlighting.

OK! now it's time to drop your dbeaver and look more hacker in the terminal

How integrate jira with my git branches with python

Andres 🐍 in 🇨🇦 — Fri, 15 Jul 2022 03:18:50 +0000

At work I have some development ruls like:

Every change on code should be comes with a Pull request and the branch name should be named like ID of jira.

Then we have branches like this:

  $ git branch -a
  remotes/origin/DEV-118
  remotes/origin/DEV-120
  remotes/origin/DEV-123
  remotes/origin/DEV-130
  remotes/origin/DEV-133
  remotes/origin/DEV-137
  remotes/origin/DEV-141
....

with that you don't know each branch what is it ... then lets make some fun:

With this you will receive:

DEV-XXX <Assigned Name> <Status> <Title> <url>

Now you can lost between branches and jira issues :>

is this the integration ? comeon zodman! show me more!

Ok ok Imagine you have assigned the ticket DEV-foo1

In git you do:

git checkout -b DEV-foo1

and start hacking .... when you finish and everything looks good. you should create the pull request then you type:


gh pr create -B dev --title "$(jira-list | grep $(git rev-parse --abbrev-ref HEAD))"

PUM! you created a pull request with ID and title from jira.

if you don't know what is gh its a github cli

Implement a antivirus clamav scanner with rest api in dokku

Andres 🐍 in 🇨🇦 — Tue, 12 Jul 2022 03:26:48 +0000

At work, I had the need of implement a virus scanner for all files uploaded to the server.

Well lets give a chance the clamav a deamon virus scanner opensource.

I found a project what implement a rest api very easy!

https://github.com/benzino77/clamav-rest-api/

I run the docker compose of the proyect and works like charm!

It use clamav in a docker container, an a api rest application to scan the file, using a network connector.

I'm not a big fan of docker, for run the docker project by hand, but dokku handle very well and can manage (deploy, maintain, etc) very well without effort.

With the good advice of the core developer from dokku. I can implement very fast the solution in dokku.

Then let's jump in....

                                                                        ┌──────┐
                                                                        │      │
     ┌───────────────────┐                 ┌────────────────────┐       │  N   │
     │                   │   clamav-net    │                    │  proxy│  G   │
     │ Clamav (clamd)    ◄─────────────────┤   Rest API  :3000  │◄──────┤  I   │
     │                   │                 │                    │       │  N   │
     └───────────────────┘                 └────────────────────┘       │  X   │
                                                                        │      │
                                                                        │  :443│
                                                                        │      │
                                                                        └──────┘

Basically this is implemented by dokku..

first the nginx proxy the http request to the rest api application which is listen at :3000 in the container, to do that needs to communicate with clamd the deamon which scan the file and return the result to the apirest.

Easy right ?

My first approach to solve this is creating a plugin doing that but @savant Jose Gonzales from dokku suggest better create two dokku apps using https://dokkupose.netlify.app

The I paste the docker-compose file to the dokkupose and for my surprise it give exactly all the commands needed to deploy in dokku. Feeling like this:

Then the only missing here is the network link between clamd and the rest-api. I handle it by hand.

I left you the script i made to create the solution with comments:

Question?

What about the signature updates ?
In container handle it by automatically and update daily

you don't know about dokku ?

dokku / dokku

A docker-powered PaaS that helps you build and manage the lifecycle of applications

Dokku

Docker powered mini-Heroku. The smallest PaaS implementation you've ever seen.

Backers

Support us with a monthly donation and help us continue our activities. [Become a backer]

Requirements

A fresh VM running any of the following operating systems:

Ubuntu 20.04 / 22.04 / 24.04 (amd64/arm64) - Any currently supported release
Debian 11+ (amd64/arm64)

An SSH keypair that can be used for application deployment. If this exists before installation, it will be automatically imported into dokku. Otherwise, you will need to import the keypair manually after installation using dokku ssh-keys:add.

Installation

To install the latest stable release, run the following commands as a user who has access to sudo:

wget -NP . https://dokku.com/install/v0.35.5/bootstrap.sh
sudo DOKKU_TAG=v0.35.5 bash bootstrap.sh

You can then proceed to configure…

View on GitHub

It's like heroku but selfhosted.

Music for programming

Andres 🐍 in 🇨🇦 — Thu, 02 Jun 2022 14:21:06 +0000

If you are a person that needs a focus music I have a list of website for you.

https://coderadio.freecodecamp.org/

https://musicforprogramming.net/latest/ (I think is the best UI ever)

https://radio.x-team.com/ (I like it because It has a chat integrated)

the last, it's interesting because with that you don't have to go to youtube and search lofi again.

https://www.lofi.cafe/

Awesome cli that you didn't know #3 mysql tools

Andres 🐍 in 🇨🇦 — Sun, 29 May 2022 03:06:23 +0000

When I was working I have projects with a BIG database in MySQL. Big Means as really BIG!!! when you can do a mysqldump it takes hours! If you have some of these problems, these tools will be very helpful.

Only for MYSQL

You have a mysqldump of Gb's of data, to load it you need to wait hours and hours. Imagine you need to update the new structure table in your local environment. To not wait hours and hours loading the new changes, only extract the table you need and load the table only. If you on that situation this tool it's for you:

Split your big database dump on tables https://github.com/kedarvj/mysqldumpsplitter

That will process the mysqldump.sql in small chunks of SQL for each table of your database. then you can take only the part you needed.

MySQL Dumper https://github.com/maxbube/mydumper

When you start FEELING of mysqldump it's not designed for BIG databases because it's very slow and does not show the progress of what happening behind.

This is a mysqldump of the year 3000!

Is a tool used for backing up MySQL database servers much faster than the mysqldump tool distributed with MySQL. It also has the capability to retrieve the binary logs from the remote server at the same time as the dump itself. The advantages of mydumper are:

Features:

Parallelism (hence, speed) and performance (avoids expensive character set conversion routines, efficient code overall)
Easier to manage output (separate files for tables, dump metadata, etc, easy to view/parse data)
Consistency - maintains snapshot across all threads, provides accurate master and slave log positions, etc

I'm fell in love!

bye bye mysqldump welcome mysqldumper

Note: It comes with the myloader the inverse of the dumper :>

DEV Community: Andres 🐍 in 🇨🇦

Don't do unit test in frontend, Getting a Code Coverage report With Selenium

Introduction

1.- Instrument the javascript code and get the __coverage__ object

2.- With the code instrumented execute the e2e test.

3.- Analyze coverage json files and get the report.

anybar - check if command has finished, succeed or failed

How to use JDownloader in headless mode and in remote server

Watch mega.nz videos with mpv

[Ask] For Server linux security is this enough ?

Get always the last version of the github/gitlab software.

dvershinin / lastversion

Find the latest release version of an arbitrary project

lastversion

Why you need lastversion

I don't use Makefile anymore! instead I use bake-cli

kennethreitz / bake

Bake — the strangely familiar workflow utility.

What's in the oven?

awesome cli that you didn't know, replacement of dbeaver

The history

Feature:

Install

mycli

pgcli

litecli

How integrate jira with my git branches with python

is this the integration ? comeon zodman! show me more!

Implement a antivirus clamav scanner with rest api in dokku

dokku / dokku

A docker-powered PaaS that helps you build and manage the lifecycle of applications

Dokku

Sponsors

Backers

Requirements

Installation

Music for programming

Awesome cli that you didn't know #3 mysql tools

Split your big database dump on tables https://github.com/kedarvj/mysqldumpsplitter

MySQL Dumper https://github.com/maxbube/mydumper

Features:

1.- Instrument the javascript code and get the `coverage` object

Why you need `lastversion`