DEV Community: Patryk

Please port your TCP applications to UDP

Patryk — Tue, 10 Mar 2020 13:49:53 +0000

Handshakes are being deprecated due to the current coronavirus epidemic.

Emulating Unicorns

Patryk — Tue, 03 Mar 2020 11:07:48 +0000

I'm sure most of you are familiar with the term Unicorn - a startup that reaches a valuation of $1 billion USD.

Those companies often offer great working conditions, and manage to recruit the top talent, meaning they have really smart engineers, and processes that work really well to develop, test, and deploy quality software. Oftentimes, they also publish their best practices on the internet for everyone to learn from.

If you are a novice, self-taught, part of a small team, etc. those practices are a great fountain of knowledge, and I encourage you to look at the way the unicorns do things, and emulating their processes in your own projects.

Of course, not everything they do should be emulated - Netflix' Chaos Engineering may be too risky for you, and microservices are probably overkill.

Today I will introduce you to two practices I have borrowed from leading internet companies.

AirBnB JavaScript Style Guide

Like any good programmer, I am lazy. This doesn't meant that I don't like writing code or work... I simply don't want to spend ten minutes thinking about how to structure said code. Thankfully, the folks at AirBnB have published a style guide that I have discovered when I started working with the Quasar framework. Using ESLint with the AirBnB rules improved my code a lot... It is now consistent, and I don't need to rack my brain about every decision - when to put a semi-colon, whether to use a trailing comma in an object, list, etc.

Of course, there are other style guides out there, and you may have your own preferences. If you don't already use a linter for your JavaScript code, do start. Gloss over the style guide, refer to it when your linter complains, and you'll see - writing in a consistent style will soon become second nature. Remember to be adaptable; if you join a company that has its own style, use that, and if you dislike some rules, feel free to change them (or use another set of linter rules).

Angular Commit Messages

I have used the word "consistent" twice in the previous section. To be honest, I really like consistency, and this tip is in the same vein. Sometimes I help people online with various questions, open their repository, and see all their commits look as follows:

fixed bug
refactored some code
changed some stuffs
added some stuff

Of course, we all continuously learn, and I have certainly been guilty of this myself in the past. Then I have read the Angular Commit Message Rules. Like the AirBnB JS style guide, this has helped me write more consistent commit messages, making my git log much clearer, more readable, and also allowing you to easily write parsers to generate CHANGELOGs.

Commit Message Rules

I encourage you to read the original document, but the TL;DR is that a commit message should have a header, consisting of a mandatory type, optional scope, and subject, followed by a body and a footer (which are not marked as explicitly mandatory, and I often skip them).

Header

The document defines some mandatory types (build, ci, docs, feat, fix, perf, refactor, style, test). Depending on your project, you can always use a different list. I mostly use feat, fix, docs, build, and test.

The optional scope includes animations, common, core, http, router, etc. As this is for the Angular library, this makes sense, but it does not for my personal projects. As I often use monorepos with Docker, Django, and a VueJS framework, my scopes are often docker, back-end, and front-end.

The subject should use present-tense (change, not changed), and not use capitalization or periods. Personally, I don't mind that style at all, and emulate it. If you prefer something else, feel free to start with a capital letter and end with a period - just make sure that you stay consistent. Consistency makes codebases easier to work with, and onboarding more pleasant.

Some examples:

feat(front-end): add component Foo
fix(docker): rename network mynte to mynet
test: add unit tests for component Foo

Conclusion

This was a brief overview, so follow the links and read the original documents - they offer far more details than this post.

As always, be pragmatic. Take best practices and apply what makes sense to your projects. If you disagree with some rules, make your own. If you work with others, communicate with them, and write your own linter rules or commit message guidelines so the entire team is on the same page. If you join a new company that has its own established practices, follow those. If they don't have any, maybe borrow them from a unicorn, and become the office hero for a little while.

What about you, have you taken inspiration from any other projects and improved the way you work?

I had a problem, so I decided to use multithreading

Patryk — Thu, 20 Feb 2020 10:50:46 +0000

I had a problem, so I decided to use multithreading.

I Now two have problems.

Read the Tests, Luke!

Patryk — Fri, 07 Feb 2020 14:01:25 +0000

This is the second installment in my Testing series. It will not be a big wall of text with a lot of theory - this one comes with exemples from my personal experience.

Intro

If you use an open-source library, or work on an open-source project, sometimes it will not be documented at all, and the author will tell you to just read the source code.

Sometimes, the documentation is good - just out of date. Other times, it will be incomplete, and in some cases, it may be awesome, and tell you everything you need to know (but still not cover the way the library works internally).

Reading the tests will often teach you more quicker than reading the source code.

Writing software requires both theory and practice. In order to become good at it, you should write a lot of code, and you need to make mistakes that you can learn from. That is the practice bit.

You also need to read a lot of code, in books, but ideally, also in open-source projects. That will give you theory, and show you how other people solve problems.

The problem with this is that oftentimes, projects are mature, and have a massive codebase, which means you may not know where to start. My recommendation: always start by running the test suite, then read the tests. It will often teach you more quicker than reading the source code, or sometimes even the documentation.

Also, reading tests will help you see how others write tests, what they test, and may help you think about your own testing strategies.

Examples

Example 1: polyquack

polyquack is a Python library for translations that I have written so I could use it in polyquack-django - an extension to help write translatable websites with Django. polyquack-django solves some pain points that I didn't like with other translation packages, mostly in that it extends PostgreSQL JSONFields to make translations dynamic, unlike some packages that need to create new tables, or columns for each language.

polyquack has a pluralization function, and the pluralization rules are briefly covered in the documentation, but it doesn't really explain everything that happens behind the scenes (note that it's an alpha stage project, so I am sure the documentation will be improved in the future).

So, how does it work? If you use the Pluralizable class, there is a function that takes a language code and a count, and returns the proper string.

# This is an example using a special Pluralizable class defined in the package.

song_forms = {
    "en": ["song", "songs"],
    "pl": ["piosenka", "piosenki", "piosenek"],
}

song = pluralization.Pluralizable(forms=song_forms)

# prints "I am listening to a song"
print(f"I am listening to a {song.pluralize_by_language('en', 1)}")

""" prints:
0 piosenek
1 piosenka
2 piosenki
3 piosenki
5 piosenek
10 piosenek
"""
for i in [0, 1, 2, 3, 5, 10]:
    print(f"{i} {song.pluralize_by_language('pl', i)}")

Internally, there is an indexed dict that returns a function based on the rule number (and languages are tied to rule numbers). The function takes one argument (a count), and returns the correct plural form. All well and good, now how does the code look?

A lot of lambda functions with an argument c (our count), and many conditionals. Some might argue that it's hard to read, but I personally find it quite elegant. Admittedly, it needs to be properly documented, and the comment at the top tells us to look at the tests.

Right away, we can see that we simply call the pluralization.get_form_from_rule() function with two arguments (the rule number, and a count), and it returns the form number (0 to n-1, with different languages having a different number n of forms). The test function names tell us what rule we are testing, and the counts, and describe the rules - same as the documentation.

It makes the actual dict really easy to comprehend, IMHO.

(The complete test suite can be found here if you're interested - I learned a lot about grammar rules in various languages while writing this library and found it fascinating.)

Example 2

This is more of an anecdote than code example. Some time ago, I was tasked with writing a Java app to decrypt a Google Pay payload using Google's Tink library.

I couldn't find any practical examples of what I needed to do, or even the functions I needed to use in the documentation. I simply opened the tink/java/src/test/ folder, finally found a working example, used that as a basis for my own code, and it worked.

Conclusion

Hopefully, you have seen another benefit of writing tests with your code. It helps document the code (for developers, not necessarily for end-users), and can make it easy to understand how the code works internally, or how to use it in practice.

Have you ever refered to tests to understand a piece of code before it finally clicked?

Will you start writing tests to document your code, not just to ensure it works?

Testing Web Apps: Types of Testing

Patryk — Thu, 06 Feb 2020 13:34:29 +0000

Intro

If I told you I am passionate about software testing, you may think I need to get a new hobby. That said, if you never tried writing automated tests, you are truly missing out on a discipline that makes writing software better in every way.

This post is first in a series that will cover the basics of testing, particularly in relation to web-apps, on the front-end (JavaScript) and back-end (python). If you use a different stack, most of the theory will still apply. If you use JavaScript (Node.js based stack) for the back-end, you can use the many of the same tools for front-end and back-end.

This part contains a lot of theory. I still encourage you to read through it - once I add practical examples, it will all make sense. For now, we will cover two types of testing - manual testing vs automated testing.

Why Testing?

Code without tests is broken by design.

~ Jacob Kaplan-Moss

As developers, we often want to deliver software as fast as possible. If we're honest and good developers or engineers, we also want to deliver software that isn't buggy. Writing software isn't just about taking an issue from the issue tracker, writing some code, and closing the issue. If we don't test the code we write, then our customers will have to test it for us, meaning that they will find bugs. That is not fun for your customers, and based on the severity of the bugs they find, they may be put off from using your services altogether.

That is why we should always test before we release software... Catching bugs earlier is always cheaper.

Manual vs Automated Testing

There are two very common approaches to testing: Manual, and Automated. Let us describe them, in case you are not familiar with them.

Manual Testing

Manual testing means that we do everything by hand. We load our website into our browser, click around it, and make sure that it acts as we expect it to.

It is still an important part of testing. (Also, it's a very fun part! Who doesn't like seeing first-hand the results of their labor?)

It used to be very common for companies to spend weeks, or even months working on some feature(s), then passing it off to a QA (Quality Assurance) team for approval. QA engineers had a list of tasks they had to execute manually, and, if they didn't find any bugs, they approved the release. Many companies still do that, but there is a much better way.

Automated Testing

Strange as it may sound, automated testing means we write code that tests our code. Many technological companies have now implemented automated testing into their workflow and, to be honest, you probably do not want to work for a company that has not.

A comparison of manual and automated testing follows, so you can see the benefits for yourself.

Comparison of Manual and Automated Testing

Property	Manual testing	Automated testing
Protects from user errors	❌	✅
Fast to execute	❌	✅
Does not add lead time	✅	❌
Prevents repetitive work	❌	✅
Integrates into CI/CD pipelines	❔	✅
Integrates into TDD	❔	✅
Prevents regressions	❌	✅
Is easy to implement	✅	❌

Protection from user errors

Humans are not as good at repetitive tasks as machines are.

While I have much respect for QA Engineers, they may easily miss a step when running through the app. When you use automated testing, a machine will run through your tests exactly as you programmed it to. For that reason, test results are more predictable and repeatable.

Rapidity of execution

How long would it take you to run 102 tests manually using the interactive python REPL shell?

In one of my projects, 102 automated tests run in under a second (disclaimer: that's only the pytest run - it takes tox a bit more to set everything up, but it's still really fast.)

Of course, not all tests are created equal. Unit tests, integration tests, end-to-end tests, smoke tests... there are many types of tests (that will be covered in a later article), and some of them are slower than others.

Still - automated testing is generally much much faster than manual testing.

Lead time

On the flip side, a common complaint about automated tests is that they are time consuming to write whereas manual testing does not get in the way of releasing software.

It is true that writing tests takes a bit more time when writing the code. That said, the amount of manual testing you have to do to ensure there are no regressions (i.e., you implement a new feature, and accidentally break an existing feature that was previously working) grows rapidly, which brings us to...

Repetitive work

Remember when I said manual testing was fun? The fun occasionally does get old. If you have to repeat tests manually, it gets really tedious. Worse, if you spend 10 hours a week on manual testing (and you are not a QA engineer, but a developer), that is 10 hours less you have for programming.

On the other hand, the computer can repeat tasks essentially forever. It is great at it, and its time is generally much cheaper than an engineer's.

Integration with CI/CD Pipelines

Continuous Integration (CI) is the practice of merging all developers' working copies to a shared mainline (e.g. the master branch) often - usually several times a day. It makes your merges much smaller, meaning that if there are conflicts in the code of different developers, you will catch the conflict sooner, meaning it is cheaper to fix. Imagine spending a month working on a feature, only to realize that it is incompatible with the work your colleague has done during the same month - you may just have wasted one man-month of work.

On the other hand, with CI, you make small incremental merges, with incomplete features.

Continuous Delivery (CD) is an approach that ensures that software can be reliably released at any time. It uses a deployment pipeline - a set of validations the software must pass in order to be released.

Automated testing helps tremendously - you can configure pipelines to run your tests every time you push to your git repository on GitLab, GitHub, etc. It is good practice to also run manual tests on your software before releasing it to production... You can use a Continuous Deployment pipeline to build a Staging version of your website, and run automated and / or manual tests against it.

That said, manual testing does not fully integrate into CI/CD Pipelines since those are meant to be automated.

Integration with TDD

Test-driven Development (TDD) is a development technique where you write tests first, watch them fail, write the minimal amount of code you need to make the tests pass, then refactor the code.

I originally was going to say you can't do TDD with manual testing, as I have never heard of anyone doing that... But I guess, in a way, we all sort of do it.

Define the functionality your website should have.
Create a basic HTML file (or Vue.js, React, Angular, etc. project).
Open it in your browser.
Add some functionality, reload the browser, and test if it works as expected.
Repeat until your website is feature-complete.

So, theoretically, I guess you can integrate TDD and manual testing.

That said, most people who claim to practice TDD do it with automated tests. A practical example of TDD will follow later in this series.

Preventing regressions

A software regression is a bug that breaks functionality that worked previously, e.g. something that works in versions 1.0-1.5, but break in version 1.6.

When using manual testing, you typically focus on testing new functionality, and the most crucial features that were known to work previously. It is very time-consuming to test all functionality manually, so it is very easy to miss a bug that suddenly appears.

With automated testing, if you introduce a new bug, a test should fail, and you should be aware of it before you release. If your tests fail to detect it, you write new tests to cover the case at the same time you fix the bug - then you can be sure the bug will not reappear.

Difficulty

Manual testing is obviously easy. As long as you can use a computer, you can test your website's functionality (although testing all edge cases may prove much more difficult than you expected).

Automated testing is honestly very difficult to get right. You need to learn the following:

Test tools
How to write tests that...
- Test the right thing
- Pass for the right reasons
- Fail for the right reasons
- Do not break when you write new code

Conclusion (part 1)

Hopefully, the difficulty will not put you off from adding automated testing to your arsenal - like all things, you will get better at it with practice.

Stay tuned for the next installments in this series, we still have plenty of things to cover, such as:

Test tools
Types of tests (unit tests, integration tests, smoke tests, end-to-end tests, regression tests, etc.)
TDD in detail
What to test
etc.

I further hope that by the time this series is complete, you will be as passionate about testing as I am 😄.

DEV Community: Patryk

Please port your TCP applications to UDP

Emulating Unicorns

AirBnB JavaScript Style Guide

Angular Commit Messages

Commit Message Rules

Header

Conclusion

I had a problem, so I decided to use multithreading

Read the Tests, Luke!

Intro

Examples

Example 1: polyquack

Example 2

Conclusion

Testing Web Apps: Types of Testing

Intro

Why Testing?

Manual vs Automated Testing

Manual Testing

Automated Testing

Comparison of Manual and Automated Testing

Protection from user errors

Rapidity of execution

Lead time

Repetitive work

Integration with CI/CD Pipelines

Integration with TDD

Preventing regressions

Difficulty

Conclusion (part 1)

References