DEV Community: Marek Dabek

Do's and don'ts before the milestones!

Marek Dabek — Mon, 12 Aug 2019 09:32:44 +0000

Overview

No matter what kind of software you are producing there will be milestones. Unless you stick to some basic rules, this may a point in time, when everything falls apart.

Here is a simple list of do's and don'ts that can help you survive.

Do's

Prepare validation

If you have validation in place (outside the CI) run full suite before the actual validation starts.
I cannot stress enough how important this test run is. There are two major outcomes expected. First, you will know which tests are failing and may analyze whether this is due to a SW bug or a test issue. Keep in mind, that when the validation cycle starts there will be SW bugs coming, so it is unlikely to spend time on debugging tests. Secondly, test run can be a baseline, to which you may compare the execution results.

Code freeze

Set a date for a code freeze. The real code freeze. Meaning, no features are allowed to enter the repository after the CF - only critical bug fixes!
In few projects I have been working on, there were pseudo code freezes. People had been changing APIs, flows, etc. that other components had relied on, and in effect more bug fixing was introduced, delivery was slipping, everyone's frustration raised to alarming levels.

Defects prioritization

If you are not doing this already it is critical to set priorities to defects and set milestones criteria based on count of defects or even list of defects that needs to be fixed. After the validation cycle you should have a list of issues, decide which issues are critical to fix before the milestone, which are important to fix, but we can release with these, and which are negligible. Don't fool yourself that you can fix everything found during testing (if you have good tests in place), but for some of the issues there are workarounds (document those in release notes), some will hit small percentage of users and so on - these are the lowest priorities issues.

Licenses

License compliance should be really planned and checked in the early phase of a project, but as the development goes on programmers tend to find new useful components which they add to the code. If any of these components has restrictive license, to which you do not comply, you may find yourself in a serious trouble. It is usually better to delay the release, to clean up the code, than deal with legal issues later.

Release notes

By the first milestone you should establish release notes process (automatic if possible), to let know users about changes, limitations and known issues. Some of us tend to read this short summary of your work, and it can save you from answering the same questions over, and over again and more importantly can save user from wasting their time on trying a feature that does not yet fully work or is somehow limited. Work on the release notes seconds before publishing the release and you will surely forget to add something important to the document.

Plan team's holidays

This is often overlooked, but for the important milestones agree with the team who is critical for the release and ask them not to plan holidays during that time. If it is not possible, make sure every critical person has a replacement. Do I even need to explain what happens if an issue is found in a SW, 5 minutes before the release, and the owner of the component is on holidays and nobody knows how this SW works?

Don'ts

Last minute infrastructure changes

This is huge no-no. Never do that! Murphy's law will hit you hard if you introduce new infrastructure (new repo, new CI, new validation framework, you name it!), there is a high chance that something will break and everyone will be yelling at you.

Postponing performance testing till the end

If your software has performance requirements (and most software should have!) do not postpone performance testing till the last minute before the release. Performance issues may come from the invalid design and changing the design is usually much more expensive than changing few lines of code.

Last minute components

As one can devise from reading above text about the licenses, it is usually a bad idea to introduce new components right before the release. Aside from the legal issues, you may find issues with 3rd party code, which you do not know or you are incapable of fixing.

Summary

I try to stick to the above list in every project I am working on. It is sometimes a matter of presenting it to the other developers and finding a common agreement on such ground rules.
Do you have your own do's and don'ts?

References

[1] cover image

Stay curious

Marek Dabek — Sat, 13 Oct 2018 22:34:51 +0000

I lead a team of very bright, young engineers. They are keen to learn new technologies, they solve difficult problems, they can present complex ideas clearly. However, there is still one thing they need to learn - staying curious.

The usual task's life cycle: we plan, we design, we implement, we measure... and then some of us are losing interest, especially when measurements go against what was planned. Some of engineers are giving up and switching to new tasks, not knowing that by investigating why the plan did not work they can further enhance their expertise. Because of impatience they are loosing their chance to improve their skills.

Just last week we designed and implemented new caching system and measured that it improved only in half of the cases. Then, the engineer, who was working on the task said that he will switch to the next task. Neither his work was completed, nor he was curious why only half of the cases were improved. I still could pass the task to another engineer, but instead I asked him to explain his results.

Curiosity is the key feature every engineer should have. Curiosity means asking questions, not settling on the status quo and not giving up when things go downhill. Staying curious is difficult, especially when working on the same problem for another hour, day, week or month, but that is how experts are being forged.

I wish you all that you'd never loose that spark that started your engineering journey.

I'd rather you did not ...

Marek Dabek — Fri, 10 Aug 2018 09:42:17 +0000

Here goes a little gem: Secure Programming Practices in C++

The main takeaway - is a list of "I'd rather you did not":

Use C
Allocate with new
Do math a lot
Trust your external input
Use pointers a lot (since they have no semantics!)
Write “clever” code
Use shared_ptr a lot
Use share state a lot

Enjoy!

P.S.
For the curious students: C++ Core guidlines

The rule of three - minus one

Marek Dabek — Wed, 25 Jul 2018 11:11:52 +0000

Many know Jeff Atwood, some know his rule of three. Yes! Yet another golden rule, you need to memorize in your coding adventure.

Ladies and gentlemen here it is The Rule Of Three

Unit tests

Now, that we have laid the basis let's consider who could be the first audience.
How about being an audience to ourselves by unit testing our API?

I hear you saying that this not by the book unit testing and I agree. According to the definition unit tests should exercise individual units of source code. These are commonly understood as a functions or class methods. However, nothing stands in the way of crafting a larger snippets of code, which would use our API in a real-life flow and integrate it into our build acceptance test suite.

Case study

Recently, I have had a chance to write an event manager for the framework I am working on. This task required me to develop multiple interfaces: producer, consumer, event manager, event messages.

Design

I have designed everything "on paper", wrote a code, compiled and was satisfied with the work being partially done.
The project I am working on has a requirement to push code only with unit tests. So I started to write my test suite. That's where I got hit by my "design decisions" or lack of them.

Issues

Unit tests I was developing were testing some basic real-life flows: add producers to event manager, add consumers, add messages, raise events etc. Since the design was mainly focused on interfaces and the implementation was done in abstract classes, I needed to implement classes that derive from producer, consumer, and messages class. That is how my dummy classes (DummyProducer, DummyConsumer, etc.) were created.
It turned out that I did not include a object lifetime and ownership into consideration, which is not a trivial issue in the C++. I was using references to pass a producer into event manager, but it turned out to be a complete failure. Some refactoring and I switched to shared pointers - which I consider a good solution (for now) - since it is safer in asynchronous event handling.
Next, the message. Sure, I can pass the message, but what about its payload? Who should control the payload lifetime? It turned out I needed to create a proxy class, that stored the payload (which can differ from event to event) and pass it to the event carrying class.
The above examples were some of the more significant issues I have found in my design, there were even more minor details I needed to polish.

Conclusion

I was working on an embedded system for the framework project and it took a while to setup unit test environment. In addition to that, my test suites seem more like an example application than standard unit test. Was it worth it to over-complicate testing like this? I would said yes! It let me gain the perspective to and put myself in API users shoes. Definitely I would do that again!

Choose boring technology

Marek Dabek — Fri, 08 Sep 2017 18:57:21 +0000

In one of Charity Majors' talks, I've seen yesterday, she mentioned, Dan McKinley's post Choose Boring Technology. I've found it and it is a real gem! Memorizing this post should be the final assignment for any programming course!

Why I think it is that important? It reminds us, engineers, what are out goals. I assure you we are not engineering for the sake of engineering. I've seen couple of times when instead of solving problems engineers were working on something just because it was new and cool.

The most striking example of "chasing shiny objects" I've seen was a proposal to remodel development process, that came from a totally fresh employee (and recent collage graduate). The guy did not have any idea why the current process was introduced, he had only a rough understanding how developers work, the only thing that mattered was that they were not using some trendy build system he happened to know. He was crushed by questions from senior engineers and they proved that his solution would take the development process 10 years back. Not only, he had not done his homework and had learnt how the current solution works, but more importantly he had not assessed the impact his changes would have had on the other's work.

Don't be this guy, read Dan's post and learn it by heart!

Yet another documentation dispute

Marek Dabek — Wed, 06 Sep 2017 07:51:52 +0000

Introduction

I have seen multiple articles encouraging developers to quit documenting while creating a software. I believe this is the direct influence of the Agile Manifesto and a quite a misunderstanding (in case you haven't noticed it says comprehensive not any in there).

I tend to disagree that we can completely drop any form of documenting our work leaving the code to speak for itself. One exception I would accept, is really simple script, application or a webpage. In other cases we are going straight off the cliff.

Why, why, why?

Software development is a form of engineering and it turns out that in other industries engineers tend to produce quite an amount of documentation.

First of all, an engineer is only a part of a delivery chain. It is really rare that an engineer (or team of engineers) provides a final product. That said, ideas need to be somehow communicated, assumptions must be made and limitations must be discovered. What is the better way to think about them and share our discoveries than writing them down? For Pete's sake, we are solving a problem here and someone needs to understand and agree to our solution!

If you plan to maintain the project till the end of its life, then you probably need to revise your plans. What is more, there will be times, when you will expand your team and lack of documentation will cause spending more and more time explaining how your system works to new team members. Documenting your work may seem irrelevant and waste of time in the early phase, but a high level overview, and some guidelines, and tips&tricks will save tons of work later.

One last thing, there is an industry wide metric called the bus factor. It defines how many employees need to be hit by a bus to lost crucial part of knowledge. Do you really want to be left alone with a pile of code which you are not familiar with, having no idea where to start digging, while your customers demand fixing issues? All of that because no one cared to draw few diagrams?!

How much?

There is a corporate knowledge, the more paper the cleaner your ... or maybe we stop here.

The number of papers and level of details depends on multiple factors:

1. Industry

If you are dealing with avionics or automotive you need to produce zillions of documents and reports, some of which may be required by the external certification committees.

On the other hand, if you are doing simple web app, a single diagram may be enough.

2. Customer

Some customers require you to produce multiple reports from the development phase: design documents, test reports, performance reports, license compliance reports etc. Not much you can do about it.

3. Size of the project

Any complex system, may need documentation on multiple levels: general system overview, specific components documentation, internal APIs, public APIs etc.

Simple projects are usually good with the general overview.

4. Team structure

If you work with localized team with great rotation of team members or your team is spread across multiple locations, you may want to create overviews, guidelines, wikis etc. It is really wasteful to start every quarter of a year with giving new members training on basics.

Summary

I know that maintaining documentation is troublesome, but that is why I am not asking to document all the code, with all the internal APIs, etc. Especially, when cooperating within a small team or small scale project. The minimum we should be doing is to note down what kind of problem we are trying to solve, and how our software is going to help us. Then architecture overview, few diagrams may be enough.

Writing down ideas will give us time to re-think possible options and get feedback from others.

Let’s not be scared, few papers won’t bite.

Save the Internet neutrality

Marek Dabek — Wed, 12 Jul 2017 18:35:34 +0000

Do you remember times before the Internet? Do you remember how difficult it was to find information back then? Did you have a chance to publish your writing before the Internet? Do you need the Internet for your daily work?

You have probably answered at least to some of the above questions, it means the Internet matters for you.
There is an FCC proposal which puts the remaining freedom and equality of the internet in danger. The proposal would allow to create the Internet for the rich and for the poor, with limited content.

If the Internet matters to you Act now!

So, you want to contribute to a specification?

Marek Dabek — Thu, 01 Jun 2017 10:10:31 +0000

Over the last 10 years I've been working for the large company, which was influencing the direction the industry was heading. I did various projects there, but some of the most gratifying were those which served a purpose of developing or enabling a technology.

Product vs technology

There is a difference, that one of my more experienced coworker made me aware of, between building a product and building a technology. In the first case you are limited to the existing specification, which is both good and bad. Known limitations lets you make sane decisions about the product requirements. On the other hand, your implementation depends on how good the specification is, if your product needs to be backward compatible, how it needs to fit into existing ecosystem and finally how to make your product different from others.

What about creating new technology? The problems are at a different level of abstraction: is it possible to implement this, does it cover all possible flows, does it give possibility to provide custom implementations, is it possible to do interoperability testing, etc. It is often impossible, to answer these questions by a single person (or even a single team). The specifications are coined in the long process of reaching consensus between competing companies. Sometimes the process fails, and the final document gives so many implementations options, that two products adhering to the same specification are incompatible.

Ways to become a contributor

Once the specification is published in a form of pre-release draft, usually there is still a lot of room for improvement. Although, at that point, the early adopters are finalizing their products, there are multiple flows and corner cases, which yet have to be discovered. Even when the specification is released in v1.0, I can assure you there are things there which are missing, unclear or simply wrong.

The best way to find them is to start working on the implementation. In case of the HW interface specification, it may be too expensive for a single person, home project to do the implementation. Depending on the technology, but the most successful ones, are driven by corporations (e.g. see list of contributors to USB specifications). If your goal is to contribute to wildly adopted technology, the easiest path is to find a corporation and a team inside, that is implementing the specification. Of course, this may also be a matter of luck - being in the right team at the right time.

What if you are not willing to work in corporation? Well, I have seen contributions from start-ups, and smaller companies, but I do not know how they gain access to early version of specification. I would guess that this is a part of membership agreement.

There is also w third way - working for a large company, but remaining independent. This is the way for the OS community member, usually Linux kernel developers.

How to contribute

Assuming that you are implementing a technology, and found an issue with a specification. First thing to do, is to try understanding the intent behind the concept. There may be deeper sense why something is designed in a specific way.

You may also need to ask about your finding on the forum. If possible use diagrams, it will significantly improve the communication. Writing a 100 pages of e-mail instead of single drawing is a straight way to get hardly any responses.

Be patient, even the brightest idea will need to be reviewed by multiple peers. Providing clean description of the issue and proposal of the solution will speed up the process.

If you are implementing a product based on a standard technology, try to attend interoperability testing event. You will find out about the scenarios you did not take into consideration and it is great place for networking.

Have you contributed to the industry specification? What is your experience?

Hi, I'm Marek Dabek

Marek Dabek — Thu, 11 May 2017 10:55:18 +0000

I have been trying to code something useful for the past 20 years. Yet, managed to deliver my 2 cents in the USB3.0 and MA-USB. Always wanted to join polish demo-scene, but did not have enough balls and brain. I live in the only livable place in Poland - Gdansk.
I am currently trying to learn Rust, which seems like a complete waste of time.

You can find me on GitHub as mdabek

Nice to meet you.