DEV Community: Lorenzo Pasqualis

Software Maintenance, Understanding the 4 Types

Lorenzo Pasqualis — Mon, 14 Jan 2019 17:12:12 +0000

This post was first published in its entirety on CoderHood as Software Maintenance, Understanding the 4 Types. CoderHood is a blog dedicated to the human dimension of software engineering.

Software systems are a collection of bits recorded on a storage device. It might seem that once a software application works, as long as the hardware it runs on is functional, it should just keep working for eternity. So, why is "software maintenance" even a thing? There is no physical wear and tear, and there are no materials that can rot or rust away. Software systems are not affected by the atmosphere, pollution, the weather, and there isn't anything that can physically break. However, this argumentation doesn't take into consideration an essential aspect: the context.

While software applications do not have moving parts that are subject to physical wear and tear, they most often have dependencies with their working environment. With very few exceptions, as the working environment inevitably changes, the assumptions the application was built upon crumble. Eventually, the app stops working or loses its value.

Examples of Context Changes

Let me give you a few examples of context changes that can cause a software system to lose its value or break:

A website that allows users to log-in using Google authentication stops working when Google changes its authentication protocol. (External dependencies change)
A weather prediction software system becomes outdated when the implemented mathematical weather models become outdated. (State-of-the-art knowledge change)
The navigation software of a car becomes obsolete when more recent navigation software products released by competitors are significantly better. (Market expectation change)
The software that manages bank accounts in a financial institution must be updated when the hardware it runs on breaks, and compatible hardware is no longer available. (Hardware change)
An application stops working when any one of the used third party libraries requires an update to a new version that is not backward compatible. (Software dependency change)
An application needs to be modified to comply with new laws and regulations. (Compliance change)
An application needs to be fixed because the definition of leap year changed. (External concepts and definitions change)
An application needs to integrate with a new and popular third-party system. (Market environment change)
Traffic is increasing on a website, and the website software needs to be rearchitected to be able to scale to the new traffic levels. (Usage patterns change)

So, in principle, it is true that software applications don't withstand physical wear and tear. However, non-physical processes cause software applications to age and expire if they are not rigorously maintained.

The Different Types of Software Maintenance

Maintenance of software systems comes in four different flavors: adaptive, corrective, perfective and preventive. Please, note that I did not make up these names. If I did, I would have chosen something different. For example, I find the term "perfective" to be cringy, but I am going to stick with it for compatibility with the standard industry jargon.

A maintenance action is always a software change, but its classification depends on the reasons that triggered it. Let's take a deep dive in the four types of maintenance, and how to recognize them.

Corrective Software Maintenance

A corrective maintenance action is a software change that you make because:

there is an acute issue that prevents the software to work as expected
the problem is actively affecting users, or you suspect that it is.

For example, if you release a software system and your users ran into a bug, a corrective maintenance action is required to fix it. Note that, if the users were never affected by the bug and you resolve it before anybody noticed it, the maintenance action is preventive or adaptive. However, if even one single user might have been affected, then fixing the problem is a corrective maintenance action.

If you spend the majority of the time dealing with corrective maintenance tasks, your engineering team is running in firefighting mode. Pay attention to that situation, as something might be wrong with your testing practices, your ability to anticipate problems, seniority and skill level of your team, the quality of your code or your engineering capacity.

Adaptive Software Maintenance

Adaptive maintenance is something you do when you change a software system because:

you have definite information that the landscape in which your software operates is changing (i.e., market, technology, laws, etc.)
you have a good understanding of the direction and timeline of the landscape change
your customers will be affected at some predictable point in the future, or they are affected now but have trivial and inexpensive workarounds

For example, let's say that you are running a website and you discover that it is not compatible with a new version of Safari that was pre-released by Apple in developer-only beta. Based on historical patterns, you know that Apple will release the incompatible version of Safari in the next few months. If you address the problem before the new version of Safari goes mainstream, then you are performing an adaptive maintenance task. That's because:

Safari is changing
you have a good guess on when it will be released
your customers are not using the developer version of Safari; if they are, they are probably aware of the risks and can switch to the generally available release.

If you wait for the new version of Safari to become generally available to the public, and your customers run into issues, then you have to take a corrective maintenance action. Knowingly delaying adaptive maintenance until it becomes necessary to perform corrective maintenance is costly and should always be avoided.

A very famous example of massive worldwide adaptive maintenance rush was caused by the "Y2K bug" in the late 1990s. The calendar year was guaranteed to change from 1999 to 2000 on 1/1/2000. Many software systems were clearly not ready for it, but customers of those systems were not affected yet. At exactly midnight on January 1st, 2000, any software maker who did not invest in the adaptive maintenance work to fix the Y2K flow was now dealing with a corrective maintenance emergency.

Perfective Software Maintenance

You perform perfective maintenance when you change a software system because you decide to improve its value by improving something that is already working. Perfective maintenance includes things like speed optimizations, UI and usability improvements, etc. Perfective maintenance is often, but not always, initiated by customer feedback. Well-run software organizations listen carefully to their customers and invest in perfective maintenance tasks until just before they hit a point of diminishing returns.

Preventive Software Maintenance

Preventive maintenance is a software change you make to avoid potential (but not guaranteed) future issues.

The difference between adaptive and preventive maintenance can be fuzzy at times. In general, preventive maintenance is not bound to a trend that is sure to occur due to changing contextual conditions; instead, it is linked to an event that might or might not happen in the future. For example, if you are concerned about traffic spikes due to unforeseen and unpredictable future circumstances, and you decide to reinforce your software to deal with it, you are performing preventive maintenance. On the other hand, if your traffic growth patterns are somewhat predictable, and you have a good idea when your systems will start having scalability issues, any change you make to avoid those issues is adaptive maintenance.

Mindmap of the Key Concepts

Why is this important?

Every time that you make changes to a software system, it is essential to identify the type of maintenance that you are performing. Keeping track of it gives excellent insights into your engineering practices. For example, the percentage of time that you spend in each of the four maintenance types gives you an idea of the maturity and skill level of your software engineering organization. It can also give you an idea of the maturity level of your product and codebase.

How much time your engineering team should spend on each of the four types of maintenance depends on many factors. For example, immediately after a release, you should not be surprised to see a spike in corrective maintenance work. Despite your best attempts to release bug-free code, issues always crop up after a release. There are no hard and fast rules, but there are general considerations and principles to keep in mind.

Software systems age. Maintenance is part of a software system cost of ownership. You can somewhat keep it under control, but you cannot avoid it.
As your software systems become bigger and more sophisticated, or as your customer base grows, the amount of maintenance required increases proportionally (but not linearly). Without growth (AKA, more people), engineering teams that keep adding new features to a software system, or supporting a growing customer base, sooner or later end up spending 100% of their time on maintenance tasks. I call that the "zero-growth" stage, and it is not a good place to be. How quickly you'll get there depends on many factors.
Investing in preventive maintenance is like taking a bet. To win you have to bet and be right more often than you are wrong.
Do not let issues go from "potential" to "predictable" to "acute." Most corrective maintenance tasks are very costly because they affect your brand. You can usually avoid them with less expensive adaptive or predictive maintenance work if you pay attention. Take bets, move quickly, don't ignore clear signals.
If you are not working on any adaptive or preventive maintenance task, you are probably not paying attention to the context in which your software operates.
There is a point of diminishing returns in perfective maintenance tasks. You can polish an apple only so much before it becomes a futile exercise.
There are always going to be parts of a software system that you prefer not to touch. That is not necessarily a bad thing; however, you should watch out for how much time you spend working around problems in those areas.
Sometimes it is not immediately obvious how to categorize maintenance work as it seems to fall into multiple buckets. That's ok. Take your best shot. The more you do it, the easier it gets.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

Product Vision, Agile, and Going to the Moon

Lorenzo Pasqualis — Thu, 12 Apr 2018 15:26:47 +0000

This post was first published on CoderHood as Product Vision, Agile, and Going to the Moon. CoderHood is a blog dedicated to the human dimension of software engineering.

Reading Time: 6 minutes

Vision:\
(noun)\
a vivid mental image, especially a fanciful one of the future.

A Clear Vision

Everyone who lived before witnessing the Apollo 11 mission on July 20th, 1969, could only dream of touching the Moon. In 1865, Jules Verne, a French novelist, had that dream in the form of a novel that he titled "From the Earth to the Moon." He told a story of a group of gun enthusiasts and their attempts to build an enormous space gun to launch three people to the Moon.

It took 104 more years for humanity to find a way to realize that dream. When it happened, it was one of humanity's proudest achievements; a glorious moment that required an immense amount of planning and work.

Many small and significant scientific milestones needed to be realized before we could take that one enormous step. First, in 1609, Galileo discovered that the Moon was a world. That discovery was necessary for us to imagine that going to it was even a possibility. In 1687 Isaac Newton reasoned that a cannon powerful enough could put a shell in orbit. If the round were fired even faster, it would leave earth and go into space. In 1903 the Russian scientist Konstantin Tsiolkovsky defined the Rocket Equation, a mathematical basis for space propulsion. The list goes on and on.

Each milestone by itself wasn't enough to get us to the destination but combined created the necessary critical mass.

The vision that propelled us forward was simple: to put a human on the Moon. The engineering vision was more complicated, but still clear: to build the technology necessary to carry humans to space, keep them healthy, deposit them on the Moon, and bring them back to earth alive. That last part was a significant complication.

Agile and Vision Are Not Enemies

I admit it; space missions and software products are very different things. However, they do have commonalities. They both consist of complex engineering challenges that need to be overcome to reach a goal.

The Apollo 11 mission was planned and driven mainly using waterfall processes. That's how we engineered most things at the time. Today, many engineering problems are approached in a more "agile" way. The agile manifesto lays the foundation for that way of thinking:

Agile Manifesto

Individuals and interactions over processes and tools.

Working software over comprehensive documentation.

Customer collaboration over contract negotiation.

Responding to change over following a plan.

Operating according to the manifesto is common in the software industry. Unfortunately, the manifesto is very vague and open to interpretation, and it is usually misinterpreted. The 12 principles of agile spell out that philosophy in greater detail, but still they leave (by design) much to the imagination:

Agile Principles

Our highest priority is to satisfy the customer through the early and continuous delivery of valuable software.

Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

Business people and developers must work together daily throughout the project.

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

The most efficient and effective method of conveying information to and within a development team is a face-to-face conversation.

Working software is the primary measure of progress.

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Continuous attention to technical excellence and good design enhances agility.

Simplicity--the art of maximizing the amount of work not done--is essential.

The best architectures, requirements, and designs emerge from self-organizing teams.

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

There is no mention of product vision or engineering vision in either the manifesto or the principles. It more or less says that the vision needs to be realized in small functional releases and that plans can and will change along the way.

Unfortunately, agile is often confused with "starting to do stuff without a plan." That is a fallacy that causes huge problems. The manifesto and the principles do not say that you should start writing code without having any idea where the product is going. In fact, beginning with a clear vision is a fundamental of "Agile Project Management 101," but it is also an often forgotten first step.

From the product vision, an engineering team should create an engineering vision. That is a high-level idea of how a product vision is going to be realized.

Taking the first step that comes to mind toward a goal is not the right approach, as it leads to many false starts and wrong paths. Let me give you an example.

Going to The Moon

Let's say that the Apollo 11 mission didn't happen and that your goal is to travel to the Moon for the first time. The "product vision" is the idea of an astronaut planting a flag on the Moon. That is what you want to achieve, or the "what." It is a vision because you can see it in your head. From that idea, you need to imagine how you are going to achieve it; I call that an engineering vision.

In an agile world, you might be tempted to make quick progress by rushing to take the very first action that comes to mind that brings you closer to achieving your goals. For example, you could start by walking in the general direction of the Moon. That might seem like a great thing to do as it gets you closer to the destination. However, while walking in the right direction reduces the physical distance between you and the Moon, it has no chance of success because you cannot walk to space.

Walking is a misguided first step toward a dead-end. Taking a bunch of first steps toward dead-ends is not being agile; it is just lacking the ability or desire to imagine how you are going to reach your goals. In other words, it is lack of vision.

A better first step is to think what it would take to get to the Moon. Imagining how you could do it, even without knowing the details, is the engineering vision that you need to have to reach the goal. That vision keeps you focused. For example, if you envision having to build a rocket, the next steps you take are going to be focused on that concrete goal.

With the focus on a rocket (engineering vision), you can take steps that have a chance to get you to the destination.

That vision is critical. If you envision the wrong thing (for example, walking to the moon), you are going to focus on taking steps in the wrong direction. That is where the agile philosophy comes into play. Applying agile methodologies once you have a vision, you achieve functional intermediate steps while keeping evaluating the course and the overall strategy along the way.

Levels of Software Product Vision

We have seen that the engineering vision depends on product vision. Similarly, the product vision depends on the company vision.

That is because the company vision tells you WHY a product should be built. The product vision tells you WHAT product should be built and, finally, the engineering vision tells you HOW the product will be developed.

Together these three levels of vision give the entire narrative of a product. Having that narrative is necessary to describe the reality of the world after the product is built.

Where Are Marketing and Sales in This Picture?

If you don't know how you are going to market or sell a product (at least at a high-level), the product is going to struggle. Look at how long it took for Twitter to figure out how to monetize their services. For that reason, marketing and sales strategies should be part of the Product Vision.

To expand on the MindMap above:

To make this more concrete, at Amazon the vision of a product starts from a mock press-release. That forces product managers to think about all customer-facing facets of a product from the very beginning. Thinking about market and sale realities has profound implications on product design, and the product design has broad consequences on engineering direction.

Conclusions

Agile methodologies have influenced the way software is built, but have also been applied in naive ways. Their goal is to eliminate unnecessary and costly over-formalizations to enable developers to make progress without having to spell out all the details ahead of time. That should not be confused with sloppily taking a step in a direction that appears somewhat right.

Agile methodologies should not be an excuse to attempt to achieve results without planning or without knowing what we want to build or how we are going to do it. A clear product vision is still critical. Small-batch vision and incremental vision are both oxymorons.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

DreamBox Learning is hiring Senior Software Development Engineer

Lorenzo Pasqualis — Wed, 04 Apr 2018 17:07:28 +0000

About Us

For more than a decade we’ve been adapting, innovating and collaborating to shape new learning possibilities for over 2 million kids and 75,000 teachers. Our mission is to inspire all students to think differently about math and love it. We have a creative and fun office environment (foosball table, awesome coffee, dog and kid-friendly) and a work-life-balanced company culture that acknowledges hard work while encouraging innovation and professional growth.

Our award-winning online math software is a pioneer in the EdTech space, and we continue to be at the forefront of the industry. Our adaptive technology, rigorous curriculum, and game-like environment work together to provide a unique learning experience that’s like no other.

About The Opportunity

We are an Agile shop, and our team is made up of a diverse group of self-motivated, passionate, and adaptive software engineers. Our Interactive Software Development Engineers create and implement interactive designs for students to experience on desktops and tablets. We are seeking to add a new senior member to the team—someone with technical and people leadership skills, and strong desire to grow with the company and improve the lives of children and learning guardians.

A typical day in this position involves working with engineers, artists, UI/UX designers, and teachers, to create, deliver and support high-quality software. You will be designing and building interactive components to provide a rich and engaging experience for students to explore and master mathematical concepts.

We are seeking someone with a passion for learning, not afraid of challenges, and a knack for solving hard problems. We are changing the lives of students, teachers, and families with our software, and you will play a critical role in expanding and supporting the content and platform that are at the heart of our product offering.

We work on the cutting edge of a lot of technologies. We need someone who is a creative problem solver, resourceful in getting things done, and productive working independently and collaboratively. Successful interactive software development engineers at DreamBox are highly-motivated and productive coders with a strong background in Object Oriented Programming and an appreciation for smooth animations, elegant UI, tasteful transitions, game mechanics, and steady frame rates. They also have experience with a variety of client-side platforms and technologies and have collaborated to ship software successfully.

This person will take on the following responsibilities:

Create content for our DreamBox Learning experience.
Help us improve our software platform.
Operate in a collaborative development environment.
Help us refine development processes and strategies.
Deliver quality work to meet our deadlines.
Support and maintain released software.
Grow into a technical and people leader.

Skill & Qualifications

5+ years of professional programming experience.
Recent and extensive working knowledge in designing and implementing interactive software.
Expertise in writing and cross-browser web applications.
Extensive working knowledge of JavaScript.
Experience with client-server technologies, like REST, HTTP, JSON.
Experience implementing designs for a user experience across desktop and tablet devices.
Solid understanding of the DOM and Canvas.
Front-end debugging experience using tools such as Charles Web Proxy, Firebug and - Chrome developer tools.
Self-motivated and passionate about software engineering and technology.

How To Apply

Click here to apply

Please feel free to leave questions in the comments section.

What is the most fun coding problem you resolved?

Lorenzo Pasqualis — Tue, 03 Apr 2018 22:03:01 +0000

I had a lot of fun coding throughout my career. There are a few problems, however, that I still remember fondly as particularly fun to resolve. They weren't the hardest ones I worked on or the most important ones. However, at the time, they gave me a great satisfaction that I can still remember today.

Here are two examples:

Ads Probability

I was working on an online advertisement system for a hosting company, back in 2000. I had a list of banners that needed to be displayed randomly anytime a web page was rendered. For each banner, I had a percentage associated with it. The percentage represented the probability of that particular banner to be shown on a website in the system.

I had to write the code to show the ads with the correct frequency. The system was getting millions of requests per day (a huge number at the time), and it had to be highly performant. I had a blast working on that problem. The solution I implemented was simple and fast (at worst O(log n)). I used variations of it over the years for different problems.

Fast Memory Allocation

The second one is from 2003 when I was trying to optimize a virtual machine designed to run on a mobile device. Mobile devices were slow at the time. Very slow. The virtual machine was not performing well on a particular device, and I spent some time profiling it. The code was written in C++, and I realized that what was slowing things down was the memory allocation system on the device (not shocking).

Any time an object was created with "new," the allocator took too long to do its job. I decided to create my own memory allocator, highly optimized for the particular application I was running, which happened to create many very small objects. The result was an immediate 95% performance boost across the board.

What about you?

There are many others, but I'll stop here. What about you? Are there problems that you remember as particularly fun to resolve? What were they?

If you enjoyed this post, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

Avoid Design by Committee: Generate Apart, Evaluate Together

Lorenzo Pasqualis — Mon, 02 Apr 2018 15:35:27 +0000

This post was first published in its entirety on CoderHood as Avoid Design by Committee: Generate Apart, Evaluate Together. CoderHood is a blog dedicated to the human dimension of software engineering.

The tech industry thrives on innovation. Building innovative software products requires constant design and architecture of creative solutions. In that context, I am not a fan of design by committee; in fact, I believe that it is more of a disease than a strategy. It afflicts teams with no leadership or unclear leadership; the process is painful, and the results are abysmal.

Usability issues plague software products designed by committee. Such products look like collections of features stuck together without a unifying vision or a unique feeling; they are like onions, built as a series of loosely connected layers. They cannot bring emotions to the user because emotions are the product of individual passion.

Decisions and design by committee feel safe because of the sense of shared accountability, which is no accountability at all. A group's desire to strive for unanimity overrides the motivation to consider and debate alternative views. It reduces creativity and ideation to an exercise of consensus building.

Building consensus is the act of pushing a group of people to think alike. On that topic, Walter Lippmann famously said:

When all think alike, then no one is thinking.

In other words, building consensus is urging people to talk until they stop thinking and until they finally agree. Not many great ideas or designs come out that way.

I am a believer that the seeds of great ideas are generated as part of a creative process that is unique to each person. If you force a process where all designs must be produced and evaluated on the spot in a group setting, you are not giving introverted creative minds the time to enter their best focused and creative state. On the other hand, if you force all ideas to be generated in isolation without group brainstorming, you miss the opportunity to harvest collective wisdom.

That is why I like to implement a mixed model that I call "Generate apart, Evaluate Together." I am going to talk about this model in the context of software and architecture design, but you can easily expand it to anything that requires both individual creativity and group wisdom. Like most models, I am not suggesting that you should apply it to every design or decision. It is not a silver bullet. Look at it as another tool in your toolbelt. I recommend it mostly for medium to large undertakings, significant design challenges, medium to large shifts and foundational architecture choices.

Generate Apart, Evaluate Together

I believe in the power of a creative process that starts with one or more group-brainstorms designed to frame a problem, evaluate requirements and bring-out and evaluate as many initial thoughts as possible. In that process, a technical leader guides the group to discuss viable alternatives but keeps the discussion at a "box and arrows" high-level. Imagine a data flow diagram on a whiteboard or a mindmap of the most important concepts. Groups of people discussing design and architecture should not get bogged down in the weeds.

At some point during that high-level ideation session, the group selects one person to be an idea/proposal generation driver. After the group session is over, that person is in charge of doing research, create, document and eventually present more ideas and detailed solution options. If the problem-space is vast, the group might select several drivers, one for each of the different areas.

The proposal generation driver doesn't have to do all the work in isolation. He or she can collaborate with others to come up with proposals, but he or she needs to lead the conversation and idea generation.

After the driver generated data, ideas, and proposals, the group meets again to brainstorm, evaluate and refine the material. At that point, the group might identify new drivers (or maintain the old ones), and the process repeats iteratively.

As mentioned, I refer to this iterative process with the catchphrase, "Generate Apart, Evaluate Together." Please, do not interpret that too literally and as an absolute. First of all, there is idea generation happening as a group, but it is mostly at a high-level. Also, there is an evaluation of ideas that happens outside of the group. "Generate apart, evaluate together" is a reminder to avoid design by committee, in-the-weeds group meetings, and isolated evaluation and selection of all ideas and solutions.

I synthesized the process with this quick back-of-the-napkin sequence diagram:

Evaluate

The "evaluate" part of "Generate apart, evaluate together" refers to one of the two different phases of evaluation mentioned above.

The Initial Evaluation is a group assessment of the business requirements followed by one or more brainstorms. The primary goal is to frame the problem and create an initial list of possible high-level architectural and technical choices and issues. The secondary objective is to select a driver for idea generation.

The Iterative Evaluation is the periodic group evaluation and refinement of ideas and proposals generated by the driver in non-group settings.

Some of the activities executed during both types of evaluation are:

Brainstorm business requirements and solutions.
Identification of the proposal generation driver or drivers.
Identification of next steps.
Review proposals.
Debate pros and cons of proposed solutions.
Choose a high-level technical direction.
Argue.
Agree and commit or disagree and commit.
Establish communication strategy.
Estimate.
Report-in, meaning the driver reports and presents ideas and recommendations to the group.

Note that there is no attempt to generate consensus. The group evaluates and commits to a direction, regardless if there is unanimous agreement or not. Strong leadership is required to make this work.

Generate

The "generate" part of "Generate apart, evaluate together" refers to the solitary or small group activities of an individual who works to drive the creation of proposals and detailed solutions to present to the larger group. Such activities include:

Study problems
Identify research strands and do research.
Collect information.
Identify solutions.
Write proposals, documentation, user stories.
Estimate user stories.
Write/test/maintain code.
Experiment.
Report-out (meaning the driver reports to the group the results of research, investigation, etc.)

MindMap

For future reference, you can remember the main ideas of this method with a MindMap:

Final Thoughts

I didn't invent the catchphrase "Generate apart, evaluate together," but I am not sure who did, where I heard it, or what was the original intent. I tried to find out, but Google didn't help much. Regardless, in this post, I described how I use it, what it means to me, and what kinds of problems it resolves. I used this model for a long time, and I only recently gave it a tagline that sticks to mind and can be used to describe the method without too much necessary explanation.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

5 Reasons Why I Love Being Wrong

Lorenzo Pasqualis — Tue, 13 Mar 2018 17:23:33 +0000

This post was first published in its entirety on CoderHood as 5 Reason Why I Love Being Wrong. CoderHood is a blog dedicated to the human dimension of software engineering.

When I was programming 100% of my time, it was my job to write robust and maintainable code. My work needed to be well documented, designed with the proper level of abstraction and as bug-free and efficient as possible. I also had to come up with good ideas for how to resolve problems and create viable products.

As a developer, I had to be right a lot of the time. Being wrong when writing code resulted in bugs and other issues. Regardless of code reviews and testing, dumb ideas in software code have a way of costing an ever-increasing tax.

In my executive role, things are very different. It is an entirely different job. My primary responsibilities are to hire great people, create the best environment for them to thrive and be productive, make sure reality is well defined and make sure that we make good decisions.

In my job, it helps if I have good ideas, but it does not mean that I have to come up with the best ones. It just means that I have to make sure the best ideas surface to the top, regardless where they are coming from or who is coming up with them. I am entirely responsible for my team's results. I am not responsible for putting a thumbprint on everything my team does.

On Being Wrong

I don't deny it. Most of the time I have an opinion. It is part of my job to have informed opinions, but I don't expect them to be always right. I hire people way smarter than me to come up with better ideas than mine.

When having discussions with my teams, I don't mind being wrong. In fact, I like it. I could even go as far as to say that I love being wrong, at least in hindsight.

I don't try to be wrong; quite the opposite. I do my absolute best to be well informed, have good arguments and learn whatever I can to ensure that I am right as often as possible. If I were wrong every time I opened my mouth, eventually my team would lose confidence in my abilities, and that would make it impossible for me to lead.

Despite trying hard, I am wrong a lot. That is the joy and the burden of hiring smart people. They don't let the boss get away with BS, and that is awesome! I would not want it in any other way.

Most importantly, when I try to be right and I realize I am not and, as a result, the right decision is made, that's when I win. It might seem like an Alice in Wonderland upside-down logic, but let me explain what I mean by giving you five reasons.

#1 -- If I challenge a good idea with a bad one, the good one gets stronger

If somebody in my team has an opinion and I test it, if it is a good idea then it grows and becomes stronger. If it is a bad idea, then it dies or takes a different direction. Either way, we made progress.

To make progress, it almost doesn't matter if my challenge is right or wrong. For things that are important, it does matter that I don't let the first idea that sounds good take hold without some more in-depth scrutiny.

I am not advocating that you should challenge everything anybody says 100% of the time. That's just annoying, and sometimes you just need to trust people. Extremes are never a good thing. I am suggesting that, when something matters, it should get scrutinized even if it sounds good.

#2 -- When I am wrong, somebody else in my team is right

When I voice an opinion, I am convinced that it is right, unless I am playing devil's advocate, which sometimes I do. However, my job is not to be right; my job is to make sure the right decisions surface. Getting to the best idea is what counts. I do need to be careful to recognize when an idea is better than mine; but, as long as I listen and keep my eyes and ears open, that's usually not a problem.

I am not THE judge of ideas either. Everybody in my team is a judge of ideas. My job is to listen to the many angles, including mine, and choose the one that passes scrutiny and is on the right path to achieving the company's high-level goals.

When somebody proves my opinion to be wrong, we both made forward progress and everybody wins. It is crucial to abandon bad and distracting ideas for better ones, no matter who was the author. That is what leads to good decisions and outcomes. If I have a bad idea that helps someone come up with a better one, we made progress, and everyone wins, including me.

The worse thing that can happen is not being wrong. The worse thing that can happen is when nobody can come up with something that can survive scrutiny. If that happens, it is on me as I apparently haven't hired the right people or I haven't provided my people the right opportunities or environment to do what they need to do.

#3 -- When I am wrong, I learn

I don't deny it. Being right about something makes me feel good, and who doesn't like to feel good? I know I do. However, if I were right all the time, I wouldn't learn anything.

As long as the team ends up making the right decision, I much prefer to be wrong and learn, than to be right and stay stagnant. Stagnation is my definition of hell.

#4 -- When I am wrong, I grow

Whenever I am wrong, I learn something, but I also grow an inch. Realizing that I am wrong is a way to become more mature and resilient to the next challenge, and it thickens my skin. Next time in a similar situation I'll be able to apply what I learned and help the team make progress.

Now, again, I am not talking about extremes. Being wrong most of the time would eventually destroy me. There is a sweet spot and a healthy ratio when it comes to "growth by being wrong." I have learned that everything in life is a bell curve, and extremes are rarely healthy.

#5 -- When I am wrong, progress is being made

Imagine a totalitarian regime where the dictator must always be right by law. Nobody is allowed to question anything the dictator says, and everyone must defend his decisions no matter how outlandish they are.

Do you think that regime would be successful, even if the dictator is a genius? It is just impossible for one person to see reality from all points of view. Everyone has blind spots, bias and a limited understanding of reality. Different ideas and a constant search for the best ones is what enables progress and innovation.

As long as there is an open debate and the best ideas survive, progress is being made, regardless who had a good idea and who was wrong.

Some Final Thoughts

In the software industry, everything is the result of teamwork. Every once in a while you need a genius to plant a fantastic and revolutionary seed, but the plant will not grow unless it is cultivated by open dialog, collaboration and an "idea meritocracy" environment.

An idea meritocracy is a decision-making environment where the best ideas win. To create it, you must give an opportunity for people to put their thoughts on the table for everyone to see, have constructive disagreements, and have a protocol to get past any conflicts that remain. That protocol could be "the boss decides." It doesn't matter as long as a decision is made that has the best chance of being the right decision. No decision is the worst decision.

In an idea meritocracy, it doesn't matter who is right or wrong. It only matters that the best ideas survive and thrive.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

How to Lead Change, a Primer

Lorenzo Pasqualis — Wed, 07 Feb 2018 20:47:24 +0000

This post was first published (complete with images) on CoderHood as How to Lead Change, a Primer. CoderHood is a blog dedicated to the human dimension of software engineering.

On Change

Leo Tolstoy famously said, "Everyone thinks of changing the world, but no one thinks of changing himself." Tolstoy's words are true today as much as they were true in his time. Life in the 1800s was very different, but people were not.

Today's tech industry, populated by smart people pushing hard to innovate one line of code at the time, is no exception. Intelligence comes with strong opinions and personalities. If you've worked in teams for a while, at some point, you probably wished that somebody could be more pleasant or reasonable.

In this dissertation, I take you on a guided tour of my model of individual and team dynamics in relation to change. In particular, I visit what I believe are essential aspects of the human condition that affect the mechanics of change for people and teams working on challenging problems with common goals. Since my expertise is in software making, I wrote with software developers and software engineering teams in mind. However, the same concepts most likely apply to a broader spectrum of realities.

What Defines An Individual

Change involves people, and people follow patterns that are rooted in what you could call "internal programming," or human firmware. To define a model of understanding of players in a team of software makers, I like to think of human firmware as defined by three main components:

Personality. You can envision personality as a lens that people use to look at the world or a set of filters that trigger our responses. Personality forms very early in life as the result of a combination of biological, parental, and social influences.
Habits. Habits are things that people repeatedly do, settled tendencies and practices.
Beliefs. Beliefs are a combination of knowledge, experience, accepted models of reality, data, facts, plans and views that we came to trust and believe.

The combination of personality, habits, and beliefs determine the behavior of an individual. Personality, habits, and beliefs are not necessarily consistent. In fact, as we'll see later, sometimes they evolve to be at odds with each other. When they are inconsistent, you have a state of cognitive dissonance which can create erratic behaviors, unhappiness, concerns, panic, and chaos.

People Resist Change

Ironically, we humans tend to resist change despite a desire to build innovations and influence others. We push for change because we need to modify the world to align it with our beliefs, thus actually reducing change. In other words, we are driven to change the world so that we don't have to change who we are.

Change erodes people's comfort zone, transforming familiar, known and understood realities into an unclear and potentially risky vacuum. Even though most things in life are unknown, unknowns are scary because they challenge our beliefs and habits; they are similar to shadows lurking in the dark that could jump out to get us.

Despite our comfort with the familiar, we do not know what is going to happen tomorrow. We like to believe that it will be just like today. Given that we operate guided in large part by habits, more often than not we are right. We'll get up at the usual time, eat the usual food, commute to the usual workplace and work on the usual stuff. Sounds boring, but it is comforting and reassuring.

We are well aware that something unexpected could happen anytime, but chances are it won't, so we like to look at the road ahead as well paved, stable and well known. We make bets on stability and inertia, and we are usually right until we aren't.

People Don't Change Easily

Change is difficult because people, at their core, don't change quickly. Habits are hard to eradicate, and personalities have deep and strong roots. However, innovation requires constant change. Moreover, in the tech industry change is inevitable. The choices are to either deliberately design it and make it happen or to let it happen organically. Experience shows that a deliberate design brings better results, in most cases.

When change is needed, leadership pops into the picture; change management is one of the fundamental reasons for leadership to exist. Since most people dislike change, it is a responsibility, privilege, and burden of leadership to push for it. It is critical for leaders, who need to stay one level above the weeds and the minute details, to learn how to make it happen without generating panic and chaos and before external or uncontrollable factors force it.

The type of change that the tech industry push for is technological. However, leaders sometimes make the mistake of blurring the lines between technology, processes, and people. The risk is falling into the trap of wishing they could change people to make them fit a mold, plan or structure. That is a fallacy that, if unchecked, can have serious consequences.

Changing People

In general, you can't force people to change; they won't, and in most cases, you don't want them to. The unique prospectives and points of view that different people bring to the table are very valuable agents of innovation. Innovation without a diversity of ideas is like a meal without a variety of ingredients: bland and not very exciting.

In some cases, changing people's way to operate is necessary for the health of an organization. However, it is essential to understand what kind of change is needed or even possible. As we have seen earlier, when it comes to people, three main aspects could be the focus of change: habits, personalities, and beliefs. Those are very different, and I am going to discuss them separately.

Changing People's Habits

When somebody displays an undesirable habit, the temptation is to eliminate it. However, when you remove a habit, you leave a void. Our brain inevitably tries to resist change, and that void becomes subject to great inner-pressure. Without significant work, old habits tend to come back to fill the gap. As a result, eliminating a habit can be difficult. It is much easier to substitute bad old habits with good new ones, neutralizing the void and avoiding the pressure.

As a specific example, a classic "bad habit" of software engineers is to sit in one-on-one meetings with an open laptop. While a computer might be necessary in some cases (for example, during working sessions), it doesn't help one-on-one meetings where a closer personal connection is more desirable than the ability to take or read notes (or emails, or who knows what else). The vertical screen creates a barrier between two people and introduces a significant distraction.

If someone has the habit of taking or reading notes using a laptop during one-on-one meetings, a solution is to substitute the laptop with a traditional paper-notepad and a pen. Once you get used to it, it becomes a natural, healthy and beneficial habit. Personally, I take notes by hand, scan them (or take a picture with my phone) and upload the images on Evernote which has excellent OCR capabilities. Once in Evernote, I can do text-based searches, making it easy to find what I am looking for, anytime. If there is information I need during a one-on-one meeting, I write it down on my notepad before the meeting. That gives me what I need and also functions as meeting preparation.

If you are helping someone to change a habit, suggesting a new one is an excellent way to make progress. However, since change is hard, the person you are helping must want it.

Changing People's Personalities

I fundamentally do not believe that changing somebody's personality is desirable or realistic. Personality is more than a habit. It is the signature of an individual, a representation of who somebody is.

In theory, if somebody's specific personality-traits get in the way of their goals, it is possible for a good coach to help to make measurable progress over an extended period. However, for somebody to be coachable on a personality trait they need to:

Understand the necessity of the change and the potential benefits, both professionally and personally.
Seek help first (instead of being forced to seek help).
Find a trusted coach.
Be willing to work with the coach.

In other words, the coach can only be available to "support" the change and provide guidance and feedback, not force it.

Otherwise, Do Not Even Try

Unless the situation satisfies the four conditions above, I do not believe in attempting to adjust somebody's personality. That is true in all situations, but especially in business. It is a questionable thing to consider for the following reasons:

Even if you could (you can't), you would not want to create a team of people with the same personality traits. Lack of diversity reinforces bad habits, kills innovation, creates group-think and slows down progress.
The investment is too great; it would take years and is probably impossible.
Who said that "your way" is better? Even if it was better in the specific business context, people have a life outside of work, and their personality is what brought them there. They are who they are. Wanting to change them is arrogant and misguided.
If in some business context somebody's personality prevents them from being able to do their job, the organization should evaluate options. Attempting to change the person's personality should, in most cases, not be one of them. If there are no options, the organization should provide a generous severance package and cut ties. It is best for the organization and the individual.

Changing People's Beliefs

Beliefs come in the form of knowledge, accepted models of reality, data, facts, and plans. Beliefs are the only aspect of people that you can genuinely influence as part of a healthy organization.

In tech, reality changes all the time and beliefs need to be adjusted continuously to describe the world accurately. Changing somebody's beliefs can take the shape of:

Education and training. Often beliefs are rooted in ignorance. If there are existing knowledge and materials that explain something, education and training is by far the best way to change somebody's beliefs.
Brainstorming. If there is not enough existing knowledge and material to study, brainstorming can be an excellent way to align the beliefs of a group of people.
Decision and commitment. Sometimes there are many different ways to look at a particular issue, and no single belief or interpretation of the data seems to be perfect. Usually, that occurs when there is not enough information. In that case, if you can't gather more information, making a decision and a commitment to a specific approach and way of thinking might be necessary. Decisions and commitments eventually become new beliefs.
Logical arguments. Most of the time there is not enough specific information to guide the formation of beliefs. However, often some logical arguments and deductions can be applied to available data in combination with observations and (shudder) some assumptions. For example, when you walk you don't know for sure that the ground in front of you will be able to hold your weight. You don't and can't have enough information to make that decision without the shadow of a doubt. However, if the ground looks stable, you can reasonably believe that it is stable. That belief, formed by the visual information you have, experience and a few low-risk assumptions, is what allows you to walk freely every day.

Changing People's Behaviors

Behaviors are due to a combination of habit, personality, and beliefs. As mentioned above, I believe that it is possible to influence the habitual or the belief parts of behavior, but it is neither practical nor desirable to change the personality aspects of it.

Leaders who need to influence somebody's behavior must be able to differentiate between those aspects. They can help morph unproductive habits into healthier ones, change incorrect beliefs into correct ones, but should avoid attempting to change people's personalities.

Change Management in The Software Industry

The software industry is, in many ways, no different from any situation where groups of smart people work together to resolve difficult problems. Intelligent people often have strong opinions, and strong opinions are usually associated with difficult personalities.

Leaders must learn to analyze patterns of people and group behavior to discern what aspects of it is due to personalities, habits, and beliefs. Change management is a subtle art that takes years of deliberate practice to master but brings great rewards.

Change is hard, but in the software industry, it is a fact of life.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

10 Top Reasons To Have a Career in Tech

Lorenzo Pasqualis — Mon, 08 Jan 2018 15:52:55 +0000

This post was first published on CoderHood as 10 Top Reasons To Have a Career in Tech. CoderHood is a blog dedicated to the human dimension of software engineering.

If you're exploring new professional opportunities, a career in tech may have caught your attention as a potential avenue. Even if you already have one, you might have reached a point where you wonder if it was the right decision.

Having been in the software industry for 30+ years, I firmly believe that a career in tech, and especially programming and software engineering, is not only accessible to everyone, but it also represents a brilliant path to take in life. I am going to give you my top 10 reasons for this sentiment.

1. Tech jobs are not going anywhere.

The world needs software engineers, desperately. There is not enough tech talent to satisfy the demand, and there are more ideas for software products than people to build them.

Programming requires creative thinking, and machines will not easily replace it; not anytime soon, at least. It is true that today, there are AIs able to generate code, but we are far from being to a point where programmers are going be automated out of existence.

Programming requires creative thinking and a deep understanding of human needs. For this reason, I do not believe that software will be able to replace humans in the art of creating software for humans.

Even if that were to happen, humans would move their focus to the next level. Instead of writing all the code, they'd write the code that is not yet possible to automatically generate, and they'd move their attention to where the human brain is required. I am actually looking forward to that kind of innovation. It will make coding even more interesting than it is today. Repetitive tasks, still very much a reality in software development, will be automated, and the most exciting parts of the work will be left to human creativity.

For all of these reasons, taking computer science and programming as a career path is a very safe bet to make. You will most likely not see a world without programmers in your lifetime. On the other hand, you might see many other careers disappear out of existence.

2. You can make good money while you learn.

Many people work menial jobs while they are studying. Minimum wage jobs are popular among college students because they help pay the bills and are accessible without an education. However, programming brings "work as you study" to a whole new level. It opens up opportunities that are not available to most: well-paid activities while still in school.

That has been the case for a while. When I was in college in the early '90s, I was making more money than my college professors selling my software online as shareware. While the term is now antiquated, the concept is very much alive. You write software, put it out there for free, and people can use it with some restriction. If somebody likes it and wants to activate all of the features, they can pay a small amount to unlock it. You might recognize that model as "demo-ware" or "freemium."

Today things are easier than in the 90's. Even if you haven't completed your CS studies, it is not difficult to create software that can be distributed and sold for profit. For example, you can develop iOS applications and sell them on the App Store. With a minimal investment, Apple gives you all the tools you need to build applications and an amazing distribution channel that could make you a fortune.

I am not claiming that it is easy to get rich that way; far from it. However, it is not difficult to make enough money to help pay for college tuition and living expenses for a while. If you are good, it might be the beginning of a future enterprise that could become your fulltime job when once you finish school. Ironically, if you are exceptionally good you might not even have to finish school (but, we all know that you should, right?).

3. You can integrate your passions with coding, whatever they are.

When I was in middle school, I wanted to be either an artist or a software engineer. I couldn't decide until I realized that I could do both. I could take software engineering as a career, and art as a hobby. Even better, it was possible to integrate art and coding, killing two birds with one stone.

No matter what your passion is, you can most likely find a way to integrate it with a career in tech. For example, if you love science, you can write software to aid scientific research. If you like writing, you can write code to help writers or a website for writers to share their work. If you love cooking, you can create applications to support cooks, or sites to share recipes. No matter what you are into, coding will integrate nicely and aid your passion.

4. Programmers are well paid.

In the US, software developers median salary in 2015 was $98,260 according to Labor Department reports. The highest-paid 10% of the profession earned $153,710, and the lowest-paid earned $57,340.

That was 2015, and numbers are going up. Today a talented junior developer with one year of experience can make as much as $80K, and even more. There are not many professions that pay that well from the beginning.

While a CS degree helps, in the USA you can aim to a high salary without a degree. After a few years as a software engineer, if you are sharp and continue to challenge yourself, you can climb as high as your potential can push you.

5. Programming is like a gym for your brain.

To build a healthy physique, you exercise by performing movements that challenge your body. Similarly, to keep your brain in shape, you need to exercise it by solving difficult mental problems. Coding is similar to lifting weights for the brain.

Research has found that learning to perform challenging tasks can keep cognitive functions sharp as we age. However, to be effective, the tasks need to be new and stimulating. In other words, you can expect to keep your mind sharp by doing the same thing over and over your entire life.

When developers work on challenging projects, they can push their brains to the limit on new and stimulating problems. Their mind has to stretch to create models and abstractions; such mental exercise makes developers very flexible thinkers.

Moreover, developers operate in a virtual laboratory where they can make small changes to solutions, and observe the effects immediately. As a result, their brain is subject to a feedback loop that challenges and corrects their mental powers at once. It is like having a personal coach who observes and corrects mistakes, continuously and tirelessly.

6. Your career in tech can grow indefinitely; the sky is the limit.

Software developers can code their entire career and never stop growing. Like martial artists, age does not prevent them from making progress. One can continue learning and growing almost indefinitely through the corporate ranks. Mastery and wisdom come with practice, and practice keeps mental functions active, preparing you for the next challenge.

Even if a developer doesn't want to code forever, there are career progressions that can lead to many different places. For example, developers often become:

Executives.
Entrepreneurs.
Consultants, helping other companies achieve their goals.
Inventors.
Speakers.
Teachers.

There are no limits to the career growth possibilities that you can have as a developer. I've even seen developers easily change careers paths to become successful lawyers, scientists, musicians, artists, etc. The mental exercise that coding provides, and exposure to many industries and problems, is a perfect launching platform to bring your influence and expertise anywhere you desire.

7. You'll never run out of things to learn.

When I started coding, the languages to learn were mostly C, Pascal, Basic and Assembly (Cobol, Fortran, and Lisp if you were interested in more specialized areas). Today many more languages, frameworks, technologies, and environments exist and continue to evolve and improve.

Evolution is faster than anyone's ability to keep up with everything, and it is accelerating at a rapid pace. Large software companies are developing specialized languages and framework designed to solve their problems more naturally, and the trend is continuing and expanding.

There is no way you can learn everything and keep up with all tech trends and new technologies. The problem is not running out of things to learn. The problem, if anything, is to keep current in at least a few areas.

8. You'll always be challenged.

Since there is never a shortage of things to learn, there is also no shortage of challenges. As a developer, there are virtually infinite problems to resolve. For each problem, there are countless ways to solve it. If you want to challenge yourself, there is always something that will do it.

In my 30 years in the industry, I have never met anyone who could claim to have mastered programming skills to the point of being bored. If people get bored, it happens because they are not looking for challenges or because they are interested in a career path change.

9. Programming is a lot of fun.

If you like to keep your mind busy and resolve problems, programming is a lot of fun. There are always problems and technologies that can challenge and capture the attention of every personality type, aspiration, intelligence level and passion.

When you get bored with a technology, you can change and move to something else. When you get tired of a company, you can move to the next one; there are countless organizations hiring developers, and most of them are desperately looking for talent. When you get bored with a particular industry, you can find a different one where you can apply your skills. In the process, you learn a lot about many different things.

For example, I have been personally involved with:

Embedded systems.
Telephone answering systems (back in the '80s)
Real estate.
Shareware.
Marketing.
Mobile applications.
Large-scale web hosting.
Web development.
Search.
Computer language design and development tools and platforms.
Virtual machines.
UI and UX.
Learning systems and narrow AI.
Big Data and reporting.
2D Image filtering and elaboration.
3D image rendering and raytracing.
Micro-controllers and IoT.
Education Technology.
Cloud computing.
SOA (Service Oriented Architectures)
SAS (Software as a Service)

For each of these areas, I just scratched the surface. There is practically no end to the depth and breadth potentials in any field, and there are so many areas and industries to explore that the fun will never end.

10. You can work everywhere in the world without having to re-learn everything.

I studied CS in Italy and worked there for many years. At 26 I got married to my American wife and moved to the US. The skills that I acquired in Italy were the same skills I needed in the US. I didn't have to relearn anything. When I came to the States I made a few phone calls, showed up for an interview, did some familiar C coding on a whiteboard, and got hired. It took about two weeks to find a well-paid job, even if it was my first time in the states.

Programming is like a lingua franca. No matter where you learn it, it stays the same everywhere you go. It is similar to a career passport for the world. It can bring you anywhere and everywhere.

The same cannot be claimed for many other career paths. For example, if you study law, you usually only learn the law of a particular country or even state (unless you specialize in international law). If you are a doctor, your license needs to transfer if you want to change geographical area. If you study political science, your knowledge mostly applies directly to a particular government. In other words, many fields are localized, and they don't easily transfer to other places.

With computer science and programming what you learn somewhere in the world is immediately applicable anywhere. There are no certifications or licenses that need to transfer, and there are no language barriers. Everything in technology is universally in English; you learn it once, and it follows you anywhere. How cool is that?

Why does it matter? Let's say that you find love in a foreign country and you decide to move there. As a programmer, once you can legally work in the country, you can just go there and do it. No extra schooling is generally needed. Coding skills open the door to interviews and jobs without geographical obstacles. It is pretty amazing. If you have the skills, you can also work 100% remotely for any company in the world without having to move at all.

Conclusions

A career in tech is fun, makes you a better thinker, keeps you mentally young, gives you a passport to companies everywhere in the world, keeps you challenged, is not going to be automated out of existence, pays well and is compatible with whatever passions you might have.

Even if math is not your strength, there is much coding you can do without advanced math knowledge. You don't necessarily need a CS degree, and there are many informal ways to learn.

If you are in tech, you made a great choice. If you are thinking about tech, I hope this list of reasons will give you a little more confidence in making that step. You won't regret it. I promise.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

Generating Art With Code: Doodling, Math And Cornucopias

Lorenzo Pasqualis — Tue, 02 Jan 2018 20:15:53 +0000

This post was first published on CoderHood as Generating Art With Code: Doodling, Math And Cornucopias. CoderHood is a blog dedicated to the human dimension of software engineering.

I have a compulsive need to create art; I also have a lifelong love for coding. The two passions are compatible and interrelated. Coding and Math can generate artistic visions. In this post, I show how I created a program to draw doodles formed by tangles of "Cornucopias" swimming in space.

Doodling for Life

If you are a reader of my blog, you know that every Tuesday I publish a hand-drawn meeting doodle from my Moleskine. Recently I have been doodling an interesting shape that, for some reason, I enjoy looking at. I call it a cornucopia because it resembles an elongated and twisted goat horn. You can see an example here.

This week Doodle is based on that concept, but it is a bit different and much more elaborate. Here it is:

As you can probably guess, I didn't draw this with a pen and paper. I wish I were that good. Instead, I "drew it" with code and math, and I am going to show you how I did it.

Doodle Generator

During the holidays, in between eggnog and Christmas cookies, I decided to do a little side project to keep my coding skills sharp and satisfy my compulsive artistic urges. I created a Processing (pretty much Java) sketch to automatically generate art pieces composed by cornucopias floating in 3D space.

I started off with a clear vision of what the final result had to be. My intent was to convey a sense of motion and a sense of "entanglement." I also wanted to generate many random variations of the doodle to have material to make a selection for the most visually pleasing one. Additionally, I tried to make the code easily very configurable.

Cornucopia Theory

A cornucopia as defined in this context is probably a bit different than what you'd expect. It is a 3D shape generated by extruding a spiral along a Bézier spline.

Spiral

A spiral is defined by the position of its center, a small radius r, a large radius R > r, an angle alpha, a number of FULL loops L >= 0 and an angle beta.

Here is an example with r=1, R=2.5, alpha=-30°, beta=60° and L=3:

You can also eliminate the need of having both L and beta by simply specifying an angle theta = (360*L)+beta. In the example above, theta = 1140°.

A cornucopia is a 3D shape, and it needs some "thickness." To create it, I simply doubled the spiral by drawing two spirals with the same r (or close to the same), alpha, beta and L, but slightly different values for R. Here is an example:

After removing the labels from the image, the shape of the spiral becomes clearer:

Bézier Spline

A cornucopia extends through a path following a Bézier Spline, that is a curve that intersects a given number of points in space. The following is a 2D example of a spline defined by 8 points:

Spiral Radius Along The Spline

The cornucopia starts with a large head and ends with a narrow tail; it kind of resembles a trumpet. After playing with various formulas, I settled for an exponential radius, which results in a shape with a section like this:

Putting all together

Finally, the cornucopia is formed as the result of an extrusion of a spiral along a spline in 3D space. Imagine stretching a spiral along a spline and creating a trumpet-like shape with an exponential radius from head to tail.

Here an example rotated in various directions:

Automatically Generated Doodle

The Doodle Generator uses all of this "theory" to create an image with a large number of random cornucopias oriented with the head in the general direction of the observer. The program also allows the selection of a texture image to finalize the effect to a pleasant composition and the positioning of the lighting source.

I showed you an example of the final product at the beginning of this post; let me show you a few more.

Without texture:

One with a single texture texture:

And one with multiple textures:

Show Me The Code!

The source code of the result can be found here.

Note: Processing sketches are generally not applications designed to be used by non-technical people. They are usually compact pieces of code built with the sole purpose of generating images or animations. My Cornucopia Art sketch is not an exception, and changing its parameters requires changing the code.

This is a work in progress that I will continue to refine over time, especially if there is interest in the topic.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

Visualizing Company Culture And Cultural Change

Lorenzo Pasqualis — Sat, 23 Dec 2017 23:47:17 +0000

This post was first published on CoderHood as Visualizing Company Culture And Cultural Change. CoderHood is a blog dedicated to the human dimension of software engineering.

My Processing code used to generate the images and the animation on this post can be found clicking on this link.

Company culture is an important parameter in the success of a business, especially in the tech space. It represents the environment and character where employees generate ideas and produce value. It is the personality of a company and it regulates the heartbeat of a functioning organization.

Despite our desire to be in control of our environment, we are often unable to do so. Many things occur naturally, pushed by forces governed by the result of a myriad of factors that are too complex to study, understand and predict.

As an example, we don't fully understand gravity. We learned to work with it and developed mathematical models to describe the results of gravitational forces. However, we don't understand in details the mechanism that causes the gravitational pull. It might seem strange that the force that keeps us with our feet on the ground every day is still not well understood. However, that is just an example.

Group Interactions Are Complicated

Another phenomenon that we struggle to understand in details is the interaction between people in a group. There are so many variables at play that it is difficult to have a clear view. We understand it at a high-level, and we learned to recognize common patterns, but we are far from having the equivalent of Newton Laws for people interactions. We only have observations and experience to guide our intuition.

For example, any time that a group of people interacts for a prolonged period, it develops a personality that we call "culture." The culture of an organization is a fascinating subject that affects our lives profoundly. Cultures form in all kinds of groups: families, governments, nations, companies, institutions, schools, online communities, etc. It is hard to reconstruct with precision the events that lead to a group's culture, but we can quite clearly experience and describe it.

For example, people tend to brag about daily little things on Facebook, complain on Twitter, act like annoying jerks on Reddit, be collaborative on dev.to, one-up each other on StackOverflow, bring out the professional side on LinkedIn, reveal a wild-illicit side on Whisper and SnapChat, be self-centered on Instagram, etc. Each group has a distinctive flavor and behavior, and people that are part of multiple groups tend to play the part depending on where they are interacting.

Culture is Energy, Like Waves in a Body of Water

Every person operating in a group modifies the group by broadcasting energy into the environment. People have different personalities that translate into different energy signatures. We are similar to wave generators immersed in an energy pool that both influences us and is influenced by us.

In this wave model, company culture can be visualized as the result of the interaction of the effects that each employee has on the environment. Culture is the energy pattern that travels through the energy field in which the group is immersed.

Since "energy field" is a concept difficult to visualize and sounds too abstract and "new agey," think of it as if it was water. Imagine that each person in a group is floating in a body of water. The movements, actions, and words of each person in the water broadcast waves with a distinctive pattern, unique to that person.

A wave has many characteristics, but in this simplified analogy I'll use only three: amplitude, interval, and speed.

Definitions And Clarifications

Some of the terms that I use in this article are similar to terms used in physics. However, I am not attempting to make an exact parallel. In some cases, the words might have a different connotation. To avoid confusion, let me define the terms I use in this context, including their meaning in the analogy with culture.

Each wave generator is a force immersed in the energy field that creates a unique wave. I will also use the term "signature" as a synonymous with wave-generator. A signature is uniquely identified by its position, amplitude, interval, and speed (see below for definitions). In the analogy:
- Each signature represents a member of a group.
- The wave generated by the signature describes the effects that said signature has on the culture.
A crest is one of the high points of a wave, and a valley is one of the low points.
- In the analogy, crests and valleys represent the peaks of influence generated by a member of the group.
Amplitude is the delta of the energy between a crest and a valley. A wave with a large amplitude is more intense than a wave with a small one.
- In the analogy, the amplitude of a wave generated by a member of the group is proportional to the intensity of the influence that such person has on the culture of the organization. Greater amplitude means a higher degree of importance.
Speed is the traveling speed of a crest (or valley). In other words, it is the quantity of energy per unit of time (velocity) pushed through the energy field.
- In the analogy, the speed of a wave represents a unique characteristic of the influence of the signature generating it.
Interval refers to the distance between two consecutive crests. Shorter distance results in more turbulence (irregular changes and movements).
- In the analogy, a shorter interval represents higher cultural turbulence.
Waves expand radially from the wave generator. When different waves interact, they sum or cancel each other, forming an interference pattern, or pattern.
- In my analogy, the interference pattern represents the culture of the group.

Company Culture Patterns

Using this analogy, company culture depends on the unique signature of each employee. If you introduce a new member to the team or modify an existing one, the pattern changes.

To create a visual representation of the concepts described in this article, I wrote a program to animate a 3D (or 2D) rendering of the interference patterns of waves. The program allows placing signatures, and change their characteristics in various ways. It also enables to rotate the view, zoom in and out, change color schemas, save and load the patterns, etc. The program is written in Processing; I called it CultureWaves and published it on GitHub under the MIT license. It is far from perfect at this point, and it has several known issues, but it is good enough to generate the visuals that I was looking for.

In the visualizations created with CulturalWaves, each signature is represented as a sphere and is labeled with a name of a person, representing a member of a group. Each sphere changes size rhythmically, pulsating at the speed of the wave it generates. Also, its maximum size is directly proportional to the amplitude of the waves it produces. The waves generated by the signatures interact, making interference patterns that metaphorically represent the culture of the group.

For example, if the spheres are employees of a company, the interference pattern represents the culture of that organization. The size of each sphere represents the influence level of each employee. You might imagine a large one as the CEO --- having lots of influence on the company culture --- and a small one as a junior employee with limited impact.

Group of One

Let's start with a group of one. Bill is an entrepreneur, and he works alone. The cultural wave pattern he generates represents a company culture of one: predictable and regular, only dependent on one personality. Here is a visualization of Bill --- the white sphere --- and the culture he generates:

From a higher altitude and a different angle you can appreciate the regular pattern that Bill generates:

Group of Two

Let's imagine that the group that Bill started is a company and that the business at some point grows. Bill hires Julie. She joins the company and the effects on the company culture are dramatic. What was a regular pattern becomes much more complicated:

The difference of the interference patterns of Bill alone compared to Bill and Julie together is a visualization of how different a culture of one is from a culture of two. With two people things get a lot more interesting and nuanced. The difference is also noticeable from a distance, and shows the far-reaching influence of a partnership:

Group of Three & Four

If a third person --- Maggie --- joins the company, the interference changes once again acquiring a much more turbulent and irregular nature:

In this early stages of a company growth, small additions of influential people radically shape the culture. In the visualizations shown so far, Bill --- being the founder --- has a relative influence level of 200, Julie 130 and Maggie 120. Those values are a comparable measure of the amplitude of the waves generated by the members of the group.

Adding Adam, a new junior employee with a relative influence level of 40, changes the company culture, but not by much:

As shown in this visuals, company cultures are shaped more by the most influential people. Also, the larger the group gets, the smaller is the impact of each person.

Dominant Influencers

A very dominant influencer can shape the company culture dramatically. For example, look what happens if Julie's relative influence level grows to 200, while Bill and Maggie drop to 30 and Adam to 20.

The effects are even more apparent when observed from a distance. Note how the culture is very Julie-centered:

Growth

Even with a dominant Julie and her influence level of 200, when the company grows, things start to become more homogeneous. For example, here is a visualization of the group with 50 people. Julie is colored in red, to show her position relative to everybody else. The other 49 employees have comparable influence levels around 40.

Note how the pattern is not Julie-centric anymore; she still has significant influence, but the people around her have the effect of generating a turbulent and nuanced company culture that is no longer centered on her. For clarity, here is a view of the same group without the name labels and the spheres:

And look what happens when Julie is removed from the group:

There is a difference, and it is noticeable, but it is not dramatic. The company culture seems to maintain a certain quality that it had before Julie was removed. The personality and energy of the people remaining maintain the culture of the organization somewhat similar to the original.

However, if we increase the relative influence of one of the remaining 49 people --- April --- to 600, which is three times Julie and 15 times the average employee, this is what happens:

Once again, one single individual has a disproportionate influence on everyone else. From a closer point of view and including name labels:

Cultural Changes Over-time

Top Driven

Since all members of a group are immersed in the culture, their behavior and personality change to conform to it. Left alone, each person both modifies the culture and conforms to it, creating a small shift and a larger consolidation effect. People join a group with a given signature, and after a while they change to conform, adding their voice to reinforce the existing culture.

To induce a cultural shift, you need to get the strongest influencers to change their signature to something designed to move the company toward a predetermined direction. If the influencers continue to broadcast a consistent message, eventually everyone will shift and adapt making the resulting cultural pattern the new normal. This is why I am a firm believer that a company culture is shaped at the top and forms at the bottom.

For example, using the graphical visualization that I adopted, let's take a group of three people who start with their natural personalities and impact on the culture:

The pattern represents the original culture of this group, where there has not been a normalization shift yet. Angel is the leader. Maggie and Jason are junior employees.

If Angel doesn't work on pushing a cultural shift toward her direction, the group might settle on this natural pattern:

Note how everybody's wave interval and speed converged around similar values. The resulting culture is mostly "accidental," and might or might not be what the company needs.

However, if Angel tends to push for a cultural shift toward a strategic direction, this is where the group might converge to:

In this case, speed and interval of the three members are closed to Angel's starting point, and the resulting pattern has a different look and feel, closer to Angel's natural preference. The difference might look subtle in this example, but it is very tangible when you are immersed in it.

Cultural Changes Driven by Hiring

Another way to achieve cultural shifts is to hire a significant number of people at once --- or a few influential people --- that do not conform to the existing culture; this typically occurs accidentally during periods of fast growth in small companies. Cultural change driven by hiring can be so dramatic that, unless top leadership takes deliberate actions to guide it, can result in the creation of an entirely different company in a small span of time. I have seen this phenomenon a few times, and it is not always pretty.

On the other hand, if a company lacks the strong leadership needed to shift the culture deliberately, and if more diversity seems to be what's required to get the company out of a rut, hiring new blood might resolve the problem. Strategic hiring to push a cultural shift can be an expensive proposition, but sometimes it can be the most effective one.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

Choosing The Proper Level of Abstraction

Lorenzo Pasqualis — Tue, 05 Dec 2017 13:11:35 +0000

This post was first published on CoderHood as Choosing The Proper Level of Abstraction. CoderHood is a blog dedicated to the human dimension of software engineering.

Models of Reality & Level of Abstraction

The physical world and the rules that regulate it are complicated and very difficult to grasp. To make sense of it, we --- humans --- attempt to explain it with mental models. Models are logical constructions defined with terms that we created and understand. We use those models as proxies for reality in an attempt to grasp and predict its complicated and multifaceted rules.

Models are not an engineering-only concept; they are fundamental to everyday life and are natural simplifications of the world that we experience. We build models for everything, every day. For example, the memory we have of someone we met is a model of that person that we constructed in our mind. It is a description that is simple enough to store away in our memory, and sophisticated enough to be nuanced.

A Model of a Rock

Given the limitation of our brains, we create logical models of many aspects of reality. For example, we have models for objects based on their most prominent characteristics. A very efficient model of rock is the word "rock" itself.

ROCK

A more nuanced model could be a simple shape, which gives us more information on what the rock looks like.

But that is still very rough. It is hard to tell if that's a rock, or an egg or something else. Anything with that shape can be described as an oval, which makes the model unable to capture the distinctive characteristics of a rock. A more nuanced model could be visualized as a drawing of it.

The drawing is not much more detailed than the circle, but it carries information to remind us that a rock does not have smooth contours, and shows some of the imperfection of the surface. Still, due to my poor artistic skills, that drawing could represent many different things.

A picture of a round rock is a better and more detailed model, which includes color, texture and the surrounding context. Also, a photograph has the advantage of being captured by a machine without the filter of human interpretation.

However, this model falls short of being a full description of the rock. For example, it is two dimensional and shows only a particular side. It does not contain any information about the weight, three-dimensional characteristics, internal composition, position in space, etc.

A full description of the rock at a particular moment in time would require mapping every one of its atoms and their relative positions in space. Given that the stone changes over time, a complete description would require such a map for every instant of the rock existence. That would require defining when the rock started existing, and it would be done only when it is no longer recognizable as a rock.

That's, clearly, an absurd model. Such a description would be infinitely complex and unusable with our limited mental processing powers. In other words, it would be vastly useless for most purposes.

Models And Abstractions

The terms "model" and "abstraction" are often overloaded; it is important to clarify their definition in the context of this document. Warning: the following definitions are more formal than I'd like them to be; however, I find it necessary to define them precisely to avoid confusion.

Domain: a specified sphere of concepts, objects, activities or knowledge.
- For example, "building" is a domain composed of all possible buildings.
Model: a set of distinctive characteristics of a subset of a domain, including their constraints. The constraints are necessary, in a well-defined model, to limit the number of variations to a finite set.
- For example, a house model could contain a set of distinctive characteristics of a house: things like color (one of the 16,777,216 8-bit RGB values), size (integers from 50 ft2 to 10,000 ft2), # of rooms (from 1 to 10).
- The subset of the building domain covered by the house model is the set of all houses that can be described by it. In the example, every possible combination of colors, sizes and # of rooms. That is a total of 1,669,332,992,000 possible houses.
Instance of a model: One particular variation that can be described by a model.
- For example, an instance of the house model could be identified as being red (RGB #FF1010), 4000 ft2 and have 6 rooms.
Explosion of a model is the set of all the possible instances of a model.
- In the house model example, all the 1,669,332,992,000 possible houses.
Model A is called an abstraction of model B and B is a model abstracted by A if the explosion of A is a superset of the explosion of B.
- For example, a house is an abstraction of a studio apartment. A studio apartment is a model in the house abstraction.

One more example

Carbon-Based Lifeform is an abstraction for many models, such as: Plant, Animal, Microorganism, etc. Animal is a model that can describe a set of distinctive characteristics of a, well, animal. One of the domains of Animal is the Carbon-Based Lifeform domain. Animal is also an abstraction of a dog and Dog*is a model in the *Animal abstraction. Dog is also an abstraction of a German Shepherd model. A particular German Shepherd named Cody is an instance of the German Sheppard model.

If you made it this far, the rest is much more comfortable. I promise.

Choosing a Level of Abstraction

The level of abstraction that you should choose when modeling something depends on what you know about the context in which you are operating, and how things might change in the future.

For example, if you wanted to create software to manage the participants of a "Best in Show" dog competition, modeling a dog would be sufficient. The chances that a dog-centric competition will expand to include other animals is so low that abstracting to all animals is not something you need to plan for.

On the other hand, if you were writing software to manage guests of a pet shelter, Dog is probably not the right abstraction, even if all the current guests are, indeed, dogs. You need to look a step ahead. Shortly the shelter might host cats, birds, lizards, and who knows what else. In this case, Animal is probably a better abstraction.

Choosing an Abstraction in Software Development

In software development, choosing the right abstraction can be tricky. If you make it too simple, it won't let you create a model to satisfy even the immediate requirements. If you make it restricted to the urgent needs, you might have to change it almost immediately to implement the next iteration of the model. However, if you make your abstraction too generic and all-encompassing, modeling solutions might get so complicated that you'll go out of business before you are finished.

In my experience, the level of abstraction that engineers choose is determined --- good or bad --- by:

Their skills and experience.
Their personality.
Data available about the domain.
Time and resources.
Accidental or deliberate assumptions.

In the rest of this post, I use a concrete example and an analogy to visualize what it means to choose an abstraction.

Modeling A Calculator

Imagine that a customer asks you to build a calculator application (like a pocket calculator); something that would satisfy the needs of grade-school students.

The Ideal Model

The customer has a high-level ideal in mind. They don't know it, and can't clearly describe all the details, but if they took the time to document the ideal model, it would look like this:

The Information You Have

However, instead of a design document, the customer invites you for lunch and gives you a verbal description that you try to jot down as best as you can; as a result, your "spec" is a sketch on the back of a coffee-stained napkin.

They tell you that they want to get something out soon, and it doesn't have to be perfect. You accept the challenge.

Your Challenge

Now you need to figure out everything else. The data you have, in this case, is a vague idea of what the goal is. Your rough sketch of the calculator gives you some information about the "reality" that you need to model, but not a complete description. You have more questions than answers.

As a developer, you need to use the information you have to create the proper abstraction. A useful abstraction is one that allows implementing a simple model to satisfy the initial needs and gives space to refine that model until it approximates the ideal.

To recap, your domain is the universe of Calculators. Your goal is to define and build the ideal model; however, you don't know what that looks like is. What you have is a vague sketch of a model scribbled on the back of a napkin.

Your Goal Is a Jagged Ink Splatter

To draw an analogy, imagine that the ideal model is this black ink splatter:

The splatter represents the functionality, features, and behaviors of the perfect solution. The ideal solution is a model that uses all aspects of an abstraction to satisfy all the needs, present and future, of the problem that you are trying to solve.

The jagged edges of the splatter represent the complications and details of the ideal abstraction. The shape is complex and has many aspects that are difficult to model correctly.

An Over-Simplification

A first simple approximation could be an application that emulates a calculator that looks like this:

That is a start, but it covers only a small part of the ideal target. The picture below shows that small part in the context of the black ink splatter analogy; visualized in blue:

The customer is not going to be satisfied with it. The model is too simple, and it covers only a few of the features of the ideal.

Simplification With Deviation

A second attempt could expand on that idea, but in the process, you might make some choices that are a deviation from the ideal concept; for example, you could choose to add some new buttons and a usage workflow that is not quite what the customer wants. The result might look like the following model:

This model covers more of the original black ink splatter compared to the over-simplification, but it also expands outside of the boundaries of the ideal.

Overall, this model is not complete, and a little off.

An Acceptable Simplification

After several iterations, a close study of the original sketch, and lots of conversations with the customer, you could come up with a better approximation of the ideal model; it is not perfect, but it captures most of the needed characteristics. It misses some of the buttons and functions, but it adds a larger screen which the customer likes.

This approximation models the ideal solution much more closely than the previous iterations. Visualized in blue on top of the black ink splatter, this model coverage of the perfect abstraction looks like this:

Over-Abstraction

Over-simplification is not acceptable, but over-engineering and over-abstraction are even more dangerous. You fall into that trap when you make things much more complicated than the ideal solution, adding unnecessary cost and missing the spirit of the original target.

As a result, if you were asked to create a calculator, you might end up with a super-computer.

Which, in the ink analogy, covers the entire black splatter and a chunk of the observable universe around it.

In this exaggerated example, the model grew much larger than the ideal abstraction, meaning that you over-abstracted the problem.

Choosing The Right Abstraction and Initial Model

When you are presented with a problem, you need to think carefully about the right level of abstraction. The goal is to create one that:

Can model an initial solution to satisfy the absolute minimum requirements.
Can be refined into new models to resolve known foreseeable future needs at an acceptable cost.
Does not attempt to solve problems that you'll never need to address.

To evaluate an abstraction and choose the initial model, you need to have two lists:

List of the minimum requirements; this list is required to determine the proper initial model.
List of all foreseeable future needs; this list is needed to choose the appropriate abstraction.

When evaluating an abstraction and an initial model based on it, ask yourself the following questions:

Does the model satisfy the minimum list of requirements? If not, the model is too simple and needs to be improved. That is, you need to expand the initial blue area on the black ink splatter.
Does the abstraction allow for an eventual expansion of the model to cover the foreseeable future needs at a reasonable cost? If not, you might need to expand your abstraction. That is, you may need to enlarge the area that the black ink splatter covers in the domain.
Is the model solving many more problems that are not in one of the two lists? If so, you might have created an over-abstraction. Consider reducing it. Over-abstraction increases long-term costs.

Conclusion

Choosing a proper level of abstraction is a science and art that you learn with experience. I hope that this dissertation will help you visualize the challenge, and give you some additional tools to improve your model abstraction skills.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!

7 Ranks of Coderhood: Coder, Programmer, Computer Scientist, Developer, Engineer, Architect

Lorenzo Pasqualis — Thu, 30 Nov 2017 15:41:17 +0000

This post was first published on CoderHood as 7 Ranks of Coderhood: Coder, Programmer, Computer Scientist, Developer, Engineer, Architect. CoderHood is a blog dedicated to the human dimension of software engineering.

The Many Names of a Software Builder

In my blog, I have been using the terms coder, programmer, software developer and software engineer interchangeably. I do that mainly to avoid linguistic repetitions. However, I believe that there are differences between those words and similar others.

In this article, I will discuss a set of nouns commonly used to refer to someone who writes code. I will also give my interpretation of how those terms relate to engineering skill levels.

Interpretation of the meaning

The definitions that I will present here are not official. There are no formal definitions that I am aware of or trusted body to create such a thing. However, there are lots people with strong opinions on the matter. My understanding of what each of the terms implies is based on three decades spent in the software industry, but I fully expect that other people will disagree with my interpretation.

My intent is not to start a debate or attempt to convince anyone who may have a strong opinion. I do not believe that there is a right or wrong at this time. However, if you have not yet formed a strong opinion on the matter, I hope that this guide will provide some clarity on a possible interpretation.

A three-pronged approach

For the sake of clarity, for each of the nouns I will do three things:

#1 - Give a Description of the Skill Level

I will provide a description of the skill level that, in my interpretation, is implied by the term.

#2 - Provide a Parallel With Martial Arts Ranks

I will make an analogy with martial arts ranks. In particular, I will equate the technical software-making skill level implied by the term with martial arts belt colors.

#3 - Give a Code Example

I will provide an example of how I expect that somebody at that level would choose to approach a simple programming problem. The definition that I will use is trivial and not intended to be a realistic example. Its purpose is to compare and discuss skill levels. The simplicity is designed to allow anyone, regardless of experience, to easily follow along. The problem that I will use is:

Calculate the sum of a set of integers.

As you are reading my dissertation, please play along and imagine this as a proxy for a much more complicated case. In this make-belief universe, this task requires serious thought and has many potential solutions and design approaches. Imagine it to be a central piece of computation for a system that needs to scale and perform.

I will use Ruby to show some trivial examples of implementation; the code is simple enough that it should be understandable even if you don't know Ruby. If you don't understand it, don't worry.

The List

The nouns that I will be discussing are:

Beginner.
Coder.
(Hacker)
Programmer.
Computer scientist.
Software developer.
Software engineer.
Software architect.

Ready to follow along? Let's get started.

Martial Arts Belts

When I was living in Italy in my teens and twenties - about 40lbs ago - I spent years practicing Judo and Kung Fu. During that time I learned that in many martial arts, the color of the belt is a symbol of the skill level of the practitioner. A typical color pattern goes from white to black, where the darkness is proportional to the experience level.

A beginner wears a white belt because they have no experience. The color white indicates "new and clean." As the martial artist trains, the belt gets darker to show progress. The color represents the grime accumulated with hard work and sweat. A martial artist with many years of experience eventually achieves a black belt rank to indicate high levels of expertise and skill.

Traditionally belts were black or white only. In recent decades things have evolved to include more colors. Today, different martial arts schools have different belt color schemas. The schema depends on the style, the dojo, and the country.

Why Am I Talking About Martial Arts?

I will use martial arts belt colors to draw a parallel between software building skills of a software professional and the fighting skills of a martial artist. For this purpose, I will use a belt color schema commonly used in Europe. In order: white, yellow, orange, green, blue, brown and black.

The following table shows the engineering levels that I mentioned. For each of them, the table shows a corresponding belt color and a title that is commonly given to someone at that skill level:

Engineering Level	Martial Art Level (Belt Color)	Example of Job Title
Beginner	White
Hacker	Street Fighter (No Belt)
Coder	Yellow	Jr.Dev
Programmer	Orange	Software Dev
Computer Scientist	Green	Software Dev
Software Developer	Blue	Sr. Software Dev
Software Engineer	Brown	Principal Dev
Software Architect	Black	Software Architect

The engineering skill level is related to the technical and teamwork abilities of the developer. The job title is an example of what the industry calls someone at that level (heavily dependent on the company and geographic area).

Beginner - White Belt

You have to start somewhere, and that is usually "no experience at all." A beginner in software development is just that: someone who is new to coding and is in the early stages of learning. Beginners cannot yet reliably write or understand simple functioning programs without consulting books, tutorials, cookbooks, or getting help from somebody.

Beginners might be able to write working code, but they often do not understand the details of why that code works. They commonly spend a lot of time searching for code snippets on StackOverflow or similar sites and stitching them together until something functions.

Powerful tools are not the same as robust skills

To make things more confusing, many "modern" languages and frameworks allow anyone to generate the structure and some of the implementation of complex programs without the need to understand what's going on behind the scenes. For example, getting a simple Ruby on Rails application up and running and accepting HTTP requests is something that anybody can do with a few command-line commands and a tiny bit of determination.

Here is how you do it on a *nix system:

$ gem install rails
...
$ rails new website
...
$ cd website
$ bin/rails server
...

Done! That is all you need to get a server responding to HTTP requests generated by a web browser. If this were a martial art fight, it would be the equivalent of showing up with an armor and a gun. The armor might allow you to survive a little longer, and with the gun, you might win the match. However, winning a fight that way doesn't make you a skilled martial artist. It just gives you tools to do something challenging without traditional training, effort or study.

Don't take me wrong. Tools like Ruby on Rails that allow a developer to get things done fast are great. In fact, I think that cutting down the time spent on any boiler-plate code is fantastic. It is an excellent start of a project, but it only requires the skill level of a white belt.

The real fight starts when the tutorial ends and when tools do not automatically generate the application you are trying to implement. To move past that point, you need to become a coder.

An example

If a beginner wanted to write a program that sums a set of numbers using Ruby, they might Google "How to sum array of numbers in Ruby?", and find this page. That's the first Google result at the time of the writing of this post. That page gives the following most voted answer, which got 524 upvotes on StackOverflow:

array.inject(0){|sum,x| sum + x }

And sure enough it works. Here is an example:

$ irb
2.4.2 :001 > array=[1,2,3]
 => [1, 2, 3] 
2.4.2 :002 > array.inject(0){|sum,x| sum + x }
 => 6

The beginner might get that to work, but they don't understand the tradeoffs of that code. Is it readable? Is it fast compared to other ways of doing the same thing? Is it maintainable? Why does it work? What exactly happens when that line gets executed? How much CPU does it use? Are the variables "sum" and "x" defined after that line of code runs?

A beginner Ruby developer is someone who does not know the answer to most of those questions.

Coder - Yellow Belt

A coder is someone who can put together many lines of computer code to resolve a range of simple problems without help. The result won't be beautiful, but a coder understands why it works and they get the job done.

The first necessary step

I called my blog "CoderHood" because every person who writes code for living reached a "coder" level at some point. Coderhood is a word I made up to indicates a developer's life in tech, starting from that first yellow belt.

The main difference between a beginner and a coder is that a coder can write code and understand it. They might not understand what's going on behind the scenes in great details, but they do know why they wrote the code they wrote.

In the industry, the title associated with a coder is something like "jr. developer" or "developer in training."

An example

I expect a "Ruby coder" to be able to come up with and know the difference between most of the following methods of calculating the sum of an array of integers:

$ irb
2.4.2 :001 > array=[1,2,3]
 => [1, 2, 3] 
2.4.2 :002 > array.inject(0){|sum,x| sum + x }
 => 6 
2.4.2 :003 > sum=0;array.each { |x| sum+= x };sum
=> 6 
2.4.2 :004 > array.sum
=> 6 
2.4.2 :005 > array.inject(0, :+)
 => 6 
2.4.2 :006 > array.reduce(0, :+)
 => 6 
2.4.2 :007 > eval array.join '+'
=> 6

In case you are wondering, some of those are terrible solutions, but they are working solutions nonetheless.

Hacker - Jeans And No Belt

I include "hacker" on the list because somebody will ask about it. However, it is not a great fit for this discussion.

Not a necessary skill

I don't believe that "hacking" is a necessary skill in a software developer's growth path. White-hacking skills are useful to learn how to test and secure software applications and systems, but I don't see it as a word describing an overall "skill level." I'd categorized it as something some people do and not as something that gives an indication of technical proficiency. In fact, a hacker's skill level can be all over the place. Some are amazing, while others not so much.

Since being a hacker is not a necessary stepping stone in the software development progression, in my analogy, a hacker does not wear a traditional martial arts belt. They are more an equivalent of a street fighter, wearing Jeans without a belt.

Some of them are angry thugs, others are survivalists, others are good guys defending people, and most are somewhere in between.

The many types of "hacker."

There are many types of hackers. Some of them can code, others can't. The meaning of the word depends on the context and who is using it. Some common definitions are:

A computer expert who adheres to the technology and programming subculture.
An individual who can compromise computer security for malicious (black-hat) or security research (white-hat) purposes.
A software developer who gets the work done in the quickest and dirtiest way possible.
Someone who studies, experiments with, or explores telecommunication systems, equipment, and systems connected to telephone networks. This type of hacker is also called a phreaker.
A skilled engineer who works very close to the metal to gain more control on a system for good reasons (i.e., to squeeze more performance out of the hardware) or malicious purposes (i.e., to exploit security holes and find ways around operating system defenses).

Some examples

Type 3

A hacker of type 3 might choose to sum an array of integers with the following "ruby code":

$ irb
2.4.2 :001 > `echo "1+2+3" | bc`.to_i
 => 6

It works, on some systems at least, but it is a... "total hack." That's the type of thing that unskilled hackers-who-code do. They do things in dubious ways, usually executing unreadable command-line commands until they somehow get the result they want.

Type 5

Hackers of type 5 operate at very low-level and close to the metal. There is something to be said for that skill; it is not easy to acquire and can be very valuable if you are trying to secure a software system or write extremely high-performance applications. I was never what you would call a "hacker," but I started coding very close to the metal (C and Assembly) and I still consider myself a low-level developer at heart.

Type 5 hackers can be fantastic street fighters, with mad skills that would let many professional programmers in the dust on some very specialized tasks. That kind of "hacker" might choose to sum an array of integers using Assembly, like this.

Programmer - Orange Belt

A programmer is someone who can write functioning applications, understands basic algorithms and possesses some computer science rudiments. A programmer can get a product to work, even if it might not be very scalable or maintainable in the long-term. A programmer typically works well alone, and might or might not be a good team player.

The majority of developers stop at this level, especially if they don't plan on studying computer science theory. Programmers can write decent code and work in the software industry at that level for their entire career.

From an industry title standpoint, a programmer is often known with a title of "Software Engineer."

In the simple "sum of an array of integers" example, a programmer is someone who might decide to write the code this way:

#!/usr/bin/env ruby

if ARGV.size==0
  puts "Usage: "
  puts "   sum [space separated list of integers]"
else
  puts ARGV.sum{|x| x.to_i}
end

This code implements a usable command-line command to sum a list of numbers. If you invoke it without parameters, it displays a helpful usage message. Otherwise, it prints the result on standard output. Here is an example of usage:

$ sum
Usage: 
   sum [space separated list of integers]
$ sum 1 2 3
6

It is a "complete solution," it is self-documenting and is somewhat abstracted since you can call it as a command-line command.

Computer Scientist - Green Belt

A computer scientist is someone who studied Computer Science, either in school or on the job and has a good understanding of concepts such as:

Base-N (N = 2, 10, 16)
Binary operations
Boolean logic
Algorithmic complexity & big-O notation
Data structures (arrays, linked lists, B-trees, red-black trees, queues, stacks, hash tables, heaps, sets, graphs)
Sorting algorithms, and when to use them.
- Basic understanding of NP-completeness
Basic multithreaded algorithms
Memory management & garbage collection (just because you code in a language that takes care of memory management for you, it doesn’t mean you can get away with not understanding it)
Pointers (you need to understand the concept at least, even if you don’t code in C or C++ ), and the difference between passing parameters by value or reference.
OOP concepts (interfaces, inheritance, constructors, destructors, classes, objects, abstractions, encapsulation, polyphormism, etc…)
OO design and patterns
Recursion
Some basic understanding of dynamic programming, greedy algorithms and amortized analysis, string matching and approximation algorithms

A computer scientist has a CS degree or has worked as a developer for many years and studied applied CS theory on the job. As you know, I don't believe that it is necessary to have a CS degree to have a successful career in software engineering.

Now, being a computer scientist alone doesn't make you a great programmer, which might seem to break the martial art analogy a little. However, it doesn't. Think it this way: even in the martial arts world, there are specialties. Some green belts are better than others at some things. The progression is non-linear. The belt color often represents a level of experience and the quantity of work that was put in the mastery of the martial art, not necessary a level of skill for every facet of it.

A computer scientist would probably code the sum of integers solution like the programmer did. However, the difference is that a computer scientist would be able to tell you immediately that the complexity of that algorithm is O(n). As mentioned, this is an elementary example, but you get the idea.

Software Developer - Blue Belt

A software developer is someone who can get larger and more complex projects done. In comparison to a programmer and a computer scientist, a software developer:

Writes code that is cleaner, better designed, more maintainable, documented and readable.
Writes fewer bugs.
Works faster.
Works better in teams, and understand the value of development processes.
Is stronger at finding and optimizing bottlenecks of code and software systems.
Has more experience (has been there and done that).

An example

In the simple "sum of integers" example, a software developer is someone who might choose to solve the problem by building a service that exposes a Web API. The API would take a set of integers, and return the sum.

I expect the application to be well documented and configurable, at least. It would also have tests, proper source code organization, and it would be easily maintainable by other developers.

The main application might look something like this in Ruby, using Sinatra:

require 'sinatra'
require "sinatra/config_file"

# Load the server configuration from config.yml:
config_file 'config.yml'

#
# EndPoints:
#
# /sum/n1+n2+n3+n4+...
# Return:
#     {result: [sum of n1,n2,n3,n4,...]}
#
# Example:
#    $ curl http://localhost:8080/sum/1+2+3+4
#    {"result":"10"}
#
get '/sum/:numbers' do |numbers|
    {result: numbers.split("+").collect{ |x| x.to_i}.sum}.to_json
end

A good software developer would be well aware of the many limitations of this solution compared to other ones. For example, this solution is limited to performing the sum of a set of numbers that fits in the URI; it has no explicit error checking, it interprets as zeros strings that do not start with a number, etc.

Software Engineer - Brown Belt

The difference between software developer and software engineer is arguable; I admit it fully. The two terms are typically used interchangeably. However, I am going to suggest that a software engineer is someone who has the CS knowledge of a computer scientist and a lot of experience as a Software Developer. The major differences are:

Ability to create more scalable systems.
Longevity of their work. Their work lasts for longer and has fewer problems.
Fewer bugs, and better code quality.
Ability to be a technical lead for projects and teams.
Possesses excellent collaboration and communication skills.
Enough software architecture skills to be able to get the job done.

In the industry, you'd find this kind of developer with titles such as "Senior Developer" or "Principal Developer."

An example

A software engineer is someone who might choose to write the "sum of integers" application by building a service and exposing an API to take a set of integers as a developer would. However, I’d expect the software engineer solution to include improvements such as:

Result Caching.
Abstraction of the concept from a sum to any mathematical expression passed in the request.
Include logging, authentication, monitoring hooks, etc.

This example gets silly

As you can tell, the simplicity of the example I chose becomes problematic and even foolish. It morphs into a discussion of how to improve an already over-engineered solution for a trivial problem.

For example, managing a cache of the results of simple operations performed on a small set of numbers is most likely heavier and slower than making the calculations in the first place. It would make sense only if the set of numbers you could pass to the API were huge; but in that case, however, the list would not fit in the URI of the request.

You could move the list of numbers to the body of the request, but then it would no longer be a RESTFUL API, and the request would no longer be cacheable. At that point, you'd be tempted to change the request to be a POST, but that would make it never cacheable. Anyway, the discussion could go on and on.

The critical part

Do you see what's going on here? As the skills of a developer improve and projects become more complex, a funny thing happens. The hard choices become less and less centered on the "core code." Instead, they become more and more centered on handling the context in which the core code operates.

As a result, highly skilled software engineers spend the majority of their time refining aspects of a system such as scalability, performance, error checking, robustness, maintainability, abstraction, portability, handling of edge conditions, etc.

Also, they learn ways to work more efficiently or improve how they interface with other developers and how they plan to approach the work to minimize risks, etc. Building software becomes less about coding and more about engineering systems and solving problems.

Software Architect - Black Belt

All software developers and software engineers need to be able to design the parts of the systems and products they are going to build. A "software architect" brings that skill to a higher level, and needs to be able to make design choices on the high-level interactions of larger software systems developed by other engineers.

An example

In our example, among other things an architect might draw this diagram to guide the implementation of the service to sum a set of integers:

Muscle memory and leadership

To be able to do so effectively, a software architect must have been trained by years and years of experience in the weeds, working at all levels. That hands-on experience must have become embedded in their muscle memory. That allows an architect to make right high-level decisions without getting stuck in details.

That said, I do not believe in pure architects, that is engineers in charge of making high-level decisions full-time. Instead, I think that reliable architects need to jump in and out the details, staying at a high level when necessary, ready to dive into the code regularly and efficiently.

In martial arts a black belt teaches and mentors. I think that teaching and mentoring is also one of the functions of a software architect. The teaching I am referring to is not direct (lectures), but is more done by example, showing the way and guiding people to make their decisions.

Conclusions

Serious martial artists study their art all of their life; serious software developers do the same. I hope that you found this discussion useful. Hopefully, it gave some context to some terms that are often poorly defined and ideally it explained how to use common terms in more precise ways.

If you enjoyed this article, keep in touch!

Find all my posts on CoderHood. Don't forget to Subscribe to receive notifications of new posts by Email.
Join my professional network on LinkedIn.
Follow me on Twitter.
Join my Facebook page.
And finally, follow me here on dev.to!