DEV Community: Spicer Matthews

Prioritizing TDD Friendly Languages and Frameworks

Spicer Matthews — Thu, 23 Mar 2023 16:38:30 +0000

Unit testing and Test-driven development (TDD) are crucial elements of software development. Unit tests ensure individual code units function correctly, while TDD emphasizes writing tests before writing code to meet expected requirements.

As someone who enjoys TDD but dislikes writing unit tests, I appreciate the importance of testing and value an efficient development process. Unit tests should make development more enjoyable and not be an extra burden. They should be tools that help developers work faster, rather than additional tasks.

"Said a different way, unit tests when written to speed up my development process are amazing, but they are super annoying when written as a chore."

When choosing a programming language or framework, prioritizing those that are easy to use with TDD is essential. Golang is excellent for TDD when developing web apps because testing is simple and fast, making the process more efficient. In contrast, testing in Javascript can be tiresome, particularly on the frontend, where writing unit tests often feels like a task rather than a helpful tool.

Changing personal development habits, such as moving from browser refreshing to running unit tests in PHP can pay huge dividends, despite taking time to adjust. This shift may not necessarily make one a faster developer on day one, however since PHP is fairly easy to test the benefits will be noticeable over time.

I was prompted to write this post because I've noticed that many projects choose programming languages that are difficult to test, or new technologies that are associated with hard-to-test languages. For example, Node.js is a popular choice for serverless computing, even though it is harder to test than some other options. This has always confused me. The argument that Node.js is great because it is the same language on both the front and backend seems weak. What happen to picking the correct tool for the job?

Of course, there are pros and cons to any technology decision, and it's not always possible to prioritize testing over other factors. However, I believe that developers should give greater consideration to languages and frameworks that facilitate TDD, as this can ultimately increase productivity.

In short, unit testing should not be viewed as a chore, but as a valuable tool for developers. By prioritizing languages and frameworks that make it easier to write and run tests, we can improve the quality of our code and enhance the overall development experience.

Nitpicking Code Quality: A Necessary Evil or a Harmful Habit?

Spicer Matthews — Thu, 23 Mar 2023 16:36:45 +0000

I have a close friend, whom we will call Frank, who has been both a work colleague and a close friend for years. However, there are some aspects of code quality on which Frank and I strongly disagree. Before delving into this further, I must admit that there may be something that I am overlooking. Frank has had a long and successful career working at some of the top tech companies. Even to this day, he has a long list of tech companies vying for his services, while none have ever approached me :(.

This post is not about Frank's "dumb" ideas versus my "brilliant" ideas. Instead, it's about the difference between the big corporate mindset and the small business mindset.

To illustrate this point, let's use an example. Frank strongly believes that not properly ordering your imports at the top of a Golang file is a sign of poor code quality. In contrast, I do not think that it's that important. I am not saying that correctly ordered imports are not important; it is. But, in my opinion, they are not worth getting hung up on. Frank has blocked many of my PRs in the past due to such nitpicky issues, arguing that these things are vital to code quality. If not addressed early on, they can snowball into more significant problems and slow down developers.

Yes, your IDE should manage your imports for you. Once again this is just an example, and IDEs are not prefect. I have failed many PR's because my IDE did not manage the imports properly.

While I am using Golang imports as an example, I am highlighting all sorts of nitpicky code quality issues. Poorly designed code is undoubtedly a showstopper that can snowball into more significant problems and should be addressed at the PR level.

My biggest argument against nitpicky style issues is that it demotivates developers. When a developer is in flow, you want to maximize their focus on the problem they are trying to solve. When a software developer finishes their work and is eager to share their work with the team, slowing down the cadence through nitpicky PRs ruins the excitement of the bigger task at hand.

In my observation, the difference between a normal software developer and a 10x developer is how giddy they are about their craft. There is a reason why we do not see tons of blog posts, Twitter conversations, side projects, and so on from big tech employees. I think working in big tech may ruin the giddiness of a craft that many of us learned alone at the terminal.

While being nitpicky about code quality is textbook proper coding, it neglects to take into account the second-order effects of developer satisfaction and the importance of keeping a developer motivated.

You may ask how not being nitpicky in the early days will not snowball into unreadable code. I combat this by subscribing to a philosophy that you should never open a file without leaving it better. During my development cycles, I often browse code, maybe even code I have never written, to understand what I need to do to complete a task. It is during these non-flow states that I might notice things like import orders not being proper, and I'll fix them. If everyone on your team has the philosophy that they never open a file without leaving it better, code quality will naturally improve over time.

If you're a big tech company with a large budget and many developers, you may be able to afford not to groom every software engineer to be a 10x engineer. However, if you are a small or medium-sized business, you cannot afford not to have your developers excited and in the flow every day.

While Frank and I would agree on almost every software development principle in a perfect world, for me, the perfect world comes at a cost that I don't believe is worth it for most projects.

Let's chat on hacker news.

Spicer's Rules on How and When To Test Your Code

Spicer Matthews — Mon, 16 Sep 2019 02:15:11 +0000

I just marked my 20th year as a professional software developer - meaning I received my first paycheck for writing code 20 years ago. I have to admit it has been a wild ride of successes, failures, crazy ideas, and immense learning. Throughout this journey, the topic of unit tests, or any kind of testing for that matter, has always been front and center for me. This topic has led countless barroom debates and even more IRC fights - yep, showing my age here. After some reflection, I think I have finally settled on my rules for how and when to unit test through the lens of business objectives.

As a side note, this post is inspired by a podcast I recently listened to: 103: Code Quality and Balancing TDD. Co-host Ben Orenstein, by all accounts, has had a very successful software development career, and recently he became CEO of his new start-up Tuple. In the podcast, he describes how as a software developer in the past he was hard and fast with his unit testing rules, always insisting on a strong testing policy with any project he worked on. He mentioned now that he is a CEO and is balancing time, budget, and progress, and he has taken a more subjective approach to unit testing. While he did not go into great detail, this has been a topic I have been thinking about for a while now. This podcast has encouraged me to write down my thoughts on how to balance good software development principles with business goals through the lens of code testing.

Rule #1: If you have money, time, and passion, unit test is everything.

The preaching of people in the “test everything” camp always brings me back to my college days listening to a professor teaching what you “should” do every time in software development. Many of my professors taught me that always testing everything with 100% coverage with no exceptions was the correct thing to do. Yeah, in a textbook that makes tons of sense. Professors will often point out that you will write less buggy software and because of that, your users will be much happier. I agree with these concepts under 3 conditions.

The software you are working on has an unlimited timeline. Testing everything requires time. If you want full and complete coverage it is not just a few minutes here and there.
Your project has a big budget. Time and money tend to go together. If you want true 100% coverage with matching integration testing, be prepared to spend money on paying highly paid developers.
Passion for testing. This is a topic that is not talked about very much. If you are a software manager and you are non-negligible on testing and your team gets a little burned out writing tests for frivolous things, then your team is going to be disengaged or this will lead to a revolving door at your company. The act of demotivating a talented software developer is far worse than not having 100% coverage.

If you have unlimited time and money, and your team is nutty for testing, I say go nuts! If not proceed to Rule #2 :).

I do acknowledge there is certain software that should be 100% tested. Maybe 200% tested. Software running self-driving cars or satellites should be tested no matter what as bugs lead to expensive or deadly mistakes. Just be careful not to convince yourself that your little SaaS app is as important as self-driving car software.

Rule #2: If you have a monolithic app with a big team, invest in testing.

If your software application is big in scope and is designed to be a monolithic application developed over many years, testing becomes important. Testing becomes even more important if you have a big team of developers working on it. It is very easy for some developers to unknowingly break some features just because they did not recall the original requirements of the functionality.

Pretty simple rule: More people + bigger codebase = more testing required.

With that said, IMHO, monolithic applications are pretty old school. Yes, there are some pros to a big monolithic application but there are plenty of cons when working in a big team. Jeff Bezos has the 2 pizza rule, I think this is a great concept to consider when building software teams.

If you have a big application built by many developers, consider microservices. Break up the monolithic app into smaller modules or separate applications. Given the smaller nature of the codebases, testing might be able to be relaxed. Often, a manager might decide to invest in testing more important microservices and relaxing on less important services. With monolithic applications, we tend to overgeneralize when it comes to setting testing policies.

Rule #3: Test-driven development is a great win-win.

Test-driven development (TDD) is the process of writing your tests first then writing your code. Everyone does this differently. In my world, I consider TDD a change in how you develop where you use your unit tests to help develop your code.

Maybe my TDD approach is better described in an example. Say you are building a web application. Traditionally you would write some code, compile your app, then hit refresh in the browser to see if your code worked. The process of recompiling and refreshing can be time-consuming. A different approach might be to write some tests, write some code, then run the test. You can use the test to debug certain things as you develop giving you the same level of verification a browser refresh gives you.

With test-driven development, a 3 step development processed could be reduced down to 2. The result can be the time you waste compiling and refreshing could mirror the time it takes to write unit tests instead. At the end of the day, you have spent the same amount of time developing but with TDD you also end up with unit tests you can run forever to verify your app continues to work as planned.

"With test-driven development, a 3 step development processed could be reduced down to 2."

By using a TDD approach you will most likely get a big chunk of your tests written without any additional time spent. Managers that keep a strong eye on timelines and budget love this approach. It is a great hybrid.

Rule #4: Don’t waste time testing stuff that is hard to test.

Some programming languages are hard to test, looking at you Javascript! In languages like Javascript, I find the TDD approach I mentioned in Rule #3 simply does not work. There is no way to use your tests as a way to support your development. I find developing the software to be one distinct task and writing tests to be another distinct task. This often doubles the development time.

Often these difficult to test languages such as Javascript tend to be UI based. There is so much to test from HTML to CSS to the Javascript logic it is nearly impossible to get 100% coverage in the sense you are testing for every possible outcome. Testing can lead to a false sense of security at the price of doubling your development time. To me, it is not worth it.

It is always good to have some tests in place. So with frontend development, I often suggest integration tests vs unit tests. Test the final product, not each method. Yes, I am fully aware a professor would tell you to do both, but it comes down to time and money. I tend to think in terms of “bang for your buck”.

Lastly, I do not think unit testing poorly designed languages is something you should never do. I tend to think you should add unit tests as you fix bugs or make minor tweaks. I promote skipping the unit tests during the core feature development. See Rule #5 for more information.

Rule #5: When you open a file always make it better.

I think David Heinemeier Hansson originally put this idea in my head. That every time you open a file you should make it better. It is very common to develop a core feature, deploy that feature, and then come back weeks or months later and need to make modifications to that original code. Maybe a bug popped up, or the feature needs to be tweaked. David suggests that every time you open a file you make it better.

"That every time you open a file you should make it better."

When you make a file better you might review the code for style, better comments, or todos that never got finished. I much prefer this approach over a hardcore code review process. Getting hung up on small things like style takes away from the core engineering task when originally developing the feature. I will post more of my thoughts on code reviews in another blog post. For now, this concept of always making a file better applies to testing as well.

Whenever you circle back to add small features or fix bugs you have fewer balls in the air meaning when you are building a feature from the ground up, there is a lot to consider vs when you’re fixing a small bug, there is less to manage. Since you have less on your plate why not take a few extra minutes and add a few more tests (or in the case of Javascript, write your first test)?

My general rule is when a bug is reported, or a feature requires a small tweak, I will take the extra time to add a test for that. Just a little insurance that this bug will never happen again.

This incremental approach to increase test coverage at the end of the day might cost the same as doing it all at once but think about it in terms of cash flow. When you are a start-up you just want to get out the door so you can get some customers paying you. If you get out the door faster and earn revenue you can then reinvest that revenue into these incremental changes. Often, time matters when making decisions to invest in test coverage, good software managers always take this into account.

"This incremental approach to increase test coverage at the end of the day might cost the same as doing it all at once but think about it in terms of cash flow."

Rule #6: Test using the services you use in production

A unit testing zealot will tell you every test is its own "thing" and should not have dependencies on other things. Sure, this is a nice non-subjective approach to unit testing, but it often leads to extra work to support.

A great example would be many people will use SQLite as a database when unit testing, and a MySql database in production. The idea is you can spin up SQLite on the fly since it is just a file vs starting a MySql database. Yes, this isolates the test to have no other dependencies, but I have seen it have more issues than they are worth.

If you are using MySql in production, you should run your unit tests against MySql. SQLite is not the same as MySql and the subtle differences will haunt you and add more time to the development process. With docker containers for everything these days, you can almost always find a solution that is close to being “pure” unit testing and match what you run in production.

Rule #7: Don’t be a unit testing purist

The unit testing purist out there will call me out on this, but I simply disagree. Don’t be afraid to put “if testing” code into your core codebase. Of course, you should not do this often, but if you are a senior software developer with good judgment, go for it.

It might be best to explain this with an example. In Golang you can fire up a goroutine (which is more or less a thread) by putting the keyword “go” in front of a function. This is handy for backgrounding the process of sending a transactional email (since the user should not have to wait for that to go through). The problem is the unit test will likely finish before the thread finishes and you have no way of testing if there were issues with that transactional email. I think it is ok to simply put an “if testing” statement in front of it and while we are unit testing, so we do not run the method in a different thread. Pretty minor, but it saves writing ugly code just to make the code more testable.

Of course, if you have a number of junior developers on your team it may be best to have less subjective hard and fast rules about testing code. It can get out of hand fast with poor judgment.

Textbooks will argue there are rules on how you should unit test. Don’t be afraid to break those rules if they make better business sense for you and your project.

I would remind you that the “book on testing” was written a long time ago. When shipping code was expensive and could not happen on the fly. These purist rules were smarter then because repercussions were greater if buggy software was shipped. In the modern web era if a mistake got shipped it could very quickly get fixed. Hence, converting textbook rules of unit testing to mere suggestions and applying your own judgment can pay big dividends in terms of cost and time.

Rule #8: Take a top-down approach to testing.

Testing nerds will say what I am describing is integration testing, not unit testing, and good developers do both. Whatever you want to call it, there are only so many hours in a day and I think you should focus on the low hanging fruit first.

Most apps I build these days are REST API backends. Meaning front-end applications call some HTTP endpoint and JSON data is returned. Normally, many sections of your application are called with every endpoint. For example, the following functionality might be called: authentication, controller, model, database, helper libraries, user analytics, and so on. You could unit test every method called in one HTTP endpoint request, but I say that is cumbersome.

With a less cumbersome approach, you could write tests that make a call to the endpoint and verifies what it returns. By default, all the functions mentioned above are tested as well. If any of them fail, the endpoint will not return what is expected.

I can tell you by taking this approach you give your application a massive amount of test coverage. However, your test coverage tools will tell you that you have low coverage, and your purist unit testing friends will tell you that you are getting a false sense of security. I claim you are getting a massive bang for your buck. If your goal is to build software that makes money, this is a great place to have some savings.

To my point in Rule #5. I do think you should add unit tests further down the stack in a more isolated manner. Just do them over time. When you circle back to make updates adding more isolated tests as you have time, it can’t hurt. I just don’t think you need them on day one.

Summary

Many reading this might jump to calling me an amateur claiming my relaxed approach to testing is not what real developers do. To my point in Rule #1, if your business case supports unwavering from all forms of testing, great. I agree this is best. Personally, I have never worked on a project that would not benefit from applying some judgment vs. sticking to hard and fast rules.

I have never worked for a big company like Google or Apple. I know these companies tend to do things by the book with little exception. Maybe on their scale, it is required, but as an outsider looking in it seems their innovation has slowed a great deal since the early days. However, a company like Amazon, which is known for rapid innovation, continues with no decrease in pace. It is my thought Amazon thinks outside of the box when building software. I have no idea what Amazon’s unit testing policies are, (I am sure they are rather good), I do think they optimize their development teams whenever possible throwing out standard rules and building custom rules for their business goals.

My major point here, while using testing to illustrate, software development should proceed in conjunction with business objectives. Often, software developer ideologies and business goals run in conflict with each other, and I believe they should be in sync. Testing is just one area where synchronization is important.

Hello World: My New Blog, Hugo, Tailwind CSS, and More

Spicer Matthews — Fri, 26 Jul 2019 02:01:03 +0000

Today, I am launching, once again, my personal blog. Starting over. Hopefully, this is my last reboot (more on that below).

I could not even begin to count the number of blogs I have had in my life. Currently, I blog for Options Cafe, Skyclerk, and Cloudmanic. However, in the past, I have had all sorts of personal blogs. Just for fun here are some of my past blogs.

All my blogs in the past have died for 2 reasons. Keeping up with the site updates were a headache, and many of my posts were better suited for company blogs.

Maintaining a Blog is a Headache

Yes, I am someone that runs websites for a living but the headache of keeping my personal site up is often annoying. Prioritizing a site that does not bring in income just never happens. It could be months between blog posts, and without constant updates content management systems grow to be out of date.

In fact, as I write this I can’t access the Cloudmanic blog because all I get is PHP errors when I try to login to the admin panel. Yes, at some point I will take 30 minutes and fix that -- it is just off-putting when you are in writer’s mode and your tools are broken. Over the years, the site neglect led me to just taking down my sites and starting fresh -- always with the latest and greatest in content management systems.

The main headache of running a content management system is staying up-to-date. Many content management systems run on PHP. Just keeping up with PHP versions can be a headache. Not to mention many versions are no longer supported.

As a side note for my business blogs (with the exception of the one currently spitting out PHP errors), I am using Craft CMS. I think Craft is one of the best CMSs out there and so far keeping my sites up-to-date has been a breeze. Since they are business blogs the effort to maintain the site is well paid for.

I am launching my new personal site with the following goals:

To stay up regardless of technology -- no need to upgrade any CMS system.
No complex hosting (ie. no PHP, Mysql, and so on to support).
Something I build from the ground up. No need to hire designers.
Something that will last me the rest of my life.
To try to meet my goals I decided to go with a static site hosted on Github pages. My final product will be nothing more than HTML, CSS, and some images. What is amazing about this is I do not need to update any software under the hood, I do not need to deploy any databases or particular webservers.

I chose to use Hugo. Hugo is just brilliant software. It is written in Golang and is super easy to use from the command line. Among other things, Hugo solves the problem of compiling shared HTML into final HTML files. For example, each page of my site might have a header and footer that is shared among all pages. With a static site, we do not want to copy this shared HTML to each file we need a tool that upon complication generates complete pages. Hugo allows me to build partial HTML snippets that will be added to each new webpage. I am really into the perks of Hugo and hope to post more on this in the future.

A Site I Can Design Myself

Every website I have ever built hiring a designer to design the site, and an developer to convert that design to HTML was part of the process. I am someone who is proud to put my name behind a nicely designed site. However, this becomes a roadblock at times because my design and HTML teams are often busy working on business-related sites. Sidetracking to a personal site is not a productive use of time. With that said one of my main goals is to build a personal site that I can build from the ground up, relying on no other help.

Since I am not a designer, nor a good HTML / CSS developer, this site might not win any awards but I think I can meet my goals of having a relatively nice home on the Internet for myself. To help with design, and HTML / CSS creation I have decided to use Tailwind CSS. I can not speak highly enough of Tailwinds CSS.

They say Tailwind is tomorrow’s Bootstrap and I could not agree more. Bootstrap is amazing, but I think its time has passed. You can build a great website with little effort with Bootstrap but that site looks like every other site on the internet (since tons of sites use Bootstrap). You can customize Bootstrap but it tends to be a fair amount of work and not very straight forward. I find Tailwind to be much more straight forward for building a unique site.

Furthermore, I find Tailwind to be more non-designer developer-friendly. It comes with nice increments in spacing out of the box, good colors with shades, and much, much more - making it so a non-designer does not shoot himself in the foot. Another nice feature is it allows the developer to tweak things very quickly within the HTML. So with little effort, one can experiment until a perfect design is found.

I am sure there are standards of design regarding paddings, fonts, margins, and more that would be second nature to a good designer. Tailwind helps a developer like me produce a nice design without knowing those rules. I really hope to share my journey with Tailwind CSS more as I play with it more. As of right now, this site is my first attempt to use it.

Going Forward With This New Site

As I mentioned above I blog on a few other blogs for business reasons and will continue to do so. For everything else I have to say I will now post here. My guess is I will likely blog tech-related stuff here but also could post my random thought of the day as well. All I know is I would like to build a site that stays with me forever. You can find lots of my random thoughts on Twitter and Facebook but those thoughts drift off as people’s timelines fill up.

Thanks for reading. Please check back in as I build up this site more, and share more thoughts.