Software Testing in a Startup

Part 1/2

The speaker discusses the significance of testing in software development, emphasizing its role in ensuring product stability and avoiding issues in deployment. It acknowledges the expense and time commitment required for comprehensive testing and recognizes the challenges faced by smaller teams or companies with limited resources. Despite these constraints, practical strategies are proposed for maintaining stability, including keeping code pieces small and mutually exclusive, building and testing features independently, and implementing processes for easy rollback. The importance of striking a balance between testing efforts and product development timelines is emphasized throughout. Overall, the narrative underscores the necessity of testing while offering pragmatic approaches for achieving stability even in resource-constrained environments.

Podcast

Check out on Spotify.

Summary

Introduction to Testing:

• Importance of testing across various scenarios and types of applications.
• Acknowledgment of the expense and time-consuming nature of comprehensive testing.
• Emphasis on the benefits of testing for avoiding regression and deployment issues.

Constraints on Testing:

• Not every team, company, or individual can afford extensive test coverage.
• Analogies drawn to illustrate the gradual process of achieving desired levels of testing.
• Acknowledgment of the need to balance testing efforts with product development and release timelines.

Strategies for Ensuring Stability:

• Maintaining stability without extensive automation.
• Importance of keeping code pieces small and mutually exclusive.
• Strategies for building and testing features independently, even within a complex architecture.
• Ensuring that issues with new features do not impact existing stable components.
• Implementing processes for easy rollback and resolution of issues encountered during deployment.

Transcript

0:01

Hey there, hope you're doing well. In today's video, let's talk a little bit about testing and the need for testing, how we could do it in a variety of different scenarios, what we can afford to do, what we should do and so on and so forth.

0:18

So, long story short, we all know how important testing is, right? I mean, we couldn't ever overstate the importance of writing test code and the more automation you have, the better off you're going to be, right?

0:35

So, whether you're building mobile apps, web apps, PR server-side components, whether they are microservices or whether they happen to be monolithic apps, it does not matter. The more test code you have, the more automation you do and the more frequently you run them, make them part of the build process, the better off you're going to be.

0:57

Right. So, having said that, so if that's the case, what's the purpose of this video, right? I'll explain why. Now, having all of those tests is the way to go, but it's expensive. I mean, getting I don't know what the actual math is, but I want to say for every line of code that you write, I think it's like one is to four.

1:18

You might need to have like 4 lines of test code on an average to make sure that all of your paths are traversed in the code. Which means that if you have a million lines of code, obviously you need like 4 million lines of test code or something of that nature. Take that mat to the grain of salt, but it gives you the idea.

1:36

Which means it's going to be very expensive and time consuming, at least when you write the test. But hopefully they'll save you time because you catch, you don't end up having regression, you don't have deployment issues, and you don't find stuff in production, right? So it's good to have it.

1:51

Obviously there's nothing more to say there, but the purpose in me making this particular video is not every team or company or individual can afford to have the kind of test coverage that they desire, at least to begin with.

2:07

We almost have to work our way to earning to be in a position where we can write tests at that level, to have that kind of coverage. It's almost like I live about 40 miles to the West of DC. We like where we live, but we would obviously like to live closer to the city, hopefully one day in New York City, but till then.

2:31

But to get every mile, you get every mile closer to DC that you want to be, it's going to cost that much more, right? So sure, we want to be there, but it's going to take time. So I don't know how great an analogy this is, but back to testing. The more you have, the merrier. But not all of us can afford to do it because we are small teams, small companies start-ups.

2:52

We want to get more things out there in production. So how can we not have as much testing as we would like to yet make sure that your product is stable? So that's the purpose of this video. It probably has taken me 3 minutes to get to the purpose. Clearly brevity has not been my four day.

3:09

I'm not going to deny that. Now let's talk about and as a minor distraction. The weather is like beautiful today. It means a lot of days are beautiful and today is no different. It's nice and cool. I'm just out here and hopefully you'll pick up a lot of this audio.

3:26

There's a new camera, so hopefully it's not too bad. Anyways, so how can we do it? Let's take an example. Right. So work for the work that we do for our own product. Again, if you haven't checked us out, do so. Go to snowpal.com, download the app on the Play Store or App Store and stay organized.

3:44

Manage it for projects and for just about everything in your personal and professional life and let me know what you think. Sorry, there's a bit of noise there. It's nice to see. We still have planes flying and hear the sound as much as I used to, but hopefully we get there soon enough, right?

4:04

OK, so back to the core purpose of the video. I'll take one example. Now we have a web app, we have a mobile web app, we have a native mobile app. So we have plenty of clients that you interact with and on the back end we have a number of servers, right?

4:25

We have an NT architecture. So there's a lot of moving parts. Now if you want to write test, all you know, enough tests or tests at all for what we have to do, it's it's it's a lot of work, right, because we have a lot of features.

4:41

It's a pretty rich platform and it's pretty fluid because we keep adding more features, more enhancements and we are a pretty small team. So here's what we do to ensure stability till we get to the point where we can have these fancy yet very, very needed automated test suites, which unfortunately we don't have at this point, right.

5:03

But again, we still maintain a sense of stability without a whole lot of automation and that doesn't happen magically, right? We have to make an effort to making that happen. Here are some ways that we've done it, and I'm sharing this with you. So if you're a smaller team and very ambitious, as ambitious or more ambitious than us, then you could probably follow these approaches and see how it goes for you.

5:26

Let me know. One thing is keep your pieces as small as possible right now. Again, no matter how small you keep them, you'll end up in hindsight, you would you would feel like you could have kept it even smaller, but at least make an effort to keep your pieces as small as possible and as mutually exclusive as possible as well, right?

5:49

So if you're building again, short of saying the word microservices because it could be misconstrued in a few different ways, even if your pieces are not independently deployable, make sure they're at least independently buildable, right? And here's what I mean by that.

6:06

When you build your features, keep them, even if they if they live in a separate repo, that's better, right? We do it as frequently as we can do that, but that's not always possible yet. We don't want to have single monolithic applications either.

6:23

So regardless, make sure that that piece that you're building, that feature is independently built. Now the dependencies may not be sorry, there's more noise, their dependencies may not be entirely independent. So if you're introducing like, say, a charting library right into the mix, sure you might.

6:42

You would be sharing the charting library with a number of other features. So the dependency

in itself is not independent, right? It's it's built into your system. That's all right, because then your deployment, you don't have to worry about how you minify it, how you package it, how you deploy it, and all of that stuff.

7:01

You do it once and you're good to go. You just need to keep upgrading the versions. But when you build your features that end up consuming our leveraging these third party libraries, make sure that your pieces are built very independently.

7:16

And again, they don't have to be independently deployable because that takes an extra piece of effort. Even if they're deployed as part of the same artifact, make sure that they are actually they can be coded to and tested. Whether you write automated tests or otherwise.

7:34

Make sure that they can be built and tested independently and in a mutually exclusive manner from all of your other pieces. To take it one step further, if you make those changes, build a feature and for whatever reason you don't like it or you, you've run into some issues when you deploy the production and you have to roll back, it should be as simple as flipping an environment variable to false so the entire feature does not show up right?

8:02

And the reason I say that is sometimes you can't wait forever to get your features out there. So you want to, you have a timeline in three weeks time. You want to get something new and cool out there, do it as well as you can. But if you do run into issues, and hopefully you don't, if you do run into issues, that should not, for one, for starters, cause any regression to any of your existing pieces that have been in production for a while and I've gotten sort of solidified and they've become well-oiled machines.

8:35

Any problems that you do notice when you deploy and implement a new feature should be isolated to that new feature, because then it's as bad as it might sound. It's not so bad because you can now fix that and then push your next deployment and make sure your deployment processes.

---

Part 2/2

The discussion delves into the critical aspects of deploying and implementing new features, emphasizing the importance of isolating issues to the newly introduced elements rather than causing disruptions across the entire system. Challenges arise when issues are discovered post-implementation, potentially leading to regression and user dissatisfaction. Despite limited automated testing, efforts are made to maintain stability, evident in numerous deployments to the web app despite architectural complexities. The narrative underscores the necessity of efficiency in development and deployment processes, drawing parallels with the restaurant industry's imperative to uphold credibility. Ultimately, stability and user satisfaction remain paramount, urging continuous efforts to streamline processes and uphold credibility in software deployment.

Podcast

Check out on Spotify.

Summary

Introduction to Deployment and Implementation

• Discusses the importance of isolating problems to new features during deployment.
• Highlights the significance of a seamless deployment process.

Challenges of Identifying Issues

• Emphasizes the potential consequences of finding issues after implementing new features.
• Outlines the problems caused by such issues, including regression and user dissatisfaction.

Efforts to Maintain Stability

• Discusses the commitment to maintaining stability in the product despite limited automated testing.
• Mentions the number of deployments made to the web app and the complexity of the architecture.
• Acknowledges the challenges but expresses satisfaction with the progress made.

Importance of Efficiency and Stability

• Reinforces the message of not letting testing limitations hinder efficiency in building and deploying new features.
• Draws an analogy with the restaurant industry to emphasize the importance of stability and credibility.

Conclusion

• Summarizes the importance of stability and maintaining credibility.
• Mentions the examples provided and hints at additional strategies for stability maintenance.

Transcript

0:00

Any problems that you do notice when you deploy and implement a new feature should be isolated to that new feature, because then it's not as bad as it might sound. It's not so bad because you can now fix that and then push your next deployment and make sure your deployment process is seamless and easy enough so it doesn't take you several hours, only a few minutes to deploy the latest changes.

0:23

But what will become really bad is if you find any issues when you deploy a new feature after you implemented, obviously. And then you start finding issues that it has caused in other pieces of your system. And now that's a big problem because one, you broke what was working, so regression. Two, it's a wasted time, effort, and money.

0:44

And three, your end users and end users are not going to be happy, right? So firstly, we go all out. We go out of our way as much as possible in our offering in our product to make sure that we keep it stable. And doing it in the context of limited automated testing is pretty challenging.

1:06

But I want to say we've done a fairly decent job so far. We've had like close to 168 deployments or maybe 169 deployments to the web app, which means we have multiple API repos that work alongside the UI apps. You can just multiply the number of deployments at least by two, and then obviously we have the native mobile app, right? So again, architecture is quite interesting just because, you know, we've built more and more pieces. I'm actually very happy to see planes fly a lot more than these days.

1:47

Yeah. So yeah, this video is getting a whole lot longer than I wanted for it to be initially. Sure, I'm sure I rambled quite a bit, but I want to convey the message saying that don't. Even if you are not, even if your test suites are not great or not there.

2:06

Don't stop that from making you be efficient in terms of building new features, building new apps, new components and deploying them right. Because from an end user standpoint, they don't care how you do it, but they expect the system to be stable and it's very, very important.

2:24

And just like a restaurant has to be clean and has to sell good food at all times throughout the year because building the credibility is hard, losing it is easy, right? So I think it's important to make sure whatever it is that works for you.

2:41

You want to keep it stable. Here. I'm just giving you a few examples of how we've gone about doing it. There's plenty more that we do to maintain that stability.