A Primer on Software Development and Its Frustrations

Keith Axline — Tue, 10 Nov 2020 17:46:35 +0000

I wrote the text below for a client and I thought it might be useful for others to use. I'm releasing under the MIT license.

If you are ever working with clients who are new to software development this can help properly set expectations and get everyone on the same page. Feel free to edit as you wish and send PRs if you think others would benefit from your changes.

It’s an open secret among software developers that there is some mystical entropy that infects all projects. Unlike other hiring situations – like hiring a plumber to fix a leak – hiring someone to build software has too many variables to contend with.

It's not pipes and fittings and building codes – it’s words and colors and interactions. It's complex systems talking to other complex systems, possibly ones with which the developer has no prior experience. All to please the moving target of business needs and human desires. It’s a mix of customer service, art, and engineering – all wrapped up in one.

The job is done not when the water stops leaking, but when the client is satisfied enough with the state of the open-ended project that they declare a success, or at least enough success for now.

Most people who hire plumbers have tried to fix the leak themselves or have had some sort of first-hand experience with the problem in question. Very few purchasers of websites and software have attempted to do what they’re asking the developer to do, and that lack of experience is a big source of suspicion and doubt. There’s no visceral way for the client to evaluate a developer’s performance.

Another thing that might not be clear to the inexperienced: It’s impossible for the developer to account for all the variables before development. There are several reasons for this. For one, the people and business needs the project is supposed to serve continue to change while the development is happening. And unlike a building, the plans are 'finalized' as the software is built, not before hand. It's the dark side of software's instantaneous and potentially limitless nature. There are no real physical constraints to rein in a project.

Also, desires change not only in parallel to development, but in response it. Problems are uncovered. Problems are created by accident. You might as well try to model reality itself. At the beginning of the project, the client (understandably) wants the developer to see all the pieces and put them together in her mind, and then tell the client if there will be any problem. That’s essentially telling the developer to completely build the website in her head before she even gives an estimate.

This state of things just doesn’t seem right, to either side. And it’s hard to point to the exact source of this entropy because, like climate change, it’s roots are complex, hard to define, and it creeps in too slowly for many people to notice.

When I was a more junior developer I thought that people were using this uncertainty as an excuse to not work hard enough or to cover up their own ineptitude. It took dozens of projects going over budget and off the rails – despite great developers working on them – for me to believe it.

Developers themselves are to blame for much of this expectation mismatch. We tend to think that we can do things super quickly and give outsiders the false sense that we are 'wizards'. This is partly because we like the awe and respect it brings us, but more so because of the lack of ability to model the whole project in our mind ahead of time. We genuinely believe that things will be easy, no matter how often we've been burned.

So unless the developer you're talking to is the developer that will ultimately be saddled with the burden of your project, and even if they are, to be honest, take what they say with a giant grain of salt. They don't mean to mislead you, and they don't think they are, but chances are they're giving you a rosier picture of your problem and requirements than is realistic.

Why Did This Other Developer Say They Could Do It In Three Months For Half the Price?

Software developers confidently give concrete estimates in order to get business. It’s hard to plan a business around uncertainty, so if one developer is telling you it’ll be two months and another is saying they don’t know how long it will be, the second one seems unreliable and unappealing even though she is being truthful and the other one is lying.

Once the developer gets her foot in the door, the project almost always goes over budget and over the deadline. While it's likely to happen, each time it happens it's usually for unique reasons and not clearly the fault of the client or developer, although each side will blame the other. Quality developers are a necessary but not sufficient condition for success. Success depends more on the framing of the endeavor, the quality of communication, and the matching of expectations than it does the adherence to deadlines and milestones.

The more experienced a client is, the less she demands from the developer in terms of timelines and budget estimates. The smart money has seen how the sausage is made so the it will put more weight on trust, dedication, and track record.

A common scenario is this. An agency needs a website, so they put out an RFP describing what they think they need. Developers all scramble to read between the lines of what the agency is saying and what their expectation of cost is. Then they evaluate whether or not the agency’s expectation is an acceptable amount for the work that the developer thinks is needed. The proposal submitted is more like a poker bet than a bid. Get the client interested enough to stay in the hand so you can increase the pot.

When a proposal is accepted by the client, it’s like an arranged marriage. The two parties don’t even get to meet each other until all the contracts have been written up, which are all based on complete conjecture about whether this will actually be a successful relationship.

What is lost in this process is the problem that actually needs to be solved for the client. To fix this, clients need to become smarter and wiser about software development and understand that the inherent entropy involved means that you can’t use your same judgments to evaluate software contractors as you would for other trades or services.

If you're primarily driven by cutting costs, you'll actually spend more because keeping things on budget and on deadline are directly at odds with properly exploring, defining, and fixing your problem.

If clients become the smart money at the table, they can get more of what they want for less money, instead of being suckered into a drawn out round of betting in which they ultimately pay more and feel swindled – which essentially is what the RFP process is.

A successful software project is one that is viewed as an exploration, rather than a building built to spec. It’s, “We’re trying to get to this idea over here and we trust you as our guide, so let’s move in that direction.” Not, “I know what this needs to be, it's your job to build it.”

But don’t just take my word for it. Lots of smart people have come to the same conclusion:

The Myth of Software Time Estimations

https://hackernoon.com/the-myth-of-software-time-estimations-576a7466d91a

Hourly Billing Is Nuts

https://jonathanstark.com/hbin

Value-based Pricing vs. Cost-based Pricing in Web and App Development

https://www.brainleaf.com/blog/running-a-business/value-based-pricing-vs-cost-based-pricing-in-web-and-app-development/

Cover image by Matias Malka on Unsplash

Do the mythical $150/hr, $200/hr, $300/hr jobs exist?

Keith Axline — Mon, 27 Jan 2020 20:12:46 +0000

I've been a contract developer for a while and noticed a pretty wide gap between perception and reality of the market. Am I way off? Is anyone out there living the dream of a laid-back client paying $300/hr for a variable commitment averaging 20 hrs per week or less?

Perception: There are so many businesses out there struggling to find software developers that it's easy to find $150+/hr jobs.

Reality: Yes, they are desperate and frustrated, but they also don't have the money to pay those rates. If they do, their expectations about what they get for that amount are so high that the stress is not worth the money. It's only the people who have gone down the cheap route enough to know that quality always costs. These clients are very hard to find and have developers banging down there door for any position they post.

Perception: Developers hold all the cards on a project and can dictate timelines.

Reality: Developers are handed deadlines after being left out of the discussions that produce them. That's because deadlines are mostly decided by politics, budgets, and business goals rather than an understanding of the problem, the proposed solution, or the technology. But luckily developers are also to blame when those timelines are invariably blown to pieces.

Perception: Some developers are good at estimating how long things will take, others are not.

Reality: No one really knows how long things will take. Whether a project delivers on time seems to mostly depend on luck, an ability to redefine success as the project develops, and client expectations. Not the skill or discipline of the people involved. Of course there are exceptions, but this seems to be the situation in the middle of the bell curve.

I thought I was one of the good ones until I got unlucky several times in a row. Even developers at the highest level find themselves uttering the refrain, "... but that took longer than I thought it would, of course."

Perception: With all the UI component kits and frameworks out there (Bootstrap! Foundation!) It's now relatively cheap and easy to make something that people like to use and looks good.

Reality: "Fast, cheap, and good. Pick two," is what a former coworker would always say, and that still holds true. Choosing a UI kit or framework really shouldn't lower the design budget at all. If you want things to make sense to users and be something the client is happy with, it's going to cost time and money. Design is something a certain type of client (perhaps the most common type) expects, does not want to pay for, and blames you for not being good.

The only way I can see actualizing the perception that we all seem to echo is this:

Some old-school, non-techinical-yet-super-profitable business suddenly decides they need a website. This website is not connected to any KPIs or revenue growth projections. The boss just thinks the company should have one of these websites he's been hearing about.

They don't know how to go about finding a developer so they ask around in the office. Your buddy happens to work there and suggests you for the job. Your buddy has heard that developer salaries start at $300/hr, so he says that's what your rate is. They don't bat an eye and have your buddy bring you in. A couple weeks later the pet project is mostly forgotten. To people at the company, any progress you show looks like magic and blows them away.

So I guess my question is, does anyone here work at this company?

DEV Community: Keith Axline

A Primer on Software Development and Its Frustrations

Why Did This Other Developer Say They Could Do It In Three Months For Half the Price?

Do the mythical $150/hr, $200/hr, $300/hr jobs exist?