DEV Community: Syberry Corporation

Making Software Development More Predictable

Syberry Corporation — Tue, 15 Oct 2019 14:34:30 +0000

Look up data on the success rate of software projects, and you’ll likely be unpleasantly surprised at how often they’re considered failures. There are many reasons software projects fail, from busting budgets to missing deadlines to failing to fulfill the clients’ goals. Whatever the ultimate reasons, the majority of those failures stem from the unpredictability inherent in software development.

In this article, we take a look at some of the causes of unpredictability in software development, and then we outline best practices to help manage that unpredictability and therefore make the project more successful.

Drivers of Unpredictability
Chaotic Systems
Software engineering projects are unpredictable by their very nature, because each one is a complex socio-engineering system. And “complex” in this context means something very specific — much like a double pendulum, these systems aren’t random, and yet they aren’t predictable, either. The double pendulum, with one pendulum attached to another, seems simple enough at first glance, as its movement is governed by a set of straightforward differential equations. But try to predict the pendulum’s position at any point in time, and you’ll realize those equations aren’t at all straightforward after all, but very difficult to solve using traditional methods.

Why? They’re extremely sensitive to initial conditions.

In our experience, software engineering projects are similar in nature. We all know which project management models to use and how to build teams, develop software, and debug systems, but these projects are highly sensitive to the contextual conditions. Those models and strategies and systems aren’t quite as “plug and play” as one might expect. But the good news is that, while the systems aren’t predictable, they have stable “equilibrium” states that we can target as we set up and implement the project.

Speed of Engineering
It seems, these days, that speed and agility are the end all, be all for software development clients. And in that haste, it’s easy to overlook a critical step in the development process: modeling a system before beginning construction.

Engineers are responsible for thinking ahead, imagining and modelling and optimizing their projects before they start building. This doesn’t mean producing detailed, bureaucratic documentation for the entire system upfront, but it does mean thinking carefully and modeling at least the next piece to be built.

When engineers dedicate the mental energy and self-discipline to thinking ahead about the architecture of a system and its components, the build process goes much more smoothly. When we consider what’s coming before we jump in, we run into far fewer road blocks and surprises along the way.

Unrepeatable Processes
Unpredictability is a two-way street: a project may fail, but it can also be a success. And when a project succeeds, the next question is, how do we repeat that? If processes aren’t sufficiently documented, then the next attempt to repeat that success — either in updating the application or replicating it elsewhere — will be just as unpredictable as the first.

As a service-based organization, Syberry has a bit of a leg up. We have a lot of different projects and products under our belts, and we and use the knowledge we gain from each to steer the next ones. When things don’t go well, properly analyzing mistakes and adjusting processes accordingly is critical to consistently improving our service.

Building a single engineering team to design and create a new application is no easy task, but it can be done. It’s like building a Tesla prototype — difficult, but doable. But as soon as you ask that team to start replicating their success, that’s like building a Tesla factory, and it requires a significant increase in resources and experience.

Making Development More Predictable
Many software developers handle all these complexities by putting them on their clients’ shoulders, providing the team of engineers and requiring the client to manage the team and the project. While this works sometimes, it often leads to subpar experiences and can’t rightly be called full-service software engineering. At Syberry, we work closely with our clients, but we are the ones managing our projects and our teams. We believe our clients hire us so they don’t have to do that work.

We know there is no single solution to the challenges of software development — or any challenges, for that matter — but we are constantly fine-tuning our processes and strategies to make software development more predictable and give our clients and our projects the highest chances of success.

Here are four of the drivers behind that success:

Organizational Culture
Culture and processes are the combination that determines an organization’s ability to deliver quality products and services. While the processes are important, they can’t account for everything. You can write books worth of rules and processes and regulations, but there will always be new scenarios, finer details, and unexpected questions that formal processes don’t account for.

This is where culture comes in, and Syberry has worked to cultivate a culture where employees are empowered to make the best decision for the client and the project anytime there is no handy rule to reference. Our employees highly skilled developers and critical thinkers, and we trust them to make informed viable decisions — or seek help when a decision is out of their realm of expertise.

Promises
It’s not so much about making promises as it is about making promises and commitments you can fulfill. We’ve all met “yes men,” who agree to anything, promise the world, and then fail to deliver. That’s not beneficial for anybody, and it’s certainly not the path to success for a software project. When we work with clients, we are careful to make realistic, thoughtful commitments based on our software expertise and our clients’ expressed needs.

To ensure we’re making promises responsibly, we implemented what we call “service models” in our organization. Similar to project management models or methodologies, service models define what we promise to our clients in different kinds of engagements. We use a variety of different models. These are just a few of the most frequent:

Custom software development: When a client comes to us with detailed requirements for a project, and the vision and core functionalities aren’t likely to change, we can promise to build a system with a specific list of features and within a specific budget and timeline.
Product development: When a project is nothing more than a vision, when the features are still flux, and when competitors are breathing down a client’s neck, we can’t make such specific promises. When time is of the essence and the goal is a moving target, we promise a high-performing team to maximize the number of features built within a given timeline. To clarify our promises, despite the flexibility, we work with the client to define clear milestones to track the project’s success.
Maintenance and support: A system that is in production requires a completely different set of promises, including SLOs around response time for troubleshooting and fixing issues or levels of system stability.
There are many types of services, but the key here is that, when we sell something, we have to be sure that both our team and our client understands exactly what we’re selling. Structuring our services this way, rather than just trading hours for dollars, protects our time and our clients’, and it makes our services clear, successful, and scalable.

Client Education
When you go to a doctor (unless you’re a doctor, yourself), you expect her to know more than you do about medicine, and you also expect her to help you understand and improve your overall health. The same is true in software development. Our clients are experts in their businesses, and we’re the experts in software development.

Our clients don’t have to be skilled software engineers or project managers — that’s our job. But it’s up to us to ensure they have a solid understanding of the process, milestones, and end goals, maximizing their confidence and minimizing surprises along the way.

Team Equilibrium
Along with our emphasis on culture, we believe that the most effective software engineering teams are driven by KPIs and evolving ideal processes, not by management. The most effective teams’ function with proper goals and framework, with management playing the role of gardener — finding the right people and giving them the tools they need to flourish on their own.

Because development isn’t predictable, teams have to be flexible and prepared for change, always aiming for equilibrium, not stasis. In other words, they need to be prepared to pivot in order to work in whatever way is best for a given project at a given time. This flexibility and constant change mean our team is always looking for better ways to achieve project goals within the constantly changing business environment.

Software development will always include a level of chaos, but that doesn’t mean it can’t be successful. When you partner with a vendor whose expertise, agile processes, and service-oriented culture minimize the inherent unpredictability, your organization’s software development project is far more likely to be a rousing success.

The article first appears at https://www.syberry.com/company/blog/articles/making-software-development-predictable

Keep Quality Assurance

Syberry Corporation — Fri, 13 Sep 2019 11:51:18 +0000

At Syberry, we pride ourselves on high quality standards, and one of the ways we maintain those standards is by keeping development and quality assurance separate on every project, from beginning to end.

Couldn’t the developers test the software themselves? Doesn’t a separate QA team just add to the cost and extend the timeline? Sure, but we’re confident that separate QA is worth the extra resources, and the upfront investment will likely save both time and money in the long run.

Why Developers Shouldn’t Perform Their Own Quality Assurance Testing
Have you ever written a report or paper, proofread it carefully several times, and submitted it, only to find a typo or grammatical error the moment it’s too late? There’s a reason writing teachers always recommend having someone else proofread your work. When we write something ourselves, painstakingly creating it from thin air and shaping it to match our vision, we’re simply too close to see it objectively. We can’t see that transition that doesn’t quite work or the missing verb in the third sentence or the typos sprinkled throughout, because we know the work so intimately that we see what we know we ought to see — not necessarily what’s there. Only a second reader, one who hasn’t lived and breathed the work like we have, can help us step back and see its flaws.

And the same is true with software development. The developers writing the code may be world class, but they’re too close to the project to be able to step back and study it objectively. “They can’t see the forest for the trees,” so to speak.

However, maintaining a separate team to conduct QA testing ensures a more objective assessment of the code’s functionality, weaknesses, and alignment with the client’s vision.

While this may require extra resources upfront, it also ensures that more bugs and deficiencies will be caught and fixed before the software is released. While a good vendor is always there to help maintain a project post-release, it’s always better to make sure the work is clean before it goes live, minimizing user experience issues that could lead to frustrated customers or stalled internal processes and, ultimately, more time and money lost.

So How Does It Work?
For some vendors, quality assurance only happens periodically, as developers wrap up each sprint on the project timeline. However, this system tends to cause more stress than it alleviates, because by the time the QA team finds any issues, there’s no time to solve them without pushing back the deadline, and depending on where the errors are, fixing them may require rewriting large portions of the code.

At Syberry, we take a more agile approach, with the development and quality assurance working in tandem. Rather than waiting for the developers to hand off large chunks of code, the QA team continuously checks the developers’ work. That way, any glitches can be fixed quickly, before they impact other pieces of the code and cause delays that set the project back.

Are You Saying My New Software Will Be Perfect?
We’ve written before about why custom software doesn’t come with a warranty, and the same principles apply here. To be truly perfect, any software must be tested by real users in the real world, and the user feedback that comes from this process will often highlight room for improvement.

That said, a good vendor doesn’t just cut and run once the software is live. Your vendor should be a partner that will work with customers to monitor and maintain software as its used, preventing or fixing any glitches that appear and modifying the code as needed to help software grow and change with the business.

Quality Assurance: A Worthwhile Investment
Having heard all this, some customers still wonder whether they need quality assurance at all. After all, if the developers are qualified, shouldn’t that be enough? But consider the same question in another context: If you were buying a new house, even if you knew the builder was highly reputable and qualified, wouldn’t you still hire an inspector to check everything before you closed? That way, you could fix any problems before you moved in to save yourself time, stress, and money in the long run.

The same is true with software. No matter how skilled the developer, it’s in the client’s best interest to bring in a separate quality assurance team to identify and fix any issues before the application goes live, ensuring a smooth launch and a low-stress user experience.

Yes, maintaining development and quality assurance as separate parts of a custom software project may add to the upfront investment, but with the increased quality it ensures and the headaches it saves later on, we hope you’ll agree that it’s a necessary use of resources.

MEAN vs. LAMP: Choosing the Right Stack for Your Software

Syberry Corporation — Mon, 29 Jul 2019 15:07:43 +0000

When we’re working with clients to create the plans and roadmaps that will guide us in converting their software visions into custom-built realities, one of the first decisions we make is which technologies will best fit the clients’ particular needs.

In early years of web applications development, a technology stack called LAMP was dominant in the field. But in recent years, evolving technologies have given rise to for dominance in the development field, businesses are asking which stack to use to reduce risks as much as possible in their systems development.

At Syberry, we love thinking through these questions with our clients, and we’re always happy to advise on the right technology stacks for your projects. But for now, let’s break down some of the key decision-making points between LAMP and MEAN in an effort to illustrate the pros and cons of each and help you start to make the decision for yourself.

First Things First: What Are MEAN and LAMP?
Conveniently enough, the stack names are acronyms that stand for the technologies used in each.

LAMP breaks down like this:

Linux: a dominant family of operating systems used for hosting web applications
Apache: a popular, configurable web server
MySQL: a relational database for data storage
PHP: a general-purpose programming language designed to create dynamic, interactive webpages
And MEAN contains its own set of technologies:

Mongo: a non-relational (NoSQL), document-oriented database
Express: a collection of best practices for building typical web application
Angular: a client-side application framework for building rich, single-page applications
NodeJS: a run-time environment allowing developers to write server-side business-logic for the system in Javascript
There is no one-to-one technology mapping between the two stacks, and both have their advantages and disadvantages depending on what a developer aims to accomplish. Let’s take a look at a few of the key distinctions.

Javascript Makes MEAN an Attractive Option
One of the most important functional differences between MEAN and LAMP is the underlying programming language. MEAN is Javascript driven, while LAMP is grounded in PHP. So what does this mean, practically speaking?

The big advantage of the MEAN stack is the usage of Javascript both in the browser and on the server. On the development end, this allows the team to be truly cross-functional, which makes it much easier to balance the workload between front- and backend engineers. Additionally, the crossfunctionality makes troubleshooting easier, as a single person can access and inspect the entire application.

On the user side, the recent extensive development of Javascript as both a technology and a tools ecosystem has made it a popular choice for building rich, interactive client-side applications.

But PHP Has its Benefits
PHP is an older programming language, and many Javascript fans perceive it as outdated or missing various features that have become standard to programming languages in recent years. However, as the standards evolve, different programming languages have converged more and more in terms of the features they support. PHP is no exception here, and all of the major advancements and functionalities are available in newer versions of the language.

While the feature sets between PHP and Javascript aren’t equal, they are close enough from an engineering standpoint that you may not see a lot of difference development from one to the other — especially at scale. What PHP lacks in modernity, it makes up for in maturity, with availability of rich frameworks such as Symfony with long-term support.

An Argument for LAMP
While the Javascript functionalities of the MEAN stack are useful for highly interactive, feature-rich web applications, the PHP-driven LAMP stack often makes more sense for classic, business-oriented systems.

In these systems, while the user experience and interface are important, the focal points are the functionality and logic of the system that supports the business. In our experience, it’s much more efficient and economically sound to build the classic way, using templates to build static HTML-based user interfaces. For these, the templating tools available in all popular PHP frameworks are sufficient, and there’s no need for Angular or other client-side-oriented frameworks that are available in LAMP.

Another budgetary benefit to this approach is that a back-end engineer may be able to create a solid user interface for a classic business system, which eliminates the need to staff a team of front-end engineers.

Additionally, Javascript is usually an asynchronous language and, therefore, writing clean, efficient code is a more complex and time-consuming task. Working with Javascript, therefore, raises the bar for quality development and engineering expertise, making hiring much more difficult.

Especially if you’re using the MEAN stack, it’s important to hire developers who understand these concepts and who recognize that it will be more difficult to scale the team to speed development.

The Devil Is in the Database
Another key difference between the two stacks is the type of database they use. MySQL, used in the LAMP stack, is a classic example of a relational database built around the concept of relations (tables) and records.

Mongo, on the other hand, the database used in MEAN is document-oriented, which means that each unit of data is stored as its own document, and different documents in the same collection may have different — or even nested — attributes.

It is important to understand that there are no inherent pros or cons to either approach, and the proper choice is dependent on the nature of the data the system stores and how users will need to work with that data.

If your system requires strong schema and a lot of join operations over multiple data sets — such as you see in reporting — a relational database is a better choice. This is usually the case with an internal business system. But if your data has weak schema and is centered around key objects such as user profiles, the NoSQL solution found in the MEAN stack might be better. Additionally, Mongo’s scalability potential is higher than MySQL’s, so if you expect your application to grow in its data usage load, Mongo will be the better choice. (Though it should be noted that, if you’re working with a sufficiently large system, you may need to consider using a combination of both types of database.)

Ultimately, Scaling Isn’t about Stack
Though the stack you use may play a factor, if you take a bird’s-eye view to the issue of scalability, you’ll see that the key factor isn’t the specific set of technologies you choose (though you’ll likely run into trouble if you choose haphazardly). In fact, with tools like Docker that allow you to move specific components and technologies into isolated units, you’re no longer even restricted to a single stack. What matters for scalability is actually your approach to the architectural design of the system.

If you plan for the system load to grow rapidly, you must plan for that from the start by building in horizontal scalability — in other words, giving yourself the option of adjusting for the workload by adding more computational power in the form of commodity server nodes (as opposed to scaling vertically by buying a more powerful server). Building for horizontal scale means being very intentional about how you organize primary components like state, business logic, and presentation layers.

Conclusion
As is the case with so many strategic questions about both software and business development, there is no one right answer as to which technology stack is superior. Each company must work with its development team to make the strongest choice for the individual situation.

The most important rule of thumb we can offer is that, the smaller the application and the scale of development team, the more specific stack choice matters. The opposite end of the continuum — large-scale projects — is all about building proper architecture rather than nailing a programming language or database choice the first time out.