DEV Community: Coherence

Staging is dead: The rise of preview environments

Coherence — Thu, 22 Dec 2022 15:28:08 +0000

While the embers of staging environments are still smoldering, like a phoenix, preview environments have risen from their ashes.

Modern software development teams are increasingly using preview environments, also known as ephemeral environments, to test and preview their features. Unlike traditional staging environments, which were typically shared by multiple teams and often led to bottlenecks and delays, preview environments are created on-demand for each developer. This allows for greater collaboration and feedback, as well as improved testing due to production parity. Preview environments are fully automated, which saves time, costs, and resources compared to manually setting up and maintaining staging.

Overall, preview environments provide a superior solution for modern software development teams who need to ship to production multiple times a day. The goal is to empower teams to ship higher quality code at a higher velocity.

Traditional staging bottlenecks

Having only a single staging server can quickly become a bottleneck for developers, as they must wait their turn to deploy to production. Maintaining a staging environment is always challenging, as it is difficult to keep it in sync with production. The challenges and frustrations associated with using a single staging environment often lead developers to devise workarounds, such as creating their own "staging" environments. This can introduce additional complexity, overhead, and security risks.

Ephemeral environments, on the other hand, eliminate these issues by providing each piece of work with its own dedicated environment.

Collaboration & feedback

Before the use of preview environments, developers often had to rely on screenshots and videos to showcase their work to stakeholders and other teams. This usually resulted in "review meetings" where various teams, such as product, design, and stakeholders, would gather to review the work. Collaboration and feedback were difficult to do asynchronously and a lot of time was often wasted in these meetings.

Instead of sharing screenshots and videos, or holding "review meetings," preview environments can be easily shared via a URL. This allows for asynchronous collaboration and feedback, saving time and resources. Sales teams can also benefit from using preview environments, as they allow for easy demos of new features to prospective customers.

Overall, preview environments provide a more efficient and effective way to review and demo work.

Testing

There are several benefits to using ephemeral environments for testing. These environments are nearly identical to production, which gives developers increased confidence that their code will work when it's deployed.

Using ephemeral environments also makes it easier for developers to perform code reviews. They don't have to spend time rebuilding their local environments, which can be time-consuming and difficult. This means that developers are more likely to test their code thoroughly, leading to better code quality and fewer bugs. Ephemeral environments require little to no maintenance, which means that teams can focus on testing their work with the level of quality and thoroughness that it deserves; eliminating the need to cut corners in testing.

Automated infrastructure

Preview environments and the underlying infrastructure that powers them are automated and require little human intervention. Whenever a developer opens a new pull request, a new environment is created specifically for that piece of work. These environments are designed to be created and destroyed quickly, so if an environment breaks for any reason, a new one can be spun up in a couple of minutes.

Since these environments are automated, your DevOps team has one less thing to worry about as we all know that DevOps teams are often under significant stress.

Cost control & savings

Preview environments are designed to be ephemeral, meaning they only exist for as long as you need them. This can save you money since you don't have to pay for environments that aren't being used. For example, you can automate these environments to spin up for every pull request and have them automatically destroyed when the request is approved and merged. The environment only exists for as long as necessary, so you only pay for what you actually use.

Preview environments and the Coherence SDLC

Preview environments are an integral part of the Coherence SDLC. After creating a new branch in Coherence, an isolated ephemeral environment is automatically created for it. Development is done via a workspace, and the new code is then pushed, via git, to the branch. Coherence automatically triggers a new build, updating the environment with the latest changes. Developers can then easily share the environment with whomever they wish via a unique URL. Once the branch has been approved and merged, Coherence will automatically destroy the environment and its underlying infrastructure.

Conclusion

While staging environments may not be dead yet, they are on life support. We’re seeing an explosion in third-party ephemeral environment providers because the demand is there. At Coherence, not only are we bullish on preview environments, we believe they’re core to an integrated development experience. The long term bet we’re making is preview environments in conjunction with cloud development environments, which we refer to as workspaces, is the real wave of the future

If you want to see what the future of software development looks like, we invite you to try one of our free sandboxes.

AI and the future of DevOps

Coherence — Fri, 09 Dec 2022 17:50:06 +0000

The release of ChatGPT by OpenAI has caused quite the stir in the software industry, as it showcased the potential of AI in an easy to digest format for the everyday consumer. With over one million users within just five days, ChatGPT has demonstrated growth that surpasses even the most popular social media platforms. This impressive adoption rate highlights the increasing demand for AI in today's world and its potential to disrupt many industries, including DevOps. However, even with such staggering growth and excitement, there is still a good amount of skepticism – these are early days, afterall.

As AI continues to evolve, we expect it to have a profound impact and practical applications in the DevOps world. AI has real potential to improve code quality, strengthen monitoring and alerting systems, enhance security measures, and increase engineering productivity.

Code quality

As AI tools like GitHub’s Copilot and Replit’s Ghostwriter continue to grow, we are already seeing more and more software written by artificial intelligence. This trend will only continue, with AI being used in DevOps to write configuration files, IAM policies, Infrastructure as Code (IaC), and more.

Here is an example of ChatGPT debugging an error with an AWS IAM policy.

AI will continue to play a significant role in code generation. While developers will still need to review and make adjustments to the code written by AI, the speed and accuracy of AI-generated code will likely surpass that of human-written code. This could shift how software is developed, with AI taking on a more central role in the process. However, the exact timeline for when this might happen is still unknown. It will likely depend on several factors, including the advancement of AI technology and the willingness of developers to incorporate AI into their workflows. It’s going to be interesting to see how quickly AI is embraced or kept at arm's length by developers.

Code reviews

As AI technology advances, it will likely profoundly impact code reviews. This is already starting to happen with the development of tools such as Amazon CodeGuru and GitLab's ModelOps. These tools automate certain aspects of the code review process, such as identifying potential errors and suggesting improvements, saving developers time and effort while also helping to improve code quality.

In the future, AI-powered code review tools may become even more sophisticated and comprehensive, taking on more complex tasks and providing more detailed feedback to developers. This could lead to more efficient and effective code review processes, helping to ensure that the code is of the highest possible quality.

AI will verify that the code is correct, test for security vulnerabilities, and ensure that it works properly. As the AI performs more and more code reviews, it will become more proficient at identifying issues and suggesting fixes. Over time, the AI will become increasingly knowledgeable about your application's code base, enabling it to make more accurate suggestions. Initially, developers will still need to review and approve any changes made by the AI, however, the need for human intervention will decrease over time.

Testing

AI will revolutionize testing. The shift-left testing approach, which focuses on identifying and fixing issues early in the development process, will be significantly impacted by AI. The traditional testing pyramid, which emphasizes the importance of different types of testing, will become obsolete as AI takes on a more significant role in the testing process. With AI, your application's code base can be thoroughly analyzed and tested at all levels, from individual units to end-to-end user scenarios. In addition, AI will test for security vulnerabilities, performance issues, edge cases, and more.

In addition to performing comprehensive code reviews and thorough testing, AI will monitor your application in production and identify any issues that may arise. By continuously monitoring the application, AI will learn from these issues and adapt its testing strategies to prevent similar problems from occurring in the future. This ability to constantly learn and improve allows the AI to provide ongoing support and protection for your application, ensuring it remains stable. Additionally, by automating many tedious and time-consuming tasks involved in testing, AI can free up your development team to focus on more critical tasks and accelerate the overall development process.

Monitoring and alerting

AI will monitor the performance of an application and automatically alert teams if any issues need to be addressed. This can help teams respond to problems quickly and prevent them from becoming more serious. We believe that an AI could soon replace most site reliability engineers (SRE’s) responsibilities. AI can analyze large amounts of data, such as logs, to identify potential problems and trends in real-time. Analyzing such large amounts of data will provide insights and recommendations that can help teams make more informed decisions.

When a bug is found in your application, AI could deploy a working version of your app without interrupting its operation. After deploying a previously functioning version of the app, the AI could create a new PR and write the necessary code to fix the bug.

AI could also identify inefficiencies in your cloud infrastructure and suggest ways to improve performance and reduce costs. This could include scaling resources up or down based on demand, consolidating underutilized resources, and more. The use of AI in cloud management and monitoring has the potential to improve efficiency, reduce costs, and prevent bugs.

Security

AI technology will play a significant role in the security of applications and infrastructure. For instance, AI could automatically detect and block malicious attacks such as DDOS, XSS, and CSRF. It will also instantly identify potential security vulnerabilities during code reviews. Additionally, AI could surface any leaked or vulnerable PII (personally identifiable information).

AI could monitor network traffic and identify suspicious patterns indicating an attempted attack. This could include analyzing the content of network packets, the frequency and location of requests, and other characteristics. Based on this analysis, an AI could automatically take actions to defend against the attack, such as blocking the source of the traffic or redirecting it to a honeypot. Additionally, AI could conduct security testing of your application during the development process, simulating various attacks and vulnerabilities to identify potential weaknesses before attackers can exploit them.

Increased productivity

AI already automates repetitive and time-consuming tasks, enabling teams to be more productive and efficient. AI will increase productivity for DevOps engineers by reducing the amount of boilerplate code for infrastructure as code (IaC), templates, and configuration files. Tools such as Copilot and Ghostwriter currently help with this and will only improve over time.

Furthermore, AI can help DevOps engineers by providing instant access to information from documentation. Instead of reading through documentation, engineers can ask AI for the information they need, allowing them to access the information they require instantly. Additionally, AI can handle tasks such as configuring infrastructure and setting up cloud services, freeing DevOps engineers to focus on more specialized work.

Coherence and AI

At Coherence, we see several opportunities to incorporate AI in a way that will materially benefit our customers. For example, AI could help us and our customers streamline the onboarding process for new applications. Since every application has unique requirements, the onboarding process can vary greatly depending on the complexity and architecture of the app. AI could analyze a customer's application and assist with creating Dockerfiles, our coherence.yml, and ensuring that the app is ready for deployment in the cloud. This would save time and reduce errors, improving the overall onboarding experience for our customers.

Artificial intelligence could help us regularly monitor our customers' infrastructure and cloud services to ensure optimal performance and cost-effectiveness. AI could enhance application security and compliance with HIPAA, SOC2, and others. Additionally, we believe that AI can assist our team with more efficient code reviews and improved code quality.

Conclusion

Companies will create DevOps-specific models that are hyper-focused and specialized, like Amazon’s DevOps Guru. Sam Altman, the CEO of OpenAI, foresees a future where several new companies and startups train existing models for specific verticals. Companies will leverage already trained and existing models but enhance them with data specific to DevOps. These AIs will improve code quality, strengthen monitoring and alerting systems, enhance security measures, and increase engineering productivity.

The use of AI in DevOps has the potential to significantly improve the efficiency and reliability of software development and deployment. We believe by leveraging the power of AI, companies can build better products faster and cheaper. If that provides a better experience for our customers and users we should embrace it.

It's time to forget what you think you know about remote development

Coherence — Mon, 05 Dec 2022 18:56:24 +0000

Over the past few years, more and more companies have migrated to remote development environments for their engineering teams. With the advent of the cloud, developing remotely provides numerous benefits over local development. While local development is still the predominant way of writing software, it is only a matter of time before remote development makes it a thing of the past, and for a good reason. Cloud development environments save time and money, prevent the “it works on my machine” problem, increases code quality, mitigates context-switching headaches, and provides better consistency between development and production.

Cloud development environments are the future

Remote development is in a similar place that the cloud was in several years ago. These days, the cloud is ubiquitous. Remote development is next.

We are not the only ones who believe this, as large companies like Google, Microsoft, and JetBrains continue to invest heavily in cloud development environments. Companies like Slack, Shopify, Meta, and Tesla are investing in cloud development environments for their engineers because the costs and benefits far outweigh local development.

“At Slack… Engineers would no longer have to maintain code or dependencies on their local laptops. They could get a fresh isolated environment on demand, ready to be used within a couple of minutes.” - Source

The dreaded "it works on my machine" problem

Developers often work with various operating systems, tools, software, and dependencies in multiple versions and states. Trying to maintain consistency across unique machines only leads to headaches and frustration. Remote development solves this problem because every development environment is the same.

The cloud is the environment.

Remote environments save time and money

"New hires can go from zero to a functioning development environment in less time than it takes to install Slack" - Source

Developers spend significant time maintaining their local development environments, often spending several hours, sometimes days, updating, maintaining, and debugging their machines.

As an application grows and becomes more complex, it will consume more resources like CPU and RAM. With cloud development environments, you can upgrade every engineer's machine with a single configuration change in seconds! Otherwise, you have to buy every engineer a brand-new laptop. How much will that cost?

"I've seen engineers lose hours or even days debugging local development environment issues. It's even worse when starting a new job. There's little more disappointing and disempowering than to excitedly begin your job and then get stalled because you can't get your local development environment working. Every company should aim to get their developers as productive as quickly as possible. How long does it take a brand new developer to get code in the hands of customers when they start at your company? Is it measured in months, weeks, days, or hours?." - Source

Cloud development environments increase code quality

Developing in the cloud improves code quality and produces fewer bugs.

When a developer is reviewing code they look at the difference between the current state of the application versus their changes. Seeing these differences is vitally important, but there is another crucial step that most skip; pulling the code down onto their machines, to ensure the code works. Most of us ignore this critical step because it is often too painful and time-consuming.

The developer performing the code review is most likely working on their feature, fixing a bug, or doing something completely unrelated to the code under review. They would have to save their code changes, switch contexts, and get their local development environment in the same working state as the code under review. This means pulling down the code under review, installing packages, libraries, version updates, etc., a recipe for yak shaving. With cloud development environments, developers can review code in the same environment that the code was written in.

No more yak shaving!

Effortless context switching

Everyone knows context switching is wrong but let's be honest with ourselves; it happens all the time. A developer can be working on a feature, suddenly open a new environment to fix a bug, and even open a third environment for code review. We are not suggesting that context switching between three different contexts is a good idea, but context switching does happen more often than we like to admit, and it is effortless when developing in the cloud.

One operating system to rule them all

Another thing to consider is that cloud development environments run on top of a single operating system. Local developers could use Mac, Windows, or various Linux distributions. Even if they use the same operating system, that does not necessarily mean everyone is running the same version. In remote environments, everything is the same, including the operating system. Your developers also benefit from developing on the same operating system used in production, providing a narrower gap between dev and production.

Common objections and pushback

Developing remotely in the cloud is not ideal for every situation or team as there is still more work to be done for it to completely replace local development. Here are some common objections and pushback developers have when it comes to developing remotely.

Remote environments are slower than my local environment

Developing remotely in the cloud is typically slower than local development. Many factors, like network speeds, internet connection, and latency, contribute to this. One of the key trade offs regarding speed is the ability to spin up fresh environments quickly, per branch, without the need to maintain local environments.

Remote environments are also becoming faster with technology like edge computing.

“But surely you can see that the latency question is a question of letting the Moore's law equivalent of cloud commoditizing infrastructure take its course.” - Source

What if I need to work offline?

If a developer regularly works offline, cloud development is not a great option, as remote environments require a fast internet connection. However, in practice, this is rarely an issue. Most developers are constantly connected to the internet as they rely upon it to do their jobs.

I don’t want to use/learn another IDE

Developers are often very opinionated and particular, especially regarding their development environments. Many have custom key bindings, aliases, shortcuts, extensions, and themes, all finely tuned and configured on their machines. They have gone to great lengths to painstakingly set up and configure their IDEs to their exact specifications and liking.

Many IDEs, like VSCode, allow developers to access cloud environments from their local machines remotely. Developers can develop remotely in the cloud using the same setup on their local machines. In this way, they get the best of both worlds.

Coherence workspaces

At Coherence, we are excited to help create this paradigm shift in software development. Our workspaces are an integral part of our software development life cycle. Workspaces are ephemeral VSCode instances (we also support Emacs and VI) that are created automatically for every branch. In addition to remote development environments, we also provide managed Infrastructure as Code, managed CI/CD, shareable preview environments per branch, staging environments, and one-click deployments to production.

We use Coherence to build Coherence, and we can't imagine ever spending the time, money, and resources necessary for local development again. As a startup, we must move at breakneck speeds without compromising our burn rate. Developing remotely in the cloud feels like a secret weapon since we take all the time and money we are saving to focus on delivering value and differentiating ourselves from the competition.

If you are interested in trying out Coherence, you can try one of our sandboxes for free.

You can also check out our docs to learn more.

Conclusion

Remote development will revolutionize software development. The benefits of cloud development far outweigh those of local development. It saves time and money, prevents the “it works on my machine” problem, increases code quality, mitigates context-switching headaches, and provides a consistent environment between development and production. With these many benefits and more, it is hard to justify writing software locally ever again.

It is only a matter of time before local development becomes a thing of the past.