DEV Community: Nimbus

These companies use Cloud Environments

Nimbus — Wed, 07 Dec 2022 20:49:37 +0000

Cloud-based development environments are popular among companies of all sizes as a way to improve collaboration and productivity among engineering teams. But most people don’t realize how much software is built on remote development environments. In this post, we want to share a dozen examples of companies that have made the transition, why they did it, and who led the change.

We’re doing this because even as founders in this space, it’s often hard to explain how and why companies make the transition. This is partly because engineering orgs often have an allergic reaction to productivity measurements. Still, the more significant issue is that developing on cloud-based environments is a different way of working: it’s hard to compare, and the driving force behind the decision differs at many companies.

If you want to jump straight to the sources, summaries, and info - you can jump straight to the “Companies building software on Cloud Environments” section. But first, a quick overview of what we’re seeing.

What companies?

Companies of all sizes, but the tipping point is 50 to 70 engineers. With 100 engineers, the benefit becomes clear to engineering leaders. Our case studies are biased toward more prominent companies (i.e., the ones with resources to maintain an engineering blog).

Why did they make the switch?

‍Primarily to simplify workflows and testing, unlock more computing power/memory to cut build, compile, and test times, and improve stability and consistency. Teams also found they could run more microservices in development, speed up onboarding times for new projects and new engineers, and reduce the context engineers need to get work done.

Who leads these initiatives?

‍At small companies, this is often led by engineering leaders. This is owned by DevOps/SRE, Developer Experience or Productivity, or Infrastructure teams at mid and larger companies.

What were common concerns?

‍Teams were worried about high latency, coding without the internet, changing workflows, and tool lock-in. Much of this stems from experience with Cloud IDEs in the past, but with products like Nimbus, you can have a local-like experience with remote development. An internet connection is required, but teams opt not to optimize for rare “coding on a flight scenario.”

Are teams buying or building?

Most of these examples have homegrown tooling, but more eng leaders are buying tooling as the market and product develop quickly. In the past, solutions failed because they limited tooling options (e.g., VSCode only) and couldn’t deploy on their customer’s cloud. But with this change, we’re seeing teams look to replace their internal tooling to relieve engineering capacity and focus on core products.

Companies building software on Cloud Environments:

Stytch - 30 engineers

Read about it here: https://stytch.com/blog/remote-dev-1/
Pains: Development needed too many services, context, people, and setup time. Issues from differing production and local configs. Difficulty in testing.
Summary: Stytch found five primary productivity drags from local development. This included: too many services, too much context needed, development was too people-intensive, and config and testing issues. They considered 3 options instead of moving to the cloud but realized that cloud-based development was the only way to meet all their requirements. Stytch found that centralization, scale, and efficiency were the most significant benefits of remote development.

Doximity - 200 engineers

Read about it here: https://technology.doximity.com/articles/github-codespaces-is-great-but-mutagen-is-better
Pains: Docker Desktop for Mac was slow and resource-intensive, and lack of observability into local development issues
Led by: Site Reliability Engineering
Summary: The SRE team created Dox Compose Cloud, a standardized remote development environment that gives increased visibility and uniformity across development and production environments. The tool is similar to GitHub Codespaces, but Dox Compose Cloud allows users to continue developing on their local machine using their editor of choice and allows SRE teams to have visibility over development environments and resolve issues more quickly.

Monday - 230 engineers

Read about it here: https://engineering.monday.com/development-environments-in-the-cloud/
Pains: Slow onboarding, resource-heavy docker composition, could not run more than a few microservices, and difficulty maintaining environments
Led by: Developer Experience
Summary: Monday.com used a third-party tool to set up cloud-based development environments. This was done for their monolith and 30 microservices, as well as Ambassador (its HTTP gateway). Monday.com’s tooling enabled developers to work on their Kubernetes environments, replicas of production environments, without the need to install or maintain tools locally. This improved the time it takes to spin up the environment from scratch, reduced resource usage, and provided a more consistent development experience.

Pipedrive - 300 engineers

Read about it here: https://medium.com/pipedrive-engineering/dev-environment-evolution-in-pipedrive-from-a-developers-perspective-cdc8a5b7e3c1
Pains: Stability and consistency issues, development environments were resource intensive, slow builds, complex setup, unintuitive alternative solutions
Led by: Developer Experience
Summary: Pipedrive’s dev environment evolved from a Docker workstation to a Kubernetes workstation to using Devbox (testboxes) to transfer the most demanding components of the dev environments to the cloud. The switch allowed for better parity between environments, resolved dependencies between microservices, and reduced developer hours troubleshooting dev env issues. It also allowed for faster onboarding and easier maintenance of the dev environment.

Plaid - 400 engineers

Read about it here: https://plaid.com/blog/scaling-with-devenv/
Pains: Docker Desktop for Mac was slow and resource-intensive, had slow build times and Docker image pull times, and poor development experience and reliability.
Led by: Developer Efficiency Team
Summary: Plaid’s Developer Efficiency Team decided to create “Devenv,” an internal CLI tool backed by a remote environment, to address several scaling challenges they were experiencing. Originally Devenv was built for local development but was moved to the cloud (backed by a standardized set of EC2 instances). Devenv allows developers to set up and validate the development environment; clone and update repositories; start, stop, and rebuild dockerized services; run unit and integration tests; perform lint checks. Nearly 2 years after the rollout of remote devenv, 87% of users exclusively use remote devenv, and 97% of all commands are run on remote devenv.

Rippling - 500 engineers

Read about it here: https://www.rippling.com/blog/how-we-used-the-cloud-to-supercharge-remote-development
Pains: Limited hardware resources, slow onboarding, slow testing times, environment consistency
Led by: Engineering Shared Services
Summary: Rippling’s application is a massive Python monolith that has grown quickly. The size and complexity made it slow and unwieldy for local development. The team experimented with third-party remote solutions but decided to build this in-house because they didn’t like the browser-based solutions they evaluated. They ultimately created a solution that synced local code changes to remote EC2 servers where the code would be executed. The main benefits were a simplified and standardized setup that accelerated onboarding and faster app and test running times.

Slack - 1400 engineers

Read about it here: https://slack.engineering/remote-development-at-slack/
Pains: Fragile and inconsistent environments, slow build times, and slow setup times
Led by: Developer Productivity
Summary: Slack’s engineering team wanted to improve the consistency and speed at which engineers could write code. As a starting point, they updated their rich CLI tools to build features that let them interact with dev environments. They found that putting environments on EC2 servers did more than just cut down setup - it simplified workflows, opened new possibilities, cut bootstrap times, and allowed engineers to focus on writing code without worrying about environment setup and maintenance. ‍

Palantir - 1500 engineers

Read about it here: https://blog.palantir.com/the-benefits-of-remote-ephemeral-workspaces-1a1251ed6e53
Pains: Lack of local compute resources, slow compile and clone times, felt productivity was limited
Led by: DevOps
Summary: Palantir was always a fan of ephemeral environments, and they discuss the benefits of working this way in a large enterprise environment. At Palantir, these cloud-based workspaces improved local build times, increased developer productivity, and reduced onboarding costs. The article also covers the complexities and challenges of implementing this type of environment and how Palantir addressed them. Interestingly, Palantir saw that this might be more expensive than local development but that productivity gains offset the costs. ‍

GitHub - 2500 engineers

Read about it here: https://github.blog/2021-08-11-githubs-engineering-team-moved-codespaces/
Pains: Long build times
Summary: GitHub used Codespaces to migrate its core repository to a remote development environment. This has shortened build times (from 45 minutes to 10 seconds), reduced the time required to recover from local development breakdowns, and improved collaboration. Developers can access development environments from any device using the remote development environment (via VSCode).
Full disclosure: Codespaces is a competitor to Nimbus.

Shopify - 4500 engineers

Read about it here: https://shopify.engineering/shopifys-cloud-development-journey
Pains: Project and product complexity had grown, there were huge monoliths, and developers lacked local resources to work effectively. Additionally, it took a long time to ship software, and it was slow to reproduce environments and collaborate.
Led by: Developer Tooling and Infrastructure
Summary: Shopify outgrew its local development environment when the products started becoming too complex, and it became slow to support developers. Shopify’s written about this many times, but in this article, they wrote about how they used a remote development environment supported by a lightweight virtual machine to deliver development environments. This allows developers to partition their work and better control supporting processes, improving performance and reducing shipping times. The remote development environment enables developers to collaborate and validate changes in running applications without duplicating the entire environment.

LinkedIn - 8000 engineers

Read about it here: https://engineering.linkedin.com/blog/2021/building-in-the-cloud-with-remote-development
Pains: Long build times
Led by: Developer Productivity and Happiness
Summary: By utilizing a remote development environment, LinkedIn’s Developer Productivity and Happiness team were able to cut the initial setup and build times for the majority of its products from up to 30 minutes to 10 seconds. The goal of the Remote Programming effort was to give all developers access to quick, dependable, consistent, and straightforward remote development environments. These remote development environments, or RDevs, are generated on fast hardware in LinkedIn’s private cloud and are containers set up for a specific product. It’s interesting to note that LinkedIn created this internally without using Codespaces. ‍

Guide to AWS Web Application Architecture

Nimbus — Mon, 28 Nov 2022 22:59:32 +0000

Engineers, educators, and government institutions are migrating to more reliable platforms that allow scalable web hosting and provide security, high performance, and availability.

As a result, AWS (Amazon Web Services), a popular cloud-computing vendor, has become the most sought-after cloud infrastructure market, offering a wide range of on-demand data storage and transportation solutions. It's a user-friendly, cost-efficient platform with over 200 integrated services for computing storage, artificial intelligence, data analytics, data lakes, the internet of things, and machine learning.

To begin with, this post will look at the AWS architecture with its essential components and benefits, and how it varies from other platform providers. We'll look at some recommended practices to consider when implementing this web application architecture and, finally, some helpful AWS tools to start with.

What Is AWS Architecture?

AWS architecture is a process of planning, designing, and integrating AWS resources into a cloud infrastructure. It represents the relationships between front-end, back-end, and network infrastructure components. Hence, cloud services can handle the large amount of data created daily and bring a lot of web traffic to apps or websites.

The AWS architecture ensures that clients build and maintain their cloud solutions in line with best practices, guidelines, and recommendations. It also showcases the basic AWS services, including EC2, Amazon S3, Elastic Load Balancing, CloudFront, Route 53, security groups, and more.

AWS is being used by big companies such as Netflix, Meta (Facebook), Airbnb, and even Adobe to confront scalability problems and also manage traffic on these sites.

Benefits of AWS Architecture

operation excellence
security
cost-friendliness
reliability
application performance

Web Application Hosting Architecture in AWS Cloud

You'll run into many technical and architectural problems while managing a web application. AWS provides a quick, easy, and cheap way to solve these problems.

The AWS architecture aims to facilitate a better alternative for on-premise systems. It offers features such as individual server mapping, configuration servers, and pricing.

When a user sends an email to the AWS server requesting to register or transfer a domain, the request will be sent to the Amazon API Gateway along with the necessary data.

The gateway then sends the user data to the AWS Lambda function, which creates an email and sends it to the third-party server using Amazon SES. A virtual private cloud is built to protect the web application, and resources are spread across availability zones to ensure redundancy during maintenance.

The AWS EC2 instances are used to host web servers, whereas the Elastic Load Balancer is used to balance the external traffic to servers. Depending on business needs, you can add or remove instances and scale them up or down based on dynamic scaling policies.

With Amazon CloudFront distribution, we can minimize latency and maintain global edge locations that serve as caches for internet and streaming content. In contrast, the Route 53 domain name service is used for internet domain registration and management.

Critical Components of an AWS Web Hosting Architecture

Load balancing with Elastic Load Balancing—This allows you to distribute load across various availability zones and AWS Auto Scaling groups for redundancy and service decoupling. It may reduce and boost load-balancing capacity by adjusting specific traffic needs. Sticky sessions are also allowed for complex routing services.
DDoS protection with AWS Shield—It automatically protects your AWS-hosted infrastructure against the most prevalent network and transport layer DDoS assaults. AWS shield also offers dynamic detection and inline mitigation to reduce application downtime and latency.
Static storage and backups with Amazon Simple Storage Service—AWS offers many storage options, including the basic storage service (S3) and Elastic Block Storage (EBS). EBS, which is equivalent to hard disks, is widely utilized when rapid storage is required. When larger file storage is requested, Amazon S3 is suggested, as it employs the cost-effective pay-per-use approach.
Firewalls with security groups—This feature provides suitable architectural security as a stateful, host-level firewall for web and application servers. It defines the protocols, ports, and source IP ranges for the EC2 system. So, you can use this component to restrict the security group's access to EC2 by configuring it using subnets or IP addresses.
Caching with Amazon ElastiCache—It manages the memory cache and is essential for memory management and successfully reducing service demands. Caching information in the database improves the architecture's performance. Using Redis or Memcached to provide caching services may help ease app and database load and also minimize latency for frequent requests.
Edge caching with Amazon CloudFront—Amazon CloudFront is a network for delivering content that uses low latency to send content directly worldwide. It uses network locations worldwide to speed up the delivery of dynamic, fixed, or streaming content. Therefore, users can request material depending on their current location.
Managed database with Amazon Relational Database Service—Relational Database Service (RDS) facilitates access to databases such as MySQL and Microsoft SQL server. It creates a multi-AZ, highly available database architecture with six alternative DB engines. It's incredibly user-friendly and straightforward to use.
AWS Lambda—The AWS Lambda function performs the back-end code and retrieves the required API gateway data. In addition to calculating service scales and managing the infrastructure necessary to execute the loaded code, Lambda computes service scales. Using Amazon cloud watch, you can also monitor a real-time visual log and matrix update.
DNS services with Amazon Route 53—Amazon DNS services are provided to facilitate domain maintenance.

Common Web Application Hosting Issues Addressed by AWS

Those in charge of administering a web application often encounter a number of challenges; AWS offers cost-effective and seamless solutions.

Unexpected traffic spikes: Traditional hosting approaches are notorious for provisioning very slowly. There are several examples of web applications going inaccessible due to an unanticipated surge in demand. This occurs when the web application fails to adapt and scale according to increased demand. The same on-demand feature that allows web apps to grow to meet typical traffic surges can also manage an unexpected load on the AWS cloud. That is to say, you can create new hosts, make them accessible within minutes, and take them offline as soon as the traffic returns to normal.
Paying for unused resources: Traditional hosting models require servers to be provisioned to handle peak capacity from the onset. During non-peak hours, these unused cycles end up going to waste. AWS-hosted web applications may take advantage of on-demand server provisioning, hence continually altering the capacity and pricing based on real traffic patterns.
Unreliable hardware costs: In a traditional hosting environment, even after production is done, you will incur extra costs at each step of the development life cycle, like creating preproduction, beta, and testing fleets to ensure the quality of the application.

When these resources aren't used optimally, the hardware lies idle for extended periods. This is not the situation in the AWS cloud, where you're free to deploy testing fleets on demand.

‍

Web Application Architecture in AWS vs. Other Platforms

Although other cloud service providers offer components that are similar to those in the AWS architecture, AWS has a more extensive user base and higher trust and reliability ratings because it has been in the cloud market the longest. Other service providers, like Microsoft Azure and Google Cloud Platform (GCP), offer 100+ and 60+ services, respectively, and are currently catching up to AWS in terms of functionality and popularity.

Here are how some common services compare across AWS, Microsoft Azure, and Google Cloud Platform:

IaaS: Amazon Elastic Compute Cloud (AWS) | Virtual machines (Azure) | Google Compute Engine (GCP)
PaaS: AWS Elastic Beanstalk (AWS) | App service and cloud services (Azure) | Google App Engine (GCP)
Containers: Amazon Elastic Compute Cloud Container Service (AWS) | Azure Kubernetes Service (AKS) (Azure) | Google Kubernetes Engine (GCP)
Serverless functions: AWS Lambda (AWS) | Azure Functions (Azure) | Google Cloud Functions (GCP)
NoSQL: Key value: Amazon DynamoDB (AWS) | Table storage (Azure) | Google Cloud Storage, Google Cloud Bigtable (GCP)
RDBMS: Amazon Relational Database Service (AWS) | SQL Database (Azure) | Google Cloud SQL (GCP)
NoSQL: Indexed: Amazon SimpleDB (AWS) | Azure Cosmos DB (Azure) | Google Cloud Datastore (GCP)
Virtual server disks: Amazon Elastic Block Store (AWS) | Managed disks (Azure) | Google Compute Engine persistent disks (GCP)
Object storage: Amazon Simple Storage Service (AWS) | Blob storage (Azure) | Google Cloud storage (GCP)
File storage: Amazon Elastic File System (AWS) | Azure file storage (Azure) | ZFS/Avere (GCP)
Cold storage: Amazon Glacier (AWS) | Azure Archive Blob Storage (Azure) | Google Cloud storage Nearline (GCP)
Peering: Direct Connect (AWS) | ExpressRoute (Azure) | Google Cloud interconnect (GCP)
Elastic Load Balancer: Elastic Load Balancer (AWS) | Azure Load Balancer (Azure) | Google Cloud Load Balancing (GCP)
DNS: Amazon Route 53 (AWS) | Azure DNS (Azure) | Google Cloud DNS (GCP)
Virtual network: Amazon Virtual Private Cloud (VPC) (AWS) | Virtual Networks (VNets) (Azure) | Virtual Private Cloud (GCP)

AWS has become the most popular cloud provider on the market because of its simplicity in offering scalability and holistic security to its consumers. However, because it's simple to connect MS tools with the Azure cloud, using the Azure cloud makes more sense for various organizations that use MS products.

Alternatively, most users choose GCP because it's cost-friendly and provides the greatest pricing model for the infrastructure that runs Google Search and YouTube.

Best Practices for AWS Web Application Architecture

Creating an AWS architecture makes it simpler to design, construct, and execute applications in the AWS cloud when deploying a solution. This stage is critical whether you develop a cloud-native architectural pattern or relocate your project to AWS.

Here are some tips to consider when using the AWS web architecture.

Update your AWS architecture Diagram regularly. AWS architecture diagrams are excellent for understanding, planning, developing, and deploying a cloud architecture solution. Keeping your AWS infrastructure up to date might be difficult since it's continually changing. However, organizations can use assisting tools in adding icons, shapes, networks, and other aspects to the diagram using design tools. Therefore, automatically, you generate a diagram for your AWS architecture.
Add data to your AWS diagrams. The AWS architecture provides an excellent depiction of your resources and their interconnectedness. Therefore, to avoid switching back and forth between your architecture and the documentation, always include more data alongside your shapes and icons.
Keep dependent resources close to each other. Consider grouping components that are related to each other to create an architecture that is straightforward to read and comprehend. This strategy also minimizes the number of intersecting arrows in your architecture, which helps to keep it simple.
Consider multiple architectures for different grouping. In a situation where you intend to share and categorize your resources according to different layers and data, consider adapting architecture drawings according to the audience. For instance, an architecture layer for the security team should include information tailored to the audience. You can group these using icons or according to shapes while focusing on a specific component.

Helpful AWS Tools for Ensuring Good Architecture

AWS SageMaker
AWS rePost
Well-Architected Framework
AWS Perspective

Conclusion of AWS Web Application Architecture

The serverless architecture of Amazon Web Services eliminates the requirement for users' servers in application deployment.

This post covered AWS, which hosts web servers, balances external traffic, and scales instances using components such as EC2, Amazon S3, Elastic Load Balancing, CloudFront, and others.

The AWS platform handles the back-end processes for building and maintaining apps. As a result, developers and engineers are no longer required to manage, scale, or provide apps manually using the AWS applications architecture.

It makes the experience seamless while offering customers' data three pillars of security—confidentiality, integrity, and availability—to give complete security and privacy.

Finally, the ultimate decision on which provider is preferred will depend on your organization's demands. Although AWS is the leading cloud service provider today, others like GCP and Azure are undoubtedly cutting-edge providers as well.

‍_Originally posted on www.usenimbus.com

‍This post was written by Anita Ihuman. Anita is a software developer with experience working with React (Next.js, Gatsby) and in the web development industry. She has proficient skills in technical blogging and public speaking, and she enjoys exchanging information. She loves contributing to open-source projects. Anita is a community manager at layer5 and creates blog posts for the community blog and she’s a content creator for the Gnome Africa Blog.

Why and How to Migrate from Heroku to AWS

Nimbus — Wed, 02 Nov 2022 22:48:57 +0000

Why and How to Migrate From Heroku to AWS

The Difference Between Heroku and AWS

Heroku and AWS are two cloud computing platforms, both with their own pros and cons. Heroku tends to be simpler and easier to get started with than AWS, but for larger projects it can quickly become costly. Because of this, it's common to eventually migrate a growing infrastructure to AWS. In this article, we'll take a look at some of the reasons why this might be advantageous as well as the basic steps of migration.

Heroku is a platform as a service (PaaS) solution based on AWS infrastructure. AWS, meanwhile, is the world's largest cloud platform. As such, it covers a lot more ground: infrastructure as a service (IaaS) and software as a service (SaaS) as well as PaaS. They're both cloud computing services and offer the same basic benefits: access to resources from anywhere in the world with a pay-as-you-go model.

The key difference is the scope of the service. While Heroku comes with a ready-made environment allowing developers to deploy directly from code without needing to work with the infrastructure, AWS covers it all from start to finish with a wide array of products and services. Depending on your project and needs, both come with advantages and disadvantages.

Should I Deploy with Heroku or AWS?

When choosing which service to use, the key factor will likely be PaaS vs IaaS. In a nutshell, Heroku offers a platform with a fixed infrastructure, meaning it's simple to use and quick to deploy, but there's less control over your infrastructure. AWS, meanwhile, offers vaster services, but you need a more solid grasp of the basics of infrastructure. Let's look at the top considerations in a little more detail:

DevOps: Because AWS works with the entirety of the infrastructure, you'll need an understanding of DevOps to set up servers. Heroku doesn't ask you to set up your own configurations. It uses Git to deploy, meaning you don't have to be versed in DevOps or AWS.
Flexibility: AWS is a massive platform, and it offers a lot more room for scaling and infrastructure changes. However, that's not the only flexibility it offers. It's also available in more regions and supports more operating systems and architectures.
Pricing: Heroku's pricing plans range from $0.01 per hour of computing for small (hobby) projects to $0.34–$0.69 per hour for high-traffic projects. Meanwhile, AWS has more robust services that move around $0.13 per hour. For larger projects, Heroku can be significantly more expensive.
Commonality: AWS has a lot of services. Many teams will already be using various AWS services for other parts of their project. In this case, it's easier to keep everything in one place, and AWS makes that easy. If you're just getting started with a small team without a lot of time or expertise for infrastructure, Heroku offers quick deployment with all the benefits of cloud computing. However, as you scale, both in terms of your team and your project, it's likely that it'll eventually grow costly and overly rigid and simplistic for your needs.

How to Migrate

You might start off with Heroku and eventually want to migrate to AWS. If you're reading this article, that's probably already the case. Let's look at the basics of one way to do so using Docker.

Create Your AWS App
You'll first want to create an AWS app to migrate to. If you're looking to start with PaaS support that's similar to what Heroku offers, there are a few AWS services you can use. Elastic Beanstalk is the closest equivalent. Once you choose your platform, it automatically creates an EC2 instance without needing much infrastructure management. AWS Lightsail also handles many of these steps via pre-configured servers that act like virtual private servers.

However, you can also create your own EC2 instance. You can do this by heading to the EC2 dashboard, where you'll find the Launch Instance section.

This will take you to the launch page. We won't go through the details here. If you don't have specific infrastructure needs, the default settings will do. Amazon's default AMI is generally the most cost-effective option, and other aspects of the setup will be app specific. You'll want to create a key pair in the key pair section and save it to your machine.

Create Docker Files and Docker Compose

You'll then want to create Docker files for your application. Heroku keeps your code on GitHub, meaning the majority of the migration will deal with the environment and configuration. Docker ensures that you're working in a consistent environment when migrating, and it makes setup and deployment on AWS significantly easier. This step can seem pretty daunting and vary according to the stack you're using, but Docker's documentation has an array of samples you can use as guidance.

As a side note, if you're not running Linux, Docker runs on a virtual machine. This can make it quite slow, but you can check out this article for tips on how to speed things up.

For the most part, in your Docker folders you'll likely want the following:

an app Docker file defining the application
an environment file with the environment variables as key-value pairs
a web Docker file defining your server
a configuration file for your server
The environment file will be a copy of your Heroku configuration variables, which you can find in the Settings tab of your Heroku dashboard:

You'll then want to create a Docker Compose file, docker-compose.yml, to manage your Docker files. This will point to your database, your application and server containers, and your environment variables. This will again depend on your specific stack, but the basic structure will be similar. You'll need a 'db', 'app' and 'web' service. Your 'web' service will depend on your 'app' service, which will in turn depend on your 'db' service. The app and web services need to be built from Docker files, while the 'db' only needs to specify the database.

docker-compose build

For more details on how to set up this file, make sure to read the Docker documentation.

You can then use Docker Compose to initiate the Docker containers with the following commands:

docker-compose up -d

docker ps

Migrate the Database

There are more elegant ways to do this, depending on your stack (for example, rake for Rails applications), but the surest way to migrate your database is to simply make a backup with the Heroku CLI, then change the DATABASE_URL to point to your new database.

Set Up Docker Containers on AWS

For this section, we'll work on the AWS CLI. Make sure to follow the steps in the documentation to get the CLI set up. Once you're done with that, you can SSH into your EC2 instance with the key you saved on your machine earlier by clicking Connect and following the instructions on the pop-up.

Then run the following commands to set up Docker:

sudo yum update -y

sudo yum install git

sudo amazon-linux-extras install docker

sudo service docker start

sudo usermod -a -G docker ec2-user

sudo curl -L https://github.com/docker/compose/releases/download/1.24.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

docker-compose --version

Then create your Docker containers:

cd /var

sudo mkdir www

sudo mkdir <app name>

sudo chown -R ecs-user <app name>

git clone <source code github repo>

cd <app name>

And finally, with Docker Compose:

docker-compose build

docker-compose up -d

‍

HTTPs Traffic

In order to allow traffic to the instance, you'll have to change a few settings. This goes into some security aspects that can get complicated, so take note that this is a very simplified approach that'll allow you to get things running and can be augmented with more precise rules and best practices.

Add an inbound rule to your security group to match the port in your container and allow any IP address to access it. This will be in the "Security" tab when you click on your instance.
Click on your instance's VPC ID to see your VPCs. From here, click Actions > "Edit DNS hostnames" and enable DNS hostnames in order to allow public DNS names And with that, you should be able to start your app on AWS!

Conclusion

Heroku can be an easy way to get started, but as your app grows and expands, it's likely that AWS will become more cost-effective and give you room to scale and further options to control your infrastructure. With Docker, you can migrate your app in stable containers and get up and running on AWS quickly and consistently and reap the benefits of AWS's services and economical rates.

‍

This post was written by Vivienne Roberts. Vivienne is a versatile medical physicist turned developer involved in all manner of projects, from web development and database management for small businesses to machine learning in agricultural automation. Their focus is on providing software for companies and community organizations in need of solid infrastructure.

Installing PowerShell on Mac

Nimbus — Wed, 02 Nov 2022 22:38:03 +0000

Installing PowerShell on Mac

PowerShell is one of the most utility-rich command-line shells developed by Microsoft. In fact, it's much more than just another command-line shell. If you've ever used a Windows system, you probably already know about it since it comes preinstalled on a Windows device by default.

However, most people assume that PowerShell is exclusive to a Windows system.

So, can you run PowerShell on a Mac? Yes, you can!

In this post, I'll walk you through how to install and use PowerShell on Mac. We'll explore some common commands as well.

I'll show you how to install this using Homebrew but you can find different other ways to install PowerShell on Mac from Microsoft's official guide here.

Make sure you have Homebrew installed. If you don't have it already, you can install it from here.

Now, open your command line and run the following command:

The above command installs PowerShell on your Mac device using Homebrew. The --cask flag is necessary due to the licenses involved in using and installing PowerShell on Mac. Once you do that, you may also be prompted to enter your device password (if any).

As your installation proceeds, you should see some messages on your terminal indicating the status and progress of the installation:

It should take a few minutes before the entire process is completed and PowerShell is successfully installed on your Mac.

Verify PowerShell Installation on Mac

Now that PowerShell is successfully installed, let's verify it. In order to verify that PowerShell is installed on Mac, run the pwsh command.

Running this command should give back the version of PowerShell installed. In this tutorial, we've installed version 7.2.6. It should also give a message pertaining to Microsoft Corporation's copyright.

It looks like we've also verified PowerShell's installation on our Mac device. You can further run the PSVersion command that will tell you the version of PowerShell installed.

It should also indicate the OS and platform of your device. In order to run this command, type the following on your command line and hit enter: $PSVersionTable.

Here is how the PSVersion Table should be returned back to you:

Awesome!

Let's explore some common commands in PowerShell now. As indicated in the above message, you can also directly run the help command to get some information on using PowerShell.

List All PowerShell Commands

The best way to learn and explore PowerShell is to browse through all the available commands that you can run. The Get-Command in PowerShell retrieved a list of all the available commands that you can run on it. Let's run the Get-Command. To do that, simply type "Get-Command" and hit enter on your terminal.

It should return a number of commands, some of which are shown below:

Notice that the command list also tells you the type of each command and the PowerShell version it's compatible with. Do you want to go ahead and try out a command from the Get-Command list?

Let's move ahead!

Check Built-in Variables

You can check the built-in variables that indicate the OS of your device. The next command that does this is Get-Variable. Simply run the following on your command line: Get-Variable Is*.

It should return a table with some booleans and their corresponding values. We're on a Mac right now, so only the isMacOS should be true, and the rest should be false.

Notice that there's another variable called isCoreCLR. This is a PowerShell variable that tells us if PowerShell was built targeting the device we're currently running it from.

List All Available Modules

You can also list all of the available PowerShell modules in the system using PowerShell. Run the following Get-Module command in your terminal:

It should then list all of the available modules in a table. It also tells you the directory where these modules are present.

Using Windows Commands

To list all the directories inside the current directory, we generally run the following command on a Mac: ls.

It then displays the directories or files and folders present inside the current directory where you're running the command from:

The equivalent of this in Windows is the dir command. With PowerShell, you can run all the Windows equivalents of your Mac commands. Now if we run the following on the terminal—dir—we get the same list of directories present inside the current directory in a table fashion.

With this, you also see some more metadata around these directories, such as when they were last updated, etc.

You can try running some more of the commonly used Windows commands here.

Getting Currently Running Process in PowerShell

The Get-Process command is commonly used to list all the running processes on your device locally. Let's run it by typing Get-Process on the command line and hitting enter.

It should then return a list of all the processes along with their process ID:

Further, you can also list specific processes by mentioning their process ID. For instance, if we want to see the process with ID 296, we can run the following on the terminal:

That should then return that particular process back:

Using .NET on PowerShell

PowerShell is more powerful than the regular command line. In general, command-line shells simply take a string as a command and return another string back as a result of the command. However, with PowerShell, you can run .NET natively on it. This means that you can pass and return complex .NET data structures like .NET objects.

Or you can simply run some .NET commands directly on PowerShell. For instance, here's how we can use the Math command to find the square of 3:

In the above command, [math] is the object and pow is the function available on the Math object. Let's take a look at the output as well:

Like any other shell, you can also run simple computations. Or you can see today's date using the Get-Date command:

You can further manipulate the above Get-Date command by returning a date four days in advance:

Using OpenSSL With PowerShell

Ever since Google has mandated the use of HTTPS for secure browsing, SSL has become an essential thing for developers. OpenSSL allows you to create digital SSL certificates, sign them, create private keys for them, and much more. All you need to do is run a bunch of OpenSSL commands from your command-line system.

PowerShell allows you to easily set up some environment variables required for OpenSSL. You can also run all the OpenSSL commands directly from the PowerShell.

PowerShell on Mac vs. Windows

You've seen how Windows commands can be easily translated to a Mac using PowerShell. However, you might think that PowerShell on a Mac behaves exactly the same as on Windows. However, that's not true. PowerShell is built on top of .NET and, hence, any OS level or platform-specific differences for operating .NET will introduce some disparities on Mac and Windows.

Windows systems use the full-blown .NET framework. However, Lunix and Unix OS only use a part of the .NET framework—that is, .NET core. Hence, if you run some scripts on a Windows machine that utilize components of the .NET framework that aren't present in .NET core, they'll fail to run the same way on a Mac device.

Conclusion

In this tutorial, we've learned how to install and verify PowerShell installation on Mac. Using Homebrew makes it really convenient to install PowerShell directly without installing it from native binaries, which can be trickier at times.

We've explored how to list all the commands and processes, and we've seen how we can run Windows-equivalent commands on a Mac using PowerShell.

Feel free to explore other commands or learn even more about what different commands mean and how else you can use PowerShell on your Mac device.

‍

This post was written by Siddhant Varma. Siddhant is a full stack JavaScript developer with expertise in frontend engineering. He’s worked with scaling multiple startups in India and has experience building products in the Ed-Tech and healthcare industries. Siddhant has a passion for teaching and a knack for writing. He's also taught programming to many graduates, helping them become better future developers

The Guide to Cloud Environments

Nimbus — Mon, 26 Sep 2022 23:28:17 +0000

Developing on Cloud Environments

Cloud development is more relevant than ever. It’s a more secure and scalable way of developing that lets your development and IT teams overcome any environment related disruption. For example, you can support the latest chips (e.g. Apple silicon) or fan out the secure and stable version of a tool or dependency with the click of a button. These flexible and dynamic environments bring consistency, resilience, and security while improving the developer experience.

That's why all the big companies are using cloud dev environments. Shopify built one to support their exponential growth. LinkedIn moved to the cloud and increased productivity with faster builds. Slack uses EC2 instances to run their application and its infrastructure for development. Palantir moved their development to the cloud to help developers deal with their complex enterprise environments.

Let’s look at cloud dev environments and see how they can help you.

What Is a Cloud Dev Environment?

A dev environment is where developers get their work done. It’s where code becomes software and has typically been set up and maintained by every developer on their local machine (e.g. laptop, PC, etc.).

The differences within each developer’s dev environment in a team and against other environments (e.g. production) have been a nuisance for developers since the dawn of time. Technologies, like Docker, have come out to help bring greater consistency between developers and environments.

A cloud dev environment takes this workspace off of developer laptops and moves it into the cloud. This changes how developers work as these cloud environments differ from local environments in many important ways.

How Do Cloud Dev Environments Differ From Traditional?

No Development on Local Systems

With a cloud dev environment, all your development moves off local systems. This fundamental change in how developers work yields many benefits.

When talking about cloud architecture, engineers often refer to having "cattle, not pets." It’s a reference to treating cloud systems as being both standardized and disposable: when a cloud system fails, another identical copy can seamlessly take its place.

When you shift to a cloud dev environment, you copy this philosophy from production and introduce it to development. Instead of building and testing code on laptops that are closer to pets than cattle, your developers work on systems that look as much like production as possible, while still being useful for development.

No Configuration Drift

So, there’s no configuration drift. Your development and production systems stay in sync since they're both managed in the cloud.

When developers work locally, a lost or broken laptop is a disaster. Even a minor configuration issue can lead to hours of stress, frustration, and lost productivity.

When a cloud development environment fails or breaks, you destroy it and create a new one in minutes. Here again, treating dev environments as “cattle” instead of “pets” make your infrastructure more robust and easier to maintain.

As a bonus, your laptops require less configuration and maintenance, too. If a developer’s laptop fails, they're back online in minutes with a replacement system.

Improved Security

The shift from local systems to cloud improves your security posture, too. A laptop with your tools and code installed is an attack vector. If it's compromised or lost, attackers can steal valuable secrets. This threat is diminished, if not eliminated, with cloud development environments since the laptop is only an access path.

Enable Collaboration

Local systems tend to become silos. A developer works alone on their "pet" until they're ready to push their code. Only then do they share their work and can collaboration begin, if at all.

Cloud dev environments enable collaboration because no one "owns" a system, and it’s easier to move from one cloud system to another. Developers can easily troubleshoot, share progress with other teams, work together on features, or push code from a shared repository.

Increased Productivity

Unlike a local system, you can size a cloud system for the task at hand. If you need a more powerful system to reduce build times, you create it. And, since it's in the cloud, you can shut it down or delete it when you’re finished.

Sizing systems appropriately also makes it easier for developers to switch contexts (e.g. switch branches) when they finish working on one issue and need to move to another.

Cloud environments also save engineers time by eliminating environment setup and troubleshooting, enabling faster feedback, and broadly improving the computing experience while developing (e.g. longer laptop battery life, cooler machines, better data transfer speeds).

Scalability and Streamlined Environment

Scalability is one of the many reasons teams move to the cloud. As companies grow, the product will also grow in complexity and size - development environments need to keep pace.

Most companies will eventually use the cloud in some part of their development workflow as they scale - and apart from the efforts of change, there's no reason you shouldn't take advantage of it in development. For example, some teams have used cloud environments to avoid building and maintaining additional UAT or staging environments that mirror production. Instead, they use cloud development environments for integration and performance testing as needed.

Benefits for Engineers and Engineering Managers

The technical advantages that cloud dev environments provide you with translate into business and management bonuses too.
First, local environments for your developers only need to reach the cloud environment and support business applications like email, calendar, and online meetings. They're cattle now, too. You no longer need to provide access to a local development environment.

Securing developer environments is simpler now, too. Instead of providing access to the local environment in offices, you can manage it from the cloud, regardless of where your developers sit.

This location independence provides you with many benefits beyond simplified security. Your hiring pool expands since you can support developers from anywhere that can reach your cloud. You can also support them if they move to a new home or want to travel.

Cloud dev environments scale with your teams, similar to how they scale for your products. Developers working on local systems have limited resources and limited control, while cloud teams can control and work together on systems more easily.

Teams have more flexibility and choice over their cloud dev systems, too. It's easy to build a cloud instance with an alternative OS version, package set, or hardware configuration. This gives them the power to experiment with new configurations while keeping the ability to quickly revert to the norm.

When should you NOT use Cloud Dev Environments?

While cloud dev environments have myriad benefits and advantages, they're not the best option for every development team.

Small teams (2 to 3 engineers) might not see the financial and organizational benefits of moving to the cloud.
Developing for mobile devices is not often cloud friendly. Typically, this requires local access to mobile devices, simulators and proprietary environments that don't work well in the cloud.
Developing games and other applications that require cutting-edge GPUs don't work well in the cloud. Your choices are often limited to what the public cloud providers offer.
Developers working in areas without internet connectivity. Though cloud environments are great for those with slow internet speeds, a stable connection is still needed.

How Do I Create a Cloud Dev Environment?

There are many ways to build a cloud dev environment and the differences are largely driven by the size, scale, and distribution of an organization, as well as its tech stack. Let’s look at the two most common scenarios.

Individual Developers

For an individual developer, scalability is less of an issue. They can work alone on a single cloud system, whether it's a dedicated host or a virtual machine. If they need to run tests that span multiple systems, such as a dedicated database engine or a message broker, they can create what they need, do the work, and destroy the extra systems.

Location is less of an issue too since they can pick the cloud region closest to them and build their dev environment there.
There are a number of great resources out there to learn how to build a cloud dev environment. We’ll list a few here:

Development Teams

Building a cloud dev environment for a team is more complicated. With more than one developer, issues like security, locations, scalability, and performance are a greater consideration.

More than one developer means more than one system, since your environment needs a mechanism for synchronizing your configuration across multiple hosts. It also means that you'll need tools for collaboration between developers, too.

Each additional developer means a new set of access credentials and represents a new security vector, so security is more of a concern than it is with a single developer. You're probably going to want to integrate the dev environment with SSO that ties into your existing IT infrastructure.

If you're supporting a large team, there's a good chance your environment needs to be global, too. This means providing team members with access to shared resources from more than one location, with acceptable performance.

So, you'll need to cluster, synchronize, or scale your source code control, CI/CD pipelines, database servers, and other shared services across each region.

Depending on the regions you need to cover, optimizing performance may require supporting more than one cloud provider in order to provide the best performance.

Not only is building a cloud dev environment solution for teams complex, but it’s also incredibly important because environments are critical infrastructure. If your environments aren’t working, the ENTIRE team is blocked - and there’s no greater fear for any development team. Given this - we don’t recommend following any online guide. Rather, go with a trusted provider (Nimbus, Github Codespaces, or Gitpod) or build a team of experienced infrastructure developers to create a customized solution for your specific team and org.

Shift to a Cloud Dev Environment

In having delivered cloud environments from companies with less than 10 engineers to those with over 1000, we’ve learned that the benefits are often underestimated. Cloud environments provide a better developer experience with more power and flexibility.

Cloud dev environments change how developers design, build and test code. It lets them use the device of their choice, onboard in seconds, and work from wherever they want. And it also makes it easier for teams to support developers, secure the codebase, and scale globally in ways that weren't possible before.

It’s the future of software development - try it out if you haven’t already!

Instantly Improve Docker Performance on Mac

Nimbus — Tue, 30 Aug 2022 21:44:57 +0000

Is macOS Good for Running Docker?

Macs are excellent development platforms. They support Java, Python, Ruby, and many other commonly used development languages. They let developers open a shell and find an environment that looks a lot like Linux. And many developers appreciate the user experience that Mac and Apple have become famous for. More importantly, Macs support the most popular development tools like VS Code, the JetBrains toolbox, and.. Docker – which is an important tool for testing and production environments.

But Docker's performance on Mac has a poor reputation. Let's talk about why that is and what you can do to improve it.

Docker runs differently on macOS than Linux. Docker uses the Linux kernel to manage resources between containers, so your Docker applications run on the Linux operating systems. But macOS doesn't have a Linux kernel, so Docker has to run in a Linux virtual machine. This affects performance -and this is exacerbated by certain tools or languages (like PHP projects that have heavy I/O consumption).

Does that make macOS a bad choice for running Docker? No.

MacOS is still a good choice for development, and you can get a lot done running containers on a Mac. It's not the right choice for performance testing, and you're going to hit real limits on the size and number of containers you can run depending on CPU and memory—just like you would on any local development machine.

But, running containers instead of native applications on a macOS computer is often a better option than using native installers or even Homebrew. You can fire up a MySQL database, Nginx web server, or InfluxDB in seconds with Docker — just as easily as you can on a Linux development system. That said, in many instances, you might need more CPU and memory than expected or on a Linux machine.

Is Docker on Mac Slow?

Because Docker runs in a virtual machine on macOS, container operations are slower than they would be on Linux. One of the most noticeable differences is with shared filesystems. Accessing large files, or large numbers of files, via a virtual machine in macOS will always perform worse than native access.

This doesn't mean that Docker on macOS is always slower than on Linux. For many compute and network operations, performance between macOS and Linux is still comparable, and performance gains delivered by Apple Silicon may help bridge the performance gap over time.

These performance issues are something that Docker is very aware of. This is evidenced by recent releases of Docker Desktop that have experimental features attempting to address performance on macOS. We'll look at them below as we cover general ways to improve Docker performance.

How Do I Speed Up Docker on My Mac?

There are a few ways you can make Docker run better on your Mac, and most of them are by simply changing a few settings.

How to get out of trouble (resetting changes that made performance worse)
Measuring and adjusting system resources
Experimental features (new virtualization framework and VirtioFS)
Switching to Minikube (instead of a single-node Kubernetes cluster)
Remote development (running environments on a remote machine)

How to Get Out of Trouble

Before we start, let's discuss what you can do if you change settings and things go badly. While not likely, it's possible that you could tweak a memory or CPU setting and your containers won't start. Fortunately, Docker's developers anticipated this and gave you an easy to reset your settings.

Start Docker Desktop and select the bug icon on the top right.

This opens the troubleshooting panel.

Let's go over these controls from top to bottom:

The first is a quick shortcut for restarting Docker on your Mac.
The second takes you to a support page with some diagnostics.
The third is a shortcut for restarting Kubernetes. We'll discuss running Kubernetes on a Mac below.
The fourth, as it says, will delete cached data and may fix issues with Docker not starting.
The fifth button will reset your settings. This is the control that should fix issues you cause by changing your settings.
Finally, you can uninstall Docker from this panel. So, if you change a setting and Docker stops working, try the Reset to factory defaults button.

Let's look at some system settings that can help with Docker performance.

Measuring and Adjusting System Resources

Now, click the settings gear.

Then, click Resources in the menu on the left.

Here's where you configure the resources allocated to Docker.

The defaults are:

Half of your available CPUs
2GB of memory
1GB of swap The disk image size and location will vary based on your system. Changing them won't help with performance.

Depending on the containers you're running, 2GB of memory may not be enough. But before you change the settings, you use the Docker command line to see how much memory your containers are using. (Note that my system is set to 7.90GB because I was using it to run a very large container.)

The docker stats command gives you a listing of containers and the resources they are using. The fourth column tells you how much memory a container is using.

Here's the output from Docker stats with three containers running:

With containers for Ubuntu, Nginx, and MariaDB running, my system is using less than 100MB of memory for containers.

If your system is under more pressure and uses close to the amount of memory allocated in Docker desktop resources, it may start to swap and performance will suffer. Adding more memory may help it run better.

Adjust Experimental Features

Sometimes adding memory isn't enough. Docker has added a pair of experimental features that will, hopefully, make it to production at some point in the future. Both of them deal directly with how Docker runs in a virtual machine on macOS.

Measuring if and how well these features help with performance depends on how you are using Docker. You can:

Time how long it takes for your containers to start.
Measure file system access operations, like database backups, writing large files, or sharing them from a web server. You'll find these features in a menu under the settings panel.

New Virtualization Framework

The first deals with the virtual machine Docker uses. Apple introduced a new virtualization framework with Big Sur, but Docker uses the hypervisor.framework by default. The new framework may perform better, but Docker doesn't provide official support for it yet. To enable the new framework, check the box next to it and click the button to restart Docker. You'll need to restart all your containers, too.

VirtioFS Directory Sharing

VirtioFS performs much better than the default file sharing mechanism and enabling it may help you with Docker's biggest performance problem on macOS. You need to enable the new virtualization framework to use it. Like the virtualization, check the box and click the button to restart Docker to use this.

Minikube

If you're running a single-node Kubernetes cluster on your Mac, you have two choices: the Kubernetes implementation that Docker includes with Desktop, or minikube.

Minikube isn't suitable for production environments, and it lacks many features when compared to a "full" Kubernetes cluster. But it uses considerably fewer system resources.

Here is the output from Docker stats with a minikube cluster started:

Here's Kubernetes:

If you're developing containers for a Kubernetes system or testing jobs for a system like Argo Workflow or Apache Airflow, minikube will do the job while using less memory and running fewer containers.

Remote development

Lastly, you always have the ability to shift your development environment to a cloud or remote machine. This is a way of working that’s been growing in popularity over the years and has become the primary way of developing at companies like Meta/Facebook, Shopify, Slack, LinkedIn, Palantir, and so many more. Full disclosure - this is the business that Nimbus is in, so if this is something you’re interested in check us out.

Remote development lets developers use a VM or container to host their development environment and to write/run their code. Docker (and other tools) are all run in this remote environment — thereby reducing local development resources which has the benefit of improving the local computing experience.

It’s quite easy to set up your own remote development server for individual use and there are tutorials online that show you how. If you’re building this for a team - this becomes increasingly complex but we’re looking to put out an article in the near future with guidance on that.

Running Docker on a Mac
We've looked at a few ways you can improve Docker performance on macOS. Docker Desktop has several built-in settings that you can adjust to help your system perform better. We looked at what they were and discussed options for testing if help. We also discussed how running lighter weight tools like minikube (instead of Kubernetes) can help, too. And, lastly, we talked about how offloading resource consumption to the cloud can be an easy solution.

All this is to say, that while Docker has to run through an extra layer of abstraction on a Mac, it's still perfectly usable for development. If you’re still having issues that you just can’t resolve, get in touch and we’ll see if we can provide some more guidance and get you building and hacking fast.

‍

‍Liusha is CEO of Nimbus. He's an engineer who's worked at two hyper-growth startups that have exited and then at Facebook/Meta. He found that the cloud dev infra at FAANG delivers an incredible developer experience but that even the most successful startups struggle with getting it right. He started Nimbus to bring this way of working to everyone else.‍

Originally posted on www.usenimbus.com

Laravel Development Environment: A Complete Introduction

Nimbus — Thu, 07 Jul 2022 23:08:52 +0000

Originally posted on usenimbus.com

PHP, Laravel, and the Development Flow Overview

PHP is one of the most popular programming languages for the web, and Laravel is one of the most popular frameworks for PHP. Laravel is a "batteries-included" framework that includes everything you need to build your application. Whether you need an API, an e-commerce website, or a console application to manage your internal infrastructure, Laravel has the ecosystem to meet your needs.

In this post, we'll briefly look at what a development environment is, and we'll list the different ways that you can set up a development environment. We'll also discuss the pros and cons of each approach so that you can make an informed decision about what works best for you.

In the context of web development, a development environment is a workspace for developers to make changes to a web application without breaking the production deployment. Typically, you run this on your own machine, but you can also run it remotely on a remote server.

As seen above, a common development flow will see a developer work on their development environment locally. Then it's deployed to the testing environment for integration with other features and testing by a wider audience. Finally, you push it to production once the build passes all the tests.

Keep in mind that development flows come in many forms and sizes, and there's no one-size-fits-all situation. You can break each step down further, depending on your individual requirements. In this post, we'll focus purely on the local part of the process. This is the part where you would normally write your code in your chosen code editor.

Where does Laravel fit in the development flow?

Back-end development environments generally require some sort of application server to serve your application. Some languages like PHP have their own built-in testing server, but they're only suitable for quick local tests. Server-side applications require a back-end development environment.

Front-end development environments provide the tools to develop and package client-side applications. Once packaged, they run in a client-side environment.

Laravel is a framework for PHP and requires a back-end development environment. Laravel doesn't dictate which JavaScript/CSS framework to use with your project. However, it does include a simple front-end tool named Mix to work with whatever JS or CSS framework you decide to go with.

Now let's look at a few ways you can set up a development environment for your Laravel application.

How to Install Laravel in Your Local Development Environment

All these approaches will require certain software in addition to Laravel and Composer. Where applicable, I'll list them explicitly.

Using Laravel with A Docker Development Environment

Requires: Docker

Docker is a popular containerization platform that allows you to separate your local environment from your development environment. This makes it easy to install different tools depending on your project requirements and not have them clash or affect your local environment.

Laravel has a handy tool called Sail that makes it easy to set up your Laravel application within Docker. Follow the steps below to create a new Laravel application using Docker.

## Copy Laravel files and setup in your local directory
curl -s "https://laravel.build/example-app" | bash

## Start Sail so that it can download the required Docker images
cd example-app
 ./vendor/bin/sail up
Note: For Windows, having WSL2 installed makes things much easier.

Sail installs all services by default, but you may need only a few services for your project. You can customize the services installed during the set-up phase as shown below or by updating the docker-compose.yml file. Use the following command to only install MySQL and Redis services along with your Laravel setup:

curl -s "https://laravel.build/example-app?with=mysql,redis" | bash

However, if you’ve ever run Docker on macOS (Docker Desktop), you’ve likely seen regular performance issues and theres a number of reasons for this covered here that we won’t go into now. These issues are heightened for heavy I/O consuming PHP projects and so you may want to look at other methods to set up your environment or cloud development infrastructure (like Nimbus) to offload the resource requirements and maintain a smooth development experience.

Now let's have a look at a different method to set up your development environment.

Using Laravel with a Homestead (Vagrant) Development Environment

Requires: Vagrant and one of VirtualBox, Hyper-V, or Parallels

‍Laravel Homestead is the officially supported prepackaged Vagrant box that contains everything you need inside one virtual machine. For the full list of software that comes with Vagrant, check out the Included Software section of the Laravel docs. Before installing Homestead, make sure you have a supported provider that Vagrant can work with. Once you have your preferred provider set up, you can run the following commands to get going:

# Clone Homestead
git clone https://github.com/laravel/homestead.git ~/Homestead
# Checkout the release version
cd ~/Homestead
git checkout release
# If you're on macOS / Linux
bash init.sh
# If you're on Windows
init.bat

Running the init script creates a homestead.yml file, which you can use to configure your Homestead installation. Generally, you won't need to fiddle around with the configuration as Homestead will take care of everything. But the following are key configuration values you should care about:

Provider. The provider key defines the virtualization provider that you'll use. This is usually one of VirtualBox, Hyper-V, or Parallels.
Folders. The folders key dictates how your local project folders are mapped within the VM. Laravel has some recommendations for how this mapping should be carried out.
Sites. This dictates how each folder corresponds to a website that you can access using your browser.
Homestead has numerous configuration options, so be sure to check out the Laravel docs page to find out how to get the most out of it.

Using Laravel With Composer

The previous methods covered setting up Laravel in self-contained environments that came packaged with everything you need. But what if you already have a development environment and you just need to install Laravel within that environment? You can use Composer to create the project for you or have it install the Laravel installer.

To install Laravel using Composer, run the following commands:

composer create-project laravel/laravel example-app
cd example-app
php artisan serve

If you create a lot of Laravel projects, you should consider installing the Laravel Installer. As with most things, there are a plethora of configuration options listed on the docs page.

composer global require laravel/installer
laravel new example-app
# If you have git configured correctly, you can create a Laravel project and initialize git in one go
laravel new example-app --git

Once you've created the Laravel project, you can serve it using Apache or Nginx. A sample starting point for the Nginx configuration is shown below:

server {
    listen 80;
    listen [::]:80;
    server_name example.com;
    root /srv/example.com/public;

     add_header X-Frame-Options "SAMEORIGIN";
    add_header X-Content-Type-Options "nosniff";

    index index.php; 

    charset utf-8;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php8.0-fpm.sock;
        fastcgi_param SCRIPT_FILENAME$realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }
}

And that's all you need to get going. Because of its popularity, Laravel has many different options for setting up a project. The options listed above are a good starting point to explore your options.

What Is the Best Development Environment for Laravel?

The answer to this question is that it depends. There's no magic bullet that works for everyone. During my development career, I've worked on all of these methods based on what I needed at the moment. Homestead is a great option when you want to have a development environment that has all your common services installed in it. Docker is a great option when you want to isolate projects but don't want the overhead of having a VM for each one. And if you already have a development environment set up, using Composer to add a Laravel project to it is very straightforward.

Going beyond traditional local setups, platforms like Nimbus can provide a fully functional and isolated development environment in the cloud for end-to-end development or take on resource-demanding segments of the workflow. You just need to set up VS Code or the remote editing capabilities of your favorite code editor, and off you go!

In this post, we covered the traditional ways to set up a development environment for Laravel. We also discussed the different tools that you would need to create these environments. The most effective way to find out which environment is the best fit for you is to try out all the different options available. This will give you insight into the pros and cons of each option based on your own needs. Happy hunting!

‍

‍This post was written by John Pereira. John is a technology enthusiast who's passionate about his work and all forms of technology. With over 15 years in the technology space, his area of expertise lies in API and large scale web application development, and its related constellation of technologies and processes.

How to Set Up a Microservices Dev Environment

Nimbus — Wed, 22 Jun 2022 22:20:22 +0000

What Are Microservices?

Microservices are increasingly popular way of developing software in which different teams build small pieces of functionality. Teams can work together to deliver an application with a wide range of different features.
Microservices aim to speed up the process of delivering software and make it easier to modify and update.

This post will look at how to get your microservices dev environment up and running with Docker.

Let's learn about microservices architecture by first establishing an understanding of a legacy monolith application architecture. With monolithic architecture, all the services and modules are part of a singular unit.

Each and every service develops, deploys, and scales together. Engineers must write the whole application in a single language with a single runtime to maintain and scale the monolith even when a particular service—say, a WebSocket server—would perform in a much more efficient manner if written in Golang, for instance.

Microservices architecture, splits thisone colossal application into into many small services, allowing engineering teams to build quicker, innovate faster, and scale only what's required. With this approach, you leave the one-size-fits-all architecture requirements behind.
Now, engineers and teams (like DevOps) are able to scale up only those required services and not everything at the same time - but that option still remains. Still, the only difference between microservices architecture and a monolith architecture is that scaling everything at once is no longer a constraint but rather a choice that teams hardly ever make.

Benefits of Adopting a Microservices Architecture

Microservices are all the rage in the world of web development. They're so popular that they've managed to infiltrate the world of enterprise software development. But why exactly are they getting so much heat? Let's look at a few benefits of microservices architecture.

It helps you cut costs rather smartly and invest in scaling up only those services that require some additional juice. If you're in the e-commerce industry, you could scale up your recommendations engine during the holidays to cater to that enormous demand.
Selecting the tech stack of your services is in your hands. For instance, you can write your API gateway in NodeJS, your web-scraping engine in Python, and your real-time chat server in Golang.
Individual deployment of all the services allows developers to quickly fix issues on any service without necessarily affecting all of the other services.

What Are the Basic Blocks of a Microservices Architecture?

Containerization

Basically, containerization is the process of using a bucket that holds all your source code, dependencies, frameworks, and libraries required to run a particular service or an application. Docker has to be mentioned when talking about containerization, as it's the easiest to set up and doesn't demand that much configuration, so even developers can dockerize their application or service quickly.

All you need is an essential Dockerfile in which you mention what base image your application is run upon. For example, if your app requires a Node runtime, Node offers official Docker base images for you to get started. Do you want to go pro? Build your app on a Linux base image with fine control over the network and port configuration, and Docker has your back.

Orchestration

Once you have all the services containerized—or, for the sake of this post, dockerized—what do you do with these buckets? How do you redeem those advantages of auto-scaling and all the shiny bells in reality? Here comes orchestration, in which Kubernetes is the industry standard for open-source container orchestration.

In simpler words, Kubernetes will take care of all your needs ranging from automated deployment scaling to managing those buckets.

"Kubernetes can scale without increasing your ops team." —Kubernetes

Orchestration also enables you to automate one of the most common incident response methods—i.e., "restarting the server will work for now"—where the developers fix the issue.
This is called self-healing in Kubernetes terms, where it does that for you. Having set appropriate monitoring and alerts, you can configure Kubernetes to reboot the containers in the event of lethal failures.

All of this is fine, as by this point you should know the basic definitions and how everything works in a production environment with all of these tools.

Let's move on to learn how to set up a microservices dev environment.

How to Set Up a Microservices Dev Environment

Note: Microservices environment, like any environment, can be set up locally or on the cloud. In this post, we’re going to cover how to set up a microservices dev environment locally or on cloud platforms that work like they are local (like Nimbus).

As developers, we have many tools and languages at our disposal to build applications and services. But when it comes to managing and deploying those services, things can get tricky. We may have to deal with technologies we don't know about; use new tools; or spend a lot of time on tedious, repetitive tasks.

Compose is here to make things easier. It's a tool defined in a very concise manner.

"Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration." —Docker

Follow the official installation guide and continue reading.

Environment Setup Example

Here's an example of setting up a dev environment with Docker Compose. To start, here are some of the example services we need to orchestrate:
client app—the web app built on React your end users use to browse inventory, talk to support, and place orders
server app—an Express app written in Node.js
real-time chat service—a WebSocket server written in Golang
database service—a Postgres instance as the RDBMS
database admin service—an instance of Adminer for a visual approach on DBA
cache service—a Redis instance for caching commonly/frequently requested data that is not updated that frequently
All of these examples and references are from various sources. They don't exactly fit together, but for the sake of this post, let's assume all these services belong to one product and proceed with writing a config for the docker-compose.

./docker-compose.yaml
 version: "3"

services:
  client-app:
    image: example/client-app:latest
    restart: always
    environment:
      - NODE_ENV=production
    ports:
      - 5000:5000`

  server-app:
    image: example/server-app:latest
    restart: always
    environment:
      - NODE_ENV=production
    depends_on:
      - database-service
    ports:
      - 3000:3000

  chat-service:
    image: example/chat-service:latest
    restart: always
    environment:
      - NODE_ENV=production
    ports:
      - 3001:3001

  database-service:
    image: postgres:13-alpine
    restart: always
    ports:
      - 5432:5432
    volumes:
      - ./.example-data/postgres-store:/var/lib/postgresql/data
    environment:
      POSTGRES_USER: root
      POSTGRES_PASSWORD: example_password
      POSTGRES_DB: example_db

  database-admin-service:
    image: adminer:4-standalone
    restart: always
    ports:
      - 8080:8080
    environment:
      ADMINER_DEFAULT_SERVER: database-service

  cache-service:
    image: redis:3-alpine
    ports:
      - 6379:6379
    volumes:
      - ./.example-data/redis-store:/data

With that the services are ready. The config for docker-compose is ready. Now what? Let’s utilize the CLI commands docker-compose offers to spin up our docker-compose project.

Use docker-compose up to create and start containers. In the command below, -d shows we'll be running docker-compose in the detached mode.

docker-compose -f docker-compose.yaml up -d

Use docker-compose down to stop and remove containers.

docker-compose -f essentials.yaml down

Use docker-compose logs to view output from the containers.

docker-compose logs

That's it. That's all you need to get a microservices dev environment up and running.

Summary

Though microservices have been around since the 2000s, they are still a new way of building software for many engineers, and it can be challenging to wrap one's head around the concept.

We hope this post has helped you understand what microservices are, how they work, and how you can use them to help your business. If you have any other questions or concerns, don't hesitate to contact us.

Thank you for reading!

Originally posted on www.usenimbus.com - the makers of fast and secure cloud environments that work like your laptop.

This post was written by Keshav Malik. Keshav is a full-time developer who loves to build and break stuff. He is constantly on the lookout for new and interesting technologies and enjoys working with a diverse set of technologies in his spare time. He loves music and plays badminton whenever the opportunity presents itself.

How to Improve Developer Experience: 7 Things to Change

Nimbus — Fri, 10 Jun 2022 00:39:42 +0000

What's Developer Experience? Why Should You Care?

In software development, we talk a lot about user experience, and rightly so. Since it's the end user who benefits directly from the software we write, it makes sense that they're front and center in all our efforts. But recently, another term has started to gain traction in the context of improving software quality: developer experience.

That's what this post is all about—developer experience. We'll open by briefly explaining what developer experience is and why all software organizations should care about it. From there, we'll jump into the practical bit by walking you through a list of suggestions you can adopt right away to improve the developer experience for your team or company.

Here's my definition of developer experience: how comfortable, straightforward, and enjoyable the experience of software development is for a given team or organization.

Why is developer experience so important? Why have people started talking about it so much recently?

It's not rocket science. Software is tremendously valuable, thus the people who create software are tremendously valuable as well. It's in the interest of any company that creates software to keep its developers happy and productive, especially in a hot market where developers are offered opportunities left and right.

If you improve the developer experience at your company, you're more likely to attract great developers and keep the ones you already have. Productivity is likely to go up, and so is the quality of output.

Developer Experience: Improve It With These 7 Tips

Without further ado, let's see seven practical suggestions you can adopt today to improve the developer experience at your organization.

1. Minimize Interruptions

Generally speaking, software developers hate interruptions. Not because we're prima donnas who can't be bothered with meetings or activities that can be described as "social" (though some of us certainly are.)

The problem with interruption is that software development is an activity that requires long stretches of intense focus. As any professional whose craft demands such a level of focus will say, achieving that state takes time and effort. However, losing it is easy: all it takes is Bob from accounting or your manager to drop by your cubicle or ping you on Slack to make that focus disappear into thin air.

To improve developer experience, minimize interruptions. In more practical terms, do the following:

Use asynchronous communication as much as possible. Try to use an internal newsletter, wiki articles, and e-mail updates instead of meetings and calls.
Adopt a meeting-free day. Pick a day of the week, or at least half a day, in which everyone's calendar is blocked and can't be booked. That will give everyone the chance to have dedicated time for deep concentration.
Optimize the time and duration of meetings. Meetings are unavoidable sometimes. When that's the case, pick a time and duration that makes it easy for people to focus. For instance, if your company's work hours end at 6:00 p.m., don't schedule a meeting that ends at 5:30. Those final thirty minutes will most likely be lost since they're not sufficient for developers to achieve the deep focus they need.

2. Invest in Technical Excellence

Most developers want to know that their work matters. They want to know that they're working for a team that creates software to the highest standards using the best possible practices available and delighting users with great features delivered at a fast pace.

Thus, investing in technical excellence is a great way to improve the experience of developers. The list of what you can do includes the following:

Adopt automated testing and, optionally, test-driven development (TDD).
Use continuous integration (CI) and continuous delivery and deployment (CD).
Have a code review process or, alternatively, pair programming.
Use static analyzers and linters to detect common issues in the code.
Gather and track quality metrics.
These are just a few suggestions about what you can do to foster a culture of technical excellence in which developers feel pride in their work and feel like their work has a purpose.

3. Give Developers Autonomy

Autonomy is a big driver of productivity and happiness at work. In order to feel fulfilled, people need to have at least some degree of autonomy over the way they do their jobs.

Generally speaking, developers should be awarded a high degree of autonomy over how they do their work. Allow teams to decide the following:

Which branching workflow best suit their needs
Whether they estimate using points, T-shirts sizes, hours, or something else entirely
How strictly they want to adhere to the ceremonies of their preferred flavor of agile (for instance, whether or not to have a daily scrum)
On an individual level, allow developers to choose whether they want to work remotely, at the office, or in a hybrid way. Give the contributor flexibility to assemble a benefits package that makes sense for them (some people have children, others do not, so benefits that cater only to people who are married with kids are bound to alienate a sizable part of your workforce.)

4. Remove the Friction When Creating Development Environments

One thing that virtually every developer hates is a manual, tedious, and error-prone process when creating a new development environment. These processes cause developers to take a long time until they're able to contribute code to the team. Worst of all, the processes are often not easily reproducible, which means differences between environments can be introduced.

As a consequence, the development environment drifts away from staging and production, making bugs more likely and creating the infamous "it works on my machine" excuse.

To improve developer experience, invest in ways to facilitate the creation of consistent and reproducible environments. Container solutions such as Docker are a great way to achieve that. Alternatively, you can leverage solutions that offer the creation of development environments in the cloud.

5. Invest in Developer Education

Developers also have a great experience when they feel like they master the tools of their trade. The feeling of being in control is simply exhilarating. It doesn't only lead to higher quality output, but it's also more enjoyable.

So, a sure way to improve developer experience is to invest in their education. There are several ways to do that:

Offer developers a monthly or annual stipend that they can freely use to invest in courses, books, or certifications.
Give developers time (as in, paid company time) during which they can study and practice
Offer company-paid learning resources such as a company library or services like Udemy or Pluralsight

6. Don't Be Cheap When It Comes to Tools

If you want your developers to do great work, you must give them the tools to do so. If your developers can't deliver great work due to a slow computer or the lack of a software license, that's on you.

Your engineers should have great hardware, such as a company-provided laptop, monitor, keyboard, and so on (and even office supplements like a great chair).

You should also provide the necessary software. That includes IDE licenses, subscriptions to services, plugins, and what have you. No matter what your developers need in order to perform their work, the company should, within reason, provide it.

7. Listen to Developers' Feedback (and Act Upon It)

Last but not least, here's the final item on our list: listen to your developers!

Here's the thing: software developers are usually smart people who care about what they do. Chances are they have pretty good ideas about how to improve not only their own experience but that of their teams and organizations as a whole.

Listen to your developers. When it makes sense, put their suggestions into practice. Besides the benefit created by the suggestion itself, this will have the additional benefit of improving the team's morale. Your developers will feel valued and like an essential part of the team. Which, of course, they are.

Improve Developer Experience, Improve Your Company

If your organization is a tech organization, then software developers are arguably your most precious assets. So, improving developer experience, besides being a decent, human thing to do, is also a great investment. It's in your best interest to keep your developers happy and productive, and as you've seen in this post, this is something you can achieve relatively easily.

The suggestions in this post are simply suggestions. Your mileage can vary (for instance, if you work in a highly regulated industry, giving developers a high degree of autonomy may not apply.) As always, use your judgment. Consider the list in this post as a starting point. Tweak it as needed, trim it, and add to it. What matters is that you improve the developer experience at your company for the sake of your developers, your end users, and the company as a whole.

‍

Originally posted on www.usenimbus.com - the makers of fast and secure cloud environments that work like your laptop.

This post was written by Carlos Schults. Carlos is a consultant and software engineer with experience in desktop, web, and mobile development. Though his primary language is C#, he has experience with a number of languages and platforms. His main interests include automated testing, version control, and code quality.