DEV Community: InRange

Move faster with Stacking PRs 🚀

Gwion Robertson — Thu, 12 Oct 2023 00:15:37 +0000

Working with large pull requests can become difficult for both the original developer and the wider team. When working on a feature branch that's thousands of lines long, keeping track of all your changes and how they affect each other is no easy task. The same goes for the reviewer, it can be a nightmare sifting through large PRs, praying you don't miss the cause of the next prod outage.

Just split the large feature into smaller features, right? But what if multiple developers are working on the same large feature? If one devs work depends on another, must they wait for the other's PR to pass review before working on their task?

Stacking PRs seeks to resolve this problem, whilst also solving many more.

What is Stacking

Stacked Pull Requests (PRs), a.k.a. dependent, incremental or chained PRs, are PRs that branch off from other PRs.

This workflow can be used in your personal projects or with a team of developers to help move faster and with more conviction.

Benefits of Stacking

The idea behind stacking is to build small coherent units, in our case PRs, to make conceptualising these changes easier.

This helps the developer as well as the reviewer for a few reasons:

Atomising your changes like this results in simpler PRs, reducing the risk of the developer burning out
Simpler PRs also reduce review time, speeding up a Team's workflow
Less code means less rebase conflicts. Stacked PRs spend less time in the review process than 'traditional' PRs
If something goes wrong, and a revert is necessary, there are less cascading side effects because there's simply less code

It also helps multiple developers work code that depends on the same feature. The devs don't have to wait for a feature to be reviewed or merged, they can just stack on top of it.

Drawbacks of Stacking

There aren't many drawbacks to stacking; however, it is worth noting that the act of splitting up a large feature can be quite time-consuming.

Knowing when a feature is small enough to be a stack is sometimes difficult to decide. As you use Stacking PRs more, you begin to get a feel of what should be its own PR.

Another drawback is that it isn't widely adopted yet, and can be confusing to other team members if you use Stacking PRs and they do not.

Stacking Example

Let us consider a real-world example. An engineer, say Alice, is asked to migrate his company's React frontend to TypeScript. We can imagine this task requires 3 features:

Initialising TypeScript
- Install TS with your preferred package manager
- Configuration (tsconfig, linting etc.)
Migrate Components
Migrate Pages

Alice begins by creating a branch called ts-init from main. She commits her changes and creates a PR and requests a review.

Now, Alice would like to begin migrating the components to TS. Traditionally, one might wait for the ts-init PR to be reviewed and merged to main before beginning a dependant feature, but with stacking, this is not the case.

Instead, Alice creates another branch called components-migration from ts-init. She commits her changes and creates a PR and requests a review as before.

Here's where it gets interesting. Another engineer, say Bob, is assigned to migrate the pages. To accomplish this, he can simply create a branch pages-migration from ts-init, and both Alice and Bob work on their respective changes simultaneously.

If, however, pages-migration was dependent on changes in components-migration, then we could inform Bob to work on something else whilst Alice finishes migrating the components. There is still the benefit that Bob could work on migrating the pages while components-migration and even ts-init are still in review.

Finally, the engineers must safely merge these feature PRs with main. The order by which each branch is merged is solely dependent on the order of creation, that is to say ts-init must merge with main first, followed by components-migration and finally pages-migration.

To reduce clutter, you might prefer this feature to read as one merge with main, then you could merge components-migration with ts-init, then pages-migration and finally merge ts-init with main.

How to stack an existing PR

There are a few different ways to convert an existing large PR into a stack of PRs, but I feel that this way is the easiest to conceptualise and track:

Unstage previous commits so that you can reassign them to different PRs. You can achieve this by:
- checkout the current PR's feature branch
- soft reset main
- restore all staged commits

$ git checkout large-feature-branch
$ git reset --soft main
$ git restore --staged .

You should rename the branch to indicate what your initial feature PR will be:

$ git branch -M initial-feature

Proceed to commit your initial changes and open up a new PR
Create a branch from initial-feauture and repeat 3-4 until there are no changes left

You've now split up a large, review nightmare of a PR into small PRs that will benefit all involved!

Conclusion

Overall, stacked PRs are a great way to break up large features into concise, ingestible PRs that will make your life easier as well as your peers'.

The process can be daunting at first, but the benefits make it all worthwhile. Though there are tools available to help with Stacking your PRs (such as Graphite) which are full of great features, I would recommend giving Stacking a go without tooling first. This will help you get a good grasp of the concept and begin to understand where the benefits come from.

Streamlining Workflows for Efficiency and Resilience: Leveraging Lambda and Step Functions.

Oluwatimilehin Odubola — Tue, 26 Sep 2023 14:21:53 +0000

Having recently participated in an enlightening workshop on Building Event-Driven App moderated by Yan Cui, I gained valuable insights into how to use AWS lambda and AWS Steps functions for orchestrating serverless workflows.

In cloud-based application development, Lambda and Step Functions are two great tools provided by Amazon Web Services for managing serverless operations.

Developers can execute code with Lambda without setting up or maintaining servers. It is perfect for doing quick, focused tasks within an application. Examples of tasks can include updating data in a DynamoDB table or executing an ETL and data transformation process in data pipelines.

In comparison, Step Functions is a service that enables the coordination of distributed applications and microservices using visual workflows. It provides a way to design, execute, and debug complex workflows, making it indispensable for managing intricate, crucial business operations.
Example of tasks where Step Functions shines includes managing multi-step approval processes in an application or orchestrating the deployment and management of cloud infrastructure, including provisioning resources, configuring services, etc.

Let's explore the unique qualities of Lambda and Step Functions below to see where each technology excels.

Using Lambda for Workflow Orchestration :

Event-Driven Execution: Lambda functions are triggered in response to events, such as changes to data in an Amazon S3 bucket or updates to a DynamoDB table. This event-driven execution model ensures efficient resource utilization, as the function only runs when needed, thus minimizing costs.
Lambda functions have extremely low startup times, allowing near-instantaneous execution. This speed and agility make Lambda suitable for quick, responsive applications where latency and responsiveness are critical factors.
Scalability: Lambda automatically scales based on the number of incoming requests. This scalability allows cost optimization by avoiding over-provisioning of resources, ensuring that you pay for only what you use, especially during low-demand periods.

Using Step Functions for Workflow Orchestration:

Reliability and Error Handling: Step Functions offer robust error handling and retries for each step in a workflow, enhancing reliability. The ability to define error recovery strategies at each step ensures smooth and consistent workflow execution in a scenario where failure occurs.
State Management: Step Functions maintain the state of each step in a workflow, making it easier to track progress and debug issues. This capability is essential for business-critical applications where tracking the status of various steps and understanding the flow of execution is crucial for successful operation.
Visual Representation and Debugging: The visual representation of workflows in Step Functions simplifies the debugging process. Developers can easily identify bottlenecks, optimize performance, and troubleshoot issues by visually inspecting the flow of steps, making debugging more accessible and effective.

Although Lambda and Step Functions can both work separately, a robust system architecture can be created to take advantage of both of their strengths in parallel. Step Functions can be utilized to orchestrate and manage the overall workflow process, seamlessly integrating with Lambda to handle each "stateful" step within the workflow.

This synergistic approach ensures efficient coordination and execution of complex workflows while leveraging the capabilities of both tools for optimal performance and reliability.

Illustration of using both tools in parallel

Summary:
Lambda functions are great for attaining operational efficiency, cost optimization, and easy workflows. Step Functions, on the other hand, are perfect for handling intricate and important business procedures, maintaining dependability, and facilitating the debugging process.

Both orchestration tools are potent instruments that meet various requirements in serverless workflow orchestration.

Streamline Your Development Process Using AWS Amplify's Deployment Environment

Oluwatimilehin Odubola — Mon, 08 May 2023 16:25:24 +0000

As a software team, it's crucial to have a streamlined process for testing and deploying new features. With AWS Amplify, deploying and testing your application becomes easy by creating separate environments for pull requests and ensuring seamless integration into the main software branch.

The development team can generate separate testing environments where work can be evaluated before going live without affecting production.

In this article, we will explore using AWS Amplify to deploy a React application and setting up a deployment environment for pull requests.

Prerequisite

Make sure to have Node/NPM installed on your computer. To install,[click here for a guide (https://nodejs.org/en/download).
Verify you have Node.js installed by using the following terminal command - type node -v .
Basic understanding of JavaScript / ReactJs
Understanding of Version Control systems i.e. Github, Gitlab, Bitbucket, etc.
An AWS account where you can access Amplify

To complete this course, you will need a React application hosted on a remote repository. To expedite the process, I will assume that you already have a React project hosted on your preferred source code repository hosting site. In this case, we will be using GitHub.

If not, you can take a look at create-react-app docs and push to a remote GitHub repository.

Now that we're done with housekeeping, so let's hop over to AWS Amplify to connect to our git repository and launch the application.

On the Amplify dashboard, we are going to be clicking on - "Amplify Hosting Option".

After which, we will connect our source code repository, as stated earlier, I am using Github, so I will be selecting Github and giving it the necessary permissions to the code repository.

Once the necessary permissions are granted, Amplify will display a list of permitted project repositories. From here, we can select the application we want to deploy, choose the branch, and set any necessary environment variables. After that, we can click on Save & Deploy to initiate the deployment process.

Now Amplify will start building and after a couple of minutes, you'll be presented with the deployed app link.
If you modify your code and push it into the master branch, it will rebuild and re-deploy your app automatically.

Finally, let's configure Amplify to work with branches created on the repository, once this is done we can have a deployment link for each branch before merging a pull request. To do this, click on Preview located on the right side of the options panel and enable it by selecting Enable Preview.

Once, that is done, we can test by creating a new branch, making some code changes, and raising a PR against the main.

As demonstrated, AWS Amplify enables the creation of a new deployment link for PR branches, allowing for testing in a separate environment before merging with the main branch. This tool can greatly enhance your development process, and I hope this article has been helpful in showcasing its capabilities.

If you have any questions or contributions, please let me know in the comments below. Don't forget to like and share. Thank you for reading!

Doubly meaningful variable names

Tom Harvey — Mon, 07 Nov 2022 17:58:00 +0000

Header image by Lucian Petrean CC

Giving meaningful names to our functions and variables helps communicate the context around our code.

Inspired by a LinkedIn post from Lowell Bander I recognised another layer of meaning in my code; one that communicates the motivation for why I code.

Here’s a snippet I was working on:

I’ve already posted about why now is the best time to work in the energy industry and it’s nice to have frequent reminders that the work is important. We’re installing solar and avoiding carbon emissions - tonnes of it!

Just as we all prefer different naming conventions, we all find motivation from different places. I hope that you can look at your code today, put aside debates about Hungarian notation, camelCasing or PEP 8 and recognise something which gives you a sense of meaning and purpose.

Put a spark in your career

Tom Harvey — Tue, 20 Sep 2022 10:02:15 +0000

The best time to work in energy is right now. It's the original high-tech industry but has been slow-moving. This stagnation, alongside recent transformations, has created the challenges and opportunities that make it a rewarding, engaging, and vital space to work.

Climate change is our generation's biggest problem, but you know that already. Solving that problem was the hook that drew me in, and it remains a daily driver, keeping me engaged. I won't be joining the chorus of voices convincing you to start your journey on a mission-led career path; you're already convinced. Right now, a confluence of factors is creating massive opportunity, and that's why now is the best time to join us, building in energy.

Decarbonization requires that the industry decentralizes generation and embraces digitization. This industry currently relies on 'test match cricket' as a communication protocol; dragging it towards current technologies is today's most exciting engineering challenge.

The existing grid operates under a century-old assumption that millions of consumers can and will rely on a few large generators. Energy flows in one direction from one place. That assumption is already broken and grows more wrong by the day. As more small local generation comes online, we each become both a source and a consumer of energy. Incumbents see this as a problem, while we see that as democratisation; and as a challenge that brings opportunity.

Commercial and industrial energy purchasers already ask for more than the industry and their energy suppliers can provide. They want real-time, high-frequency data from a supplier who measures latency in days and frequencies in half-hours. The physical grid operates at 50Hz, and as a software engineer, you're no stranger to serving inside of 20 milliseconds.

These are seismic shifts for something that underpins everything in our society; these shifts are only now beginning to accelerate, and InRange is taking advantage.

Image by mckaysavage is licensed under CC BY 2.0

At InRange, we use AI to understand, forecast, and optimize generation and consumption within local energy markets, extracting more value from each electron. The more value we can generate, the quicker we grow and decarbonize.

We're using modern data pipelines to enable high-frequency measurement of renewable generation, hyper-local consumption, and carbon avoidance. Energy consumers need this level of data for real-time, 24/7 matching of their demand with zero-carbon energy.

We're productizing a buying experience that has traditionally been a slow enterprise-led process. Our product pairs the user with AI to deliver our business proposition quickly, efficiently, and with minimal input. Business buyers have tremendous purchasing power, so giving them access to great software allows them to buy more faster.

These technologies and approaches have never been applied to the electricity industry like this before. Our success will write the operating system for the emerging energy market - decentralized, digitized, and decarbonized.

Across the industry, there are exciting opportunities in product, data, and engineering. Beyond "just" a chance to work on an exciting and vital problem, at InRange, we're building a team that holds its values above all else. Here's what we're looking to hire. We'd love to see you join us.

Running remote GUI apps with the infinite power of the cloud.

Tom Harvey — Tue, 20 Sep 2022 09:42:48 +0000

Remote development environments allow us to shift our computing needs to the infinite processing and high bandwidth offered by the cloud. It creates a shared, collaborative space that's close to our production environment. The growing toolchain around them is an exciting part of a new developer experience. But I'd never used it with a GUI app until now.

The below takes about 10-15minutes, and you get a remote, powerful desktop environment ready to run whatever Linux GUI tool you need; scientific research, 3D modelling, or just renting a faster computer for a while.

QGIS is a tool to help load, analyse and visualise geospatial data. But they don't have a build for my M1-based Mac; plus, I want to load a lot of data into it.

I'd turn to VSCode remote development or a SageMaker Studio Notebook for command line or scripting work. For a GUI app, I need to add a desktop environment to Ubuntu and access that through VNC.

Start with an Ubuntu 22.04 EC2 instance.

I've used CDK to spin this up, but go with the console, the AWS CLI, or whatever is comfortable for you.

instance = ec2.Instance(self, "DevInstance",
    instance_type=ec2.InstanceType('t3.micro'),
    machine_image=ec2.MachineImage.generic_linux(
        {'eu-west-1': 'ami-0d75513e7706cf2d9'}
        # The above is Ubuntu22.04 in the Ireland region
        # If you use another region, you'll want to find
        # the AMI for your region
    ),
    block_devices=[
        ec2.BlockDevice(
        device_name="/dev/sda1",
        volume=ec2.BlockDeviceVolume.ebs(
        )
    ],
    vpc=ec2.Vpc.from_lookup(self, "VPC", is_default=True)
)
instance.role.add_managed_policy(
    policy=iam.ManagedPolicy.from_aws_managed_policy_name(
        "AmazonSSMManagedInstanceCore"
    )
)
CfnOutput(self, "InstancePublicIp",
    value=instance.instance_id,
)

Gain CLI access to the EC2 instance.

I'm using AWS's SSM Session Manager, so I don't need to worry about ssh keys or opening ports.

aws ssm start-session --target $AWS_INSTANCE_ID

Install a desktop environment and a VNC server.

I'm using Tiger VNC and securing it with a password. Do this as the ubuntu user

sudo su
su ubuntu

sudo apt update

sudo apt install -y \
    xfce4 xfce4-goodies \
    tigervnc-standalone-server tigervnc-common tigervnc-tools

You'll now need to do a "first run" of the vnc server to do some initial configuration - like setting a password:

vncserver

This will prompt you to create a password and optionally create a "view only" password so that user can only view. I didn't set a view only, you probably don't need that either.

We can now continue with the setup of the VNC server:

cat > /home/ubuntu/.vnc/xstartup<< EOF
#!/bin/sh
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
/usr/bin/startxfce4
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
x-window-manager &
EOF

chmod u+x /home/ubuntu/.vnc/xstartup

echo ':1=ubuntu' | sudo tee -a /etc/tigervnc/vncserver.users

sudo systemctl start tigervncserver@:1.service
sudo systemctl enable tigervncserver@:1.service

Allow access to port 5901 on the server.

I've used SSM Session Manager to create a tunnel. This way, localhost:5901 tunnels across to remote:5901; saving me from opening that port in AWS and making everything more secure.

aws ssm start-session --target  $AWS_INSTANCE_ID \
    --document-name AWS-StartPortForwardingSession \
    --parameters \
    '{"portNumber":["5901"], "localPortNumber":["5901"]}'

Open a VNC client connection.

macOS comes with one! Or, download a VNC client for your machine

In the Mac's finder, use the "Go" menu, select "Connect to server" (or press command-K), and add the VNC connection path:

vnc://127.0.0.1:5901

Now, I have the desktop environment of that Ubuntu server in a window on my local Mac. I can install QGIS on that server or whatever GUI app I want. That remote machine has an ultra-fast network connection and only needs to stream the (low bandwidth) screen activity down my network connection. I can keep costs down by using a small AWS instance, or if I need to load more data, I can have up to 24TB of RAM and hundreds of CPUs! My lowly laptop can really be a wolf in sheep's clothing.