DEV Community: Jay Clark

Pipelines, Placeholders, and Plans - Oh my! Day 1 of Building in Public

Jay Clark — Wed, 29 Mar 2023 00:40:23 +0000

Day 1 of building in public - I’m excited!

Last year, I built a proof-of-concept app to help me plan behavioral interview answers, practice them on camera, and share the videos with friends for feedback. I found it incredibly helpful in interview preparation but I had to put it aside when I started a full-time job at Amazon. I was laid off recently and now I'm using the extra time during my job search to build Prepple, a more production-ready version of my original app concept.

I’m building in public because I think it’s incredibly helpful to be transparent about the development process. I’ll share the insights I gain and obstacles I encounter along the way, in the hopes that it can help others. It's a shame that when we work for corporations, the NDA’s we sign mean that we rarely get to share our day-to-day struggles and triumphs. This is a rare moment in my career where I can share everything as it unfolds, and I'm excited to take advantage of it.

This week I’ll be getting my design docs in order and setting up key pieces of infrastructure for the core app.

What We're Starting With

Some parts of the framework are set up already: I have a functioning deployment pipeline, and functioning REST API gateway to a serverless Java Spring app. I also have a spiffy logo and have acquired the domain name for my app.

I’m using AWS CDK to define and provision my architecture and AWS CodePipeline to deploy. AWS CDK is an infrastructure-as-code system that lets you define CloudFormation stacks in a language of your choice, and then generate & deploy the templates to automatically provision and interconnect the AWS resources you need.

AWS CodePipeline integrates with GitHub, which means a deployment through the pipeline automatically triggers when new code is merged into the target branch of the repo on GitHub. The pipeline builds the CloudFormation templates, uploads code assets, provisions the infrastructure, deploys the code assets to the infrastructure, and runs various pre-defined tests, advancing to the next stage only if all required tests pass. My pipeline is pretty standard, with beta, gamma, and prod stages. (For more context, this article gives a great overview on what each stage typically involves: https://aws.amazon.com/builders-library/automating-safe-hands-off-deployments/. Mine is a little different in that the alpha stage will be standalone and nonblocking, but I’ll go into more detail on the reasoning behind that in a later post.)

I also have a functioning boilerplate Java Spring app, currently running via a Lambda function and accessible by API Gateway. I’ve set it up so that I can test updates locally via the SAM CLI & Postman. It’s just the scaffolding of an app right now (one unauthenticated route, serving up “hello world”), but that’s all I need right now.

What's Next

Here's my game plan for getting started:

Highest priority: design docs. Coherent, well-written design docs will help avoid feature creep and ‘one-way door’ decisions (infrastructure or feature decisions that are difficult to undo.)

Next highest priority: AWS billing alerts. Before I provision any more resources in the Prepple AWS account, I’ll set up a Billing alert & spend threshold. Better safe than sorry!

Stretch goal: deploy minimal versions of a few other components the core app will need. I’d love to have a minimal auth service, a billing service, video ingest, and front-end running by the end of the week.

What do you think of this plan? I’d love to hear any thoughts, feedback, or questions you have. Leave a comment below or send me a message!

How to open 100 PRs in 100 days

Jay Clark — Thu, 03 Mar 2022 16:20:02 +0000

I recently finished an open source challenge where I contributed 100 Pull Requests to open source projects in 100 Days. I cannot recommend this style of challenge enough!

My experience level fast-forwarded so much from the challenge, I worked with dozens of new languages and tech stacks on real-life projects, and it was easier to see the commonalities across them because I was working on them in such a short period of time. I became confident in making contributions even when I felt I didn't know the language or framework well enough, while also staying constantly aware of just how much I don't know yet about coding.

I've been struggling to figure the most helpful way to share what I learned, but one of the questions I'm most often asked is: "what steps did you take to go from issue to completed PR?"

This post outlines the steps in that process. I'll add additional posts for each of them over time, with more detailed explanations. (For background on the terminology, see this previous post on what to know before you start.)

The Five Steps

In general, each pull request you do will to follow the same 5 step process:

Search GitHub issues to identify potential issues to help address
Read the contributing guidelines and eliminate any repositories with too many or too few contributing guidelines
Build your local development environment and eliminate issues where you're unable to build the environment after three tries
Do the work (write the code)
Submit your work (open the pull request)

And of course Step 6 (usually after some back & forth with the issue owner about changes to your code): celebrate your PR being merged!

Each time, you start with many potential issues and narrow them down as you go along, so that only the ones you're most likely to be able to help on are your "to do" list for the day.

Step 1: Search the issues

GitHub lets you search all issues across all repositories. Rather than try to find repositories first and then look for issues in them, it's easier to start by searching open issues. My go-to search is is:open is:issue archived:false no:assignee label:bug,"good first issue","help wanted" , followed by a keyword related to what type of work I'd like to do that day (i.e. react or responsive or logic.) It looks for open issues without an assignee (i.e. no one is publicly working on it yet) that have labels that indicate they're good for junior or first-time contributors.

Your goal in looking through the results is to find 5-10 issues that:

have clear and detailed descriptions
have keywords that you're familiar with
are narrow in scope
are less than 6 months old
have no pull requests open against them yet (the branch icon in the screenshot below indicates that a pull request has already been opened for the issue)

You should also avoid issues that are only a few days old. Many repos allow anyone to submit an issue. The project maintainers usually need time to validate new issues - you don't want to start working on a 'bug' that turns out to be a feature!

Step 2: Read the contributing guidelines

Now that you've identified some issues, you've still got vetting to do. Read the contribution guidelines and make sure you can participate without jumping through too many hoops. Some repositories are only looking for long-term contributors and want you to email them beforehand, get set up with a mentor, have regular meetings, etc. Those projects are not a good fit for this challenge -
your goal is breadth of experience, not depth.

Other projects might have zero instructions on how to contribute or set up the project locally for development - that's also not ideal.

You might find that you drop drop about 2 out of every 3 'promising' issues at this stage.

Step 3: Build your local development environment

Once you've found an issue you can contribute to, you'll need to:

Fork the repository
Clone the forked repository to your local machine
Follow the repository's directions for building your local development environment.

This is another step where you'll cut a lot of issues from your list - if you can't build the development environment after three tries, move on to the next issue. Over time, you'll learn to recognize and navigate around common issues in this step. But at the start, focus your efforts on projects where the dev environment build is successful within the first few tries. Clear instructions in the repository help a lot with this step.

If you are studying a specific stack, building environments for that stack will be the easiest for you. Start with what you know, but branch out into trying new stacks when you feel a little more confident.

Step 4: Do the work

Once you have a working local development environment, you can start to explore the repository and try to identify where the issue is coming from. If you're lucky, the person who posted the issue explained where to find it in the code. If not, you'll have to do some searching to narrow it down. Remember that GitHub has an option to search the whole repository for a word, and VSCode has the option to search all files as well.

Before you make changes, checkout a new branch in git. If there's no branch naming rules in the contributing guide, you can look at past pull requests in the repo to see whether there is an unwritten rule that people are following.

Step 5: Submit the work

When you think you've solved the issue, take a last pass through your code. Remove any console.logs, TODO's, and unused variables. You want to deliver something clean and readable! Run the repo's preferred linter to make sure you've formatted your code the right way, and make sure it passes any tests that are available for you to run.

Push your changes to your fork on GitHub and navigate to that repository in your browser. If your push was successful, you'll see a notification that has an option to open a pull request.

Once you click on the 'compare & pull request' button, you'll see fields to add a title and some descriptive content. Many repositories have templates or specific formats - follow these if they exist! If not, be succinct in your post but try to keep a friendly tone. (This is the issue owner's first time meeting you!)

Once you're satisfied with your title and comments, click the 'Create pull request' button to open your pull request.

Success!

That's it - you've opened your pull request! Give yourself a pat on the back that you've completed the five steps and have made a contribution!

This process was what I used every day to determine which issues I'd work on during the challenge. Let me know if you've found a different approach that works for you!

One thing to keep in mind: Since issues are constantly changing and being assigned, addressed, or closed out, if you embark on this challenge you'll want to get into a flow of doing these steps regularly. You can't, for example, identify an issue on a Monday and trust that it will still be in the same 'available for me to work on' state on Thursday.

In the follow up posts over the next few weeks, I'll go over each of these steps in a little more detail - thanks for reading and I hope this helps guide you on your journey!

100 Pull Requests in 100 Days: What to know before you start

Jay Clark — Thu, 24 Feb 2022 21:11:14 +0000

It's been more than a month since I finished one of the most intense challenges of my life, and I'm still struggling to figure out the best way to write about it. I thought taking a week to reflect would help, but then life got in the way - so here I am five weeks later, still not sure how to start talking about the firehose of information and insight that I experienced in those 100 days.

I'm torn between documenting my journey vs providing advice on how to take the challenge on yourself. And I certainly don't have enough space in a single post to do either of those topics full justice.

So I thought I'd do a series of posts, starting the basic foundational concepts you need to know before starting your own 100 PRs in 100 Days challenge.

(This assumes you're relatively unfamiliar with git, GitHub, and/or Open Source Software. If you're already a wiz at all three, you most likely don't need this overview and can skip to the next post!)

Fundamentals

Open Source Software:
You've probably heard this term a fair amount, but you may not be aware that open source typically means not only that the source code is freely available, but that in many cases anyone in the world can contribute to the project, subject to approval by the project's maintainers. Different projects have different rules about contributions, ranging from very restrictive (only employees can contribute, but the source code is still available to anyone) to very broad (anyone can contribute and there are relatively few rules about what types of work totally new contributors can do.) Most fall somewhere in the middle.

git
This is popular version control software. You'll need it installed on your local development machine in order to complete this challenge. You don't need to be an expert at using it yet - this challenge help you to learn and build muscle memory! Before you do this challenge, you should make sure you do know the very basic concepts, like what a repository is, how to initiate one, how to clone one, how to make a new branch and switch among branches, and also the general process of staging and committing files. (It's OK if you're still not entirely sure WHY or WHEN you should be staging & committing changes at this point.)

Github.com
This is an online community where you can sync your local git repositories, share your work with the world, and collaborate on projects. In most projects the GitHub repository serves as the "single source of truth" that contributors' local git repositories feed into and pull from. If you're a developer in training, you've very likely set up a GitHub account, but you may not know why it's useful or how to use it yet. This challenge can also help with that.

Pull Request (PR)
When you ask for your code to be incorporated into a repository on github.com, you open what is known as a Pull Request - a formal request for one of the repository maintainers to approve incorporating your code changes into the indicated branch of the main repository. The maintainer will look at your code (eventually) and usually do one of the following: approve the request, ask for changes, or close the request. If approved, your changes will be merged into the main repository and your code will become part of the project. A good thing about PRs is that you can practice them on your own repositories... for example, you can create a branch called develop and then open a request (to yourself) to pull the changes on that branch into main. So you have the option of doing a few practice runs before you try to open a PR against an external repository.

Open Source Software Concepts

Contributors: This is the set of users who have provided code, documentation, or discussion to the project in some way. Different projects have different guidelines about how can and cannot contribute, or what steps need to be taken before contributing. It's important to look for this information before doing any work, so that your efforts aren't wasted.
Maintainers: This is a subset of contributors who have administrative privileges in the repository, like the ability to approve pull requests, merge branches, or trigger workflows.
Contributing Guidelines: Many of the larger open source software projects have an official CONTRIBUTING.md file that goes over the rules of who can contribute and how, and provides helpful information on how to get started developing on the project. On smaller projects, this information is often included in the overall README.md file for the repository instead. In some cases, you may need to reach out to the repository owner for more information on whether you can contribute and how you can set up your local development environment.
Templates: Large projects will often ask you to submit your PR using a specific template, or to name your commits or branch following a particular structure. It's important to know that these rules may exist, and follow them if you can find them, so that your PR isn't rejected.

Git Concepts

Commits: These are snapshots of a project's file structure and the contents of those files. Instead of tracking every single change you've ever made to a file, git allows you to make as many changes as you want and then simply commit the difference between your current file and the last snapshot (last commit) into the repository's "memory." This can be counterintuitive if you're new to version control software -- you need to actively commit your changes in order for the software to be able to remember them after you do additional work in the future! Your commit history is public on any public projects you contribute to (including your own), so keep that in mind when deciding what commit message to write. Though we all have at least one less-than-stellar commit message written out of haste or frustration (see below.)

Rebasing & Fast-Forwarding: You may hear these terms as you progress in your project. Going in, all you need to know is that they are ways of reconciling branches that have different commit histories. For example, maybe the primary repository has a develop branch that you copied and have added Commit A to, and you are working on Commit B. But, while you were working on Commit A, Commits C, D, and E were merged into the primary repository's develop branch, and they change enough that it would make sense to have those changes in the version of the software that you're working on (but you're also not quite ready to ask to merge Commit A back into the primary.) In this situation, you will have to decide the best way to incorporate Commits C, D, and E into your local repository.
Merge Conflicts: When you open a Pull Request, or when you try to pull new commits from the primary upstream repository into your own, you may encounter something called a merge conflict. This means that the changes are significant enough or the branches are divergent (different) enough that the version control software can't figure out how to incorporate your changes in. Many merge conflicts can be resolved by the right rebasing or fast-forwarding strategy, but sometimes it's necessary to manually resolve the conflict by directly editing each file where a conflict exists.
Branches: When working on a large project, it's customary to make your own branch before you start making your particular changes. Many repositories have guidelines on what to name your branch (usually some combination of your initials, what type of work you're doing, and a brief description - i.e. jc/fix-octal-literal-duplicate-error) Paying attention to these rules will help things go smoothly and will make you look like more of a pro.

GitHub Concepts

Issues: The easiest way (and often the only way) to contribute as a totally new community member is to open a PR that fixes an issue in the repository. Issues track a variety of topics ranging from feature requests & general discussion, to bugs (something isn't working as intended) and chores (something needs to be refactored or upgraded to prevent obsolescence and avoid future bugs.)
Labels: Issues can be tagged with different labels that can help you assess whether they are something you might help fix or not. For example, "good first issue" usually means that something can probably be tackled by a first time contributor. Sometimes you'll also see issues tagged with the label "easy" or "low effort". The label "help wanted" often also means that contributions are welcome.
Assignees: The assignees of an issue are shown in the top right corner. If an issue is assigned to someone, that often means that person is working on fixing the issue - and that if you were to begin working on it, you would potentially be duplicating their efforts. Since most of this work is on a volunteer basis, no one likes to commit their time to something only to learn that it was in vain! At the same time, if you see an issue that is assigned to someone and there have been no draft PRs or discussion in several months, most likely that assignee has abandoned the issue and you can ask the maintainers if it can be reassigned to you instead.
Forks: To work on an open source software project on GitHub, the first step after identifying an issue you want to tackle is to fork the repository - this adds a copy of the main repository at that particular moment in time to your personal GitHub account. (As the primary repository has commits added to it through pull requests, you'll need to update your copy accordingly by pulling in and merging the changes.) Once you've forked, you can clone the repository to your local development machine, create a new branch for your changes, and start work using your preferred IDE.

So there you have it -- some fundamentals that you need to know before you start your challenge. In my next post, I'll walk through of the 'find issue -> fork -> develop > contribute' process now that we've got the vocabulary down, as well as some of the variation you might experience from repository to repository in terms of how things work.

Use this mnemonic to catch bugs in your code.

Jay Clark — Wed, 22 Dec 2021 16:14:43 +0000

I wasn't joking when I said I nerd out on mnemonic devices. If a checklist can be turned into an acronym, I will find a way. Having ready-to-go lists of things to do to get back on track when I hit a challenge is one of the best ways I've found to boost my productivity.

Today the mnemonic I'd like to share is one for beginners. It will help you identify what's going wrong with your code when here aren't helpful enough cues from the console, your IDE, or the test suite.

SCRUPLES

Is your syntax right?
Did you call the function you need, or just declare it?
Does the function return what it needs to?
Could there be unintended consequences to your code?
Did you put things in the right order?
Do your loops increment correctly?
Is there a way for your loops to end?
Are there issues with scope?

Syntax

When you're starting out, 99% of your problems are going to be solved right here. The browser console & your IDE provide error messages about syntax, but they aren't always clear and sometimes can be misleading. Make sure there isn't a , where there should be a ;, and that your {} and [] are nested correctly. Little things like inconsistent capitalization of variables, plural/singular typos, or incorrect formatting of fat arrow callback functions can also cause issues that can be hard to locate. If you're doing web development and setting styles, check that the values you're passing are modified with the right units (px, %, vw, etc.)

Call Your Functions

I still make this mistake more than I'd like to admit. I'll write an incredibly complicated function, then fail to ever actually call it in the code. Think of the function declaration like a recipe - you can write down the recipe for grandma's peanut butter fudge, but writing down the recipe (declaring the function) is not the same as actually making the fudge (calling the function.)

Return something from the function

If you need a function to return something, make sure it actually returns that thing. Setting the variable result = true; inside your isUserLoggedIn() function does you no good unless isUserLoggedIn() actually returns that value!

Unintended consequences

Watch out for unintended consequences. Certain methods mutate the original variable while others return a copy. If you have a function that increases a counter or increases the value of a variable, that effect will take place even if you are just logging the result of the function to the console. You'll learn over time, with practice, what to look out for. I had to learn a hard lesson about Array.forEach() and splicing/deleting array elements. I still forget that String.replace() needs to be assigned to a variable and doesn't mutate the original string.

Put things in the right order

Function declarations can appear later in the code than where they are called, but fat arrow function expressions must appear before they are called. Variables need to be declared earlier in the code than when they are used or mutated. Beyond these functional issues, consider trying to write your code like a story so that when you return to it in the future you have an easier time figuring out what you were trying to do and how.

Loops

Make sure your loops increment the way they're supposed to. This is especially important for while loops - if a counter isn't increasing or something isn't being mutated each loop, you're unlikely to ever hit the condition to exit the while loop.

End your loops

Somewhat related to the last one - make sure your loops have a way to exit. Don't be me and start a for loop with for (let i = arr.length - 1; i >= 0; i++). If your loop is taking a very long time to run, check whether you can add in a way for it to end early (if you're searching a 20,000 element array for the first index of a specific value, and you find that at i = 2, make sure you're not iterating through the entire remaining 19,997 values before ending the loop.)

Scope

Check that the scope of your variables and functions works for where you are trying to use them. If you define a helper function inside of a function declaration, that helper function is not going to be available outside of that declaration. If you're looking for the highest value in an array, but declare the variable highestValue inside the for loop you're using to scan the array, that variable will get torn down and rebuilt with each loop.

Above all, be kind to yourself. Don't beat yourself up over any of these mistakes. As I've interacted with more and more senior people in the industry, I've learned that everyone makes these mistakes. You'll be much more successful at learning to quickly identify and correct mistakes like these, than you will at training yourself not to make these mistakes at all, so focus your effort where it will make the most impact.

But my code still doesn't work!

This list isn't comprehensive. It's a checklist of the most common errors I had to train myself not to make when starting out. Your mileage may vary. You might also be less of a beginner than you think (and are encountering more complex issues.)

Think of it as a way to organize your thoughts & analysis. Especially in moments of frustration, I find it helpful to have a plan of attack.

What do you think - are there common errors I missed? Do you have your own version of 'scruples' that you use?

Last night I dreamt that somebody merged my PR...

Jay Clark — Fri, 29 Oct 2021 19:55:55 +0000

I created my first open source pull request (PR) on October 11. When I saw the automated tests transition one-by-one from processing to passed, I was already hooked. The notification came through a few hours later that my PR had been approved and merged, and it was official: I wanted to contribute to more projects, learn new languages, and master every new-to-me developer productivity tool under the sun.

A classmate had recently completed a '100 Days of Code' challenge. I'd considered doing something similar, but hadn't come up with a good focus. After that first PR merge, I found my focus: create 100 non-trivial pull requests in 100 days.

(Why specify 'non-trivial'? It's surprisingly easy to automate PRs that correct common typos or formatting errors without ever even cloning the repository. I was determined that each PR in my challenge would require me to build and use a local development environment.)

I'm almost 20 days into my challenge. I've created 30 pull requests in 29 repositories, and 26 of them have been approved and merged. (3 are awaiting review and 1 was rejected.) The projects I've contributed to range from personal portfolios and practice projects, to large open-source apps and chrome extensions used by millions of people.

Here are some things I've learned:

100 PRs in 100 days is a breakneck pace

It limits contributions to relatively easy tasks, like hunting down a small UI bug or adjusting CSS breakpoints to make a layout more responsive. Harder tasks like refactoring take a few days. I'm OK with my contributions being relatively minor in this challenge, because I'm going for breadth at this stage instead of depth.

Everything breaks, all the time

If a project has contributing guidelines that outline how to set up your local development environment, follow them TO THE LETTER. You never know when completing steps in an order that shouldn't break the setup process in theory will, in practice, break the setup process. Guides about setting environment variables should be read very closely.

Contributors are flaky

I've seen so many issues that have been claimed and then abandoned. The issue owner will check in with the assignee and get no response, then eventually un-assign it. A few days later someone new will come along asking to be assigned, and the same process will start again. Best practice is to write a note to set expectations on timeline and blockers. If solving the issue will involve multiple commits, it might also be a good idea to create a draft pull request after the first commit so that its public knowledge that progress is being made.

Reviewers are flaky too

It took 8 days for my simple formatting changes in O3DE to be reviewed. I'm still waiting (15 days and counting) for an answer & assignment on a Public Labs project. I've learned to not be too upset with radio silence - I just move on and appreciate the experience.

Every community is different

Some projects have no contributing guidelines but expect you to know & follow a strict community code. Others have delightfully extensive guides to help onboard new contributors. Repositories sometimes have strict PR templates that must be followed to the letter; almost all projects require commits to be signed off on (thankfully there is a setting in VSCode that makes that automatic.) Becoming familiar with & following multiple of these divergent community rules each week is a bigger challenge than I'd anticipated.

Contributing is incredibly rewarding

In just a few weeks, I've had the opportunity to practice so many parts of what the onboarding process will entail at a new job - setting up the environment, learning the codebase, working with new people, finding issues & tickets that are challenging enough to teach you something new, but not so challenging that you'll delay others taking too long to complete them. I am grateful every day that I've found a way to practice these skills. If I can contribute 100 PRs in 100 days, I can certainly get fully up to speed on 1 project in the same time frame.

TL;DR

Creating 100 pull requests in 100 days as a SWE-in-training is far tougher than I expected. I'm not certain the format is ideal, either - the time spent in any one repository feels so fleeting. I thought many times about altering the challenge, but ultimately the only change I made was allowing myself two PRs against the same repository (if the issues were significantly different or exceptionally complex for my skill level.)

The thing is, a challenge is supposed to be difficult. If it's easy, it's not a challenge. And I'm still in the early stages, feeling just as overwhelmed as I was in the first couple weeks of learning JavaScript, R, or C. The beauty of a challenge like this is that it turns something that seems difficult and insurmountable into a regular routine.

I can't wait to see where it takes me.