DEV Community: Jasper Gabriel

How I Contributed to Open-Source While Learning Svelte

Jasper Gabriel — Sun, 21 Jan 2024 09:10:02 +0000

If you're looking to start contributing to open-source projects, or you want to make more open-source contributions, you might already be looking at different open-source projects, seeing where you can help out.

Sometimes, contributing to open-source projects comes naturally. You might even find opportunities to improve a project's documentation while reading, or in my case while learning Svelte.

In this article, I'll share how I ended up contributing to Svelte while learning through their interactive tutorials.

My background: deciding to learn Svelte

Before we continue, let me share my background.

Throughout my career, I've mostly done backend development. I wanted to learn frontend development to build my personal website.

I was deciding between React and Svelte. After browsing through multiple blogs (including this one) and Reddit threads comparing both, I was ready to choose.

Based on my quick research, React is ideal for complex and large-scale applications while Svelte is more beginner-friendly since I don't have to learn JSX and virtual DOM.

As someone who is new to frontends and just wants to build a personal website, I ended up choosing Svelte for its simplicity.

Learning Svelte through the interactive tutorials

Now that I've chosen to learn Svelte for building my website, the first I did was visit Svelte's website. Then I went to Svelte's interactive tutorial.

The tutorial is composed of four parts, covering both Svelte and SvelteKit, which is a UI framework for Svelte.

💡 Note: Here's my notes on GitHub while learning Svelte and SvelteKit.

Finding possible improvements in the tutorial

I went through the tutorials during my free time. The first part was pretty straightforward.

Once I got to the second part, in the slots section, I noticed a difference between the docs on the left, and the code on the right.

In the left part, the docs show <div class="card ">, while the code snippet on the right shows <div class="card">. The difference is that there's an extra whitespace on the class value.

Here's a side-by-side comparison to see the difference more clearly:

<div class="card "> (Left part: docs)
<div class="card"> (Right part: code snippet)

I usually notice small details like this, so I paused and looked to improve the tutorial page.

Improving the tutorial documentation

Websites and documentation that are open-source usually have an edit page button so readers can edit the page when they find issues or possible improvements.

Fortunately, the maintainers at Svelte also make it easy to edit the interactive tutorial pages.

While still on the slots tutorial page, I clicked the Edit this page link at the bottom.

Clicking the Edit this page button brought me to the GitHub repository of Svelte's interactive tutorial.

Reviewing the contributing guidelines

While reviewing Svelte's repository, I didn't see any CONTRIBUTING.md or any other open-source contributing guidelines, which can also be found in the repository's README.md.

What I did instead was review the previous commit messages to see if there was a specific commit format that I should follow.

There was no standard commit format, so I went with the conventional commits format.

Forks, commits, and pull requests through the GitHub website

Now, I'll show you the steps on how I contributed my changes through the GitHub website.

💡 Note: The screenshots I will show for forking, comitting, and creating a pull request on GitHub were made only to show the process.

I won't create an actual pull request adding a random white space. 😉

While still on the slots page README.md file, I clicked the edit button to edit the file directly on GitHub.
Since it was my first time contributing to Svelte's tutorial, I was prompted to fork the repository.
Once I forked the repository, I edited the file with the improvements. Then I clicked Commit changes.
I was prompted to enter my commit message, along with the optional description and commit email. Then I clicked Propose changes.
After making a commit, GitHub will automatically create a new branch with patch-<number> for the branch name. My branch name is patch-3. Then I reviewed the changes before clicking Create pull request.
Finally, I was prompted to enter a pull request title and description. Then I clicked Create pull request.

Making changes on the GitHub website works for small commits, but I'd still recommend cloning the project to your local machine to test your changes.

It's ideal to make a habit of testing your changes before making a pull request to open-source projects.

So that's exactly what I did. I cloned the project and followed the README.md docs for setting up the project locally.

Once I've tested out that my changes didn't break anything, I made the pull request.

Pull request merged!

After creating my pull request, it's up to the maintainers if and when they'll approve and merge it.

After a few days, one of my pull requests was merged.

Making pull requests that introduce small changes and have good descriptions increases your chances of having your pull request approved.

It also makes it easier for the open-source project maintainers to review your pull request, since they are usually busy with a lot of things.

Other possible improvements I've found and some tips

I'd like to share that I made three pull requests, but only one has been approved and merged as of writing this.

Here are the other pull requests if you're curious:

Based on my observations, the less visited pages of a website usually have more areas for improvement. In the case of Svelte's interactive tutorials, not everyone will finish the tutorials.

As a result, the less visited tutorial content might have more areas for improvement, since fewer people can spot potential issues.

So if you're looking to improve websites and documentation, you can start looking at the least visited pages first. 😉

Conclusion

To recap, I've shared my story on how I made an open-source contribution while learning frontend development with Svelte.

Hopefully, my story shows that sometimes, you can contribute to open-source even if it's not your first intention. You just have to follow one of the scouts' principles which is to "leave the campground cleaner than you found it".

When you see issues in documentation, you can improve it as long as it's open-source. Even if it's not open-source, you can still contact the project's support team or message the author directly.

Thank you for reading and I would love to hear your open-source experiences. Feel free to comment or message me here.

Good luck on your open-source journey!

Resources

How to Make Your Awesome GitHub Profile

Jasper Gabriel — Sun, 07 Jan 2024 13:17:35 +0000

If you're new to GitHub or have mostly worked with private GitHub repositories, chances are you don't have a GitHub profile yet.

A GitHub profile helps provide basic information for those visiting your profile. Having a well-made profile can even help you stand out, especially when you start contributing to open-source projects and people start noticing you.

In this article, I'll show how to create your own GitHub profile. I'll also share where to get inspiration for your profile. Finally, I'll share resources and tips to help you create an awesome GitHub profile!

Creating Your GitHub Profile

Before you can start customizing your GitHub profile, you'll first need to create one.

Here's a short guide from GitHub on how to set up your profile.

But all you need to do is to:

Create a new repository that is the same as your GitHub username.
Add a README.md file to your new repository.

For example, my GitHub username is jdg2896. To create my profile, I need to create a repository also named jdg2896, then add a README.md file.

After setting up your repository and README.md file, verify that your profile is visible by going to your GitHub profile at https://github.com/YOUR-USERNAME.

In my case, it would be https://github.com/jdg2896.

Customizing Your GitHub Profile

Now that you have a GitHub profile, it's time to get creative!

The key here is to let your personality show on your profile. Your GitHub profile doesn't have to be too formal like LinkedIn.

I'd also suggest starting simple. This helps get your GitHub profile up and running. You can always improve your profile later when you have new ideas.

GitHub Flavored Markdown, Formatting, and HTML

For customizing your GitHub profile's README.md, you'll be using GitHub Flavored Markdown. If you've written markdown content before, formatting should be easy for you.

If it's your first time writing in markdown, you can go to GitHub's documentation to familiarize yourself with the available formatting options.

You can also use HTML for additional formatting options for your profile.

I've found the following HTML tags to be useful:

non-breaking space: nbsp;
div center align: <div align="center"> </div>

You can use most HTML tags, but GitHub Flavored Markdown filters out the following HTML tags:

<title>
<textarea>
<style>
<xmp>
<iframe>
<noembed>
<noframes>
<script>
<plaintext>

💡: To learn more, here's the GitHub Flavored Markdown Spec related to HTML blocks.

Finding Inspiration

To help you get started, I suggest looking at other awesome GitHub profiles for ideas. You can go to awesome-github-profile-readme, where I've found inspiration when making my profile.

Since the profiles are open-source, you can use some of the good ideas for your awesome profile!

You can also check out my profile for some ideas. 😉

Adding Badges

For adding badges to your profile, you can check out markdown-badges. The repository has a wide selection of badges to choose from, ranging from programming languages to streaming platforms like Netflix.

If you can't find what you're looking for or want to create custom badges, you can go to shields.io, which is what markdown-badges use.

Here's an example where I used markdown-badges on my profile.

Adding Icons

For adding a skills or tech stack section to your profile, I recommend using skill-icons which provide beautiful icons.

If your icon is not supported, you can go to simpleicons, which has over 2900 SVG icons for popular brands.

Here's an example where I used skill-icons for my profile's tech stack section.

Using Emojis

In GitHub Flavored Markdown, you can use emojis. To see the full list of supported emojis, you can go to this emoji-cheat-sheet.

If you want to get the list of supported emojis yourself, you can use GitHub's Emoji API.

Going to https://api.github.com/emojis on your browser should show a JSON response of all supported emojis.

{
  "+1": "https://github.githubassets.com/images/icons/emoji/unicode/1f44d.png?v8",
  "-1": "https://github.githubassets.com/images/icons/emoji/unicode/1f44e.png?v8",
  "100": "https://github.githubassets.com/images/icons/emoji/unicode/1f4af.png?v8",
  "1234": "https://github.githubassets.com/images/icons/emoji/unicode/1f522.png?v8",
  "1st_place_medal": "https://github.githubassets.com/images/icons/emoji/unicode/1f947.png?v8",
  "2nd_place_medal": "https://github.githubassets.com/images/icons/emoji/unicode/1f948.png?v8",
  "3rd_place_medal": "https://github.githubassets.com/images/icons/emoji/unicode/1f949.png?v8",
  "8ball": "https://github.githubassets.com/images/icons/emoji/unicode/1f3b1.png?v8",
  ...

Here's an example where I used emojis for my profile.

Adding GitHub Stats

For adding cards and stats for your GitHub activity, I recommend using github-readme-stats. You can customize your stat cards with different layouts and themes.

Here's an example where I added GitHub stats to my profile.

Adding Quotes

Adding random quotes to your profile can add a nice touch for visitors. I found github-readme-quotes to be useful for doing just that.

Here's what it looks like on my profile. I personally like to add quotes to provide some value to my profile visitors.

Improving Accessibility

When customizing your profile, make sure that it is viewable by as many people as possible. Not everyone can view or load images. Some people have disabilities, while others have slow internet connections.

One way you can improve the accessibility of your profile is by adding descriptive alt text to your images.

<!-- Markdown Image -->
![Image Alt Text](image-source)
<!-- HTML Image Tag -->
<img alt="Image Alt Text" src="image-source" />

Then to test your profile's accessibility, you can try to disable image loading on your web browser. Here's a guide on how to disable image loading for Google Chrome.

Here's what my profile looks like with image loading disabled on Google Chrome.

More Ideas

For adding more infographics to your profile, I recommend checking out metrics. This is one of the most starred repositories on GitHub with the github-profile topic, so I couldn't leave this out.

Then I found this beautiful resource beautify-github-profile, where you can find more ways to customize your profile.

If you're also feeling adventurous, you can explore the github-profile topic here. The repositories are sorted by the number of stars by default.

Feel free to explore repositories with the github-profile topic. You might even find ones that aren't used as much but are just what you need.

GitHub Profile Achievements

While this is not related to customizing your GitHub profile's README.md, I feel the need to include it.

If you go to your GitHub profile, you'll notice an Achievements section on the left sidebar.

These achievements are fun to collect and can improve your overall GitHub profile.

To learn more about what achievements are available and how to get them, check out the list of GitHub profile achievements.

Conclusion

To recap, we walked through how to create your GitHub profile. Then I showed how to format your profile with GitHub Flavored Markdown and HTML. After that, I shared where you can get inspiration for your own profile. Finally, I gave tips and resources on ways to customize your profile.

I hope this can help you in making your awesome GitHub profile. I'd love to see what you can come up with!

Thank you for reading and feel free to comment or connect with me here.

Resources

Reflections and Goal Setting as a Software Engineer

Jasper Gabriel — Sun, 31 Dec 2023 02:05:49 +0000

As software engineers, we're often busy year-round developing new features, fixing production bugs, learning new tech stacks, interacting with different teams, and many more. As a result, we tend to feel tired towards the end of the year. We would use our holiday breaks and vacation leaves to shut off completely, spending time with family and friends, traveling to new places, and doing hobbies like gaming, reading, and whatever makes us feel good.

Unwinding during our break is completely normal, and is even needed to recharge for the next year. But the break also allows us to take a pause, reflecting on what happened in the past months. Then it gives us the time and space to plan and set big goals, making sure we have something meaningful to work towards and not just run endlessly in the hamster wheel of software engineering.

In this article, I'll guide you on your journey towards reflecting on past experiences, along with planning for the future with big, ambitious goals.

Reflecting on what happened this year

“We do not learn from experience...we learn from reflecting on experience.” ― John Dewey

Often, we let the year pass by without really taking a pause. After a tiring year, we just want to binge our favorite shows, play our favorite video games, and so on.

By reflecting on our past experiences, we can appreciate our achievements, while also remembering the lessons brought by failures and mistakes.

This holiday break provides an opportunity to do exactly this, reflecting on what happened this year.

A framework for reflecting

For your reflections, I suggest starting simple, especially if you haven't done this before.

Allot a few minutes of your time to answer these questions:

What have I done this year?
What have I learned this year?

For question one, think of all the things you've done, no matter how small. Then remember what you felt after finishing the things you've done.

By remembering all the good that you've done, you appreciate yourself more. You did a good job this year!

Once you've listed all the things you've done and experienced, it's time for question two.

From those experiences, what are the lessons or improvements that you can make?

Think about what you could've done better or wish you'd done if you were able to do those things again.

Take your time in answering these two questions.

My reflections

To give you a reference that might spark ideas of your own, here are my reflections.

During 2023, I've managed to achieve the following:

Written 12 articles (including this one).
Contributed to two open-source projects.
Met up with two professionals (a representative at AWS and a founder) over coffee.
Experienced being at an early-stage startup.
Watched 13 startups (including ours) pitch live at a startup incubator's demo day.
Finished the AWS Startups Build Accelerator program.
Passed the AWS Certified Cloud Practitioner exam.
Learned Svelte and SvelteKit by going through the interactive tutorials.

Then here's what I learned from these experiences:

Writing is not just about writing itself, but all the pre-writing that comes with it. (I'm practicing Mise en Place Writing.)
You can contribute to open-source projects, even if it's just one line of code.
I should prepare more when meeting up with professionals to lessen the awkward moments.
If I can buy something instead of building it, then I should buy it. (Using existing solutions for your app's login and authentication.)
When learning new technologies, make it a habit to take notes as you learn. (Writing helps you learn faster, while also having valuable notes that you can share with others.)

Planning for the new year

“You do not rise to the level of your goals. You fall to the level of your systems.” - James Clear, Atomic Habits

A framework for planning and setting goals

For 2024, I've adapted Sahil Bloom's annual planning template, which I recommend checking out for yourself.

For setting your goals, there are two main categories, namely professional and personal. We'll be focusing on setting your professional goals, but the framework is also applicable to personal goals.

There are three steps in setting your professional goals:

Choose 1-3 big goals, they should be SMART (specific, measurable, attainable, relevant, time-bound).
Set up 1-3 checkpoint goals for each big goal you have.
Plan 2-3 daily actions that will give you progress towards your big goals.

The idea here is that you choose your big, ambitious goals that feel impossible to achieve. Then work backwards from your big goals.

After choosing your big goals, it's time to set checkpoint goals for each of your big goals. These checkpoint goals are meant to be shorter-term goals, making your big goals easier to work towards.

For example, I'd like to read 12 software engineering books for next year. In order to achieve this goal, I'll set up 2-3 checkpoint goals. So by April 1, I should have already read three books, then by July 1, six books, and by October 1, nine books.

Checkpoint goals help keep track of your progress, making sure you don't stray away from your big goals.

Lastly, the most important part of this framework is having daily actions and systems that will slowly get you towards your big goals.

To continue with the example, if I want to read three books every three months, I'd have to start with my reading habits. I'll start by reading for 30 minutes daily.

This is not a strict rule, and some days I'd find myself feeling lazy. On those days, I'd try to read one page at most.

The important thing is to do your daily habits, even if you don't feel like doing it. Remember, anything above zero compounds!

My goals

To give you an example, here's one of my big goals for 2024:

Big goal

Continue writing at least 1 article per week.

Checkpoint goals

Written at least 13 articles by April 1.
Written at least 26 articles by July 1.
Written at least 39 articles by October 1.

Daily systems

Write around 1000 words per day.
Take notes when learning new stuff.
Capture notes when consuming articles.
Document experiences when trying out new stuff.

Conclusion

To recap, we've walked through the process of reflecting on your past experiences and achievements, along with what you've learned from them. We also covered frameworks for setting your big, ambitious goals, and how to make sure that you're making daily progress towards them.

I hope this inspires you to take a pause and go on your journey of reflection and planning.

Thank you for reading and if you have any questions or feedback, I'd love to hear from you. Feel free to comment or connect with me here.

How to Create Real-Time Alerts for Blockchain Events with AWS and QuickNode

Jasper Gabriel — Sat, 23 Dec 2023 12:55:27 +0000

Blockchains have revolutionized how we store data by introducing the concept of immutable records. This feature allowed Ethereum and other blockchains to create smart contracts, which are code that can't be changed. Smart contracts ensured that conditions always execute in the same way, which enabled decentralized applications.

One downside of blockchains is that transactions take time to process. This is the price of guaranteeing the security and immutability of records. Knowing that blockchains will take time to process transactions, it is important to have real-time updates, especially for customer-facing applications.

In this article, I'll show you how to create real-time alerts for blockchain events with AWS and QuickNode.

What to expect

To give a brief overview, you'll deploy AWS infrastructure with AWS SAM, which consists of a webhook that sends email alerts.

Then you'll create a QuickAlert notification on Ethereum mainnet.

Once both are deployed, you will receive an email whenever there are transfers of at least 1 million USDT or USDC.

Here's a high-level diagram on how QuickAlert notifications work.

Note: QuickNode only support webhooks as of writing. In the future, they will support email alerts which will make this solution unnecessary.

Though the solution that I'll give you is flexible since we'll be using a Lambda function, which you can customize to fit your own use cases.

Before we start

To follow along with this guide, make sure you have:

An AWS Account
A QuickNode Account

Then knowledge of the following is helpful:

With that out of the way, let's continue.

Creating a webhook with AWS Lambda and SNS

Setting up and configuring AWS SAM CLI

Before proceeding, make sure to setup and configure AWS SAM CLI.

Then verify that you have installed AWS SAM CLI by running sam --version on your terminal.

SAM CLI, version 1.105.0

After you've verified AWS SAM CLI, it's time to create your webhook with AWS.

Deploying the sample application

I have provided a sample application in GitHub to help you get started.

The application uses AWS SAM to deploy the following resources:

Then to setup the project:

Clone the sample application.
Install Node.js dependencies by running npm install.

Add your email to the SNS Topic subscription in the template.yaml file.

Resources:
  SnsTopic:
    Type: AWS::SNS::Topic
    Properties:
      TopicName: SnsTopicEmailAlert
      DisplayName: SnsTopicEmailAlert
      Subscription:
        - Endpoint: <YOUR EMAIL HERE>
          Protocol: email

Build the application by running sam build.
Deploy the application by running sam deploy.

You should be prompted to deploy the changeset, enter y to proceed.

Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]:

If the deployment is successful, you should see something like this.

CloudFormation outputs from deployed stack
-------------------------------------------------------------------------------------------------
Outputs
-------------------------------------------------------------------------------------------------
Key                 WebEndpoint
Description         API Gateway endpoint URL for Prod stage
Value               https://<redacted>.execute-api.ap-southeast-1.amazonaws.com/Prod/
-------------------------------------------------------------------------------------------------

Successfully created/updated stack - aws-lambda-sns-poc in ap-southeast-1

Copy the endpoint shown in the output. This will be the webhook URL for your QuickAlert notification later.

Note: You can test your webhook with Postman or other API testing tools to make sure it is working.

Creating a QuickAlert notification

Now that you've successfully created a webhook, it's time to create a QuickAlert notification.

Once you have an account, here are the steps for creating your QuickAlert notification.

Select QuickAlerts on the left sidebar, then click Create QuickAlert.
Configure the following:
- Name your notification to My QuickAlert or any name you like.
- Select Ethereum and Mainnet for chain and network.
- Select Blank Template.
Then click Next to proceed.
Before continuing, you need to get a block number to test your QuickAlert expression.

Go to Etherscan and choose any transaction you'd like to test.
Once you've chosen a transaction, copy the block number.
After you've copied the block number:
- Paste it on the Target Block.
- Then click Get Data.

I've created a sample expression to filter out transfers of at least 1 million USDT or USDC.

Copy this expression and paste it into the Create Expression field.

(
  tx_logs_address == '0xdac17f958d2ee523a2206206994597c13d831ec7' ||
  tx_logs_address == '0xa0b86991c6218b36c1d19d4a2e9eb0ce3606eb48'
) &&
tx_logs_topic0 == '0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef' &&
tx_logs_topic1 =~ '^0x[0]{24}[a-fa-f0-9]{40}$' &&
tx_logs_topic2 =~ '^0x[0]{24}[a-fa-f0-9]{40}$' &&
tx_logs_topic3 == '' &&
tx_logs_data_int >= 1000000000000

Then click Test Expression.

Note: You should see your block number in the Blocks tested against section.

If there's a check mark, it means that your QuickAlert expression is working.

After testing the expression, click Next to proceed.

Now to create a Destination, click Create Destination.
Select Webhook, then click Continue.
Configure the following for your webhook:
- Webhook name to My Webhook or any name you prefer.
- Paste your webhook from earlier to the URL field.
- Select POST for the request type.
- Select 5 - Matched Receipts for the payload type.
Then Click Create Webhook.
Click the slider on the right to enable your webhook.

Then click Deploy Notification.

Testing the QuickAlert Notification

Now that you've created a QuickAlert notification, wait for it to deliver an alert. It usually takes a few seconds up to a few minutes depending on your filter expressions.

After waiting, refresh the page.

You should see changes to the Last delivery, which shows the time since the last notification was triggered.

Then click anywhere on your QuickAlert notification to view the settings.

Here, you can edit the following:

Notification Name
Expression
Destinations

Scroll down to the bottom of the page to see the events.

Once you're at the bottom of the page, you should see the Events section.

This shows all the blockchain events sent by your QuickAlert notification to your webhook.

Checking your email for alerts

Now, check the email you've configured earlier to see if you've received any emails.

You should see something similar.

Then click the Etherscan link to view the transaction.

You should see at least 1 million USDT or USDC was transferred.

After you're done testing, don't forget to disable or delete your QuickAlert notification to avoid wasting your credits.

Conclusion

To recap, you've successfully created a QuickAlert notification that sends an email whenever there are transfers of at least 1 million USDT or USDC.

I hope this can help you with integrating real-time blockchain event notifications into your applications.

Thank you for reading and if you have any questions or feedback, feel free to comment or connect with me here.

Resources

5 Things I’ve Learned from the AWS Startups Build Accelerator 2023

Jasper Gabriel — Sun, 17 Dec 2023 12:50:23 +0000

Before Y Combinator was founded in 2005 and ran its first successful summer batch, startup accelerators weren't a thing. Now, there are hundreds (if not thousands) of startup accelerators around the world. One of the accelerator programs startups can join is the AWS Startups Build Accelerator, which is a 10-week virtual program that provides training and support for launching MVPs to market.

For the 2023 cohort, out of 2000+ startups that applied, 500+ startups were selected for the program including ours. The program provided $2,000 in AWS credits, self-paced courses on AWS Skill Builder, a Slack community with other startup founders, and live AMA sessions with AWS Solution Architects.

In the 10 weeks the program ran, the following topics were covered:

Understanding Customers
Frontend Development
Backend Development
Security
Scalable Architectures
Databases
Best Practices
Generative AI
Leveraging Vendors
Innovating Iteratively Post-Launch

With the AWS Startups Build Accelerator for 2023 finally wrapped up, it's time to reflect on the lessons learned.

In this article, I'll be sharing five things I've learned throughout the program. The lessons cover what I feel is essential for building successful MVPs, which can lead to further funding rounds and startup growth.

Even though the accelerator program was intended for technical founders, non-technical founders and fellow software engineers can also find some valuable lessons applicable to their own startup journeys.

1. Focus on the customer

It's easier to make things people want than to make people want things.

One of the recurring themes in the weekly sessions was the emphasis on bringing value to customers. It makes sense since startups try to provide solutions to problems that customers are willing to pay.

In order to provide value to customers, we first need to define who our potential customers are. One way to do this is by building a Customer Persona.

Here are some questions to answer when building your customer persona:

Who are our customers?
What are their pain points?
Why are they trying to solve the pain?
How can we help them?

Once you've defined who your customer is, it's time to build something functional that adds value to customers.

You should also focus on your Unique Value Proposition (UVP), which defines how your potential customers will benefit from your solution.

A framework for defining your UVP is, "We help X do Y by doing Z".

When presenting your UVP to your potential customers, focus on the benefits, and reinforce it with features.

Another thing to keep in mind is the customer's data. Make sure to securely handle customer data, especially Personally Identifiable Information (PII).

Remember, data security is part of the user experience.

2. Scope your MVP

Don't over-engineer or over-innovate.

Once you've defined the customer pain points you're trying to solve, it's time to scope your MVP.

Oftentimes, we make the mistake of thinking too far ahead in the future, losing focus on what's important.

Remember, the point of building an MVP is to validate if your idea will solve the customer's pain points, and if customers are willing to pay to use your product.

For scoping out your MVP, here are some guidelines:

Who are your target customers?
What is the problem that you're trying to solve?
How will you solve that problem?

Make sure to focus on features that will provide value to your potential customers.

3. Leverage existing solutions for non-core functionality

Don't reinvent the wheel.

After defining both your customers and the scope of your MVP, it's time to start building.

One mistake that I've encountered a lot is trying to build every part of the system. This can be a costly mistake, especially for early-stage startups where your main goal is to validate your idea.

By trying to build parts of your system that don't provide value to customers, you risk losing your ability to move quickly and pivot if your assumptions are wrong.

There's also an opportunity cost since instead of building actual features that customers want, you're wasting time building something that could easily be built with an existing solution.

An example of where you can use existing solutions is authentication and user management. Building one from scratch will take a lot of time and won't be as secure as the existing solutions.

The exception here is if your startup is focused on providing authentication services, then it's a core part of your business and you should build it.

Here are some examples of services where you can use existing solutions:

Authentication and User Management:
- Cognito
- Auth0
- Okta
AI Solutions:
- OpenAI
- AWS
- Google Cloud
- Azure

By using existing solutions, you allow your startup to go to market faster, and actually validate your ideas and get customer feedback.

Just remember, every code you write becomes technical debt.

Become a Great Buyer

For guidelines on selecting 3rd-party vendors, the AWS Well-Architected Framework can be applied here.

Operational Excellence - Consider differentiation, switching cost, and criticality.
Security - Consider compliance attestations in the short to long term.
Reliability - Consider reputation, funding, and existing traction.
Performance - Consider product use patterns, frequency of need, and scalability.
Cost Efficiency - Consider vendor costs in the pricing model.

Use these guidelines when evaluating which vendor to use.

Outgrowing an existing solution

Eventually, if your idea is correct and you've found loyal customers, your startup will go through a rapid growth phase.

In this case, you might outgrow solutions from existing vendors. You can also use the Well-Architected Framework to re-evaluate if the 3rd-party solutions are still worth it, or if it's time to move to another vendor or build a custom solution.

It might be time to move away from a vendor if:

Pricing is prohibitively expensive at scale.
Security is not compliant with your business requirements (HIPAA, ISO, SOC2).
Scalability is not being supported, vendor solution is unstable.
Performance is not predictable.

Leveraging existing solutions with the startup I'm working with

Here is an example of leveraging existing solutions in the startup I'm currently working with.

We have a service that allows customers to sell their crypto and receive fiat in their bank accounts or e-wallets.

There are two things in this flow where we opted to use a 3rd-party vendor:

Transaction gas fee computation
Blockchain smart contract events alerting

In order to deposit crypto into the smart contract, we used Web3.js to interact with the smart contract.

For sending transactions, we needed to properly compute gas fees. In testing our MVP, we've found that sometimes the default gas values provided by Web3.js weren't enough, which caused some transactions to fail.

Luckily, MetaMask's Gas API was recently released to the public. Since MetaMask is a leading crypto wallet and its Gas API is battle-tested in production, we decided to use their Gas API instead of building our own solution.

Then after the crypto is deposited to the smart contract, we are using smart contract events to start the process of sending fiat to the customer.

We originally implemented our own solution to listen and send smart contract events, which I documented here. Later, we found out that it was causing problems and was harder to maintain at this early stage, so we switched to a 3rd-party solution with QuickNode.

In hindsight, we should've done this from the start, but that's what happens when you reflect on your experiences. You discover better ways to approach a problem or improve an existing solution.

4. Aim for iteration, not perfection

Iterate towards an MLP (Minimum Lovable Product).

After launching your MVP, you might think the journey is over and you can just wait to gain new customers. Well, not quite.

The work is just getting started after you launch your MVP. You'll need to continuously test, gather customer feedback, and improve your existing product. That's the only way you'll get and retain loyal customers, and in turn, grow your startup.

Testing your MVP

Before conducting tests with real customers, you first need to make sure that you have sufficient monitoring in place for your systems.

Ideally, you should be monitoring the health of your systems and key metrics that define success for your MVP. By having monitoring in place, you don't rely on customers (you shouldn't) to tell you that your product is not working.

After ensuring you have proper monitoring, you can conduct testing for your MVP.

Here are some approaches you can try for testing your MVP:

And here are additional ways to capture customer feedback:

Survey tools:
- Survey Monkey
- Typeform
Customer interviews
Focus groups

One tip for gathering customer feedback is to make it convenient. Make it easy for customers to provide feedback.

Then make sure you also have a presence in other channels such as:

Chatbots
Cloud-based call center
Notifications (push, SMS, email)
Social channels

Iterating your MVP

Once you've gathered enough customer feedback, it's now time to iterate your product.

Before you can quickly iterate your product, you need to have the proper infrastructure set in place to support fast iterations. Consider setting up Continuous Integration / Continuous Deployment (CI/CD) pipelines and versioning.

Having a CI/CD pipeline allows you to automate repetitive tasks, such as building deployment files, running unit and integration tests, and so on.

Versioning on the other hand allows you to keep track of your deployments, and if an issue occurs, you can quickly pinpoint which version caused the issues.

After addressing the infrastructure, it's time to review customer feedback. Make sure to allocate time to address issues raised by customers.

Then here are ways to deal with different types of feedback:

Positive feedback
- Can be published as customer testimonials (with permission)
- Builds credibility and trust
Negative feedback
- Opportunity to learn, adapt, and adjust approach to better meet customer expectations

Make sure to reflect the customer's voices into your products, be adaptable, and iterate as needed.

5. Document wins and failures

“We do not learn from experience... we learn from reflecting on experience.” ― John Dewey

Lastly, we should try to put as much as possible into writing. This means adding proper READMEs to code, creating diagrams on the system architecture and user journeys, and documenting decisions with RFCs and ADRs.

I haven't used RFCs and ADRs much, but there should be enough resources to help you get started in the links above.

Also, here's a useful resource for RFCs, Companies Using RFCs or Design Docs and Examples of These.

My lack of experience in using tools like RFCs and ADRs showed me the bad side of not properly documenting architecture, code, and decisions.

Some of the negatives when you don't document wins and failures:

harder to onboard new engineers.
knowledge silos, where information is isolated to a few members of the team.
issues repeat themselves (because of a lack of proper documentation).

By starting to document wins and failures, only then can we learn from our experiences and be better engineers, be better startups.

Conclusion

To recap, I shared what I've learned in the AWS Startups Build Accelerator 2023, which are:

bringing value to customers
scoping out MVPs
leveraging existing solutions from vendors
testing and iterating towards an MLP (Minimum Lovable Product)
documenting wins and failures.

I hope you've picked up an idea or two from the lessons I shared, and may it help you in your own startup journey.

Thank you for reading and if you have any questions or feedback, feel free to comment or connect with me here.

Resources

Starting My AWS Certification Journey as a Certified Cloud Practitioner

Jasper Gabriel — Sun, 10 Dec 2023 13:58:05 +0000

Introduction

As software engineers, we need to keep up with the pace of technology evolution by constantly learning. One example of evolving tech is cloud computing, which saw the rise of cloud providers like AWS being used in modern-day applications. Since cloud technologies are constantly evolving, certain standards for validating one's skills in the cloud are needed, which is where AWS Certifications come in.

Getting a certification isn't necessarily needed for most jobs, but having one gives you a sense of accomplishment and something to brag about.

In this article, I'll share how I got my first AWS certification, starting from my Google Cloud certification, my experience with AWS, scheduling and canceling my AWS Certified Solutions Architect - Associate exam, and finally taking and passing the AWS Certified Cloud Practitioner exam.

Before starting my AWS certification journey

Before I share my AWS certification journey, it would be helpful to share my professional background since my previous experiences helped me in getting my first certification.

Back in 2019, I passed my Google Cloud Certified Associate Cloud Engineer exam.

This was my first experience taking a certification exam at a testing center near my previous workplace. Then for my exam preparation, I mostly did hands-on labs in Qwiklabs. These hands-on activities helped me retain cloud concepts since Google Cloud (GCP) was my first experience with cloud computing.

Then in 2020, I started working with AWS. My first two years with AWS were mostly interacting with the Node.js apps I've deployed in EC2 and reviewing logs since we had a DevOps engineer who managed the cloud infrastructure.

After two years, I moved to a Web3 startup where I was given a lead software engineer role. This new role gave me more hands-on experience with AWS, where I've learned to implement serverless technologies like Lambda and DynamoDB.

After a year, I joined a new startup founded by my previous colleagues, where I managed to explore AWS more deeply since I'm the only backend engineer. I architected backend systems using serverless technologies for the POCs and MVPs.

Here's a summary of the technologies I've used in AWS to give you a sense of my knowledge before taking my AWS certification exam:

With that out of the way, let's continue with my journey.

Scheduling and canceling my AWS Certified Solutions Architect - Associate exam

Back in September, I saw a Facebook ad for the Get AWS Certified: Associate Challenge where you can get a 50% discount on the exam fee. The fee costs 150 USD at the time of writing, so you only need to pay 75 USD.

I signed up for the challenge and received an email with the voucher code.

Then I scheduled my exam in October and reviewed the landing page and resource hub to learn more about the AWS Certified Solutions Architect - Associate certification.

For my preparation, I did the following:

Reviewed the exam guide.
Answered the sample exam questions
Took the following AWS Skill Builder free courses:

For the official practice question set, I scored 50%.

Then for the Solutions Architect Knowledge Badge Assessment, I didn't pass. My score was 62.3% and the passing score was 80% out of 50 questions.

If you didn't pass, you can always retake this assessment after 24 hours.

Given the results of the assessments above and my exam schedule getting nearer, I didn't feel confident enough to proceed with the certification exam.

Then I decided to reschedule my exam to a later date. When I still felt that I lacked preparation, I finally canceled my certification exam and got a refund.

Scheduling my AWS Certified Cloud Practitioner exam

After the Get AWS Certified: Associate Challenge ended in October, I saw another ad for the Get AWS Certified: Cloud Practitioner Challenge where you can get a 25% discount on the exam fee. The fee for the AWS Certified Cloud Practitioner exam costs 100 USD at the time of writing, so you only need to pay 75 USD, which was the same price I paid for the AWS Certified Solutions Architect - Associate exam.

I signed up for the new challenge and received an email with another voucher code.

Preparing for my AWS Certified Cloud Practitioner exam

To be honest, I only had one day of preparation for this exam. I scheduled the exam on a Monday and ended up cramming my review on a Sunday night.

But as I've mentioned earlier, I already have over three years of professional experience with AWS. I have also previously attempted to take the AWS Certified Solutions Architect - Associate certification, where the preparation I did carried over.

For my short preparation, I reviewed the AWS Certified Cloud Practitioner page and took the following AWS Skill Builder free courses:

And here are my results on the assessment tests.

For the AWS Cloud Practitioner Essentials course, I skimmed through the transcripts, finishing the course in three hours.

Having passed both assessments on the first try, I was more confident going into exam day.

There's also the Exam Prep Standard Course: AWS Certified Cloud Practitioner, which I didn't take due to lack of time.

Taking my AWS Certified Cloud Practitioner exam

On the exam day, I received a text message saying that I could take the exam earlier from 11:00 am onwards if I wanted to. My exam is scheduled at noon and I'm already planning to go to the testing center one hour early to give myself time to settle down.

After arriving at the testing center, I did the registration steps; verifying my identity, signing terms and conditions, and emptying my pockets. Then I took a few deep breaths before asking the proctor to start my exam. The proctor showed me to my computer, then I started the test.

The exam has 65 questions with a 90-minute time limit. After completing and submitting my answers, I filled up a post-exam survey. Then, the final screen showed that I had passed the exam!

I finished the exam in 30 minutes, which surprised the proctors, and then I happily shared that I passed the exam.

Some tips I found helpful when taking the exam

For taking certification exams, or any exam in general, here are tips that I've found helpful.

Rest well the night before the exam day.
Read the whole question and understand what is being asked.

When you're not sure about your answer to a question:

Skip questions you're not sure of and flag them for review later.
Spend one minute max per question, if you still can't answer, review it later.
Eliminate choices when you know it isn't the answer.

Sometimes, our brains fill in the gaps which ends up misinterpreting the question. That's why I recommend being intentional when reading the question.

Then for questions you're not sure of, always answer with your best guess and avoid spending too much time on one question. It's important to keep moving since you have limited time and resources.

For eliminating choices, let's say "a company requires an object storage solution" and you're given four choices:

Amazon DynamoDB
AWS Lambda
Amazon S3
Amazon RDS

Continuing with the example, you're not familiar with object storage solutions in AWS, but know what DynamoDB, Lambda, and RDS are:

DynamoDB is a serverless key-value NoSQL database.
Lambda is a serverless function.
RDS is a fully managed relational database.

Knowing that none of the services you know are "object storage", you can eliminate them when deciding on your best guess. Then you'll end up answering Amazon S3, which is the correct answer.

This is a simple example. In actual exams, you might find it harder to eliminate more than one choice for difficult questions. The point is increasing your odds of having a correct answer for questions you're not sure of. This in turn maximizes your chances for passing the exam.

Passing the AWS Certified Cloud Practitioner exam

When I got home, I received an email for my digital badge on Credly.

I also checked my AWS Certification account and saw the results along with the perks for passing a certification exam, which includes a 50% discount voucher for my next certification exam.

Here are my exam results, where I scored 739/1000, which is just a little above the passing score of 700.

The exam results showed me that my one-day preparation was good enough to pass the exam, while also realizing that I need to have better preparation for my future certification attempts. Ideally, I'd like to pass my future certification exams with a higher buffer from the passing score, scoring at least 800 and above.

And finally here's my digital certificate.

What's next for me?

In starting my AWS certification journey, I achieved what I set out to do. By passing the AWS Certified Cloud Practitioner foundational certification, it helped me familiarize myself with the certification process and gave me a boost in confidence, while also showing areas for improvement in my preparation and review processes.

For 2024, I'm targeting the Application Architect certification path shown here, which consists of the following certifications:

Then I'd also want to take the AWS Certified Security - Specialty certification if I still have time, since security is important for cloud-based solutions.

I hope my experience can help and give you ideas for your own AWS certification journey.

Thank you for reading and if you have any questions or feedback, feel free to comment or connect with me here.

Resources

AWS Certified Cloud Practitioner

AWS Certified Solutions Architect - Associate

A Developer's Journal: Simplifying the Twelve-Factor App

Jasper Gabriel — Sun, 03 Dec 2023 14:06:00 +0000

Introduction

In my journey of constantly learning, while going through week 7 of the AWS Build Accelerator 2023 on software engineering best practices, I encountered the concept of the Twelve-Factor App. Naturally, I was curious and read the document to learn about the best practices derived from the authors' experiences in building, deploying, and scaling production applications.

If you're a developer looking for a quick read on what the Twelve-Factor app is, then you've come to the right place.

In this article, I try to simplify the concepts that I've read from the Twelve-Factor App. I've also included high-level illustrations that use practical examples and AWS services.

Then, let's define what a Twelve-Factor App is.

What is the Twelve-Factor App?

The Twelve-Factor App is a collection of best practices for building scalable applications. These were documented from the experiences and observations of contributors from Heroku.

According to them, Twelve-Factor Apps:

automate setup processes to minimize friction when onboarding new developers
are portable between different execution environments
can be deployed on modern cloud providers such as AWS, GCP, and Azure.
minimize differences between development and production environments and use CI/CD.
can scale out without significant changes to the architecture.

To make our applications compliant with the definition of Twelve-Factor Apps, there are twelve guidelines to consider.

1. Codebase

One codebase tracked in revision control, many deploys

A Twelve-Factor App is:

tracked in a version control system (VCS).
- Git
- Mercurial
- Subversion
correlated to one codebase.
deployed to different environments using the same codebase.
- Production
- Staging
- Local development

A Twelve-Factor App does not:

have multiple codebases.
- Multiple codebases are distributed systems.
- Each component of the distributed system is an app.
share the same code with different apps.
- Use shared libraries for sharing code to multiple apps.

2. Dependencies

Explicitly declare and isolate dependencies

A Twelve-Factor App:

explicitly declare dependencies.
uses package managers.
- npm (JavaScript)
- pip (Python)
doesn't rely on system-wide packages.

Not relying on system-wide packages means that you should declare all dependencies needed to run your application.

An alternative here is using Docker to build your application, then you can be sure that your app has access to packages in the operating system.

By following the guidelines on app dependencies, developers can run the app locally with only the language runtime and dependency manager (JavaScript and npm, Python and pip).

3. Config

Store config in the environment

A Twelve-Factor App:

stores configs in the environment (.env file).
has different configs for each deployment.
has no hard-coded configs in the codebase.
- A codebase doesn't have hard-coded configs if you can open-source it without leaking credentials.

4. Backing services

Treat backing services as attached resources

A Twelve-Factor App can:

use local and third-party services.
- The app can use a local PostgreSQL and has no issues using a cloud service like Amazon RDS.
add, remove, or replace backing services as needed.
- If there is a database issue, you can spin up a new database restored from a recent backup.
- There should be no code changes needed.

Backing services are used by the app over the network.

Examples of backing services are:

Data Stores (Amazon RDS, MySQL, PostgreSQL)
Messaging/Queueing Systems (Amazon SQS, RabbitMQ, Beanstalkd)
SMTP (Amazon SES)
Caching Systems (Amazon ElastiCache, Memcached, Redis)

5. Build, release, run

Strictly separate build and run stages

The Build stage converts code into an executable bundle, fetches dependencies, and compiles binaries and assets.

A build is triggered by developers when new code is deployed.
The build stage allows complex workflows such as adding unit and integration tests.

The Release stage combines the build with deployment configs.

A release should have a unique release ID, with a timestamp, or an incrementing number.
Deployment tools have release management tools, where you can roll back to a previous release.

The Runtime stage runs the app in the execution environment.

6. Processes

Execute the app as one or more stateless processes

A Twelve-Factor App:

is stateless.
never assumes that anything cached will be available for a future request.
stores data that needs to be persisted in a stateful backing service like a database.
stores session state in a session store like Memcached or Redis.

The memory or filesystem can be used as cache for a single transaction.

7. Port binding

Export services via port binding

A Twelve-Factor App:

exposes HTTP endpoints by port binding.
listens to incoming requests from the port.

With port binding, an app can be the backing service for another app by providing the backing service URL. For example, a frontend app using a backend app.

8. Concurrency

Scale out via the process model

A Twelve-Factor App:

scales out (horizontal scaling) when needing more capacity.
handles diverse workloads.
- HTTP requests are handled by a web process.
- Long-running tasks are handled by a worker process.

Scaling out (horizontal scaling) is the preferred scaling solution since if you follow the guidelines of the Twelve-Factor App, your application can support more requests by adding more instances.

Scaling up (Vertical scaling) is limited in scalability since it only upgrades the instance configuration (memory, CPU, GPU), not to mention you also need to stop that instance to perform the upgrade.

When building scalable applications, a mix of scaling up and scaling out is ideal.

9. Disposability

Maximize robustness with fast startup and graceful shutdown

A Twelve-Factor App:

can be started or stopped quickly.
minimizes startup time.
shuts down gracefully.
can handle unexpected, non-graceful terminations.

For shutting down gracefully:

process current requests before shutting down.
worker process should return unprocessed jobs to the queue.
jobs should be reentrant and workers should be idempotent.

10. Dev/prod parity

Keep development, staging, and production as similar as possible

A Twelve-Factor App:

supports continuous deployment.
minimizes time gaps, personnel gaps, and tools gaps.
has similar backing services for development and production.

Backing services like database, queue, and cache should have dev/prod parity.

Types of development process gaps:

Time gap - A developer works on a feature or bug fix, which takes time.
Personnel gap - A developer writes code, while another developer deploys it.
Tools gap - Different tools are used in development and production.

To minimize development process gaps and support continuous deployment:

Developers should be able to write code and deploy it after a few minutes or a few hours.
Developers are closely involved in deploying and observing their code in production
Development and production should be as similar as possible.

Tools like Docker and Vagrant can be used to allow local environments to mimic production environments.

11. Logs

Treat logs as event streams

A Twelve-Factor App:

doesn't route or store it's output logs.
doesn't write or manage log files.
stream logs to different services for analysis and monitoring.

12. Admin processes

Run admin/management tasks as one-off processes

A Twelve-Factor App:

runs admin processes in identical environments.
deploys admin process code along with the application code.

Admin processes can include:

running database migrations.
running console commands via SSH.
running one-time scripts (like fixing bad records).

Conclusion

To recap, we've learned how we can make our applications more secure and scalable by following the Twelve-Factor App guidelines.

The Twelve-Factor App consists of:

Codebase – One codebase tracked in revision control, many deploys
Dependencies – Explicitly declare and isolate dependencies
Configuration – Store configuration in the environment
Backing services – Treat backing services as attached resources
Build, release, run – Strictly separate build and run stages
Processes – Execute the app as one or more stateless processes
Port binding – Export services via port binding
Concurrency – Scale-out via the process model
Disposability – Maximize robustness with fast start-up and graceful shutdown
Dev/prod parity – Keep development, staging, and production as similar as possible
Logs – Treat logs as event streams
Admin processes – Run admin/management tasks as one-off processes

I recommend reading through the Twelve-Factor App since it's always a good thing to read through the source material. It should take you about an hour or two to finish reading the entire document.

Thank you for reading and if you have any questions or feedback, feel free to comment or connect with me here.

How I Contributed One Line of Code to Ethereum

Jasper Gabriel — Sun, 26 Nov 2023 14:18:26 +0000

Introduction

As a software engineer who has never contributed to open-source before 2023, I've been looking for ways to contribute. Being passionate about Web3, I looked for interesting projects in this space. It was not hard to find one where I could contribute since Web3 is about openness and decentralization. Eventually, I found a project where I made my first open-source contribution which is Taiko, a type 1 ZK-EVM.

After my first open-source contribution, I looked for other projects where I could contribute. One of those projects was Ethereum, which is a blockchain network that enables Decentralized Finance, tokenization of assets, and an open internet. While reading through their contributing guidelines, I found an issue in their documentation and ended up fixing it, making my second open-source contribution.

In this article, I'll be sharing my journey on how I contributed exactly one line of code to one of Ethereum's open-source repositories, Ethereum.org.

Looking for ways to contribute to Ethereum

Ethereum has a lot of open-source repositories on their GitHub.

Some notable repositories I was looking at were:

Geth - Official Go implementation of the Ethereum protocol.
Solidity - Smart contract programming language.
Remix - Browser-based compiler and IDE for Ethereum smart contracts written with Solidity.
Ethereum.org - Official website and online resource for the Ethereum community.

When choosing which repository I could contribute to, I had to consider my current skill sets.

Being proficient in JavaScript and TypeScript, Geth and Solidity are out of the question, since Geth requires knowledge of Go, while Solidity requires knowing C++.

Remix on the other hand is mostly written in TypeScript, while Ethereum.org uses Markdown. These are the Ethereum projects that I focused on contributing to.

Finding a bug on Ethereum.org's website

After choosing Ethereum.org as a potential repository where I could contribute, I reviewed their website's contributing section.

While browsing through the different ways to contribute to Ethereum.org, I saw their translation program and checked if I could contribute by translating their documentation. As I was reading through the translation program resources page, I noticed a broken Discord link in the Office hours for translators section.

The Discord link should redirect to Ethereum.org's Discord server. Instead, I got an Invite Invalid error message.

Reviewing Ethereum.org's open-source contribution guidelines

Before proceeding to fix the invalid Discord invite link, I reviewed Ethereum.org's CONTRIBUTING.md file to make sure I was following their open-source contributing guidelines.

The guide pointed to the contributing sections in Ethereum.org's website and README.md.

Creating an issue on Ethereum.org's GitHub repository

After reviewing the contributing guidelines, I looked at existing issues to see if someone else had already reported the broken Discord invite link on the translation program resources page.

Once I verified that no one else reported the issue, I created an issue on Ethereum.org's GitHub repository.

Creating an issue was easy since there are templates for different types of issues. In my case, I needed to create a Bug report issue.

Forking Ethereum.org's repository and testing the fix

After creating the issue, I forked Ethereum.org's GitHub repository and followed their local environment setup guide. Once I've set up my local environment, I started working on the fix.

The fix was easy since I only needed to replace the Discord invite link with a working one.

I tested the fix locally to verify that the translation page is still working and that my commit fixed the issue.

Creating a pull request on Ethereum.org's GitHub repository

After verifying that my commit fixed the issue, I created a pull request to Ethereum.org's repository with the fix for the invalid Discord invite link.

Pull request reviewed and merged!

After creating the pull request, I waited for the code owners and maintainers to review the pull request.

A few days later, corwintines reviewed my pull request. In his code review, he gave valuable feedback and updated my code changes by replacing the Discord invite link with the official Ethereum.org Discord invite link.

Then, he approved and merged my pull request. He also thanked me for my contribution, which left a warm feeling even though I only contributed one line of code (less than that if you consider that he updated my code changes).

After the pull request was merged, I also got a GitPOAP, as recognition for my contribution to Ethereum.org in 2023.

Conclusion

In this journey, I've learned that it is possible to make small contributions to large projects such as Ethereum. The beauty of open-source is that anyone can make changes, no matter how small. In fact, the majority of open-source contributions fix small issues like the one I encountered. These small changes are also easier for the open-source maintainers to review.

As the scouts' saying goes, "Always leave the campground cleaner than you found it". For us open-source contributors and software engineers, we should also leave the codebase better than we first saw it. That means if you see a mistake on an open-source documentation, you can suggest a fix to the maintainers.

If you're looking to make open-source contributions yourself, I hope my story has inspired you to do the same.

Thank you for reading and if you have any questions or feedback, feel free to comment or connect with me here.

How to Implement and Deploy a Smart Contract Event Listener with AWS CDK

Jasper Gabriel — Sun, 19 Nov 2023 04:26:06 +0000

Introduction

When you think of blockchain technologies, Ethereum, and smart contracts come to mind. Ethereum is one of the leading decentralized blockchains that pioneered smart contracts, which are programmable contracts that automatically execute the contract's terms.

A powerful feature of smart contracts is their ability to emit events after successful transactions. For example, a Transfer event is emitted when a cryptocurrency like USDT is transferred from one person to another. This feature opens up a lot of possibilities for developers, such as implementing real-time updates into their applications.

This tutorial will guide you through setting up a smart contract event listener for Ethereum's USDT contract. You'll also learn to deploy the smart contract event listener to AWS using AWS CDK. After that, I'll share my personal experiences in using a smart contract event listener.

Before starting

This guide assumes that you're familiar with:

JavaScript, TypeScript, and Node.js
Ethereum and Smart Contracts

Then you should be familiar with AWS resources and already have an AWS account.

If not, then you can follow this guide on setting up your AWS account.

Knowledge of the following is helpful, but not required:

With that out of the way, let's start with setting up the smart contract event listener.

Setting up the smart contract event listener

Before we start, make sure to check or clone the sample repository, which contains all the necessary code and files for the smart contract event listener. We'll refer to this repository throughout the guide.

By the end of this section, you'll have a working smart contract event listener that is ready for deployment.

Let's get started!

Setting up the environment variables

To listen to smart contract events, you'll first need to configure two environment variables:

WebSocket endpoint
Contract address

Getting the WebSocket endpoint

For this guide, we'll use a WebSocket endpoint from Alchemy, which looks like wss://eth-mainnet.g.alchemy.com/v2/<YOUR_API_KEY>.

You can get a WebSocket from popular providers such as Alchemy and Infura. Here's a guide for using WebSockets with Alchemy.

Getting the smart contract address

For the smart contract address, we'll use Ethereum's USDT contract, with an address of 0xdAC17F958D2ee523a2206206994597C13D831ec7.

Alternatively, if you prefer to use a different smart contract, you can:

Find Ethereum smart contracts on Etherscan.
Locate ERC20 token addresses on Coingecko.
Or, use your own deployed contract address.

Important: Remember to update the src/abi.json file with the ABI for your chosen smart contract.

Depending on your use case, you can choose to listen to events of popular smart contracts or your deployed smart contracts. You can also choose to listen to smart contracts in different blockchain networks as long as it's EVM-compatible and you have a WebSocket endpoint for that blockchain network.

Configuring the .env file

Once you have both the WebSocket endpoint and the smart contract address, you can configure your .env file like this:

ETH_WSS_ENDPOINT=wss://eth-mainnet.g.alchemy.com/v2/<YOUR_API_KEY>
ETH_SMART_CONTRACT_ADDRESS=0xdac17f958d2ee523a2206206994597c13d831ec7

Reviewing and testing the code

Now that you've set up the environment variables, let's review the code to understand how the smart contract event listener works. We'll also test the code to verify that it works before we deploy it to AWS.

Reviewing the smart contract event listener code

The code can be divided into several parts:

Imports and Workarounds:
- We import the following libraries:
  - dotenv - to load environment variables from .env files for local development.
  - abi.json - the ABI of the smart contract you plan to listen to. In this case, an ERC20 ABI.
  - web3 - library to allow subscription to smart contract events.
- Then a workaround is implemented for BigInt to allow JSON.stringify() of smart contract event logs, which improves the readability of CloudWatch logs when we deploy to AWS later.
```
import 'dotenv/config';
import * as erc20abi from './abi.json';
import Web3, { Contract, WebSocketProvider } from 'web3';

/* 
Workaround for JSON.stringify() event logs with BigInt values. 
We need to stringify event logs for more readable logging in CloudWatch.
https://github.com/GoogleChromeLabs/jsbi/issues/30
*/
(BigInt.prototype as any).toJSON = function () {
return this.toString();
};
```

Main Function - startEventListener():

This function creates a WebSocket connection using your WebSocket endpoint.
Then, the function subscribes to the smart contract address that you've specified.
Finally, the function subscribes to the Transfer and Approval events using the subscribeToEvent() function.

/**
 * Starts the smart contract event listener. 
 * Websocket Provider config: https://docs.web3js.org/api/web3-providers-ws/class/WebSocketProvider
 * @param chain - Name of the blockchain network for logging purposes.
 * @param wssEndpoint - Websocket endpoint for the blockchain network.
 * @param contractAddress - Smart contract address.
 */
const startEventListener = async (chain: string, wssEndpoint: string, contractAddress: string) => {
const provider = new WebSocketProvider(
    wssEndpoint,
    {},
    {
    autoReconnect: true,
    delay: 10000, // Default: 5000 ms
    maxAttempts: 10, // Default: 5
    },
);

provider.on('connect', () => {
    console.log(`Connected to ${chain} websocket provider`);
});

provider.on('disconnect', error => {
    console.error(`Closed ${chain} webSocket connection`, error);
});

const web3 = new Web3(provider);

/*
    Smart contract event listeners

    Listening to events:
    - Transfer
    - Approval
*/
const contract = new web3.eth.Contract(erc20abi, contractAddress);
await subscribeToEvent(chain, contract, 'Transfer');
await subscribeToEvent(chain, contract, 'Approval');
};

Helper Function: subscribeToEvent():

This makes the event subscription reusable, especially if you plan to listen to multiple smart contract events.
After creating a smart contract event subscription, we listen to these subscription events:
- connected - it means that you've successfully subscribed to a smart contract event.
- data - every time an event is generated by smart contract interactions, you'll receive event logs here.
- changed - if for some reason the event has been changed or reverted by the blockchain network, you'll receive event logs here.
- error - if there's an error while listening to smart contract events, you'll receive the error details here.

/**
 * Subscribes to a smart contract event.
 * @param chain - Name of the blockchain network for logging purposes.
 * @param contract - Smart contract address.
 * @param eventName - Name of the event to subscribe to.
 */
const subscribeToEvent = async (chain: string, contract: Contract<typeof erc20abi>, eventName: string) => {
const subscription = await contract.events[eventName]();

subscription.on('connected', subscriptionId => {
    console.log(`${chain} USDT '${eventName}' SubID:`, subscriptionId);
});

subscription.on('data', event => {
    console.log(`${chain} USDT '${eventName}'`, JSON.stringify({ event })); // cannot json.stringify BigInt...
});

subscription.on('changed', event => {
    // Remove event from local database
});

subscription.on('error', error => {
    console.error(`${chain} USDT '${eventName}' error:`, error);
});
}

Starting the Listener:

Finally, the startEventListener() function is called to start listening to smart contract events.

/*
Start smart contract event listeners

Chains:
    - Ethereum
*/
startEventListener('Ethereum', process.env.ETH_WSS_ENDPOINT!, process.env.ETH_SMART_CONTRACT_ADDRESS!);

Here is the full snippet of the smart contract event listener code.

src/app.ts

import 'dotenv/config';
import * as erc20abi from './abi.json';
import Web3, { Contract, WebSocketProvider } from 'web3';

/* 
  Workaround for JSON.stringify() event logs with BigInt values. 
  We need to stringify event logs for more readable logging in CloudWatch.
  https://github.com/GoogleChromeLabs/jsbi/issues/30
*/
(BigInt.prototype as any).toJSON = function () {
  return this.toString();
};

/**
 * Starts the smart contract event listener.
 * Websocket Provider config: https://docs.web3js.org/api/web3-providers-ws/class/WebSocketProvider
 * @param chain - Name of the blockchain network for logging purposes.
 * @param wssEndpoint - Websocket endpoint for the blockchain network.
 * @param contractAddress - Smart contract address.
 */
const startEventListener = async (chain: string, wssEndpoint: string, contractAddress: string) => {
  const provider = new WebSocketProvider(
    wssEndpoint,
    {},
    {
      autoReconnect: true,
      delay: 10000, // Default: 5000 ms
      maxAttempts: 10, // Default: 5
    },
  );

  provider.on('connect', () => {
    console.log(`Connected to ${chain} websocket provider`);
  });

  provider.on('disconnect', error => {
    console.error(`Closed ${chain} webSocket connection`, error);
  });

  const web3 = new Web3(provider);

  /*
    Smart contract event listeners

    Listening to events:
      - Transfer
      - Approval
  */
  const contract = new web3.eth.Contract(erc20abi, contractAddress);
  await subscribeToEvent(chain, contract, 'Transfer');
  await subscribeToEvent(chain, contract, 'Approval');
};

/**
 * Subscribes to a smart contract event.
 * @param chain - Name of the blockchain network for logging purposes.
 * @param contract - Smart contract address.
 * @param eventName - Name of the event to subscribe to.
 */
const subscribeToEvent = async (chain: string, contract: Contract<typeof erc20abi>, eventName: string) => {
  const subscription = await contract.events[eventName]();

  subscription.on('connected', subscriptionId => {
    console.log(`${chain} USDT '${eventName}' SubID:`, subscriptionId);
  });

  subscription.on('data', event => {
    console.log(`${chain} USDT '${eventName}'`, JSON.stringify({ event })); // cannot json.stringify BigInt...
  });

  subscription.on('changed', event => {
    // Remove event from local database
  });

  subscription.on('error', error => {
    console.error(`${chain} USDT '${eventName}' error:`, error);
  });
};

/*
  Start smart contract event listeners

  Chains:
    - Ethereum
*/
startEventListener('Ethereum', process.env.ETH_WSS_ENDPOINT!, process.env.ETH_SMART_CONTRACT_ADDRESS!);

Testing the smart contract event listener code

Now that we've reviewed the code, let's test the code to make sure everything is working as expected.

There are multiple ways to test the code locally:

TypeScript: Run the TypeScript file using ts-node:
- npm run start:dev
JavaScript: Build and run the JavaScript file using node:
- npm run build
- npm run start
Docker:: Build the Docker image and run the container:
1. Build the TypeScript file: npm run build
2. Build the Docker image: docker build -t smart-contract-event-listener:latest .
3. Run the Docker container in detached mode: docker run -d --env-file .env --name smart-contract-event-listener smart-contract-event-listener:latest
4. Get the Docker container logs: docker logs smart-contract-event-listener
5. Stop the Docker container: docker stop smart-contract-event-listener

I recommend building the Docker image and running the container locally to make sure the code works as expected before deploying to AWS.

After testing, you should see some Ethereum USDT Transfer and Approval events in your logs.

Ethereum USDT 'Transfer' {"event":{"address":"0xdac17f958d2ee523a2206206994597c13d831ec7","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x0000000000000000000000001fbcb0803529aea20d6b4af5845fd041e54c50d0","0x0000000000000000000000009a4b7d5916d750b9a864f316b9a2483576412bb1"],"data":"0x000000000000000000000000000000000000000000000000000000000ee6b280","blockNumber":"18597353","transactionHash":"0x61adfbd4a045c02d1b5fea0cce8747dab9b969d552db6091c19168176da6d04c","transactionIndex":"116","blockHash":"0xdc592d23957f1a9fe63e2bd7e03c394b759b7bf48139845877bf0098e82aa8c4","logIndex":"445","removed":false,"returnValues":{"0":"0x1fBCb0803529AeA20d6B4AF5845FD041E54c50d0","1":"0x9A4b7D5916D750B9A864F316b9a2483576412BB1","2":"250000000","__length__":3,"from":"0x1fBCb0803529AeA20d6B4AF5845FD041E54c50d0","to":"0x9A4b7D5916D750B9A864F316b9a2483576412BB1","value":"250000000"},"event":"Transfer","signature":"0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","raw":{"data":"0x000000000000000000000000000000000000000000000000000000000ee6b280","topics":["0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef","0x0000000000000000000000001fbcb0803529aea20d6b4af5845fd041e54c50d0","0x0000000000000000000000009a4b7d5916d750b9a864f316b9a2483576412bb1"]}}}

Ethereum USDT 'Approval' {"event":{"address":"0xdac17f958d2ee523a2206206994597c13d831ec7","topics":["0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925","0x00000000000000000000000062844dc43c064253e21b2b8ac830b28f307184f0","0x000000000000000000000000000000000022d473030f116ddee9f6b43ac78ba3"],"data":"0x000000000000000000000000000000000000000000000000000000012a05f200","blockNumber":"18597353","transactionHash":"0x29de66457b8cd4895e354aee842404e66ed34e176fdd3582bb3b9352d9ac7c8f","transactionIndex":"129","blockHash":"0xdc592d23957f1a9fe63e2bd7e03c394b759b7bf48139845877bf0098e82aa8c4","logIndex":"465","removed":false,"returnValues":{"0":"0x62844Dc43C064253E21B2b8aC830B28f307184F0","1":"0x000000000022D473030F116dDEE9F6B43aC78BA3","2":"5000000000","__length__":3,"owner":"0x62844Dc43C064253E21B2b8aC830B28f307184F0","spender":"0x000000000022D473030F116dDEE9F6B43aC78BA3","value":"5000000000"},"event":"Approval","signature":"0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925","raw":{"data":"0x000000000000000000000000000000000000000000000000000000012a05f200","topics":["0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925","0x00000000000000000000000062844dc43c064253e21b2b8ac830b28f307184f0","0x000000000000000000000000000000000022d473030f116ddee9f6b43ac78ba3"]}}}

Deploying the smart contract event listener

Now that you have a working smart contract event listener, we'll deploy the resources to AWS using AWS CDK, which is an Infrastructure as Code (IaC) tool. AWS CDK allows you to configure, deploy, and manage AWS cloud resources using popular programming languages such as TypeScript.

The sample repository I shared with you earlier is generated with AWS CDK. The smart contract listener code and cloud resources are added after.

For more info on generating your own AWS CDK TypeScript boilerplate, check this AWS documentation.

Installing the AWS CLI

If you don't have AWS CLI installed, you can go to the Get Started section and Install/Update section of the AWS CLI documentation.

Configuring a named profile in AWS CLI

Now you'll configure your AWS CLI credentials so that you have the permissions needed to deploy AWS resources in the region that you specify.

You can either configure a default profile or use a named profile for your AWS CLI credentials.

I recommend configuring a named profile especially if you're working with multiple AWS accounts.

Make sure you already have your own AWS IAM account or access keys. If not, then you can refer to this guide.

To configure an AWS CLI named profile:

Run the aws configure --name profile-name.
Add your AWS IAM account Access Key and Secret Access Key.
Specify the region where you'll deploy your AWS resources.

Then to use the credentials in the named profile:

For Linux or macOS: export AWS_PROFILE=profile-name
For Windows: setx AWS_PROFILE profile-name

For more information, refer to the AWS CLI documentation.

Installing the AWS CDK CLI

Now that you've installed and configured AWS CLI, it's time to install AWS CDK CLI.

To install AWS CDK:

Run the install command: npm install -g aws-cdk.
Then check that AWS CDK is installed correctly: cdk --version

For more information, refer to the AWS CDK documentation.

Reviewing the AWS CDK code

Before we deploy the smart contract event listener, let's review the AWS CDK code and the AWS resources that we'll deploy.

The first file bin/smart-contract-event-listener.ts imports the SmartContractEventListenerStack, then instantiates a CDK app where we can specify the stack name and optional properties.

If you noticed in the comments, you can also configure your AWS account and region in the properties. But since you've already configured your AWS CLI credentials earlier, you don't need to uncomment these.

bin/smart-contract-event-listener.ts

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { SmartContractEventListenerStack } from '../lib/smart-contract-event-listener-stack';

const app = new cdk.App();
new SmartContractEventListenerStack(app, 'SmartContractEventListenerStack', {
  /* If you don't specify 'env', this stack will be environment-agnostic.
   * Account/Region-dependent features and context lookups will not work,
   * but a single synthesized template can be deployed anywhere. */
  /* Uncomment the next line to specialize this stack for the AWS Account
   * and Region that are implied by the current CLI configuration. */
  // env: { account: process.env.CDK_DEFAULT_ACCOUNT, region: process.env.CDK_DEFAULT_REGION },
  /* Uncomment the next line if you know exactly what Account and Region you
   * want to deploy the stack to. */
  // env: { account: '123456789012', region: 'us-east-1' },
  /* For more information, see https://docs.aws.amazon.com/cdk/latest/guide/environments.html */
});

Then the lib/smart-contract-event-listener-stack.ts file defined the AWS cloud resources that we'll deploy.

The SmartContractEventListenerStack will deploy the following resources:

lib/smart-contract-event-listener-stack.ts

import 'dotenv/config';
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as ec2 from 'aws-cdk-lib/aws-ec2';
import * as ecs from 'aws-cdk-lib/aws-ecs';

export class SmartContractEventListenerStack extends cdk.Stack {
  constructor(scope: Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    const vpc = new ec2.Vpc(this, 'SmartContractEventListenerVPC', {
      vpcName: 'SmartContractEventListenerVPC',
      // EIP soft limit is 5, need to increase limit to increase AZ
      maxAzs: 1,
    });

    const cluster = new ecs.Cluster(this, 'SmartContractEventListenerCluster', {
      clusterName: 'SmartContractEventListenerCluster',
      vpc: vpc,
    });

    const taskDefinition = new ecs.FargateTaskDefinition(this, 'SmartContractEventListenerTaskDef', {
      family: 'TaskDef',
    });

    taskDefinition.addContainer('SmartContractEventListener', {
      containerName: 'Container',
      environment: {
        ETH_WSS_ENDPOINT: process.env.ETH_WSS_ENDPOINT!,
        ETH_SMART_CONTRACT_ADDRESS: process.env.ETH_SMART_CONTRACT_ADDRESS!,
      },
      // can be .fromContainerRegistry() if available to ECR
      image: ecs.ContainerImage.fromAsset(''),
      logging: new ecs.AwsLogDriver({
        streamPrefix: 'SmartContractEventListener',
        mode: ecs.AwsLogDriverMode.NON_BLOCKING,
      }),
    });

    new ecs.FargateService(this, 'SmartContractEventListenerFargateService', {
      serviceName: 'FargateService',
      cluster,
      taskDefinition,
    });
  }
}

Deploying the AWS resources using AWS CDK

Now that we've reviewed the AWS CDK code and AWS resources, it's time to deploy your smart contract event listener.

To deploy your smart contract event listener:

Build the source files: npm run build.
Deploy the AWS resources: cdk deploy.
Review the changes, then when prompted with Do you wish to deploy these changes (y/n)?, enter y.

You should see a progress bar of the AWS resources that are being deployed.

The deployment should take around 5 minutes to complete, so in the meantime, you can take a quick break.

Once the deployment completes, you should see the deployment time and the CloudFormation stack ARN.

Note: If your deployment takes longer than 5 minutes and seems to be stuck, then you can check the ECS service logs, as shown in this section.

If the logs show runtime errors, then you can do the following:

Follow this guide to delete the SmartContractEventListenerStack in CloudFormation.

Review and test the code and Docker build locally to verify it's working as intended.

Build and redeploy using the above steps.

Check the smart contract event listener logs in AWS

Now that you've successfully deployed a smart contract event listener, it's time to verify if it's receiving events from the Ethereum USDT contract or your contract.

Here are the steps to check the logs from your smart contract event listener:

In the AWS console, go to the ECS, then click on SmartContractEventListenerCluster.
Then, go to Services, and click FargateService.
Finally, click Logs.

You should now be seeing the logging of events from the Ethereum USDT contract or the smart contract you specified.

Cleaning up the AWS resources (Optional)

After testing the smart contract event listener, it's recommended to clean up the deployed AWS resources especially if you're just doing a proof of concept.

To delete the deployed AWS resources, run the cdk destroy command.

This makes sure that you avoid incurring unnecessary charges on your AWS account, since running an ECS cluster is not part of the AWS Free Tier, and would incur charges if it's left running in the cloud.

Some caveats and improvements to consider

We've verified that the smart contract event listener is working. Now I'll go through some of the caveats and improvements to consider since I've personally implemented a smart contract event listener at the startup I'm currently working at.

Integrating smart contract events into other services

You might be wondering how you can integrate smart contract events into your backend systems, so here are some ideas:

Trigger a Lambda with CloudWatch Logs
Add code in the smart contract event listener subscriptions to trigger other parts of your system.

I would recommend the first idea since Lambda is flexible and can fit a lot of use cases. You can also trigger Lambda functions every time a smart contract event is logged into CloudWatch.

At RAILS by P33R, we use this approach. The app allows customers to off-ramp their crypto stablecoins (USDT, USDC, and DAI) into fiat currencies (SGD, PHP).

A high-level implementation of the app's crypto-to-fiat use case is shown below. The flow is as follows:

Customer deposits crypto stablecoins to the escrow smart contract.
Escrow smart contract emits a Deposit event.
Smart contract event listener receives the Deposit event, and logs to CloudWatch.
CloudWatch log triggers the Fiat Disbursement Lambda function.
Customer receives fiat in their bank account or e-wallet.

This is an example implementation that you can reference for your own use cases.

Health checks and error handling

If you plan to use smart contract event listeners in production, monitoring the health of the service and handling errors is a requirement.

At the time of writing this, I hadn't figured out how to implement health checks for the smart contract event listeners in ECS. A workaround for this is to implement CloudWatch alarms. You can send email alerts when the smart contract event listener throws an error.

I also encountered a specific issue related to the block limit when querying smart contract events over 10,000 blocks. This was an issue with QuickNode, but other providers also have a 10,000 block limit for querying logs and events. For more information, here is QuickNode's support article on this topic.

This 10,000 block limit error happens because when you create a smart contract event subscription in Web3.js, it automatically filters for smart contract events, starting from the block number at the time of the subscription. Let's assume that the current block number is at 1,001. Then the smart contract event listener can safely query for events until block number 11,001. The WebSocket provider will throw an error when the block number exceeds 11,001.

Fortunately, whenever the smart contract event listener throws an error, the ECS cluster will automatically restart the service. This is the default behavior since ECS is a fully managed AWS service for container orchestration. Even with this, it's still important to improve the health checks and error handling, and not fully rely on AWS error handling.

Improving reliability with the AWS Well-Architected framework

As we're building cloud infrastructure in AWS, it's important to follow the AWS Well-Architected framework. One of the pillars of the Well-Architected framework is Reliability. This means removing single points of failure in the smart contract event listener, resulting in more resilient architectures.

One way of improving the reliability of the smart contract event listener is to support multiple availability zones in the ECS cluster. This ensures redundancy in the event of a failure in one availability zone. For more information, here's a blog on Amazon ECS availability best practices and AWS' Shared Responsibility Model for Resiliency.

Once you've implemented the smart contract event listener cluster to be in multiple availability zones, you'll need your systems to be idempotent. This means that when you receive the same smart contract event multiple times, it should only be processed once in your system.

Conclusion

To recap, we've gone through the process of implementing and deploying a smart contract event listener with the following steps:

Configuring the WebSocket endpoint and smart contract address.
Reviewing and testing the smart contract event listener code.
Deploying the smart contract event listener to AWS using AWS CDK.
Verifying the deployed smart contract event listener is working.

I've also shared some caveats and improvements to consider as you build on top of the listener:

Integrating the smart contract event listener with different AWS services such as Lambda functions.
Improving reliability by deploying the smart contract event listener to multiple availability zones (AZ).
Setting up CloudWatch alarms for error monitoring.

Listening to smart contract events enables developers to integrate smart contracts into different APIs and enable real-time updates. AWS CDK allows you to deploy reliable cloud infrastructure with just a few lines of code, giving you more confidence in your deployments, and allowing you to focus on solving real-world problems.

If you've made it this far, thank you for reading! If you have any questions, feel free to comment or contact me here.

OpenAI DevDay and GitHub Universe 2023: What It Means for Us Software Engineers

Jasper Gabriel — Sun, 12 Nov 2023 15:13:37 +0000

Introduction

Earlier this week, the team and I rented an Airbnb to prepare for our upcoming demo day on November 23rd. We had internal discussions and started preparing our pitch deck. Then while on a break, looking for videos to watch on YouTube, we happened to see a recent video on OpenAI's DevDay. After watching the video, we were amazed, and we started discussing the possibilities enabled by OpenAI's new features. After a few days, GitHub Universe 2023 started. We binged the keynotes which added to our excitement and slight worries.

Our feelings of excitement stemmed from discussions on how we can leverage AI to improve developer workflows and stay lean at our startup. Meaning not needing to hire additional engineering resources while we build out our MVPs. There are also new features that make it easier to integrate AI assistants into our future applications, which helps in providing a modern experience.

Our worries came from the disruptions caused by AI advancements. We can't help but think about our future job security since AI is becoming more capable of doing development tasks. We also thought about how certain startups can become obsolete after OpenAI's announcements.

If you're a software engineer wondering what these developments mean for you, then you've come to the right place. In this article, I'll recap what we've witnessed from OpenAI and GitHub. Then I'll share some thoughts on its impact on us software engineers.

OpenAI DevDay

OpenAI had its first DevDay, which generated a lot of buzz throughout the AI and developer communities. Let's go through the key features.

GPT-4 Turbo

The first announcement introduced GPT-4 Turbo, the latest generation model from OpenAI that had the following updates:

Four times more context length than the previous GPT-4 model (32k tokens) at 128k tokens.
Cheaper pricing for input and output tokens.
More recent knowledge cut-off date of April 2023.
Support for function calling and JSON mode.
Support for code interpreter.
Reproducible outputs with seed parameter.

Context length is important in GPTs because the more details and instructions you can provide, the more accurate the GPTs' responses will be. Combined with cheaper pricing, you can provide more context that is cost-effective.

Support for function calling and JSON mode allows for easier API integrations when using GPT-4 Turbo and other supported models.

Then support for code interpreters allows GPTs to generate and run Python code as needed. This allows for use cases that need to parse and process files, calculate data, and a lot more.

"GPTs"

The next announcement was about "GPTs", where users can create their own GPT versions for different use cases without having to write code. Sam Altman demoed this functionality by creating a "Startup Mentor" GPT since he usually gets similar questions from early-stage startup founders during YC office hours.

Screenshot from OpenAI's DevDay video on OpenAI's YouTube channel.

A GPT store will also be rolled out later this month allowing users to share their work with the public and possibly earn money depending on their GPTs' usage and popularity.

Assistants API

After that, they showed the new Assistants API. With similar capabilities to GPTs, this enables developers to leverage GPTs programmatically for their use cases.

The demo shown using this feature was on a sample travel website, where users can use an AI assistant to plan their travels.

Screenshot from OpenAI's DevDay video on OpenAI's YouTube channel.

GitHub Universe 2023

After a few days, GitHub Universe 2023 opened up with some exciting announcements. Let's walk through the key highlights.

Copilot Chat

The first announcement made was with Copilot Chat, which is coming in December.

Screenshot from GitHub Universe 2023 video on GitHub's YouTube channel.

This brings different features such as:

Using GPT-4 models
Code-aware guidance and code generation
Iterate on code in line
Shortcuts to generate unit tests, fix suggestions, pull request descriptions, and much more.

All of these features are integrated into popular IDEs such as VS Code, which helps with staying in the development flow.

Copilot Enterprise

While Copilot Chat is integrated per repository, Copilot Enterprise is instead integrated with your organization's codebase.

This helps developers quickly get insights, especially in large organizations with thousands of repositories and billions of lines of code.

AI-Powered Security

Then, an announcement on AI-powered security was made. This helps minimize insecure coding practices by detecting and correcting hard-coded credentials and secrets, and vulnerabilities such as SQL injection.

What These AI Announcements Mean for Us Software Engineers

The Positives

Most of the announced features are aimed at improving developer experiences. Whether you're a front-end, back-end, full-stack, or any other engineering role, these changes can help you be more productive and allow you to focus on building creative solutions to engineering problems.

Here are some ways these AI tools help:

Integrate AI assistants in your applications with OpenAI Assistants API.
Create GPTs for your specific use cases (Code reviewer, Mentor, etc.).
Implement Test-Driven Development (TDD) with GitHub Copilot.
Improve developer workflows with GitHub Copilot Chat and Copilot Enterprise:
- Build documentation
- Suggest code based on open files and repositories
- Generate pull request summaries
- Review pull requests
- Fix bugs and issues
- Remediate security vulnerabilities
- Ask Copilot Chat questions about the codebase

The Not So Positives

While these AI tools improve developer workflows and productivity, you can also argue that this might lead to a reduction in engineering resources for some companies. Some startups might even become obsolete if they do not pivot or solidify their unique value proposition (UVP), which will inevitably cause some job losses. Startups have always been risky, and disruptions are always bound to happen. It's just that advancements in AI accelerated this disruption cycle.

Even with these concerns, I wouldn't worry too much about the job market, especially if you're continuously improving yourself. While there is a surplus of software engineers, there's also a shortage of talented software engineers. If you start learning and leveraging these AI tools in your everyday workflows, you can keep up with the latest trends in tech, and in turn, be a valuable asset for any tech company.

Conclusion

The opening keynotes from OpenAI's DevDay and GitHub Universe 2023 showcased new and improved AI tools that software engineers can use to make their development experiences more enjoyable, from prompting ChatGPT and Copilot Chat to generating documentation and unit tests. As tech keeps on evolving, it's important for us as engineers to also adapt, innovate, and continuously learn to keep up with the trend.

Our team and I have already availed ChatGPT Plus, OpenAI Platform APIs, and GitHub Copilot. I'll be playing around and experimenting with these new features and how they can enhance my developer workflows, and I'll be sure to write about interesting stuff I learn along the way.

As we part ways, I thank you for reading and I'll leave you with one question, How will you leverage these new AI tools in your development workflows?

How to Reliably Read QR Codes in Node.js

Jasper Gabriel — Sat, 04 Nov 2023 11:16:02 +0000

Introduction

In today's digital landscape, QR codes are widely used. Thanks to the QR code's efficiency in storing and reading data, you can see QR codes being used in restaurants, public transportation, app downloads, survey forms, and a whole lot of other applications.

Reading QR codes is usually handled by frontend applications via scanners or image uploads, but if you need a way to read QR codes in the backend, then you've come to the right place!

Why I made this blog post

I was figuring out how to read QR codes in Node.js, so I've tried following existing guides from LogRocket and GeeksforGeeks. They both had similar solutions using qrcode-reader.

Upon testing the solution multiple times, I've encountered occasional bugs with qrcode-reader, which made it unsuitable for production workloads.

The qrcode-reader package is no longer maintained and the owner recommended jsqr as an alternative, which is what we're going to use.

QR Code Reader Solution

Now let's walk through the solution for reading QR codes in Node.js. You can follow along or check the sample repository here.

Installing Jimp and jsQR

For the npm packages, we'll be using jimp for the image loader and jsqr for the QR code reader.

npm install jimp jsqr

TypeScript Code (src/app.ts)

The sample QR code src/qr.png is in the sample repository. You can also replace src/qr.png and use any QR code image if you want.

import Jimp from 'jimp';
import jsQR from 'jsqr';

const decodeQR = async (): Promise<string> => {
    try {
        // Load the image with Jimp
        const image = await Jimp.read('src/qr.png');

        // Get the image data
        const imageData = {
            data: new Uint8ClampedArray(image.bitmap.data),
            width: image.bitmap.width,
            height: image.bitmap.height,
        };

        // Use jsQR to decode the QR code
        const decodedQR = jsQR(imageData.data, imageData.width, imageData.height);

        if (!decodedQR) {
            throw new Error('QR code not found in the image.');
        }

        return decodedQR.data;
    } catch (error) {
        console.error('Error decoding QR code:', error);
    }
}

// Test the decodeQR() function
const run = async () => {
    console.log(await decodeQR());
}

run();

Optional: If you need to read base64 string QR codes, replace the Jimp.read() function with this.

const base64 = 'data:image/png;base64,iVBORw0KGgo...' // replace with your base64 string
const qrBuffer = Buffer.from(base64.replace(/^data:image\/[a-z]+;base64,/, ''), 'base64');
const image = await Jimp.read(qrBuffer);

Testing the Code

To test the solution, you can build and start the function, which will retrieve the data from the src/qr.png image file.

npm build (or tsc)
npm start (or node src/app.js)

> qrcode-decoder-nodejs@1.0.0 start
> node src/app.js

Sample QR code data for testing.

As you can see, we decoded the data and received "Sample QR code data for testing.".

Conclusion

To recap, we implemented a QR code reader in Node.js using jimp and jsqr. We also tested the decodeQR() function, which decoded the sample QR image. Also, here's the sample repository with the code shown in this guide if you've missed it.

I hope this helped you with implementing a QR code reader in Node.js. Thank you for reading, and feel free to connect with me on Twitter and LinkedIn.

My First Open Source Contribution: How I Did It

Jasper Gabriel — Tue, 08 Aug 2023 13:50:16 +0000

Introduction

Contributing to open source can seem intimidating, especially to those who've never done open source before. I know I felt intimidated, thinking that open source is only for talented and senior developers.

While that may be true for those who've made packages and tooling used by thousands (or even millions) of devs, it's not true if you want to contribute to an existing open source project. I've realized that and I'm here to share how I made my first open source contribution.

If that sounds interesting, then let's continue!

Finding an Open Source Project to Contribute to

There are a lot of open source projects out there, but where do you start?

I narrowed it down using these three helpful prompts:

What specific area of tech am I interested in?
What are my current skills, or skills I want to improve?
What's the tech stack of the open source project?

What specific area of tech am I interested in?

Looking for projects that fit your interests would help you get started. Your interests can be anywhere from frontend templates, developer tools, AI, smart contracts, cybersecurity, etc.

I'm passionate in Web3, so I tried to find Web3 projects that are open source. Web3 also encourages open source so it wasn't hard looking for one.

What are my current skills, or skills I want to improve?

As for skills, what are you currently good at? or want to gain more experience at?

For me, I'm a backend developer that specializes in Node.js and TypeScript. I've integrated Web3 libraries to Node.js apps, and I've also written technical documentations for previous projects.

With that, I'd look for Node.js and TypeScript projects, where I'm already good at. I won't look for a Go or Rust project where I don't have the necessary skills to contribute.

What's the tech stack of the open source project?

Combining my interests and skills, I've looked at a few open source projects and their tech stacks.

Ethereum scaling solutions and zero knowledge (ZK) are two areas I'm interested in Web3. I've read Vitalik's article on the different types of ZK-EVMs. I saw Taiko as one of the current Type 1 ZK-EVMs in development.

Reviewing Taiko's monorepo tech stack, I saw that they use TypeScript. Their documentations use MDX, which is an improved version of markdown. I've used markdown before so contributing to their docs in MDX is doable.

With that, I started preparing to make my first open source contribution with Taiko.

Contributing to Taiko

Reviewing contributing guidelines

Before anything else, I first looked at Taiko's CONTRIBUTING.md file to orient myself on their contributing guidelines:

how to contribute
coding standards
documentation standards
helpful tips

Then I saw two ways to contribute in the Make a contribution section:

Opening a new issue
Working on an existing issue

I've chosen to work on an existing issue.

For coding standards, I looked at the standards and conventions used in the project. I saw that they are using conventional commits for pull request titles and commits.

Then for branch naming, I saw that there was no particular convention, so a short branch name describing what it is for should be enough.

Finding an existing issue to work on

I've tried to look at some good first issues, since these are usually not difficult issues, and are meant to help onboard first time contributors.

I ended up choosing the issue to improve their "Node running guide for setting RPC endpoints".

Then I reviewed the issue, and looked at the description and comments. I also reviewed the website where the docs are located.

I made sure I understood what was needed.

Forking, cloning, and working on the changes

After reviewing the issue, I forked and cloned Taiko's monorepo. Then I started working on a proof of concept.

Locating the code that needs improvement was pretty straight-forward, since the project is structured as a monorepo. The website package contains Taiko's guides, so I worked on that.

After I was able to run my local setup and test the changes, I made some comments on the issue and asked for a review to verify if the changes would solve the issue.

After getting approval from Dave (d1onys1us), I proceeded to open a pull request.

Going through the pull request review process

The review process was interesting. While waiting for reviews, I've noticed that there were changes on the main branch that was merged to my feature branch.

Then I reviewed the changes, and it affected my changes, which didn't make sense when combined with the changes from the main branch. So I decided to take the initiative to improve the section and propose the changes on my ongoing pull request review.

The team liked the change, so I proceeded to commit the changes.

Then another reviewer commented that I had API keys exposed on the guide images, so I quickly fixed these to resolve the review comment.

After all of that, my pull request was finally approved and merged!

Finally, I can say that I've contributed to open source! Got a cool GitPOAP to prove it too! 🙂

My Takeaways (Conclusion)

Making my first open source contribution helped me realize what open source meant. The value of open source is in its contributors and their contributions, no matter how big or how small.

Being an open source contributor can also be compared to a boy or girl scout, where the scouts leave the campsite cleaner than when they've found it. When a contributor encounters a bug in a feature or a typo in the documentation of an open source project, they can create an issue along with the fix, leaving the code cleaner than when they've first encountered it.

I also want to thank the team at Taiko for being welcoming, supportive, and making my first open source contribution a fun experience. Do check out the project if you're interested!

If you've missed it, here's the issue that I've worked on at Taiko and the pull request that solves the issue.

And here's a summary of what I did:

Found an open source project:
- What specific area of tech am I interested in?
- What are my current skills, or skills I want to improve?
- What's the tech stack of the open source project?
Reviewed contributing guidelines (CONTRIBUTING.md)
Looked at good first issues
Chose an existing issue, then reviewed the description and comments (if any)
Forked and cloned the repo
Set up my local environment
Worked on and tested the changes
Opened a pull request
Went through the review process and resolved comments (if any)
Pull request got merged!

I won't stop there. I'm excited on the journey ahead and looking forward to be a more consistent open source contributor. One contribution at a time.

Thank you for reading and as always, I'm open to suggestions, feedback, and comments. Also feel free to contact me at my email (jasper.d.gabriel@gmail.com), and I'm available on Twitter, LinkedIn, and GitHub.