DEV Community: Dhruv Patel

What I Learned After Building a Redis Queue Feature in SyncFlow

Dhruv Patel — Sun, 28 Jun 2026 10:07:31 +0000

While working on SyncFlow, a Shopify embedded app, I recently completed US-002 — Redis Queue Foundation.

The purpose of this feature was simple:

Move inventory sync work away from the main request flow and process it in the background using Redis-backed queues.

At first, the feature looked complete. The queue files were added, Redis configuration was created, producers and workers were implemented, retry handling was introduced, and logging was added.

But after reviewing the uncommitted changes properly, I learned something important:

A feature is not complete when the code is written.
A feature is complete when it is reliable, reproducible, testable, and safe to run.

What the Feature Was Supposed to Do

The goal of US-002 was to create a foundation for background job processing.

The feature included:

Redis connection setup
Queue configuration
Primary sync queue
Retry queue
Dead-letter queue
Queue producer
Queue worker
Queue health check
Structured queue logging

The idea was to make inventory sync more reliable.

Instead of blocking the app request while sync logic runs, the app can now publish a job to a queue. A worker can process that job separately. If the job fails, it can be retried. If it keeps failing, it can move to a dead-letter queue.

That sounds good in theory.

But the review showed that implementation details matter a lot.

Lesson 1: TypeScript Errors Can Block the Entire Feature

One of the first issues found was a TypeScript syntax error in the queue health file.

Because of that, npm run typecheck failed before the app could build.

This was a clear reminder that type checking is not optional in a TypeScript project.

Even if the logic looks correct, invalid types can stop the whole application from being production-ready.

Before committing a feature, I should always run:

npm run typecheck
npm run build

For backend-heavy features, I should also run the worker process locally and confirm that the app and worker can both start correctly.

Lesson 2: Dependency Management Must Be Reproducible

The feature added new packages like:

bullmq
ioredis
pino
tsx

But the lockfile was not updated correctly.

There was also an untracked pnpm-lock.yaml, while the project was using package-lock.json.

This creates a serious problem.

If another developer pulls the repo, installs dependencies, and runs the project, their environment may not match mine.

The lesson is simple:

Pick one package manager and stay consistent.

For example, if the project uses npm, commit:

package.json
package-lock.json

And remove accidental lockfiles like:

pnpm-lock.yaml

A feature is not reproducible if dependencies are not locked correctly.

Lesson 3: Health Endpoints Should Report Failure, Not Crash

A health endpoint should help debug the system.

It should not become another reason the app crashes.

In this feature, the health route had two problems.

First, it imported from a package that was not declared in the project. Second, the Redis connection code could throw immediately if REDIS_URL was missing.

That means the health endpoint might crash instead of returning a clean unhealthy response.

A better health endpoint should return something like:

{
  "status": "unhealthy",
  "redis": "missing_config",
  "worker": "not_ready"
}

With an HTTP status like:

503 Service Unavailable

The lesson:

Health checks should be defensive. Their job is to explain what is broken, not fail silently or crash the route.

Lesson 4: Every Queue Needs a Consumer

The retry queue was created, and jobs could be added to it.

But only the main sync queue had a worker.

That means retry jobs could sit inside the retry queue forever.

This is a common design mistake in queue-based systems.

Creating a queue is not enough.

Every queue must have a clear data flow:

Who writes to it?
Who reads from it?
When does the job move?
What happens if it fails?
What happens after max retries?

For a retry queue, there are usually two options.

One option is to use the queue system’s built-in retry mechanism.

Another option is to create a separate retry worker that moves jobs back to the main queue after a delay.

Without that, retry logic only exists on paper.

Lesson 5: Job Types Should Be Strict

The job payload allowed the job type to be any string.

But the producer always added the job as an inventory sync job.

That creates a mismatch.

A product sync payload could accidentally be queued as an inventory sync job.

This is where TypeScript should help.

Instead of using a loose string, the job type should be constrained.

Example:

type SyncJobType = "INVENTORY_SYNC" | "PRODUCT_SYNC";

Or better, use an enum or constant map.

The lesson:

If the system only supports specific job types, the type system should enforce that.

Loose strings make the code flexible, but also easier to break.

Lesson 6: Environment Variables Need Validation

Some queue environment variables were typed as required strings, but the runtime code treated them as optional and provided defaults.

There was also a risk with numeric environment variables.

For example:

QUEUE_CONCURRENCY=abc

If the code uses Number(process.env.QUEUE_CONCURRENCY), this becomes:

NaN

That can create unexpected worker behavior.

The better approach is to validate environment variables clearly.

For example:

If REDIS_URL is required, fail startup with a clear error.
If QUEUE_CONCURRENCY is optional, document the default.
If a number is invalid, reject it early.

The lesson:

Environment variables are part of the application contract. They need validation, not just typing.

Lesson 7: Public Health Routes Can Leak Internal State

The queue health endpoint exposed operational details publicly.

Queue counts may not be secret, but they still reveal internal system information.

For a public Shopify app URL, this matters.

Health endpoints can expose:

Whether Redis is connected
Whether workers are running
Queue names
Job counts
Failure patterns

For internal monitoring, that is useful.

For public access, that can become unnecessary exposure.

The lesson:

Operational routes should be protected.

Possible solutions include:

Shared secret header
Admin-only access
Internal-only route
Reduced public response
Separate public and private health checks

Lesson 8: Small Untracked Files Matter

There were accidental empty files in the project.

They looked like command artifacts.

This may seem small, but repository hygiene matters.

Before committing, I should always check:

git status

And remove files that do not belong in the project.

A clean commit should contain only intentional changes.

My New Feature Completion Checklist

After this review, my definition of “feature complete” has changed.

Before committing a backend feature, I should verify:

npm install
npm run typecheck
npm run build
git status

Then I should manually confirm:

App starts successfully
Worker starts successfully
Required environment variables are documented
Invalid environment values fail clearly
Health endpoint returns useful output
Every queue has a consumer or processing strategy
Failed jobs have a defined retry path
Dead-letter jobs are visible for debugging
Logs do not expose sensitive data
Lockfile matches the selected package manager

Final Takeaway

Building the Redis Queue Foundation taught me that backend features are not just about writing logic.

They are about designing reliable systems.

A queue feature needs more than a producer and worker. It needs clear failure handling, strict types, reproducible dependencies, safe health checks, clean logs, and predictable runtime behavior.

The biggest lesson:

Code that works locally is not always production-ready.

A production-ready feature must be easy to run, easy to debug, safe to expose, and hard to break.

That is the real difference between just implementing a feature and engineering a feature properly.

Understand Code Instead of Just Memorizing Syntax

Dhruv Patel — Thu, 25 Jun 2026 02:28:14 +0000

Many junior developers feel the same pressure:

“I need to learn JavaScript.”
“I need to learn TypeScript.”
“I need to learn React.”
“I need to learn Node.js.”
“I need to learn testing, Git, DevOps, databases, deployment…”

The list keeps growing.

At some point, it starts feeling like you need to finish multiple full language books before you can call yourself a developer.

But in real development, the goal is not to memorize everything.

The goal is to understand code well enough to read it, explain it, debug it, improve it, and use it inside real projects.

That is a much more practical target for juniors.

The Problem Most Juniors Face

A lot of beginners learn programming like this:

Watch a tutorial
Copy the code
Make it work
Move to another tutorial

This feels productive, but there is one issue:

If the code breaks, they do not know why.

That is the real gap.

A junior developer does not need to know every advanced feature of a language immediately. But they should learn how to look at code and answer basic questions:

What is this code trying to do?
What is the input?
What is the output?
Where does the data change?
What can break?
How can I test it?
How can I explain it?

This is where real learning starts.

Learn Code in Three Layers

Instead of learning everything randomly, juniors can divide their learning into three layers.

1. Language

This is the foundation.

For JavaScript or TypeScript, this includes:

variables
functions
arrays
objects
loops
conditionals
async/await
error handling
modules
types and interfaces

You do not need to know every feature on day one. But you should be comfortable enough to write logic without copying every line.

For example, you should be able to build small functions like:

calculate cart total
filter active users
search products
validate form data
group items by category
handle API response data

If you can do this, your language foundation is growing.

2. Framework

Frameworks help you build real applications faster.

For frontend, this may be React or Next.js.
For backend, this may be Node.js and Express.

But frameworks are not magic. They are just structured ways to use your language.

That is why juniors should not only ask:

“How do I use React?”

They should also ask:

“What JavaScript concept is React using here?”

For example:

React state uses JavaScript values.
React props use objects.
React lists use arrays.
API calls use promises and async/await.
Forms use events and state updates.
Components use functions.

When you connect framework concepts back to the language, learning becomes easier.

3. Environment

This is the part many juniors ignore, but companies care about it.

Environment means knowing how code runs, breaks, and gets shipped.

This includes:

terminal
Git and GitHub
npm
environment variables
package.json
debugging tools
browser DevTools
Postman
deployment
logs
CI/CD basics
Docker basics

A developer is not only someone who writes code.

A developer should also know how to run code, debug code, test code, deploy code, and fix errors when something fails.

A Simple Checklist to Judge Code

When juniors read or write code, they can use this checklist.

1. Correctness

Does the code do what it is supposed to do?

This is always the first question. Clean-looking code is useless if it does not solve the problem.

2. Readability

Can another developer understand it quickly?

Readable code is usually better than clever code.

3. Naming

Are variables, functions, files, and components clearly named?

Good names make code easier to understand.

Bad example:

function calc(x) {
  return x * 1.13;
}

Better example:

function calculateTotalWithTax(price) {
  return price * 1.13;
}

The second version explains itself.

4. Simplicity

Is the code more complicated than needed?

Junior developers sometimes try to write advanced-looking code. But in real teams, simple and understandable code is often better.

5. Edge Cases

What can break?

Ask questions like:

What if the array is empty?
What if the input is null?
What if the API fails?
What if a field is missing?
What if the user enters invalid data?

This habit makes your code stronger.

6. Error Handling

What happens when something fails?

For example, if an API request fails, the user should not see a broken screen. The application should handle the error properly.

7. Data Flow

Can you follow how data enters, changes, and exits?

This is one of the most important skills for juniors.

In React, ask:

Where does the data come from?
Which component owns the state?
Which component receives props?
Where is the API called?
Where is the result displayed?

In backend code, ask:

Where does the request come from?
Which route handles it?
Which controller runs?
Which database operation happens?
What response is returned?

8. Separation of Concerns

Is each function, component, or file doing one clear job?

If one function validates data, updates the database, sends emails, and formats the response, it may be doing too much.

Good code is easier to change when responsibilities are separated.

9. Security

Is user input handled safely?

For junior developers, basic security awareness is enough to start:

do not hardcode secrets
use environment variables
validate user input
protect private routes
check authentication
check authorization
never trust client-side data blindly

10. Tests

Can you prove the code works?

Tests do not need to be advanced in the beginning.

Start by checking:

Normal case
Empty case
Invalid input case
Error case

Even simple tests improve your thinking.

How to Practice Daily

Here is a simple daily routine for juniors.

Read Code

Take one function or component and explain it in plain English.

Write:

Purpose:
Input:
Output:
Main logic:
What can break:
How I would improve it:

This builds code-reading skill.

Modify Code

Do not only copy tutorials.

Change something.

Add a feature.
Rename variables.
Split a function.
Add validation.
Improve error handling.
Refactor repeated logic.

Modification is where understanding becomes real.

Debug Code

Take working code and intentionally break one thing.

For example:

wrong import
missing return
wrong API URL
missing dependency in useEffect
undefined variable
wrong database field
invalid token
wrong route path

Then read the error and fix it.

This is one of the fastest ways to become confident.

Explain Code

After building something, explain it like this:

“This component receives products as props. It stores the search text in state. When the user types, it filters the products array and displays only matching items.”

If you cannot explain your code simply, you probably do not understand it fully yet.

When Can You Say You Learned a Language?

You do not need to know 100% of a language to say you know it.

A practical definition is this:

You know a language at a junior level when you can use it to build features, debug errors, read existing code, and explain your decisions.

For JavaScript, that means you can work with:

arrays and objects
functions
async/await
API calls
error handling
modules
DOM or React usage
common debugging problems

You are not finished learning the language. But you are useful with it.

That is the real goal.

What Juniors Should Focus On for Interviews

For junior developer interviews, focus on practical fluency.

You should be able to explain:

your projects
your role in the project
how frontend talks to backend
how data moves through the app
how authentication works
how you handled errors
how you tested the feature
how you debugged a difficult issue
what you would improve next

You should also practice basic coding problems using arrays, strings, objects, loops, and hash maps.

The goal is not to sound like a senior engineer.

The goal is to show that you can think clearly, learn fast, debug issues, and contribute to a codebase.

Final Thought

Junior developers do not need to learn everything at once.

Start with this goal:

Read code.
Understand data flow.
Build small features.
Debug daily.
Explain your work clearly.
Improve code step by step.

That is how syntax becomes skill.

And that is how learning becomes real software development.