DEV Community: Awosise Oluwaseun

Decoding Life One Cell at a Time: A Journey Through Single-Cell RNA Sequencing

Awosise Oluwaseun — Sat, 27 Dec 2025 14:00:04 +0000

When 10,000 Cells Tell 10,000 Different Stories

Imagine you're trying to understand a bustling city. You could look at it from space and see general patterns—where the downtown is, where the suburbs spread out. But to really understand the city, you need to meet its inhabitants: the barista brewing your morning coffee, the doctor saving lives at the hospital, the teacher shaping young minds. Each person has a unique story, a unique purpose.

Your body is that city, and cells are its inhabitants. For decades, scientists studied cells in bulk—like viewing that city from space. But what if we could listen to each cell individually? What if every cell could tell us its unique story through the genes it's expressing at that very moment?

Welcome to the world of single-cell RNA sequencing (scRNA-seq), where we do exactly that.

🎵 The Power of Listening to Individual Voices

Traditional bulk RNA sequencing is like recording a symphony orchestra but only having one microphone placed in the middle of the concert hall. You hear music, sure, but can you distinguish the violins from the cellos? The flutes from the trumpets?

Single-cell RNA-seq is like giving each musician their own microphone. Suddenly, you can:

✅ Identify who's playing what
✅ Discover that one amazing violinist playing a slightly different melody
✅ Realize the person you thought was a trumpeter is actually playing the saxophone

In our bodies, this means discovering:

Rare cell types that were invisible in bulk measurements
Cell states (like cells transitioning from resting to activated)
Disease mechanisms hidden within specific cell populations
How cells respond differently to the same treatment

🎯 Our Mission: Decoding Human Blood Cells

For this journey, we're analyzing peripheral blood mononuclear cells (PBMCs)—the immune cells floating in your bloodstream right now. These are your body's defense force: the soldiers, scouts, and command centers of your immune system.

We start with approximately 10,000 cells captured using 10x Genomics technology, which works like this:

Individual Cell → Droplet → Barcode → Sequence → Digital "Fingerprint"

Each cell gets trapped in its own microscopic droplet with a unique barcode, like giving each person a numbered ticket. When we sequence the RNA, we know exactly which cell each piece of genetic code came from.

📊 The Data We're Working With

📊 Dataset: 10,000+ individual PBMC cells
🧬 Genes tracked: ~20,000 per cell
📈 Total measurements: 200+ million data points
💾 File size: Several hundred megabytes

Think about that for a second: we're measuring the activity of 20,000 genes in 10,000 individual cells. That's like reading 200 million different sentences to understand the story of your immune system!

📓 Full Tutorial: The complete Jupyter notebook with all code and visualizations is available on GitHub. Follow along and run the analysis yourself!

Part 1: The Setup - Building Our Laboratory in the Cloud ☁️

Since we're working with massive datasets, we need a computational laboratory. We're using Google Colab, which is essentially a computer in the cloud that we can access from anywhere.

Why Google Drive Integration Matters

from google.colab import drive
drive.mount('/content/drive')

This simple act of "mounting" Google Drive is like claiming a permanent workspace. Without it, every time our coding session ends (Colab disconnects after a few hours of inactivity), we'd lose everything—installed packages, downloaded data, analysis results. It's like being a scientist whose lab vanishes every evening and reappears empty the next morning!

🔧 Our Toolkit: The Scientific Instruments

We're assembling quite the collection:

install_if_missing("scanpy")      # The Swiss Army knife of scRNA-seq
install_if_missing("anndata")     # Our data container
install_if_missing("igraph")      # For finding communities
install_if_missing("celltypist")  # AI-powered cell identification
install_if_missing("decoupler")   # Pattern recognition engine

Each library is like a specialized instrument:

Scanpy: Your all-in-one analysis platform (like a high-powered microscope)
AnnData: The filing system that keeps everything organized
igraph: A social network analyzer for cells (who hangs out with whom?)
Celltypist: An AI that learned what different cell types look like
Decoupler: A pattern matcher trained on thousands of experiments

scverse / scanpy

Single-cell analysis in Python. Scales to >100M cells.

Scanpy – Single-Cell Analysis in Python

Scanpy is a scalable toolkit for analyzing single-cell gene expression data built jointly with anndata It includes preprocessing, visualization, clustering, trajectory inference and differential expression testing The Python-based implementation efficiently deals with datasets of more than one million cells. For datasets too large to fit into memory, many scanpy functions are now compatible with dask (warning: experimental).

Discuss usage on the scverse Discourse. Read the documentation. If you'd like to contribute by opening an issue or creating a pull request, please take a look at our contribution guide.

scanpy is part of the scverse® project (website, governance) and is fiscally sponsored by NumFOCUS If you like scverse® and want to support our mission, please consider making a tax-deductible donation to help the project pay for developer time, professional services, travel, workshops, and a variety…

View on GitHub

Part 2: Quality Control - Separating Signal from Noise 🔍

Not every cell we capture is high quality. Some are damaged during collection, some are actually two cells stuck together (doublets), and some are dying as we speak. We need to filter these out.

The Three Red Flags 🚩

1. Mitochondrial Genes (The Dying Cell Signature)

fresh_blood_adata.var['MT'] = fresh_blood_adata.var_names.str.startswith("MT-")

Why this matters: Imagine a sinking ship. The hull breaches, and water rushes in. The cargo floats away, but the heavy machinery stays on board. That's what happens to dying cells:

The cell membrane becomes leaky (the hull breaches)
Cytoplasmic RNA escapes (cargo floats away)
Mitochondrial RNA stays trapped inside (heavy machinery)

If more than 10-20% of a cell's RNA is mitochondrial, it's probably dying. We mark these cells for exclusion.

Healthy Cell:
🔵🔵🔵🔵🔵🔵🔵🔵🔵🔴 (10% MT = good)

Dying Cell:
🔵🔵🔴🔴🔴🔴🔴🔴🔴🔴 (80% MT = bad)

2. Ribosomal Genes (The House Cleaning Noise)

Ribosomal proteins are like the cleaning staff of the cell—essential but not particularly informative about what makes a T cell different from a B cell. Too much ribosomal RNA can drown out the interesting biological signals.

Why this matters: Imagine you're trying to identify different professions by listening to what people talk about:

Doctor: "surgery," "diagnosis," "patient care" ✓ (distinctive)
Teacher: "curriculum," "students," "lesson plans" ✓ (distinctive)
Everyone: "commute," "lunch," "email," "meetings" ✗ (not distinctive)

Ribosomal genes are the cellular equivalent of "everyone talks about this." They're housekeeping genes involved in protein synthesis—absolutely critical for cell survival, but expressed at high levels in ALL cells regardless of type.

The ribosomal noise problem:

Gene Expression Profile:

Biological Signal Genes:         Ribosomal Genes:
CD3D, CD4, CD8 (T cell markers)  RPL/RPS genes (protein machinery)

📊 PROBLEMATIC CELL (High Ribo):
Ribosomal: ████████████████████████████████ 70%
T markers:  ██████ 20%
Other:      ███ 10%
           ↑ Signal buried in noise!

📊 CLEAN CELL (Low Ribo):
T markers:  ████████████████████ 60%
Other:      ███████████ 30%
Ribosomal: ███ 10%
           ↑ Signal clearly visible!

Common ribosomal genes we filter:

RPL3, RPL4, RPL5... (Large ribosomal subunit proteins)
RPS2, RPS3, RPS4... (Small ribosomal subunit proteins)

These can represent 30-50% of total mRNA in some cells!

Visual comparison:

Cell Type Distinction WITHOUT Ribo Filtering:

     Expression Level
        │
   High │ ████ ████ ████  ← All cells look similar
        │ ████ ████ ████     (dominated by ribosomal signal)
    Low │ ░░░░ ░░░░ ░░░░
        └────────────────
         T    B    NK

Cell Type Distinction WITH Ribo Filtering:

     Expression Level
        │
   High │ ████           ← Clear differences!
        │ ░░░░ ████      (distinctive markers visible)
    Low │ ░░░░ ░░░░ ████
        └────────────────
         T    B    NK
        CD3  CD79 NKG7

The filtering strategy:

# Mark ribosomal genes
fresh_blood_adata.var['RIBO'] = fresh_blood_adata.var_names.str.startswith(('RPS', 'RPL'))

# Remove cells with >2% ribosomal content
adata = adata[adata.obs['pct_counts_RIBO'] < 2]

Why 2% threshold?

Distribution of Ribosomal RNA Content:

Cells│     Normal        Technical artifacts
     │       ╱╲          ╱╲
     │      ╱  ╲        ╱  ╲
     │     ╱    ╲______╱    ╲___
     │____╱                      ╲____
     0%   1%   2%   3%   4%   5%  →
          ↑
          2% cutoff
     Keep ✓   Remove ❌

Most healthy cells: 0.5-1.5% ribosomal RNA
Problematic cells: >2% (technical issues during capture)

3. Hemoglobin Genes (The Contamination Signal)

In blood samples, red blood cells can break open and spill their hemoglobin everywhere. It's like trying to study fish in an aquarium but the water is cloudy with algae. We need to filter this contamination.

Why this matters: Red blood cells (erythrocytes) are the most abundant cells in blood—about 5 million per microliter! But they're not immune cells and shouldn't be in our PBMC sample. When they lyse (break open) during sample processing, they release massive amounts of hemoglobin that can contaminate other cells.

The contamination cascade:

DURING SAMPLE PROCESSING:

Step 1: Blood draw
  🔴🔴🔴🔴🔴🔴🔴  ← Millions of red blood cells
  🔵 🟢 🟡        ← PBMCs (what we want)

Step 2: Density gradient separation
  ─────────────
  🔵 🟢 🟡        ← PBMCs float (good!)
  ─────────────
  🔴🔴🔴🔴🔴      ← RBCs sink (good!)

Step 3: Collection (sometimes incomplete)
  🔵 🟢 🟡 💥🔴  ← Some RBCs break open
               ↓
         Hemoglobin spills out!

Step 4: Hemoglobin contamination
  🔵💧 🟢💧 🟡   ← PBMCs coated with hemoglobin
     ↑
  HBA, HBB genes detected in immune cells (BAD!)

Hemoglobin gene expression pattern:

📊 CONTAMINATED CELL:
Hemoglobin: ██████████████████████ 50%  ← Red flag!
(HBA1, HBA2, HBB, HBD)
Immune:     ████ 10%
Other:      ████████ 40%

📊 CLEAN IMMUNE CELL:
Hemoglobin: ░ <0.1%  ← Should be near zero!
Immune:     ████████████████ 70%
Other:      ██████ 30%

Common hemoglobin genes:

HBA1  ─┐
HBA2  ─┤ Alpha globin chains
       │ (Should NOT be in lymphocytes!)
HBB   ─┤
HBD   ─┘ Beta globin chains

The detection pattern:

UMAP Plot - Before Hemoglobin Filtering:

        │  🔵🔵🔵          ← T cells (good)
        │  🔵🔵
        │
        │        🟢🟢      ← B cells (good)
        │        🟢🟢
        │
        │              ❌❌❌  ← Mystery cluster
        │              ❌❌❌     (HIGH HBA/HBB!)
        │              ❌❌❌
        └────────────────────

Gene Expression in Mystery Cluster:
╔═══════════════════════════════════╗
║ Gene      | % Expressed | Avg Exp ║
╠═══════════════════════════════════╣
║ HBA1      |    95%      |  9.2    ║  🚨
║ HBA2      |    92%      |  8.8    ║  🚨
║ HBB       |    89%      |  8.1    ║  🚨
║ CD3D      |     1%      |  0.1    ║  (no T markers)
║ CD79A     |     2%      |  0.1    ║  (no B markers)
║ CD14      |     3%      |  0.2    ║  (no myeloid)
╚═══════════════════════════════════╝

Conclusion: Red blood cell contamination or ambient RNA!

Ambient RNA problem:

What is "ambient RNA"?

During scRNA-seq:

  Droplet with healthy cell:
  ┌─────────────┐
  │  🔵         │ ← Immune cell
  │             │
  │  Clean!     │
  └─────────────┘

  Droplet with contamination:
  ┌─────────────┐
  │  🔵  💧💧   │ ← Cell + hemoglobin
  │      💧💧   │    floating in solution
  │  Problem!   │
  └─────────────┘

The hemoglobin RNA gets captured along with the cell's RNA!

Filtering strategy:

# Mark hemoglobin genes
fresh_blood_adata.var['HB'] = fresh_blood_adata.var_names.str.contains('^HB[^(P)]')

# Remove cells with >0.1% hemoglobin
adata = adata[adata.obs['pct_counts_HB'] < 0.1]

Why 0.1% threshold?

Hemoglobin Content Distribution:

Cells│     Clean cells    Contaminated
     │                    
     │ ╱╲                 
     │╱  ╲                ●  ●
     │    ╲___           ● ● ●
     │        ╲_________●______●___
     0%  0.05% 0.1%  0.5%  1%  5%  →
               ↑
            0.1% cutoff
        Keep ✓    Remove ❌

Clean immune cells: <0.05% hemoglobin
Contaminated cells: >0.1% hemoglobin
Pure RBC debris: >1% hemoglobin

Real-world impact:

CASE STUDY: The Phantom Cluster

Initial Analysis (no HB filtering):
  15 clusters detected
  Cluster 12: ~400 cells

Cluster 12 characteristics:
  ✗ No T cell markers
  ✗ No B cell markers  
  ✗ No NK markers
  ✗ No monocyte markers
  ✓ High HBA1, HBA2, HBB

Gene Ontology Enrichment:
  → "oxygen transport"
  → "hemoglobin complex"
  → "erythrocyte differentiation"

Conclusion: Not a real immune cell type!

After HB filtering:
  14 clusters detected ✓
  Cluster 12: REMOVED
  Clean analysis of actual immune cells!

The triple filter visualization:

Quality Control Cascade:

START: 10,000 cells
  │
  ├─ Filter MT > 20% ────→ Remove 800 dying cells
  │
  ├─ Filter RIBO > 2% ───→ Remove 500 noisy cells
  │
  ├─ Filter HB > 0.1% ───→ Remove 400 contaminated cells
  │
  ↓
END: 8,300 high-quality cells ✓

Cell Quality Score Formula:
Q = -w₁(MT%) - w₂(RIBO%) - w₃(HB%) + w₄(n_genes)

Where:
  w₁ = 2.0  (MT% most important - indicates death)
  w₂ = 1.0  (RIBO% moderate - indicates noise)
  w₃ = 1.5  (HB% high importance - indicates contamination)
  w₄ = 0.5  (n_genes - more genes = better)

Biological interpretation:

Why each filter matters for cell type identification:

WITHOUT FILTERS:                WITH FILTERS:

  "T cell" cluster               True T cell cluster
  Mixed signals:                 Clear signal:
  ├─ CD3D (real T marker)       ├─ CD3D ✓
  ├─ HBA1 (contamination)       ├─ CD3E ✓
  ├─ RPL3 (noise)               ├─ CD4 ✓
  └─ MT-CO1 (dying cells)       └─ IL7R ✓

  Confusing! ✗                  Perfect! ✓

The Quality Control Thresholds: Finding the Sweet Spot

After identifying these red flags, we need to decide where to draw the line. This is like airport security deciding how suspicious someone needs to be before additional screening.

# Set our quality control thresholds
fresh_blood_adata = fresh_blood_adata[
     (fresh_blood_adata.obs['pct_counts_MT'] < 20) &
     (fresh_blood_adata.obs['pct_counts_RIBO'] < 2) &
     (fresh_blood_adata.obs['pct_counts_HB'] < 0.1) &
     (fresh_blood_adata.obs['n_genes_by_counts'] > 200) &
     (fresh_blood_adata.obs['n_genes_by_counts'] < 6000)
]

Visual representation of our filtering criteria:

                         CELL QUALITY SPECTRUM

     ❌ TOO LOW                    ✅ JUST RIGHT                    ❌ TOO HIGH

     <200 genes              200-6000 genes detected           >6000 genes
     (empty/broken)          (healthy cells)                   (doublets)

     ┊                       ┊                                 ┊
     ▓░░░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓░░░░░░░▓
     ┊                       ┊                                 ┊

     Mitochondrial RNA %:

     ❌ >20%                 ✅ <20%
     (dying cell)            (healthy cell)

     🔴🔴🔴🔴🔴🔴🔴🔴🔴🔴     🔵🔵🔵🔵🔵🔵🔵🔵🔴🔴
     (80% MT-bad)            (20% MT-good)

The Filtering Impact: Before and After

Here's what happens to our dataset:

📊 BEFORE FILTERING:
╔════════════════════════════════╗
║  Total cells: 10,000           ║
║  Mean genes/cell: 2,847        ║
║  Median MT%: 4.2%              ║
║  Suspected doublets: ~500      ║
║  Low quality: ~800             ║
╚════════════════════════════════╝

           ↓ APPLY FILTERS ↓

📊 AFTER FILTERING:
╔════════════════════════════════╗
║  Total cells: 8,200 ✓          ║
║  Mean genes/cell: 3,124 ✓      ║
║  Median MT%: 3.1% ✓            ║
║  Clean, high-quality data! 🎉  ║
╚════════════════════════════════╝

What we removed:

~500 doublets (two cells masquerading as one)
~800 low-quality/dying cells (high MT%, low gene counts)
~500 contaminated cells (high hemoglobin)

Why this matters: Garbage in, garbage out. If we include dying cells in our T cell cluster, we might falsely conclude that T cells express death-associated genes when they actually don't!

Interactive Visualization: The Filtering Dashboard

Imagine looking at a scatter plot where each point is a cell:

          Mitochondrial % vs Genes Detected

     MT%
     30│                           ❌ ← dying cells
       │                    ❌  ❌
     25│              ❌ ❌     ❌
       │          ❌          ❌
     20├─────────────────────────────── Filter threshold
       │    ✓✓✓✓✓✓✓✓✓✓✓
     15│  ✓✓✓✓✓✓✓✓✓✓✓✓✓
       │ ✓✓✓✓✓✓✓✓✓✓✓✓✓✓     ← healthy cells
     10│✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓
       │✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓
      5│✓✓✓✓✓✓✓✓✓✓✓✓✓✓
       │✓✓✓✓✓✓✓✓✓✓✓✓✓
      0└─────────────────────────────→ Genes detected
       0   1k  2k  3k  4k  5k  6k  7k
                                           ↑
                                   doublets ❌

The beauty of this scatter plot:

Bottom left: Empty droplets/debris (low genes, low MT)
Middle band: Healthy cells (moderate genes, low MT) ✓
Top right: Dying cells (variable genes, high MT)
Far right: Doublets (too many genes)

Gene-Level Filtering: Removing Uninformative Genes

We also filter genes, not just cells:

# Remove genes detected in fewer than 3 cells
sc.pp.filter_genes(fresh_blood_adata, min_cells=3)

Why remove rare genes?

Gene X detected in 2 cells out of 8,200:

Scenario 1: Technical noise
  Cell_0042: "X" ← Sequencing error
  Cell_5891: "X" ← Random spurious detection

Scenario 2: Extremely rare (can't analyze statistically)
  Even if real, 2 cells is insufficient for robust analysis

The filtering cascade:

          GENE FILTERING CASCADE

Start: 20,000 genes
  ↓
Remove genes in <3 cells
  ↓ (remove ~5,000 rare/noise genes)
  ↓
15,000 genes remain
  ↓
Keep highly variable genes (later step)
  ↓ (select ~2,000-3,000 informative genes)
  ↓
Final: 2,000 genes for analysis

This is like going from every word in the dictionary
to just the keywords that define each cell type!

The Scrublet Score Distribution

When we run doublet detection, we get a score for each cell:

     Doublet Score Distribution

Cells│     Real cells        Doublets
      │        ╱╲               ╱╲
      │       ╱  ╲             ╱  ╲
      │      ╱    ╲___________╱    ╲
      │     ╱                       ╲
      │____╱_________________________╲___
      0.0  0.1  0.2  0.3  0.4  0.5  0.6
                 Doublet Score →

      ├─────────┤├──────────────────┤
       Keep ✓    Remove (threshold) ❌

Reading this plot:

Left peak: Single cells (low doublet scores) ✓
Right peak/tail: Likely doublets (high scores) ❌
Threshold (typically 0.25-0.30): Decision boundary

The Complete Quality Control Workflow Visualized

                         QC WORKFLOW

     📥 Raw Data
     10,000 cells
           │
           ├──→ [Calculate QC metrics]
           │    • Count genes per cell
           │    • Calculate MT%
           │    • Calculate RIBO%
           │    • Calculate HB%
           │
           ├──→ [Detect doublets]
           │    • Run Scrublet
           │    • Flag suspicious cells
           │
           ├──→ [Visualize distributions]
           │    • Violin plots
           │    • Scatter plots
           │    • Histograms
           │
           ├──→ [Set thresholds]
           │    • MT% < 20%
           │    • Genes: 200-6000
           │    • RIBO% < 2%
           │    • HB% < 0.1%
           │    • Doublet score < 0.25
           │
           ├──→ [Apply filters]
           │    • Remove 1,800 cells
           │
           ↓
     ✅ Clean Data
     8,200 high-quality cells
           │
           ↓
     [Continue to normalization...]

Real Example: The Contamination Story

Here's a real scenario from our analysis:

CLUSTER 7: The Mystery Cluster

Initial observation:
  • 450 cells
  • High hemoglobin genes (HBA1, HBA2, HBB)
  • Low immune markers
  • Suspiciously uniform

Investigation:
  ╔═══════════════════════════════════╗
  ║ Gene      | % Expressed | Avg Exp ║
  ╠═══════════════════════════════════╣
  ║ HBA1      |    92%      |  8.7    ║  ← Red flag!
  ║ HBA2      |    89%      |  8.2    ║  ← Red flag!
  ║ HBB       |    87%      |  7.9    ║  ← Red flag!
  ║ CD3D      |     2%      |  0.1    ║  ← Should be higher
  ║ CD79A     |     1%      |  0.0    ║  ← Should be higher
  ╚═══════════════════════════════════╝

Conclusion: Red blood cell contamination!

Action: Filter out this cluster entirely

Result: Clean immune cell analysis ✓

The Mathematical Beauty of QC

Quality control involves elegant statistical reasoning:

For each cell, we calculate a quality score:

Q = w₁(genes_normalized) - w₂(MT_pct) - w₃(doublet_score)

Where:
  genes_normalized = (genes - min) / (max - min)
  MT_pct = mitochondrial percentage
  doublet_score = Scrublet output
  w₁, w₂, w₃ = weights (importance of each factor)

High Q score → Keep cell ✓
Low Q score → Remove cell ❌

Visual representation:

Quality Score Distribution:

      │        Keep these →  ╱╲
Cells│                     ╱  ╲
      │                    ╱    ╲___
      │    Remove these → ╱          ╲
      │___________________╱____________╲____
      Low                              High
                 Quality Score →

📈 Visualizing Quality Metrics

Here's what our quality control plots reveal:

        Number of Genes Detected
             🎻
        │    ╱ ╲
        │   ╱   ╲
   Cells│  ╱     ╲___
        │ ╱          ╲___
        └──────────────────────
           0  2000  4000  6000

This violin plot shows:

✅ Most cells: 2,000-4,000 genes detected
❌ Low-quality cells: <1,000 genes detected
⚠️ Possible doublets: >6,000 genes detected

🎭 Doublet Detection: Finding Cellular Impostors

sc.pp.scrublet(fresh_blood_adata)

Doublets are when two cells end up in the same droplet. It's like conducting interviews but accidentally interviewing two people at once—their combined answers make no sense!

How Scrublet works:

Simulates artificial doublets by merging random pairs of real cells
Compares each real cell to these fake doublets
Flags cells that look suspiciously similar to the fakes

Cell A + Cell B = Doublet
  🔵    +   🟢   =   🔵🟢
 (T cell)  (B cell)  (Fake "hybrid" cell)

Part 3: Normalization - Making Apples Comparable to Apples 🍎

Different cells captured different amounts of RNA—not because they're biologically different, but due to random technical variation. Some droplets caught "talkative" cells, others caught "quiet" ones.

The problem: Cell A might have 50,000 RNA molecules, Cell B only 5,000. Does this mean Cell A is more active? Not necessarily! Cell A might have just been captured more efficiently.

The solution: Normalize!

# Scale each cell to the same total count
sc.pp.normalize_total(fresh_blood_adata)
# Log-transform to stabilize variance
sc.pp.log1p(fresh_blood_adata)

This is like adjusting volume levels so every musician can be heard equally. Now we can compare their actual melodies, not just their loudness.

Why Log Transformation?

Gene expression follows a power law distribution—a few genes are massively expressed, most are moderate or low. Log transformation compresses this range so we can see patterns in both highly and lowly expressed genes.

Before log:
Gene A: ▏
Gene B: ▍
Gene C: ▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓  (drowns out everything!)

After log:
Gene A: ▍
Gene B: ▍▍
Gene C: ▍▍▍▍  (all visible!)

Part 4: Feature Selection - Finding the Genes That Matter 🎯

Out of 20,000 genes, which ones actually help us distinguish cell types?

Most genes are "housekeeping genes"—genes that every cell needs to survive, like the genes for making ribosomes or processing energy. These genes are like everyone in the city wearing pants—sure, it's universal, but it doesn't tell you whether someone is a doctor, teacher, or artist.

We want the distinctive genes—the ones that make cell types unique:

CD3D, CD3E → T cell markers (like a doctor's stethoscope)
CD79A, CD79B → B cell markers (like a teacher's chalk)
NKG7, GNLY → NK cell markers (like a soldier's uniform)

sc.pp.highly_variable_genes(fresh_blood_adata, n_top_genes=1000)

This selects the top 1,000 most variable genes—genes that change dramatically between cells, indicating biological differences rather than random noise.

📊 Highly Variable Genes Plot:

   Dispersion
      ↑
   ●  │  ●  Selected genes
   ●  │ ●● ●  (high variance)
 ●    │ ● ●
  ● ● │
────●─┼────────→ Mean Expression
      │ ○ ○ ○  Not selected
      │○  ○    (low variance)

Part 5: Dimensionality Reduction - From 1,000 Dimensions to 2D 📐

We now have 1,000 genes per cell. That's 1,000 dimensions—impossible to visualize! Imagine trying to draw a 1,000-dimensional graph. Your brain would melt.

Enter PCA (Principal Component Analysis)

PCA is like finding the best camera angles to photograph a complex 3D sculpture. Instead of taking 1,000 photos from random angles, we find the 30-50 angles that show the most interesting features.

sc.tl.pca(fresh_blood_adata)

How PCA works (simplified):

Find the direction with the most variation (PC1)
Find the next direction with the most variation perpendicular to PC1 (PC2)
Continue for 50 components

Original 1,000 dimensions:
🎯🎯🎯🎯🎯🎯🎯🎯🎯🎯... (×100)

Compressed to 50 PCs:
🎯🎯🎯🎯🎯 (captures 80%+ of the information!)

UMAP: The Final Visualization

PCA gives us 50 dimensions—better, but still not visualizable. Enter UMAP (Uniform Manifold Approximation and Projection):

sc.tl.umap(fresh_blood_adata)

UMAP is like taking a complex network map of a city with all its roads, buildings, and connections, and drawing a simplified 2D map that preserves the important relationships:

Neighborhoods stay together
Connected areas remain connected
Distances approximately preserved

         UMAP Plot
    ╭────────────────╮
    │  🔵🔵🔵         │
    │  🔵🔵           │
    │                │
    │        🟢🟢🟢  │
    │        🟢🟢    │
    │                │
    │ 🔴🔴🔴          │
    │ 🔴🔴            │
    ╰────────────────╯

🔵 = T cells
🟢 = B cells
🔴 = NK cells

Part 6: Clustering - Discovering Cell Types 🔬

Now we can see clear islands of cells in our UMAP plot. But which island is which cell type?

The Leiden Algorithm: Finding Communities

sc.tl.leiden(fresh_blood_adata, resolution=0.5)

Think of your cells as people at a conference. The Leiden algorithm is like observing who talks to whom:

People cluster into groups (research fields)
Similar people group together naturally
Some people bridge multiple groups

How it works:

Build a network where similar cells are connected
Find communities where connections are dense internally but sparse externally
Iteratively optimize these communities

Network Graph:

T cells:        B cells:
 ○─○─○          ○─○
 │ │ │          │ │
 ○─○─○          ○─○
   ↓ (weak connection)
NK cells:
 ○─○─○
 │ │ │
 ○─○─○

🔍 Resolution: Choosing the Right Magnification

The resolution parameter is like choosing how close to zoom in:

# Low resolution (0.02): See major cell lineages
sc.tl.leiden(adata, resolution=0.02)  
# → 3-5 clusters (T cells, B cells, Myeloid)

# Medium resolution (0.5): See cell types
sc.tl.leiden(adata, resolution=0.5)   
# → 10-15 clusters (CD4+ T, CD8+ T, Memory B, Naive B...)

# High resolution (2.0): See cell subtypes
sc.tl.leiden(adata, resolution=2.0)   
# → 30+ clusters (Th1, Th2, Th17, Tregs...)

It's like looking at a forest:

🌲 Far away: You see the forest (low resolution)
🌳 Closer: You see individual tree species (medium resolution)
🍃 Very close: You see oak subspecies and age classes (high resolution)

Part 7: Cell Type Annotation - Putting Names to Faces 🏷️

We have clusters, but what ARE they? This is where Decoupler comes in.

Decoupler + PanglaoDB: The Cell Type Detective

markers = dc.op.resource(name="PanglaoDB", organism="human")
dc.mt.ulm(data=fresh_blood_adata, net=markers, tmin=3)

PanglaoDB is a database containing thousands of known cell type markers from published studies. It's like having a library of "wanted posters" for different cell types:

╔═══════════════════════════╗
║      B CELL MARKERS       ║
║  - CD19  - CD79A          ║
║  - CD79B - MS4A1          ║
╚═══════════════════════════╝

╔═══════════════════════════╗
║      T CELL MARKERS       ║
║  - CD3D  - CD3E           ║
║  - CD3G  - CD4            ║
╚═══════════════════════════╝

How Decoupler annotates cells:

For each cell, check how many B cell markers it expresses
Check how many T cell markers it expresses
Check all other cell types
Assign the cell to the type with the highest score

Cell X:
  B cell score:  0.2 ⭐
  T cell score:  8.7 ⭐⭐⭐⭐⭐⭐⭐⭐
  NK cell score: 0.1 ⭐

→ Cell X is a T CELL! 🎯

Statistical Ranking

We go one step further by statistically testing which cell type signature is most enriched in each cluster:

fresh_blood_adata_rank = dc.tl.rankby_group(
    score, 
    groupby="leiden_res0_02", 
    reference="rest", 
    method="t-test_overestim_var"
)

This is like asking: "For each cluster, which cell type signature is significantly higher here than everywhere else?"

Part 8: Validation - Trust, But Verify ✅

Automated annotation is powerful, but we still need to validate with known marker genes. Let's visualize!

The Dot Plot: Expression at a Glance

sc.pl.dotplot(fresh_blood_adata, marker_genes_dict, "cell_type")

A dot plot shows two things:

Dot size: What percentage of cells express this gene?
Dot color: How strongly is it expressed?

             CD3D  CD79A  NKG7
T cells       ●●●    ·     ·
B cells        ·    ●●●    ·
NK cells       ·     ·    ●●●

● = large, red dot (high expression)
· = tiny dot (low/no expression)

If our annotation is correct, we should see:

✅ T cell markers HIGH in T cells, LOW elsewhere
✅ B cell markers HIGH in B cells, LOW elsewhere
✅ NK cell markers HIGH in NK cells, LOW elsewhere

The Violin Plot: Distribution Stories

sc.pl.violin(score, keys=["B cells memory"], groupby="leiden_res0_02")

Violin plots show the full distribution of expression levels:

Cluster 0:  🎻 (wide, high median → B cells!)
Cluster 1:  🎵 (narrow, low → not B cells)
Cluster 2:  🎵 (narrow, low → not B cells)

The Heatmap: The Big Picture

sc.pl.heatmap(fresh_blood_adata, marker_genes_dict, groupby="leiden_res0_02")

A heatmap shows everything at once:

           CD3D CD3E CD79A MS4A1 NKG7 GNLY
Cluster 0: 🟦🟦 🟨🟨 🟦🟦   (T cells)
Cluster 1: 🟦🟦 🟨🟨 🟦🟦   (B cells)
Cluster 2: 🟦🟦 🟦🟦 🟨🟨   (NK cells)

🟨 = high expression
🟦 = low expression

Each row should show a distinct pattern, like a unique barcode for each cell type.

🧬 The Biology Behind the Code: What We Discovered

After all this analysis, what did we actually learn about these blood cells?

The Major Players in Your Immune System

🔵 T Cells (CD3+): Your adaptive immune system's quarterbacks

Coordinate immune responses
Kill infected cells
Remember past infections
Represent ~70% of lymphocytes in blood

🟢 B Cells (CD19+, CD79A+): Your antibody factories

Produce immunoglobulins
Create immunological memory
Present antigens to T cells
~10-15% of lymphocytes

🟡 NK Cells (CD56+, NKG7+): Your innate immune system's assassins

Kill virus-infected and tumor cells
Don't need prior exposure
Rapid response team
~5-15% of lymphocytes

🔴 Monocytes (CD14+, CD16+): Your mobile macrophages

Engulf pathogens and debris
Differentiate into macrophages and dendritic cells
Present antigens
~2-10% of white blood cells

Subtypes and Complexity

Within each major type, we find subtypes:

T Cells:

CD4+ Helper T cells (orchestrate immune response)
CD8+ Killer T cells (destroy infected cells)
Regulatory T cells (prevent autoimmunity)
Memory T cells (remember past infections)

B Cells:

Naive B cells (haven't encountered antigen yet)
Memory B cells (remember past infections)
Plasma cells (antibody-producing factories)

This heterogeneity is exactly WHY single-cell analysis is revolutionary—we'd miss all this complexity in bulk sequencing!

💻 The Code That Makes It Happen

Let me show you the elegant simplicity of the core workflow:

# 1. Load and QC
adata = sc.read_10x_h5(filename)
sc.pp.calculate_qc_metrics(adata, qc_vars=["MT", "RIBO", "HB"], inplace=True)
sc.pp.filter_cells(adata, min_genes=1000)
sc.pp.filter_genes(adata, min_cells=1000)

# 2. Normalize
adata.layers["counts"] = adata.X.copy()  # Save raw counts
sc.pp.normalize_total(adata)
sc.pp.log1p(adata)

# 3. Feature selection
sc.pp.highly_variable_genes(adata, n_top_genes=1000)

# 4. Dimensionality reduction
sc.tl.pca(adata)
sc.pp.neighbors(adata)
sc.tl.umap(adata)

# 5. Clustering
sc.tl.leiden(adata, resolution=0.5)

# 6. Cell type annotation
markers = dc.op.resource(name="PanglaoDB", organism="human")
dc.mt.ulm(data=adata, net=markers)

# 7. Visualize
sc.pl.umap(adata, color=["leiden", "cell_type"])
sc.pl.dotplot(adata, marker_genes, "cell_type")

That's it! About 15 lines of code to go from raw data to annotated cell types. Of course, the devil is in the details—choosing the right parameters, validating results, interpreting biology—but the core workflow is remarkably elegant.

📓 Get the Full Code: Check out the complete Jupyter notebook on GitHub with step-by-step explanations and all visualizations!

🌟 Real-World Applications: Why This Matters

1. Cancer Immunotherapy 🎗️

Single-cell analysis revealed why some patients respond to immunotherapy and others don't:

Responders have more "exhausted" T cells that can be reactivated
Non-responders lack these cells or have immunosuppressive cells
Led to combination therapies targeting specific cell states

2. COVID-19 Severity 🦠

During the pandemic, scRNA-seq showed:

Severe cases had dysregulated interferon responses
Certain monocyte subtypes predicted worse outcomes
Helped identify therapeutic targets

3. Autoimmune Diseases 🔄

Discovered that:

Regulatory T cells are dysfunctional, not absent, in many autoimmune conditions
Different autoimmune diseases have distinct cellular signatures
Opened new avenues for targeted therapy

4. Developmental Biology 👶

Mapping how embryonic stem cells become specialized:

Identified intermediate cell states
Revealed decision points in cell fate
Guided strategies for lab-grown tissues

🚀 The Future: Where Are We Headed?

Multi-Omics Integration

Imagine measuring not just RNA, but also:

Proteins (what's actually doing the work)
Chromatin accessibility (which genes CAN be turned on)
DNA methylation (epigenetic memory)
TCR/BCR sequences (immune receptor repertoire)

All in the SAME cell. This is happening now!

Spatial Transcriptomics

The next frontier: measuring gene expression while preserving WHERE each cell sits in the tissue:

        Traditional scRNA-seq          Spatial

             ●●●●●●                   🔵🔵🟢🔵🔵
             ●●●●●●                   🔵🔵🟢🔵🔵
        (lost spatial info)          🟢🟢🟢🟢🟢
                                     🔴🔴🔴🔴🔴
                                (preserved positions!)

This reveals:

Cell-cell communication networks
Tissue architecture
Microenvironments
Gradients and zones

AI-Powered Analysis

Machine learning models are being trained on millions of cells to:

Automatically annotate any cell from any tissue
Predict drug responses
Identify disease signatures
Generate hypotheses

Real-Time Clinical Diagnostics

The dream: rapid scRNA-seq in hospitals:

Patient presents with immune disorder
Draw blood
scRNA-seq analysis in 24 hours
Precise diagnosis and personalized treatment plan

🤔 The Philosophy: Why Small Things Matter

There's something profound about single-cell analysis. For centuries, we studied populations, averages, bulks. We asked "What does THE cell do?" as if there was one representative of each type.

Single-cell analysis taught us humility. There is no "THE cell." Every cell is an individual with its own history, state, and destiny. The T cell next to its sibling might be:

Responding to infection while its neighbor rests
Expressing different genes despite identical DNA
On a path to memory while its twin approaches exhaustion

This is true for cells, and perhaps, it's true for life. We are not averages. We are distributions. We are heterogeneous. And that heterogeneity is not noise—it IS the biology.

When we measure 10,000 cells individually, we're not just generating data. We're acknowledging that:

Rare cells matter (the 0.1% might save your life)
Transitions matter (cells in-between states reveal mechanisms)
Variation matters (diversity enables adaptation)

🎓 Getting Started: Your Own Single-Cell Journey

Want to try this yourself? Here's your roadmap:

1. Learn Python Basics (2-4 weeks)

Variables, loops, functions
Pandas for data manipulation
Matplotlib for visualization

2. Understand the Biology (ongoing)

Cell biology fundamentals
Immunology basics (if analyzing immune cells)
Read reviews on scRNA-seq

3. Work Through Tutorials (4-8 weeks)

Scanpy tutorials
Seurat tutorials (R alternative)
Published workflows

4. Analyze Public Data (2-3 months)

10x Genomics public datasets
Human Cell Atlas
GEO database

5. Design Your Own Experiment (∞)

Define biological question
Plan experimental design
Collaborate with experts

📚 Resources

Essential Reading:

"Current best practices in single-cell RNA-seq analysis" (Luecken & Theis, 2019)
Scanpy documentation
Single Cell Best Practices online book

Datasets to Practice:

Communities:

Biostars forum
r/bioinformatics
Single-cell analysis Slack/Discord channels

🎼 Final Thoughts: The Symphony of Life

When I first looked at a UMAP plot of 10,000 cells, each dot representing an individual cell, I was struck by the beauty of it. These aren't just data points. Each one is a living entity with:

Billions of proteins working in concert
Thousands of molecular machines running simultaneously
A complete copy of your genome deciding what to become
A past (where it came from) and future (what it will become)

And we can listen to each one's unique molecular signature—the genes it chooses to express at this moment in time.

It's like conducting a 10,000-piece orchestra where:

Every musician plays their own score
The music changes moment by moment
Patterns emerge from seeming chaos
And somehow, together, they create the symphony of life

That's what single-cell RNA sequencing lets us do: listen to the individual voices while appreciating the collective harmony.

Your immune system—right now, as you read this—contains billions of these cells. Each one performing its role, some fighting infections you'll never know you had, some remembering that cold from three years ago, some just quietly waiting for their moment to act.

We're just beginning to learn their stories. And with every cell we sequence, every algorithm we refine, every biological insight we gain, we get a little closer to understanding the fundamental question:

What does it mean to be alive, one cell at a time?

🙏 Acknowledgments & Further Exploration

This analysis was performed using:

HackBio Intership Analysis
Scanpy (Wolf et al., 2018)
AnnData data structure
Decoupler (Badia-i-Mompel et al., 2022)
PanglaoDB marker database
10x Genomics technology and datasets

📓 Access the Full Tutorial

The complete Jupyter notebook with all code, visualizations, and detailed explanations is available on GitHub:

View the Human Diseased PBMC Notebook on GitHub

What's included:

✅ Complete analysis pipeline
✅ All code cells with explanations
✅ Quality control visualizations
✅ Clustering and annotation workflows
✅ Interactive plots and heatmaps

🌐 Connect & Learn

🐙 GitHub Repository
💬 Questions? Drop a comment below!
🔔 Follow for more bioinformatics tutorials
📧 Reach out for collaborations

"In the world of single-cell analysis, every cell has a story. Our job is to listen."

Bioinformatics: PackagesNotFoundError on osx-arm64 Platform

Awosise Oluwaseun — Tue, 09 Apr 2024 23:51:52 +0000

I was following the Biostar Handbook up until where I was supposed to create an environment and install the Bioinformatics tools required for the rest of the project when I encountered the error that brought you here: PackagesNotFoundError.
I had to surf the internet for solutions, and I found bits and pieces of them, which took quite some time. Don't worry; all of the research has been done to streamline the implementation process for you, allowing you to save more time.

Why the Error
The error is a result of the architecture/platform (osx-arm64) introduced to the M-chip MacBook systems, as opposed to the osx-64 platform in the preceding Intel versions. The packages obtained from channels like conda-forge and bioconda are not yet available on the osx-arm64 platform, hence the error.

The Solution
The solution is to find a way to set the platform for the environment created to osx-64 platform (which has all the required packages downloadable via the channels) by following the steps below in this order:

Install miniconda.
Add channels to fetch packages like bioconda and conda-forge to the base environment in that order using conda config --add channels bioconda and conda config --add channels conda-forge.
Create an environment, e.g., bioinfo, using the code line conda create --name bioinfo -y. This is done to containerize the packages to avoid conflicts like version conflicts.
Activate the created environment using conda activate bioinfo.
After that, set your environment platform to osx-64 instead of the default osx-arm64 to enable the installation of packages, as they have not yet been migrated to the osx-arm64 platform. The CLI command used is: conda config --env --set subdir osx-64.
Then, install the required package, specifying the version if needed. For example, to install Python version 3.6 on the environment, use the command conda install python=3.6.
Finally, install all the tools required for bioinformatics in the environment using curl http://data.biostarhandbook.com/install/conda.txt | xargs conda install -y.

Setting tmp Folder Using express-fileupload before Deploying to Cyclic.sh

Awosise Oluwaseun — Wed, 01 Feb 2023 21:16:18 +0000

Error: EROFS: read-only file system, '/var/task/tmp/tmp...'

You are here because you probably just finished a file upload feature that runs well locally but on deploying it to cyclic.sh, you encountered the error above.

Don't fret, there are 2 key things to know about this error:

You can only write to a tmp folder which must be in the root folder of your project directory.
Cyclic.sh tries to navigate the tmp folder relative to the current working directory which is not the root folder of your project. Hence, it appends /var/task/ to the tmp directory itself.

The simple solution is to set the value of one of the parameters in your express-fileupload package as shown below. I would assume you have installed both express-fileupload and express, if not, run the following command:

 npm install express-fileupload express

// The Solution
const express = require('express');
const fileupload = require('express-fileupload');

app.use(fileupload({
    useTempFiles: true,
    tempFileDir: "/tmp",
}))

It is important that you set the value of the tempFileDir property to be \tmp. This way, cyclic.sh is able to navigate the tmp folder relative to the root directory. You can read more on express-fileupload.

I hope this helped. Thanks for reading.

Rendering Mongoose Error(s) to View(s) using Bootstrap5 and EJS

Awosise Oluwaseun — Sat, 28 Jan 2023 16:14:08 +0000

I recently completed a project that required me to send validation errors generated in the model schema to the user. Validation errors are generated when a user's input does not conform to the rules set to save such data. For example, if I set a password to not be more than 6 characters while signing up, a validation error is thrown when the user tries to set a password less than 6 characters.

The picture above summarizes what we are trying to achieve in this article. We will be working on just this page to achieve this. The link to the whole project is here and is still a work in progress as of the time of publishing this article.

Let's get started

Prerequisite

To follow through, you will need basic understanding of the following:

Since we are going to try to render errors from the server to our views, we must have a basic understanding of Express.js in setting up a server.

Setting your Model to send out Errors

Mongoose allows us to design our Schema by defining certain properties that help to validate the kind of data that is sent by the user.

npm install mongoose

const mongoose = require("mongoose")
const { Schema } = mongoose
const { isEmail } = require("validator")

const AuthSchema = new Schema({
  email: {
    type: String,
    required: [true, "Please enter an email"],
    unique: true,
    lowercase: true,
    validate: [isEmail, "please enter a valid email"]
  },
  firstName: {
    type: String,
    required: [true, "Please enter your first name"],
    capitalize: true
  },
  lastName: {
    type: String,
    required: [true, "Please enter your last name"],
    capitalize: true
  },
  password: {
    type: String,
    required: [true, "Please enter password"],
    minlength: [6, "Minimum password length is 6 characters"]
  }
})

Taking the email field for example in the above code, it has several properties with set values but we are going to look at the required and the validate properties whose values are in an array. The second element of the array represents the error message.

Property	Error message returned	What triggers error message
required	please enter an email	when email field is left empty
validate	please enter a valid email	when email an invalid email is supplied

Handling the Error that comes from Mongoose

Before the error is passed on to a controller, we need to have a function that modifies the error in order to make it easier to use down the line.

const handleErrors = (err) => {
  let errors = { email: "", firstName: "", lastName: "", password: "" }

  // duplicate error
  if (err.code === 11000) {
    errors.email = "the email is already in use"
    return errors
  }

  //validation errors
  if (err.message.includes("auth validation failed")) {
    // destructuring inside the forEach, similar to (err) => {err.properties}
    Object.values(err.errors).forEach(({ properties }) => {
      errors[properties.path] = properties.message
    })
  }
  return errors
}

module.exports = handleErrors

The function above handles the long error message returned from by mongoose, exacts everything we need and converts them into a more readable object.

Sending the Errors to the View(s)

Upon receiving the error message, the sign up controller sends the error message(should there be one) to the views. Let's talk about the sign up controller but before then, let's look at these concepts:

Session: A session is generated and saved on the server side when a request is sent from the client side. The session has a unique ID that is sent, along with any data it holds, to the client side to be stored in a cookie. This cookie is then sent with every subsequent request to the server. This process is accomplished using the express-session middleware package. This video does a good job of illustrating this concept.
Locals: The res.locals property is an object that contains response-local variables, which are only available to the view(s) rendered during that request/response cycle. This means that any variables that are attached to the res.locals object will not be available to other requests or views. It's a way to pass data from the server-side to the client-side to be used in the views. This concept is often used in template engines such as EJS and Jade to render dynamic content on the client-side.

Having talked about both concepts, let's see how they help with sending error messages to the view(s)

npm install express express-session

const express = require("express")
const session = require("express-session")

const app = express()


app.use(
  session({
    secret: process.env.SESSION_SECRET,
    saveUninitialized: true,
    resave: false
  })
)

app.use((req, res, next) => {
  res.locals.content = req.session.content
  delete req.session.content
  next()
})

// set teplate engine as EJS
app.set("view engine", "ejs")

The express-session middleware gives us access to the req.session. The req.session object gives us access to the session data and the ability to set new properties on it. In the example you provided, the req.session is used to set a content property, which is then passed to the res.locals.content property for use in rendering views. The req.session.content is deleted after it is assigned to res.locals.content.
It is important to note that res.locals is only available to the view(s) rendered during that request/response cycle.

Let's now talk about the sign-up controllers.

module.exports.signupPost = async (req, res) => {
  const { email, firstName, lastName, password } = req.body
  try {

    const user = await User.create({ email, firstName, lastName, password })
    res.status(200).json(user)

  } catch (err) {

    const errors = handleError(err)
    req.session.content = {
      email: email,
      firstName: firstName,
      lastName: lastName
    }

    res.render("pages/signup", {
      title: "Signup Page",
      error: errors,
      content: req.session.content
    })

  }
}

The above code is only an abstract of the entire project. Let's focus on the catch block as that is where our errors will be caught and passed on to the view(s).

req.session.content from our explanation above assigns the data stored in it i.e email, first_name and last_name to res.locals.content. This allows us to get back the previously inputted field along with our error message as we have in image at the beginning of this article.

res.render takes two parameters, the first parameter is the path to the EJS file to be rendered while the second parameter takes all the variables containing the data to be used to run the page dynamically.
This is where our error is rendered after been handled by our error handling function (handleError) described above.

The View

The view combines our understanding of HTML, bootstrap and EJS which are prerequisites to understanding this article.

<%- include("layout/authheader")  %> 

<div class="container">
    <div class="row">
        <div class="col-lg-6 mx-auto mt-4">
            <div class="card shadow">
                <div class="card-header bg-secondary">
                    <h3 class="text-light">Sign Up</h3>
                </div>
                <div class="card-body p-4">
                    <form action="/signup" method="post" id="add-form" enctype="application/x-www-form-urlencoded">
                        <div class="mb-3">
                            <label for="email" class="form-label">Email</label>
                            <input type="email" name="email" class="form-control form-control-lg <%= error.email ? 'is-invalid' : '' %> " value="<%= content ? content.email.toString() : '' %>" placeholder="Enter email" required />
                            <% if (error.email !== "") { %>
                             <div class="invalid-feedback">
                                <%= error.email %> 
                             </div>
                            <% } %>
                        </div>
                        <div class="mb-3">
                            <label for="firstName" class="form-label">First Name</label>
                            <input type="text" name="firstName" class="form-control form-control-lg <%= error.firstName ? 'is-invalid' : '' %> " value="<%= content ? content.firstName.toString() : '' %>" placeholder="Enter first name" required />
                            <% if (error.firstName !== "") { %>
                                <div class="invalid-feedback">
                                   <%= error.firstName %> 
                                </div>
                               <% } %>
                        </div>
                        <div class="mb-3">
                            <label for="lastName" class="form-label">Last Name</label>
                            <input type="text" name="lastName" class="form-control form-control-lg <%= error.lastName ? 'is-invalid' : '' %> " value="<%= content ? content.lastName.toString() : '' %>" placeholder="Enter last name" required />
                            <% if (error.lastName !== "") { %>
                                <div class="invalid-feedback">
                                   <%= error.lastName %> 
                                </div>
                               <% } %>
                        </div>
                        <div class="mb-3">
                            <label for="password" class="form-label">Password</label>
                            <input type="password" name="password" class="form-control form-control-lg <%= error.password ? 'is-invalid' : '' %> " placeholder="Enter password" required />
                            <% if (error.password !== "") { %>
                                <div class="invalid-feedback">
                                   <%= error.password %> 
                                </div>
                               <% } %>
                        </div>
                        <div class="mb-3 d-grid">
                            <input type="submit" name="submit" value="Sign up" class="btn btn-secondary btn-lg" />
                        </div>
                    </form>
                </div>
            </div>
        </div>
    </div>
</div>

<%- include("layout/footer")  %>

The above code shows a simple HTML form styled with bootstrap. The boostrap link and script are contained in the layout/authheader and layout/footer respectively. This can be checked in the project source code.

        <div class="mb-3">
            <label for="email" class="form-label">Email</label>
            <input type="email" name="email" class="form-control form-control-lg <%= error.email ? 'is-invalid' : '' %> " value="<%= content ? content.email.toString() : '' %>" placeholder="Enter email" required />
            <% if (error.email !== "") { %>
                <div class="invalid-feedback">
                    <%= error.email %> 
                </div>
            <% } %>
        </div>

Taking the email input field for example, a ternary operator <%= error.email ? 'is-invalid' : '' %> is used to change the border color of the box should there be any error.
When a post request is sent, this code first checks if there is an error message for the email field (if errors.email exists), and if so, it displays the error message using the <div> tag.
Another thing to quickly note is that when there's no error message, an error is raised by EJS, so to avoid this, we have to render the same page(when a get request is sent for the page) with error set as an empty string as shown below.

module.exports.signupGet = (req, res) => {
  res.render("pages/signup", {
    title: "SignUp Page",
    error: "",
    content: ""
  })
}

That's it! You now know how to render validation errors from your Mongoose model to your views. You can also customize this process to fit your specific needs by modifying the handleErrors function and the way you pass the errors to the views.

Introduction to Building your First REST API with Node.js

Awosise Oluwaseun — Tue, 10 Jan 2023 20:01:00 +0000

If you are here, it means you probably just finished learning NodeJS and it frameworks and you are looking to get your hands dirty with your first Node.js project.

Welcome to this project on building your first REST API with Node.js! In this project, we'll walk through the steps of creating a simple, yet fully functional REST API using Node.js and the Express.js framework using a blog API as an example. By the end of this project, you'll have a solid understanding of how to build a REST API with Node.js and be able to incorporate some of the features in your own projects.

Before getting into the technical aspects of the REST API, let us take a look at what REST APIs are and a broad overview of this project.

What is REST API?

REST APIs (Representational State Transfer APIs) are a way for applications to interact with each other through the exchange of HTTP requests and responses. These APIs are designed to be lightweight, fast, and easy to use, making them a popular choice for developers looking to build modern, scalable applications. I also like to refer to it as a convention that has been adopted by many developers to writing APIs. The use of HTTP verbs or methods (suchs as "GET", "POST") and paths to access resources in the database(endpoints).

Overview of the Project

In this tutorial, we'll be building a simple REST API using Node.js and the Express.js framework. We'll cover the following topics:

Setting up a Node.js development environment.
Installing and configuring required packages and environment.
Designing our blog API routes and features.
Connecting to a database and implementing the blog API designs.
Testing and Deploying the API to a production environment.

Prerequisites

To follow along with this project, you'll need to have a basic understanding of the following:

You should also have Node.js and npm (the Node.js package manager) installed on your machine. If you don't have these tools already, you can download them from the official Node.js website.

Setting up the Development Environment

The first step in building a REST API with Node.js is to set up a development environment. This will allow us to write, test, and debug our code as we build the API.

To set up the development environment, follow these steps:

Create a new directory for the project and open it in your code editor of choice(preferably VS Code).
Open the terminal in your VS Code using ctrl + ` . The back tick is below your escape button.
Initialize a new npm project by running npm init in the terminal. This will create a package.json file in the project directory.

npm init

The above images show our folder and file structure, as well as the dependencies section of our package.json file on VS Code. Node environment has already been setup here.

Installing and Configuring Required Packages and Enviroment

In order to start building our application, we must first install all necessary packages. There are two ways of going about this:

Running npm install [package names separated by space]
Copy the dependencies below and replace it in your package.json file then run npm install

"dependencies": {
    "bcrypt": "^5.1.0",
    "body-parser": "^1.20.1",
    "cors": "^2.8.5",
    "cross-env": "^7.0.3",
    "dotenv": "^16.0.3",
    "express": "^4.18.2",
    "express-async-errors": "^3.1.1",
    "express-rate-limit": "^6.7.0",
    "helmet": "^6.0.1",
    "joi": "^17.7.0",
    "jsonwebtoken": "^9.0.0",
    "moment": "^2.29.4",
    "mongoose": "^6.7.0",
    "morgan": "^1.10.0",
    "passport": "^0.6.0",
    "passport-jwt": "^4.0.0",
    "passport-local": "^1.0.0",
    "swagger-ui-express": "^4.6.0",
    "winston": "^3.8.2",
    "yamljs": "^0.3.0"
  }

The two ways mentioned above will add all the required library to the node_modules directory and update the dependencies section of the package.json file.

As we proceed, we would talk more about all of the packages installed.

Another thing we need to consider is setting up our .env and .gitignore file. The .env file was screated in the config folder in our project folder structure.

In the .env file, add the following as you would be assigning values to them as we go on:

MONGODB_URI=
TEST_MONGODB_URI=
PORT=
JWT_SECRET=

In the .gitignore file, add the following:

node_modules
.env

Now that we have setup our enviroment, Let's talk about the design process for our API.

Implementing your Blog API Designs

Awosise Oluwaseun — Tue, 10 Jan 2023 20:00:00 +0000

After going through the API design and features we'd be implementing in our previous article, let us delve straight into the full implementation in codes

Setting up our Server

Before we set up our server, let's take a look at connecting to our database. In our config folder, let's create a database.js file and copy and paste the code below:

const mongoose = require("mongoose")
const dotenv = require("dotenv")
dotenv.config({ path: "./config/.env" })

const MONGODB_URI =
  process.env.NODE_ENV === "test"
    ? process.env.TEST_MONGODB_URI
    : process.env.MONGODB_URI

const connectDB = async () => {
  await mongoose
    .connect(MONGODB_URI, {
      useNewUrlParser: true,
      useUnifiedTopology: true
    })
    .then((conn) => {
      console.log(
        `Conected to Mongo! Database name: ${conn.connections[0].name}`
      )
    })
    .catch((err) => console.error("Error connecting to mongo", err))
}

module.exports = connectDB

The dotenv package allows us to read values from the .env file we created earlier using the process.env.VARIABLE_NAME command and mongoose is an ORM that helps relate with the MongoDB database. The MONGODB_URI is gotten from mongodb atlas. You can check out this video for refesher. The database is exported as a module to be used in another file.

Let's link this up with our server code shown below.

const app = require("./app")
const connectDB = require("./config/database")
const http = require("http")
const path = require("path")
const dotenv = require("dotenv")
dotenv.config({ path: path.join(__dirname, "./config/.env") })

const PORT = process.env.PORT || 4040
const server = http.createServer(app)

connectDB().then(() => {
  server.listen(PORT, async () => {
    console.log(`Server running on port ${PORT}`)
  })
})

The connectDB() function is called and on resolving, the server is spawned. This ensures that we're connected to the database first in order to interact with it

This is a server running on our local machine and on the value of PORT from our .env file or 4040. The app in our server is the heart of all of our logic and this is where all the several pieces that makes up our blog API come together.

Having talked about setting up our server and connecting to the database, we also want to know who our database looks like, how the information collected are stored in an organised Schema created with mongoose

Models

Two Schemas(blog.js and users.js) were created for this project in the models folder in our project folder structure. One for the Blog and one for Users.

The Blog Model:

const mongoose = require("mongoose")

const Schema = mongoose.Schema
const objectId = Schema.Types.ObjectId

const BlogSchema = new Schema(
  {
    title: {
      type: String,
      required: [true, "Please enter a title"],
      unique: true
    },
    description: {
      type: String,
      required: [true, "Please enter a description"]
    },
    owner: {
      type: String
    },
    author: {
      type: objectId,
      ref: "User"
    },
    state: {
      type: String,
      default: "draft",
      enum: ["draft", "published"]
    },
    readCount: {
      type: Number
    },
    readingTime: {
      type: Number
    },
    tags: [String],
    body: {
      type: String,
      required: [true, "Please provide the blog content"]
    }
  },
  { timestamps: true }
)

//! Add the blog reading time before saving
BlogSchema.pre("save", function (next) {
  let blog = this
  let titleLength = blog.title.length
  let descriptionLength = blog.description.length
  let bodyLength = blog.body.length
  let totalLength = titleLength + descriptionLength + bodyLength
  let totalTime = Math.round(totalLength / 200)

  blog.readCount = 0
  blog.readingTime = totalTime == 0 ? 1 : totalTime

  blog.tags = blog.tags.map((tag) => tag.toLowerCase())

  next()
})

//! Delete certain fields before returning result to client
BlogSchema.set("toJSON", {
  transform: (document, returnedObject) => {
    delete returnedObject.__v
  }
})

const Blog = mongoose.model("Blog", BlogSchema)

module.exports = Blog

The User Model:

const mongoose = require("mongoose")
const bcrypt = require("bcrypt")

const Schema = mongoose.Schema
const ObjectId = Schema.Types.ObjectId

const UserSchema = new Schema(
  {
    email: {
      type: String,
      required: true,
      unique: true
    },
    firstName: {
      type: String,
      required: true
    },
    lastName: {
      type: String,
      required: true
    },
    password: {
      type: String,
      required: true
    },
    repeatPassword: {
      type: String,
      required: true
    },
    userType: {
      type: String,
      default: "user",
      enum: ["admin", "user"]
    },
    blogs: [
      {
        type: ObjectId,
        ref: "Blog"
      }
    ]
  },
  { timestamps: true }
)

//! Encrypt password before saving it to database
UserSchema.pre("save", function (next) {
  let user = this

  if (!user.isModified("password")) return next()

  bcrypt.hash(user.password, 8, (err, hash) => {
    if (err) return next(err)
    user.password = hash
    user.repeatPassword = hash
    next()
  })
})

//! Compare inputted password with the password in the database
UserSchema.methods.comparePassword = function (pword) {
  const passwordHash = this.password
  return new Promise((resolve, reject) => {
    bcrypt.compare(pword, passwordHash, (err, same) => {
      if (err) {
        return reject(err)
      }
      resolve(same)
    })
  })
}

//! Delete certain fields before returning result to client
UserSchema.set("toJSON", {
  transform: (document, returnedObject) => {
    returnedObject.id = returnedObject._id.toString()
    delete returnedObject._id
    delete returnedObject.__v
    delete returnedObject.password
    delete returnedObject.repeatPassword
  }
})

const User = mongoose.model("user", UserSchema)
module.exports = User

Both models represent the model of the information we want to recieve from our users and store in our database. The User model is used for authentication. They both also have pre hooks that run just before the data is saved to the database. The user password is hashed or encrypted before it is saved to the database using the bcrypt package. The user model has a method that is also used to compare user's inputted password with that in the database when trying to login.

The next thing we want to talk about are the endpoints but before then, let's look at some of the middleware codes we implemented so we can get the flow of request objects through the middlewares and the eventual request handler or controller.

Let's create a middleware folder as in our file structure and put inside the folder we create the auth.js, filter.js and paginate.js. This leads us to explaining the implementation of the middleware features discussed in the previous article.

Authentication

This middleware code is contained in the auth.js file we just created. It contains the following codes:

const passport = require("passport")
const localStrategy = require("passport-local").Strategy
const UserModel = require("../models/users")

const JWTstrategy = require("passport-jwt").Strategy
const ExtractJWT = require("passport-jwt").ExtractJwt

const path = require("path")
const dotenv = require("dotenv")
dotenv.config({ path: path.join(__dirname, "../config/.env") })

const { BadRequestError, UnauthenticatedError } = require("../errors")

const opts = {}
opts.secretOrKey = process.env.JWT_SECRET
opts.jwtFromRequest = ExtractJWT.fromAuthHeaderAsBearerToken()

module.exports = (passport) => {
  passport.use(
    new JWTstrategy(opts, async (token, done) => {
      try {
        const user = token.user
        return done(null, user)
      } catch (err) {
        done(err)
      }
    })
  )

  passport.use(
    "signup",
    new localStrategy(
      {
        usernameField: "email",
        passwordField: "password",
        passReqToCallback: true
      },
      async (req, email, password, done) => {
        try {
          const { firstName, lastName, repeatPassword } = req.body
          const userObject = {
            firstName,
            lastName,
            email,
            password,
            repeatPassword
          }

          const user = new UserModel(userObject)
          const savedUser = await user.save()

          return done(null, savedUser)
        } catch (err) {
          done(err)
        }
      }
    )
  )

  passport.use(
    "login",
    new localStrategy(
      {
        usernameField: "email",
        passwordField: "password"
      },
      async (email, password, done) => {
        try {
          if (!email || !password) {
            throw new BadRequestError("Please provide email or password")
          }
          const user = await UserModel.findOne({ email })

          if (!user) {
            throw new UnauthenticatedError("Please provide valid credentials")
          }

          const validate = await user.comparePassword(password)

          if (!validate) {
            throw new UnauthenticatedError("Please provide valid credentials")
          }

          return done(null, user, { message: "Logged in Successfully" })
        } catch (err) {
          done(err)
        }
      }
    )
  )
}

Passport handles our authentication using the JWT strategy (a token based authentication). It also allows us to use the local Strategy to run bothe the sign up and login feature of the application. The JWT strategy allows for a token to be generated after it has been verified to release a payload containing information that can be used to access the next request handler.

Filter

This middleware code is contained in the filter.js file we just created. It contains the following codes:

const filterByPublished = (req, res, next) => {
  req.filterObject.state = "published"
  return next()
}

const filterAndSort = (req, res, next) => {
  const { tag, author, title, state, sort } = req.query
  req.filterObject = {}
  req.sortObject = {}


  try {
    if (tag) {
      const searchTag = Array.isArray(tag)
        ? tag.toLowerCase()
        : tag.split(", " || " " || ",").map((t) => t.toLowerCase())
      req.filterObject.tags = { $in: searchTag }
    }
    if (author) {
      req.filterObject.owner = { $regex: author, $options: "i" }
    }
    if (title) {
      req.filterObject.title = { $regex: title, $options: "i" }
    }

    if (sort) {
      const sortList = sort.split(",").join(" ")
      req.sortObject.sort = sortList
    } else {
      req.sortObject.sort = "createdAt"
    }

    if (state) {
      req.filterObject.state = state
    } else {
      req.filterObject.state = { $in: ["draft", "published"] }
    }

    return next()
  } catch (err) {
    next(err)
  }
}

module.exports = {
  filterByPublished,
  filterAndSort
}

When we try to search for published blogs based on certain criteria such as authors, tags, titles, we have to add query to our request. These queries are collected in this middleware and passed on to the next request handler in order to be used.

Paginate

This middleware code is contained in the paginate.js file we just created. It contains the following codes:

const paginate = (req, res, next) => {
  req.paginate = {}
  const blogsPerPage = 20
  try {
    if (req.query.p) {
      const page = req.query.p
      const numOfBlogsToSkip = (page - 1) * blogsPerPage

      req.paginate.blogsPerPage = blogsPerPage
      req.paginate.numOfBlogsToSkip = numOfBlogsToSkip
      return next()
    }
    const page = 1
    let numOfBlogsToSkip = (page - 1) * blogsPerPage

    req.paginate.blogsPerPage = blogsPerPage
    req.paginate.numOfBlogsToSkip = numOfBlogsToSkip
    return next()
  } catch (err) {
    next(err)
  }
}

module.exports = paginate

The paginate middleware sends the number of blogs per page and number of blogs to skip per page to the next request handler.

Now that we are done with the middlewares that preceed our controllers, let us talk about our Routes and Controllers.

Routes

Let's not forget the summary of the endpoints in our previous article in this series. We are going to be linking the routes in our endpoints to a particular controller and/or middleware
Let's take a look

The Blog Routes

const express = require("express")
const passport = require("passport")
const {
  createBlog,
  getAllBlogs,
  updateBlog,
  updateBlogState,
  getAllUsersBlogs,
  getBlogById,
  deleteBlog,
  getBlogByIdAuth
} = require("../controllers/blog")
const { filterAndSort, filterByPublished } = require("../middlewares/filter")
const paginate = require("../middlewares/paginate")
const validateBlog = require("../validators/blog.validator")

const blogRouter = express.Router()

blogRouter
  .route("/home/blog")
  .get(filterAndSort, filterByPublished, paginate, getAllBlogs)
blogRouter
  .route("/home/blog/:id")
  .get(filterAndSort, filterByPublished, getBlogById)

blogRouter.use("/blog", passport.authenticate("jwt", { session: false }))

blogRouter
  .route("/blog")
  .get(filterAndSort, paginate, getAllUsersBlogs)
  .post(validateBlog, createBlog)

blogRouter
  .route("/blog/:id")
  .get(filterAndSort, getBlogByIdAuth)
  .put(validateBlog, updateBlog)
  .patch(updateBlogState)
  .delete(deleteBlog)

module.exports = blogRouter

The blog route makes use of the the express Router() method and we can see how it links our various routes to middlewares and controllers. It is important to note that the order of the middlewares are important and should be followed strictly. The user route follows the same pattern.

The input validator used in thr validateBlog function is a middleware that helps validate the user input withe the help of a package known as joi. You can check the validator folder in the source code link belw for it implementation.

The Controllers

They handle all of our CRUD operations. There are quite a number of controllers in this project and I would be picking three of them to talk about. One that requires authentication, one that does not then one that requires parameter.

The POST or create a new blog controller

const createBlog = async (req, res, next) => {
  const { id } = req.user
  const { title, description, body, tags } = req.body

  const user = await UserModel.findById({ _id: id })

  try {
    const blog = new BlogModel({
      title,
      description,
      owner: `${user.firstName} ${user.lastName}`,
      body,
      tags,
      author: id
    })

    const savedBlog = await blog.save()

    user.blogs = user.blogs.concat(savedBlog.id)
    await user.save()

    return res.status(201).json(savedBlog)
  } catch (err) {
    next(err)
  }
}

The req.user is the payload handed down from the authentication middleware and we can see how we used our BlogModel to create a new blog and at the same time save the reference ID of the blog to the list of blogs created by that user.

The GET all published blog controller

const getAllBlogs = async (req, res, next) => {
  const filters = req.filterObject
  const { sort } = req.sortObject
  const { blogsPerPage, numOfBlogsToSkip } = req.paginate

  try {
    const blog = await BlogModel.find(filters, { title: 1, description: 1 })
      .sort(sort)
      .skip(numOfBlogsToSkip)
      .limit(blogsPerPage)

    return res.status(200).json({ count: blog.length, blogs: blog })
  } catch (err) {
    next(err)
  }
}

This controller does not require authentication and we have the filterObject, sortObject and paginate attached to requests coming their respective middlewares. It returns just the title and description of all the blogs.

The GET blog published blog by ID controller

const getBlogById = async (req, res, next) => {
  try {
    const { state } = req.filterObject
    const { id } = req.params
    const blog = await BlogModel.findOneAndUpdate(
      { _id: id, state: state },
      { $inc: { readCount: 1 } },
      { new: true }
    )

    if (!blog) {
      throw new NotFoundError(`No blog with id ${id} to update`)
    }

    return res.status(200).json(blog)
  } catch (err) {
    next(err)
  }
}

Getting blog by id requires that a parameter is added to the route in order to open that specific blog.

Everything we have discussed individually are brought together in the app.js file which can be checked in the source code provided below.

Wrapping Up

All of the codes above were abstracted from the main source code and should not be run separately on their own. The source code should be checked to further understanding and also to be able to link how all the modules work hand in hand.

As time goes on, specific parts of the code will be spoken upon with reference to this.

Let's talk about testing and deployment in our next article.

Testing and Deploying your Blog API

Awosise Oluwaseun — Tue, 10 Jan 2023 20:00:00 +0000

Welcome to the last part of the series where I'd be talking about testing the routes and deploying.

Testing

In order to test this api, I created an api documentation with swaggerUI. It allows you to supply values based on the schema defined and also returns the appropriate response, error messages and status codes.

Deployment

The deployment was done on cyclic.sh. It is as simple as signing up with your github account then linking the repository containing your project then you finally set your environmental variables.
Voila! Your project is already deployed and ready for use.

Conclusion

Kudos to you for reaching the last article of the series. I believe you have been able to pick one or two concepts to apply to your next project. Kindly do well to reach me on twitter if there's any part you would like me to write extensively on.
I also owe all of this to leading tech institution in Africa, AltSchool Africa.

Designing your Blog API

Awosise Oluwaseun — Tue, 10 Jan 2023 20:00:00 +0000

Having gone through some of the overviews of the project in our previous article such as setting up our node enviroment, .env, .gitignore and installing all required packages for the project. Let's talk about the Blog API!

The Blog API allows you to perform CRUD (Create, Read, Update and Delete) operations. For the Create, Update and Delete operations, you as an authenticated user can create your own blog and have it as either draft (which is the default) or published, update and delete only your own blog. The owner of the blog is also allowed to update the state of the blog from draft to published

For the Read operation, both authenticated and unauthenticated users can read published blogs. Only authenticated users can view their own drafts.

A summary of endpoints based on the decription given above

HTTP Method	Routes	Authenticated?	Description
GET	/api/v1/home/blog	No	Get all published blogs
GET	/api/v1/home/blog/:id	No	Get all published blogs by ID
GET	/api/v1/blog	Yes	Get all published and drafted blogs created by the user
GET	/api/v1/blog/:id	Yes	Get all published and drafted blogs by ID created by the user
POST	/api/v1/blog	Yes	Post blog as an authticated user
PUT	/api/v1/blog/:id	Yes	Update blog as an authenticated user
PATCH	/api/v1/blog/:id	Yes	Update blog state as an authenticated user
DELETE	/api/v1/blog/:id	Yes	Delete blog as an authticated user

Having drafted out the endpoints, let us look at other features that we will be implementing and the packages that will help us achieve them.

Setting up our Server

The server is the link between our client and the database. It allows us to run logic that helps us in interacting with the database. All of our CRUD operations run on the server.

Authentication

The type of authentication used in this API is the Token Based Authentication. The user creates an account by signing up then uses the credentials to sign in. This then generates a token that the user will use to access protected Routes.
The packages used:

Summary of endpoints required for authentication

HTTP Method	Routes	Description
POST	/api/v1/signup	signup user
POST	/api/v1/login	login user

Input Validation

Even with a proper API documentation, humans are bound to make mistakes while filling in or supplying details to be sent to the database. So, in order to curb this error, we have packages joi and mongoose that help us do this. They allow us to run some validator checks on inputted data to ensure uniformity in the data collected in the database.

Error Handling

There are several errors generated from the API and they were properly handled using the error handler middleware and the express-async-errors package. Some of the errors that were includes:

Error Type	Error Status Code
Bad Request Error	400
Page not Found Error	404
Unauthenticated Error	401
Unauthorised Error	403

You can read more on HTTP status codes.

All of these error modules are found in the error folder shown in the project folder structure.

Middlewares

Middleware functions are functions that have access to the request object (req), the response object (res), and the next middleware function in the application’s request-response cycle. These functions are used to modify req and res objects for tasks like parsing request bodies, adding response headers.
In this project we used middleware functions to handle certain tasks like:

Input validation
Authentication
Filtering blog by query parameters
Pagination
Error handling

Rate Limiting

The rate limiting middleware limits the number of requests that come from a specific user, IP address or location within a given window (period of time). This can be achieved using a package called express-rate-limit as middleware

Cross Origin Resource Sharing (CORS)

This allows us to be able to set sites that can have access to making direct request to our API. These sites that are allowed are whitelisted.This can be achieved using a package called cors as middleware. You can read more on it

We have learnt the major features we'd be implementing in this project. Let's get into the full implementation of some of these features in our API with codes. BRACE UP! for you're about to enter the world of codes!

The JS Promise Errand

Awosise Oluwaseun — Fri, 19 Aug 2022 22:11:00 +0000

I promised to continue contributing to open source in my previous post, and I've decided to keep that Promise by writing about a recently learned concept called javascript PROMISE!

So it was a long and stressful day for me, and it was also quite hot outside. I needed a cold drink but couldn't leave what I was doing, so I asked my brother to run the errand of getting a cold drink for me while I continued working. I realized at the end of the day that I had previously practiced the Promise concepts.

The Breakdown of the Errand

This is a breakdown of everything that happened during the errand;

I called my brother and asked him to assist me in getting the drink.
I continued working while he ran the errand.
I was expecting him to either return with a cold drink or not.
What my reaction is to whatever point 3 above is.

The Errand and JS Promise

The errand is related to how javascript Promise works:

The Creation of the Promise

The first 3 points above will be linked to how Promise works using the block of code below:

const coldDrink = true;

const brother = new Promise((resolve, reject) => {
  if (coldDrink) {
    const coldDrinkBought = 'I was able to get the cold drink';
    resolve(coldDrinkBought);
  } else {
    const coldDrinkNotBought = 'I was unable to get the cold drink';
    reject(coldDrinkNotBought);
  }
});

I mentioned that I had to call my brother, which is analogous to creating Promise or promisifying my brother. I created a promise by calling my brother to help me get a cold drink which is shown in this line of code const brother = new Promise((resolve, reject) => {...};.
I also didn't stop working while my brother ran the errand, explaining Promise's asynchronous nature.
While I was still working, I was anticipating two outcomes from my brother, whether he was able to obtain the drink or not. These two possibilities are handled by the parameters of the function within the Promise, which are resolve (which handles getting the cold drink) and reject (which handles not getting the cold drink).

The consumption of the Promise

This part explains my response to both possibilities of getting the cold drink from my brother.

 brother()
    .then(iGotTheColdDrink => {
      console.log(iGotTheColdDrink);
    })
    .catch(iDidNotGetTheColdDrink => {
      console.log(iDidNotGetTheColdDrink);
    });

If my brother had gotten the cold drink, it is resolved from his end, and I can then receive the drink with .then method. If he was unable to obtain the drink, indicating it was rejected, I can obtain my money back from him using the .catch method. At this point, I think I'll have a drink.

Summary

The .then method handles the result of the Promise's resolve response.
The .catch method handles the result of the Promise's reject response.
This is an illustration to explain few of the concepts, to know more, check out node.js official site.

With my illustration, I hope I've fulfilled my Promise of explaining the fundamentals of Promise. Catch you next time (pun intended).

I got my first Pull Request merged!

Awosise Oluwaseun — Fri, 12 Aug 2022 22:04:19 +0000

Words cannot express my joy when I got the notification about the merge above, and I owe it ample time to my current school of Engineering, AltSchool Africa.
Before now, I have been introduced to open source many times, told about the importance in the tech space, and even attended open source conferences (e.g., OSCAFest). I have all the instant excitement to want to start, but imposter syndrome sets in on opening GitHub to create something.
Fast forward to Monday, the 8th of August, 2022, when I watched Bolaji's video on contributing to open source. I felt pumped again, but I wanted to apply what I learned, so I noted some steps.

The Steps

I made up my mind I was going to contribute to a project.
was focused on a site (good first issue) to pick my first project from, which I filtered to suit my skill level. I kept opening the next page till I found one.
I made sure I was equipped with the required git and github knowledge to get the project done.

The Project

After long hours searching for projects, I finally found one titled; Ensure no missing alt attributes. I was to give descriptive alt values to images from the site. Alt values in images help to improve the accessibility of the site such that screen readers can provide a detailed description of the image to, say, a visually impaired person. Easy right? Yes, but if I didn't make up my mind to get the first contribution, I wouldn't find it, and Open source will continue to be a myth to me.
I was still pumped until I discovered it was from MDN; wait, same MDN? As in Mozilla developer? Will they merge my contribution even with how seemingly easy it looks? Imposter syndrome sets in.

Upon checking the issue, I saw that people were already contributing; I summoned courage and started reading about it. Taking my time to read and understand the project and the manner of approaching the issue was another challenge I had to overcome.

The project is as easy as you try to understand it.

So, I started, and I picked two images, to begin with. I gave alt values to them, committed my changes then made a Pull Request. The time between when I made the pull request and when I got the approval mail was full of self-doubts. Should I close the pull request? This is MDN ooo... Well, it's not coding... What if I don't get merged? I might never contribute again. All it took to clear all of the doubts were the emails I got from my reviewer below;

I was indeed delighted, and this spawned me to check for more. It gave me the courage I needed to request to solve issues.

Summary

A few lines I'd love you to take home from this article are;

Open source is for all. Do you see that typo on that site you just visited? You helping to correct it is a way of contributing.
No skillset is too small. A basic understanding of HTML was what I needed to contribute.
Only you can stop yourself from contributing.
The first contribution is all you need to set the ball rolling.

I hope you have been able to pick something from my story and apply it today. This is another space I'd like to keep contributing to, so see you in my next post, and happy open sourcing!!