DEV Community: James Ononiwu

JavaScript Array Methods: Sort, Map, Filter, and Reduce

James Ononiwu — Tue, 28 Jan 2025 18:12:41 +0000

JavaScript arrays are a cornerstone of modern web development, providing robust tools to manage and manipulate data. Among these tools, the methods sort, map, filter, and reduce stand out for their versatility and power. In this article, we’ll explore these methods in detail, offering explanations and practical examples to deepen your understanding.

1. Array.prototype.sort

The sort method organises the elements of an array based on a specified order. By default, it sorts elements as strings in lexicographical order. However, with a custom comparison function, you can implement advanced sorting logic.

Syntax

array.sort([compareFunction])

compareFunction: A function that defines the sort order. It takes two arguments (a, b) and returns:
- A negative value if a should come before b.
- Zero if a and b are equivalent.
- A positive value if a should come after b.

Example: Sorting Numbers in Ascending Order

const numbers = [42, 7, 19, 73, 23];

numbers.sort((a, b) => a - b);

console.log(numbers);
// Output: [7, 19, 23, 42, 73]

Example: Sorting Objects by Property

const books = [
  { title: 'Book A', year: 2001 },
  { title: 'Book B', year: 1998 },
  { title: 'Book C', year: 2015 }
];

books.sort((a, b) => a.year - b.year);

console.log(books);
// Output: [
//   { title: 'Book B', year: 1998 },
//   { title: 'Book A', year: 2001 },
//   { title: 'Book C', year: 2015 }
// ]

The sort method modifies the original array.
Always provide a compare function for numeric or complex sorting to avoid unexpected results.

2. Array.prototype.map

The map method creates a new array by applying a provided function to each element of the original array. It’s commonly used for transforming data.

Syntax

array.map(callback(element, index, array))

callback: A function executed on each element, receiving:
- The current element.
- Its index.
- The original array.

Example: Transforming an Array of Numbers

const numbers = [1, 2, 3, 4, 5];

const squares = numbers.map(num => num ** 2);

console.log(squares);
// Output: [1, 4, 9, 16, 25]

Example: Formatting Data

const users = [
  { firstName: 'John', lastName: 'Doe' },
  { firstName: 'Jane', lastName: 'Smith' }
];

const fullNames = users.map(user => `${user.firstName} ${user.lastName}`);

console.log(fullNames);
// Output: ['John Doe', 'Jane Smith']

Working with Sparse Arrays: The map method will execute the callback for every index, including those that are undefined in sparse arrays. For example:

  const sparseArray = [1, , 3]; // Sparse array with a missing element
  const result = sparseArray.map(x => x || 0);
  console.log(result);
  // Output: [1, 0, 3]

Preserves Array Structure: Even if some elements are undefined or removed, map preserves the original array structure:

  const array = [10, undefined, 30];
  const doubled = array.map(x => (x !== undefined ? x * 2 : x));
  console.log(doubled);
  // Output: [20, undefined, 60]

map is non-destructive; it returns a new array without modifying the original.
It’s ideal for transforming data structures. The map method creates a new array by applying a provided function to each element of the original array. It’s commonly used for transforming data.

3. Array.prototype.filter

The filter method creates a new array containing only the elements that satisfy a given condition. It’s often used to extract subsets of data.

Syntax

array.filter(callback(element, index, array))

callback: A function that determines whether an element should be included (returns true) or excluded (returns false).

Example: Filtering Even Numbers

const numbers = [1, 2, 3, 4, 5, 6];

const evens = numbers.filter(num => num % 2 === 0);

console.log(evens);
// Output: [2, 4, 6]

Example: Filtering Objects by Condition

const products = [
  { name: 'Laptop', price: 1200 },
  { name: 'Phone', price: 800 },
  { name: 'Tablet', price: 500 }
];

const expensiveProducts = products.filter(product => product.price > 700);

console.log(expensiveProducts);
// Output: [
//   { name: 'Laptop', price: 1200 },
//   { name: 'Phone', price: 800 }
// ]

Sparse Arrays: Unlike map, filter skips over missing or undefined elements in sparse arrays entirely. For example:

  const sparseArray = [1, , 3];
  const filtered = sparseArray.filter(x => x !== undefined);
  console.log(filtered);
  // Output: [1, 3]

Removing Falsy Values: You can easily remove all falsy values (e.g., false, 0, null, undefined, NaN, "") by using the Boolean constructor as the callback:

  const mixedArray = [0, "hello", false, 42, undefined, "world", null];
  const truthyValues = mixedArray.filter(Boolean);
  console.log(truthyValues);
  // Output: ["hello", 42, "world"]

filter does not modify the original array.
It’s a powerful tool for extracting relevant data or cleaning up arrays.

4. Array.prototype.reduce

The reduce method applies a function against an accumulator and each element of the array (from left to right) to reduce it to a single value.

Syntax

array.reduce(callback(accumulator, currentValue, index, array), initialValue)

callback: A function executed on each element, with parameters:
- The accumulator.
- The current element.
- The index of the current element.
- The original array.
initialValue: The initial value of the accumulator (optional but recommended).

Example: Summing Numbers

const numbers = [10, 20, 30, 40];

const total = numbers.reduce((sum, num) => sum + num, 0);

console.log(total);
// Output: 100

Example: Grouping Data by Property

const people = [
  { name: 'Mike', age: 25 },
  { name: 'Larry', age: 30 },
  { name: 'Charlie', age: 25 }
];

const groupedByAge = people.reduce((acc, person) => {
  if (!acc[person.age]) {
    acc[person.age] = [];
  }
  acc[person.age].push(person.name);
  return acc;
}, {});

console.log(groupedByAge);
// Output: {
//   25: ['Mike', 'Charlie'],
//   30: ['Larry']
// }

Flattening Multidimensional Arrays: You can use reduce to flatten nested arrays into a single array:

  const nestedArray = [[1, 2], [3, 4], [5, 6]];
  const flattened = nestedArray.reduce((acc, curr) => acc.concat(curr), []);
  console.log(flattened);
  // Output: [1, 2, 3, 4, 5, 6]

Implementing Map/Filter-like Transformations: In some cases, reduce can mimic the behaviour of map or filter while combining multiple transformations in a single pass:

  const numbers = [1, 2, 3, 4, 5];
  const transformed = numbers.reduce((acc, num) => {
    if (num % 2 === 0) {
      acc.push(num * 2); // Double even numbers
    }
    return acc;
  }, []);
  console.log(transformed);
  // Output: [4, 8]

reduce is highly flexible, capable of producing arrays, objects, or single values.
It’s a powerful tool for custom transformations, data aggregation, and more.
Always provide an initial value for better predictability.

Combining Methods

Let’s analyse sales data and determine the total revenue from high-value transactions, considering how combining methods like map, filter, and reduce can be optimised for performance. For example, when working with large datasets, one can avoid multiple iterations by chaining the logic within a single reduce operation. This reduces overhead and can be particularly beneficial when dealing with real-time or resource-constrained scenarios.

const sales = [
  { product: 'Laptop', price: 1500, quantity: 2 },
  { product: 'Phone', price: 800, quantity: 5 },
  { product: 'Tablet', price: 500, quantity: 10 }
];

// Step 1: Calculate revenue for each product
const revenues = sales.map(sale => ({
  product: sale.product,
  revenue: sale.price * sale.quantity
}));

// Step 2: Filter high-revenue products
const highRevenue = revenues.filter(item => item.revenue > 3000);

// Step 3: Calculate total revenue
const totalHighRevenue = highRevenue.reduce((sum, item) => sum + item.revenue, 0);

console.log(revenues);
// Output: [
//   { product: 'Laptop', revenue: 3000 },
//   { product: 'Phone', revenue: 4000 },
//   { product: 'Tablet', revenue: 5000 }
// ]

console.log(highRevenue);
// Output: [
//   { product: 'Phone', revenue: 4000 },
//   { product: 'Tablet', revenue: 5000 }
// ]

console.log(totalHighRevenue);
// Output: 9000

Conclusion

JavaScript’s sort, map, filter, and reduce methods are indispensable for efficient and elegant data manipulation. Understanding their mechanics and applications empowers developers to write cleaner, more maintainable code. Experimenting with these methods will enhance your programming skills!

Decentralisation: Does It Matter in the Modern World?

James Ononiwu — Fri, 17 Jan 2025 20:32:26 +0000

When people talk about blockchain and Web3, one word tends to pop up over and over: decentralisation. It’s a cornerstone of the movement, but what does it really mean? And why is everyone so excited about it?

Let’s cut through the jargon and uncover why decentralisation isn’t just a buzzword, it’s a revolutionary idea with real-world impact.

What is Decentralisation?

At its simplest, decentralisation means there’s no single point of control. Instead of one authority or entity running the show, power is distributed among multiple participants.

Think about the internet. Many of the platforms we use daily, social media, cloud services, online marketplaces are controlled by centralised entities. These companies hold our data, control the rules, and can make decisions that affect millions with the stroke of a keyboard. Decentralisation flips that script.

In a decentralised system, control is shared. Participants work together to verify and validate actions (like transactions or data changes), ensuring transparency and reducing reliance on a single authority.

How Decentralisation Works in Blockchain

In a centralised financial system, your bank records all your transactions in its ledger. The bank is the sole authority. But what happens if that ledger is hacked or corrupted? Worse yet, what if the bank decides to freeze your account?

In a decentralised blockchain, every participant (or node) has a copy of the ledger. Transactions are validated through consensus mechanisms, everyone agrees that an action is legitimate before it’s added to the record. There’s no single point of failure, and control is distributed across the network.

Why Does Decentralisation Matter?

1. Resilience and Security

In a centralised system, a single point of failure can bring everything crashing down. Imagine a cyberattack on a centralised server hosting sensitive data. With decentralisation, multiple copies of the data exist across the network. Even if some nodes go down, the system remains functional.

Decentralised finance (DeFi) platforms operate without traditional banks. Users maintain control of their funds, and even if one platform is compromised, the wider ecosystem remains unaffected.

2. Transparency and Trust

When a single entity controls a system, you have to trust that they’re acting in your best interest. Decentralisation removes the need for blind trust. Instead, actions and records are visible to everyone in the network.

As an instance, public blockchains like Ethereum allow anyone to view transactions, ensuring accountability.

3. Empowerment and Accessibility

Decentralisation puts power back in the hands of individuals. It removes gatekeepers, making systems more inclusive and accessible.

Using the financial systems as example, traditional remittance services charge high fees and take days to process transactions. Decentralised cryptocurrencies enable people to send money instantly across borders at a fraction of the cost.

Existing Applications of Decentralisation

Decentralised Social Media

Platforms like Mastodon and Lens Protocol are challenging traditional social networks. Instead of a single company owning your posts and data, decentralised platforms let users control their content and connections.

Decentralised Cloud Storage

Services like Filecoin and Storj distribute data storage across a global network of computers. Users rent out unused storage space, creating a secure and cost-effective alternative to centralised providers like Google Drive.

Decentralised Autonomous Organisations (DAOs)

DAOs are groups governed by smart contracts rather than hierarchical structures. Members vote on decisions, ensuring collective ownership and participation.

Challenges of Decentralisation

Of course, decentralisation isn’t perfect. Here are some of the growing pains:

- Scalability: Decentralised systems can be slower than centralised ones because consensus mechanisms take time.

- Complexity: Participating in decentralised systems often requires technical know-how.

- Governance Issues: Distributing power is great, but reaching consensus among a large group can be tricky.

Why the Hype Around Decentralisation?

At its heart, decentralisation is about empowerment. It’s about creating systems that are fairer, more transparent, and less prone to abuse. While the journey toward a fully decentralised world isn’t without obstacles, the potential benefits make it worth the effort.

So the next time you hear someone drop the word "decentralisation", you’ll know it’s not just a tech jargon, it’s a vision for a better, more equitable future.

What is Blockchain?

James Ononiwu — Mon, 13 Jan 2025 00:21:18 +0000

Blockchain. It’s a buzzword that gets thrown around in conversations about cryptocurrency, Web3, and digital innovation. But what is blockchain, really? If you’re new to the term, don’t worry—we’ll break it down into simple, digestible pieces. By the end of this article, you’ll not only understand blockchain but also why it’s considered one of the most revolutionary technologies of our time.

The Simple Definition

At its core, a blockchain is a digital ledger. Imagine a traditional ledger—a book where transactions are recorded. Now, make that ledger digital, distributed across multiple computers (called nodes), and nearly impossible to alter. That’s blockchain.

To break it down:

Block: A collection of data, such as transaction details.

Chain: A series of blocks linked together in a specific order, creating a continuous record.

How Does Blockchain Work?

Think of blockchain as a train. Each car on the train represents a block of data, and the coupling that connects the cars is the chain. Once a block (or train car) is added, it’s there for good, and new blocks are attached sequentially.

Key Features:

Decentralisation: Unlike traditional systems where one central authority holds all the records, a blockchain is maintained by a network of computers. This makes it more resilient and transparent.

Immutability: Once data is recorded in a block, it’s nearly impossible to change without altering every subsequent block—an infeasible task.

Transparency: All participants in the network can view the entire blockchain, ensuring accountability.

Why Does Blockchain Matter?

Blockchain’s unique characteristics make it a powerful tool for solving problems of trust, transparency, and security.

Real-World Example: Bitcoin

Bitcoin, the first cryptocurrency, runs on a blockchain. When Jane sends 1 Bitcoin to John, that transaction is added to a block, verified by miners (think of them as digital accountants), and added to the blockchain. The result? A permanent, transparent record of Jane’s transaction with John.

But blockchain’s potential goes far beyond cryptocurrency.

Real-World Use Cases

Supply Chain Transparency: Imagine you’re buying coffee labeled as fair trade. With blockchain, you can trace the journey of your coffee beans from the farm to your cup, ensuring ethical sourcing.

Healthcare: Patient records can be securely stored on a blockchain, accessible only to authorised individuals, and immune to tampering.

Voting: Blockchain can ensure fair and transparent elections by creating an immutable record of votes.

Addressing Common Misconceptions:

Is Blockchain the Same as Bitcoin?

No. Blockchain is the underlying technology. Bitcoin is just one application of it.

Can Blockchain Be Hacked?
Hacking a blockchain is theoretically possible but practically infeasible due to its decentralised and cryptographic nature.

Why Should You Care About Blockchain?

Blockchain is not just a passing trend. Its applications span industries and have the potential to reshape how we think about trust, ownership, and value in the digital world.

Whether you’re a curious observer, a tech enthusiast, or someone working in industries like finance, logistics, or healthcare, understanding blockchain gives you a front-row seat to the future.

Closing Thought

Blockchain is often called "the internet of value" because it enables us to exchange assets as easily as we exchange information online. As the technology evolves, it’s poised to disrupt industries and open up new possibilities for innovation.

Prompt Engineering (For Lazy Programmers): Getting Exactly the Code You Want (and Even More, Out of ChatGPT)

James Ononiwu — Thu, 24 Oct 2024 23:00:37 +0000

Bill Gates has said it all... be a lazy programmer!.

As a programmer, there’s nothing better than code that works right off the bat—no bugs, no endless debugging. by following certain prompt techniques, you can get ChatGPT to write not just code, but optimised, fully functional, and documented code, complete with edge cases, tests, and even performance optimisations.

But first...

What is Prompt Engineering?

Artificial Intelligence, particularly ChatGPT, has become a powerful tool for content creation, coding assistance, and answering complex questions. Yet, many people don’t tap into its full potential. The magic lies in how you phrase your requests—what we call prompt engineering. In this article, we'll explore techniques that can make your ChatGPT experience more fruitful, highlighting some tricks that can take your prompts from average to brilliant, as a software programmer.

Why Does Prompt Engineering Matter?

ChatGPT is trained on vast amounts of data, but its responses are shaped by the quality of the prompt you give it. Poorly worded or vague prompts can lead to irrelevant or generic answers. Conversely, well-structured prompts can generate thoughtful, accurate, and creative responses.

This guide will help lazy programmers (like myself 🤓) get high-quality, optimised, and even more functional code from ChatGPT using prompt engineering. These will exclusively make ChatGPT do the heavy lifting—automating code generation, improving it, and even suggesting optimisations and tests. Below, I’ve also included examples that compare the normal prompts versus optimised prompts and the results they generate.

So let's get started!.

Triggering Auto-Refinements for Better Code

Ask ChatGPT to automatically refine its responses by embedding triggers to spot inefficiencies, suggest optimisations, or add features.

Basic prompt: "Write a Python function to reverse a string."

def reverse_string(s):
    return s[::-1]

# Example usage:
print(reverse_string("hello"))  # Output: "olleh"

Optimised prompt: "Write a Python function to reverse a string. If the input is too large or inefficient, suggest a better way to handle large strings efficiently."

def reverse_string(s):
    return s[::-1]

# Trigger: Refining for large strings
def reverse_large_string(s):
    # Handle large strings in chunks for efficiency
    chunk_size = 100000  # Process in 100k chunks
    return ''.join(s[i:i+chunk_size][::-1] for i in range(0, len(s), chunk_size))

# Example usage:
large_string = "a" * 1000000  # 1 million characters
print(reverse_large_string(large_string))

The basic function is simple and works for small inputs.
The optimised, introduces an efficient chunk-based solution for reversing large strings without consuming too much memory.

Contextual Memory for Multi-Step Projects

When you need code for multi-step projects, ChatGPT can remember earlier steps. This is ideal for more complex workflows where you build one layer on top of another.

Basic prompt: "Write a Python function to create a user profile."

def create_user_profile(name, age, email):
    return {"name": name, "age": age, "email": email}

# Example usage:
user_profile = create_user_profile("Alice", 30, "alice@example.com")
print(user_profile)

Optimised prompt: "Write a Python function to create a user profile. Now, extend this to also store user preferences and connect it with a database to save the profiles."

def create_user_profile(name, age, email, preferences):
    return {"name": name, "age": age, "email": email, "preferences": preferences}

# Extend with a database connection
import sqlite3

def save_user_profile(profile):
    conn = sqlite3.connect('users.db')
    cursor = conn.cursor()
    cursor.execute('''CREATE TABLE IF NOT EXISTS users
                      (name TEXT, age INTEGER, email TEXT, preferences TEXT)''')
    cursor.execute('''INSERT INTO users (name, age, email, preferences)
                      VALUES (?, ?, ?, ?)''', 
                   (profile['name'], profile['age'], profile['email'], str(profile['preferences'])))
    conn.commit()
    conn.close()

# Example usage:
user_profile = create_user_profile("Alice", 30, "alice@example.com", {"theme": "dark"})
save_user_profile(user_profile)

The first function from the basic prompt creates a simple user profile.
The second prompt creates a database-connected user profile manager with extended functionality for saving data.

Request Debug Mode for Error-Free Code

You can ask ChatGPT to act as if it’s running in debug mode, where it checks and corrects potential issues in the code before delivering the final output.

Example simple prompt: "Write a Python function to calculate the square root of a number."

import math

def square_root(n):
    return math.sqrt(n)

# Example usage:
print(square_root(16))  # Output: 4.0

The Trick: "Write a Python function to calculate the square root of a number. Activate debug mode to check for errors like negative inputs, and rewrite the function if necessary."

import math

def square_root(n):
    if n < 0:
        return "Error: Cannot calculate square root of a negative number"
    return math.sqrt(n)

# Debugged version handles errors properly.
# Example usage:
print(square_root(16))   # Output: 4.0
print(square_root(-16))  # Output: "Error: Cannot calculate square root of a negative number"

The basic function works fine for valid inputs but crashes with negative numbers.
The optimised prompt ensures error handling for negative inputs, returning a custom message instead of breaking the code.

Role-Playing: Act as a Code Reviewer

You can ask ChatGPT to role-play as a senior code reviewer to provide feedback and suggest best practices for your code.

Example prompt: "Act as a senior developer and review my Python function that checks if a number is prime. Suggest improvements for performance and readability."

def is_prime(n):
    if n <= 1:
        return False
    if n == 2:
        return True
    if n % 2 == 0:
        return False
    # Only check odd numbers up to the square root of n for efficiency
    for i in range(3, int(n**0.5) + 1, 2):
        if n % i == 0:
            return False
    return True

# Review:
# - Optimised the loop to check divisibility only up to the square root of n.
# - Reduced checks for even numbers to improve performance for large inputs.

# Example usage:
print(is_prime(5))  # Output: True
print(is_prime(4))  # Output: False

The prompt delivers a more optimised version, only checking odd numbers up to the square root, which dramatically improves performance.

Use Layered Prompting for Multi-Function Outputs

You can stack layers of functionality in a single prompt, asking ChatGPT to handle multiple related tasks in one go.

Basic Prompt: "Write a Python function to generate a random password."

import random
import string

def generate_password(length):
    return ''.join(random.choices(string.ascii_letters + string.digits, k=length))

# Example usage:
print(generate_password(10))  # Output: Random 10-character password

Optimised version : "Write a Python function to generate a random password. The password must meet the following criteria: at least 12 characters, contains uppercase, lowercase, numbers, and special characters. Also, write a validation function to check if the password is strong."

import random
import string
import re

def generate_password(length):
    characters = string.ascii_letters + string.digits + string.punctuation
    password = ''.join(random.choices(characters, k=length))
    return password

def validate_password(password):
    # Check length
    if len(password) < 12:
        return "Password too short"
    # Check for uppercase, lowercase, digit, and special character
    if (not re.search(r'[A-Z]', password) or not re.search(r'[a-z]', password) or
            not re.search(r'[0-9]', password) or not re.search(r'[\W_]', password)):
        return "Password must contain uppercase, lowercase, digit, and special character"
    return "Strong password"

# Example usage:
password = generate_password(12)
print(password)  # Output: Random 12-character password
print(validate_password(password))  # Output: "Strong password" or feedback on improvement

The basic prompt generates a random password.
The optimised gives a complex password generator and includes a validation function to check password strength.

Test-Driven Development: Generate a Complete Test Suite

You can ask ChatGPT to write the code along with a full test suite in one go, ensuring your code is ready for production with minimal effort. (If you must ask for help, make sure to ask for a lot 🤓).

Basic Prompt: "Write a Python function to check if a string is a palindrome."

def is_palindrome(s):
    return s == s[::-1]

# Example usage:
print(is_palindrome("madam"))  # Output: True
print(is_palindrome("hello"))  # Output: False

Getting more: "Write a Python function to check if a string is a palindrome. Also, write a full test suite using pytest with edge cases like empty strings and spaces."

def is_palindrome(s):
    s = ''.join(c.lower() for c in s if c.isalnum())  # Remove non-alphanumeric characters
    return s == s[::-1]

# Test suite
def test_is_palindrome():
    assert is_palindrome("madam") == True
    assert is_palindrome("hello") == False
    assert is_palindrome("A man, a plan, a canal, Panama") == True
    assert is_palindrome("") == True  # Edge case: empty string
    assert is_palindrome(" ") == True  # Edge case: only spaces

# Example usage:
print(is_palindrome("A man, a plan, a canal, Panama"))  # Output: True

The basic version checks for palindromes but misses edge cases.
The hidden trick not only refines the function by ignoring spaces and punctuation but also provides a comprehensive test suite using pytest.

By mastering these techniques, you can extract high-performance, error-free, and production-ready code from ChatGPT, all while doing less work. With auto-refinements, memory triggers, error handling, and complete test suites, you’ll code smarter, not harder.

How I Turned My Resume Into a Podcast with Google’s NotebookLM AI

James Ononiwu — Mon, 30 Sep 2024 23:59:40 +0000

In the age of AI, where technology and creativity intersect, there’s a tool that might just change the way you think about personal branding—Google’s NotebookLM. I recently had the pleasure of experimenting with it, and let’s just say, the results were beyond fascinating. Imagine turning a static document, like your resume, into an audio story, a podcast that captures not only your skills and achievements but your personal journey. Yes, that’s exactly what I did—and I’m excited to share how!

Discovering NotebookLM Deep-Dive

I stumbled upon Google’s NotebookLM while exploring tools for summarising text and automating content generation. NotebookLM is a next-gen AI tool designed to help users organize, synthesize, and create content from large volumes of text. It’s like having a personal research assistant at your fingertips. For someone like me, constantly juggling multiple research projects-whether in tech or other research areas, the idea of this tool doing the heavy lifting was irresistible.

But then, I had an even crazier idea: What if I fed it my CV, links to my LinkedIn profile, Github, and a few of my blog posts and asked it to create a podcast version of my professional story?

The Experiment: Input Meets Intelligence

To start, I uploaded my resume, links to my LinkedIn profile, Github, A few blog posts and a little text for fine tuning the whole process as sources into NotebookLM. These documents, though familiar to me, became the foundation of a much more dynamic narrative in the hands of the AI. I wanted the tool to not just list my qualifications or regurgitate bullet points, I wanted it to craft a flowing, engaging audio story that could be shared as a podcast.

The process felt like magic. NotebookLM parsed through my documents, picking up on key milestones in my career and even nuances from the blog posts that reflected my personal philosophy and approach to work. It wasn’t just about summarising, it was creating a unique angle.

From Data to Dialogue: The Podcast is Born

What amazed me most when I clicked the load button to create a podcast from the summary, was how NotebookLM managed to pull themes from my work that even I hadn't explicitly identified. For example, my focus on building businesses in tech and software development, as well as my thoughts on leadership and innovation, became central threads. It wasn’t just reading off my resume, it was curating my story in a way that was conversational, personal, and impactful.

NotebookLM helped organise this into podcast-friendly scripts, bite-sized segments that covered everything from my early career to my recent ventures in software development. It even suggested moments where a "personal reflection" or anecdote could be inserted, making the podcast sound more like an intimate conversation than a lifeless resume readout.

And that’s where it got intriguing: It felt as if my CV came alive, not as a formal document but as an audio journey, my journey.

AI-Powered Storytelling

What’s most exciting about NotebookLM is its ability to go beyond simple data aggregation. The AI identifies underlying themes and turns static information into dynamic content. For my podcast, it didn’t just stop at the job titles or skills. It connected the dots between them to form a coherent and compelling narrative.

Take this for example: I had a section on my resume that highlighted a project where I implemented an automated system with microservices. NotebookLM didn’t just focus on the technical details of the project, but also on the broader context of why I was passionate about automation and efficiency. It captured the ‘why’ behind my work, transforming that simple bullet point into a storyline that talks about microservice architecture.

The Final Product: More Than a Podcast

The final product? A seamless podcast that not only showcased my professional achievements but also conveyed my personality, passions, and aspirations. all I had to do next was click on the download button.

I’ve shared this with a few people in my network, and the feedback has been overwhelmingly positive. Some even joked that my podcast resume was more engaging than my LinkedIn profile, which in hindsight, might actually be true.

Why This Matters: A Glimpse into the Future

What’s truly exciting about this experiment is that it represents a shift in how we present ourselves to the world. The traditional resume, while functional, is limiting. With tools like NotebookLM, we can move beyond that, creating content that tells a story rather than just listing facts.

Imagine submitting a podcast resume to a potential employer, a vibrant, audio-based narrative where they can hear the passion behind your projects, the tone of your voice, and the excitement in your journey. This kind of personal branding is no longer a futuristic dream—it’s here.

Conclusion: AI, Creativity, and You

Using NotebookLM to turn my resume into a podcast wasn’t just a fun experiment; it was an eye-opening experience in what AI can offer in personal branding. By feeding it the right inputs... my CV, LinkedIn profile, and blog posts, I unlocked a storytelling tool that brought my professional journey to life in ways I couldn’t have imagined.

The future of AI-driven creativity is limitless. And with NotebookLM, I caught a glimpse of how it can transform not just how we work, but how we present ourselves to the world. Whether you're a tech entrepreneur, an artist, or someone just starting out, this kind of AI innovation is a game-changer and I can’t wait to see what comes next.

I generated a resume from my LinkedIn profile and added it as a background to video I made with the downloaded podcast.

If you haven’t explored NotebookLM yet, you’re missing out on something special. there's a whole lot you can do with it, you can dissect your research papers into different sections and have a chat with each section, upload not just text documents but audio as sources of information.
Also, Your resume, your story, could be the next big podcast sensation.

10 Python programming optimisation techniques.

James Ononiwu — Sun, 18 Aug 2024 23:35:23 +0000

Optimised code is essential because it directly impacts the efficiency, performance, and scalability of software. Well-written code runs faster, consumes fewer resources, and is more maintainable, making it better suited for handling larger workloads and improving user experience. It also reduces operational costs, as efficient code requires less processing power and memory, which is particularly crucial in environments with limited resources, such as embedded systems or large-scale cloud applications.

Poorly written code, on the other hand, can lead to slow execution times, increased energy consumption, and higher infrastructure costs. For example, in a web application, inefficient code can slow down page loads, leading to a poor user experience and potentially driving users away. In data processing tasks, inefficient algorithms can significantly increase the time it takes to process large datasets, delaying critical insights and decisions.

Moreover, optimised code is often more straightforward to maintain and extend. By adhering to optimisation best practices, developers can ensure that their codebase remains clean and modular, making it easier to update or scale the application as needed. This becomes increasingly important as software projects grow in complexity and as the demands on the system increase.

Let’s explore 10 Python programming optimisation techniques that can help you write more efficient and performant code. These techniques are crucial for developing robust applications that meet performance requirements while remaining scalable and maintainable over time. These techniques can also be applied to other programming languages by following the best practices.

1. Variable Packing

Variable packing minimises memory usage by grouping multiple data items into a single structure. This technique is critical in scenarios where memory access times significantly impact performance, such as in large-scale data processing. When related data is packed together, it allows for more efficient use of CPU cache, leading to faster data retrieval.

Example:

import struct

# Packing two integers into a binary format
packed_data = struct.pack('ii', 10, 20)

# Unpacking the packed binary data
a, b = struct.unpack('ii', packed_data)

In this example, using the struct module packs integers into a compact binary format, making data processing more efficient.

2. Storage vs. Memory

Understanding the difference between storage (disk) and memory (RAM) is crucial. Memory operations are faster but volatile, while storage is persistent but slower. In performance-critical applications, keeping frequently accessed data in memory and minimising storage I/O is essential for speed.

Example:

import mmap

# Memory-mapping a file
with open("data.txt", "r+b") as f:
    mmapped_file = mmap.mmap(f.fileno(), 0)
    print(mmapped_file.readline())
    mmapped_file.close()

Memory-mapped files allow you to treat disk storage as if it were memory, speeding up access times for large files.

3. Fixed-Length vs. Variable-Length Variables

Fixed-length variables are stored in a contiguous block of memory, making access and manipulation faster. Variable-length variables, on the other hand, require additional overhead to manage dynamic memory allocation, which can slow down operations, particularly in real-time systems.

Example:

import array

# Using fixed-length array for performance
fixed_array = array.array('i', [1, 2, 3, 4, 5])

# Dynamic list (variable-length)
dynamic_list = [1, 2, 3, 4, 5]

Here, array.array provides a fixed-length array, offering more predictable performance than dynamic lists.

4. Internal vs. Public Functions

Internal functions are those intended to be used only within the module where they are defined, often optimised for speed and efficiency. Public functions are exposed for external use and may include additional error handling or logging, making them slightly less efficient.

Example:

def _private_function(data):
    # Optimized for internal use, with minimal error handling
    return data ** 2

def public_function(data):
    # Includes additional checks for external use
    if isinstance(data, int):
        return _private_function(data)
    raise ValueError("Input must be an integer")

By keeping the heavy computation in a private function, you optimise the code's efficiency, reserving public functions for external safety and usability.

5. Function Modifiers

In Python, decorators serve as function modifiers, allowing you to add functionality before or after the function's main execution. This is useful for tasks like caching, access control, or logging, which can optimise resource usage across multiple function calls.

Example:

from functools import lru_cache

@lru_cache(maxsize=100)
def compute_heavy_function(x):
    # A computationally expensive operation
    return x ** x

Using lru_cache as a decorator caches the results of expensive function calls, improving performance by avoiding redundant computations.

6. Use Libraries

Leveraging libraries allows you to avoid reinventing the wheel. Libraries like NumPy are written in C and built for performance, making them far more efficient for heavy numerical computations compared to pure Python implementations.

Example:

import numpy as np

# Efficient matrix multiplication using NumPy
matrix_a = np.random.rand(1000, 1000)
matrix_b = np.random.rand(1000, 1000)
result = np.dot(matrix_a, matrix_b)

Here, NumPy's dot function is enhanced for matrix operations, far outperforming nested loops in pure Python.

7. Short-Circuiting Conditionals

Short-circuiting reduces unnecessary evaluations, which is particularly valuable in complex condition checks or when involving resource-intensive operations. It prevents execution of conditions that don't need to be checked, saving both time and computational power.
Since conditional checks will stop the second they find the first value which satisfies the condition, you should put the variables most likely to validate/invalidate the condition first. In OR conditions (or), try to put the variable with the highest likelihood of being true first, and in AND conditions (and), try to put the variable with the highest likelihood of being false first. As soon as that variable is checked, the conditional can exit without needing to check the other values.

Example:

def complex_condition(x, y):
    return x != 0 and y / x > 2  # Stops evaluation if x is 0

In this example, Python’s logical operators ensure that the division is only executed if x is non-zero, preventing potential runtime errors and unnecessary computation.

8. Free Up Memory

In long-running applications, especially those dealing with large datasets, it’s essential to free up memory once it’s no longer needed. This can be done using del, gc.collect(), or by allowing objects to go out of scope.

Example:

import gc

# Manual garbage collection to free up memory
large_data = [i for i in range(1000000)]
del large_data
gc.collect()  # Forces garbage collection

Using gc.collect() ensures that memory is reclaimed promptly, which is critical in memory-constrained environments.

9. Short Error Messages

In systems where memory or bandwidth is limited, such as embedded systems or logging in distributed applications, short error messages can reduce overhead. This practice also applies to scenarios where large-scale error logging is necessary.

Example:

try:
    result = 10 / 0
except ZeroDivisionError:
    print("Err: Div/0")  # Short, concise error message

Short error messages are useful in environments where resource efficiency is crucial, such as IoT devices or high-frequency trading systems.

10. Optimize Loops

Loops are a common source of inefficiency, especially when processing large datasets. Optimising loops by reducing iterations, simplifying the logic, or using vectorised operations can significantly improve performance.

Example:

import numpy as np

# Vectorised operation with NumPy
array = np.array([1, 2, 3, 4, 5])

# Instead of looping through elements
result = array * 2  # Efficient, vectorised operation

NumPy (Numerical Python) is a popular Python library used for numerical and scientific computing. It provides support for large, multi-dimensional arrays and matrices, along with a collection of mathematical functions to operate on these arrays.

It can be installed with pip by running
pip install numpy.

Vectorisation eliminates the need for explicit loops, leveraging low-level optimisations for faster execution.

By applying these techniques, you can ensure your Python or other programming language programs run faster, use less memory, and are more scalable, which is especially important for applications in data science, web and systems programming.

PS: you can use https://perfpy.com/#/ to check python code efficiency.

Enhanced Image Generation With Stable Diffusion and Roop.

James Ononiwu — Wed, 13 Sep 2023 02:18:57 +0000

In this post, we will go through the installation process and usage of stable diffusion for text-to-image or image-to-image generation.
text-to-image generation has to do with generating realistic images by using just text prompts, while image-to-image takes the game a little bit further by using a sample image in addition to test prompts to generate ultra realistic images that has features of the sample image.
Adding roop AI model to the picture further enhances the art by improving the quality of image generated to a perfection level. we will see how shortly, so let's started.

Install Stable Diffision

First, we need to install AUTOMATIC111 for stable diffusion on our system. i will be going through with the installation for mac, but not to worry windows and linux has similar process, you only need to use their respective package managers as AUTOMATIC111 runs solely on the terminal with a web ui on the browser.

Step 1: Install homebrew package manager on mac
If homebrew is not yet installed on your mac, please open a command line like iterm2 and run the commands below to install homebrew.

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Step 2: install additional packages
After the installation completes, we need to install several other packages like python(if not installed), cmake, protobuf, rust, git, and wget. run the command below to install them.

brew install cmake protobuf rust python@3.10 git wget

Step 3: Clone AUTOMATIC111 Repo
To make AUTOMATIC111 work on the local environment which is our macbook, windows or linux, we need to get a copy of it saved locally. we can achieve that by cloning it. cd into the home directory (cd $HOME) and run the command below to do that.

cd $HOME && git clone https://github.com/AUTOMATIC1111/stable-diffusion-webui

Step 4: Download the model file.
Once we have gotten a copy of AUTOMATIC111 for stable diffusion, next would be to download the model. please note that it is about 4GB of data, so you might want to make a coffee while it downloads.
v1-5-pruned-emaonly.ckpt

Step 5: Move models file to model directory
When download is completed, you need to move the model .ckpt file to stable-diffusion-webui/models/Stable-diffusion folder which is inside the AUTOMATIC111 we cloned.

After doing so, the folder will contain two files as above.

Step 6: Download Roop model
Download Roop inswapper model and place it same way as above but in the models/roop folder. when done the folder will look as below.
inswapper_128.onnx

Finally: Run AUTOMATIC111
To run AUTOMATIC111 and load the web ui in browser, enter the command below. please note that this might take a while on the first run as there are additional packages to be downloaded.

cd ~/stable-diffusion-webui
./webui.sh --precision full --no-half --skip-torch-cuda-test

please skip the additional options if you are using a GPU enabled device. just run

./webui.sh

the image below on the terminal..

indicates that the webui has been loaded successfully.

finally we can start generating some cool images. let's enter a simple prompt "image of monalisa in a wonderland, photo realistic, colorful, hyper sensitive with birds and windy background".
enter the prompt in text2img box and click on generate button. then relax a bit while our image gets generated.

After some minutes, our image gets successfully generated as you can see. Did pretty well for a painting right?, it also took into consideration the elements in the prompt. but we can do much better.

Adding Roop to the picture
This is where Roop comes into play, to generate high quality content, we need to enable roop extension and use img2img to feed in a sample image we can use for the generation.
click on "extension tab", then "install from url". enter github repo link of roop sd-webui-roop and click "Install".

next click on img2img select an image, scroll down and click on roop. upload yet another image also make sure "enable" is checked then enter some prompt and click on generate. you will be amazed at the result. i generated the image below following the steps with a simple prompt "as a pirate".

Besides, the two great men in the source photos are Salva and Julien. my mentors at ALX/Holberton school. I don't even know if doing this is legal 🤭.

Conclusion
Roop with stable diffusion brings a whole new level to image generation with AI, there's a lot of options you can try to see which works best...like selecting a sampling method, batch count range, an upscaler and so on. I encourage you to check them out.
Also note that with AI there's always room for improvement and new things to learn.

Thanks for reading 🤗.

Postmortem: Nginx Server Failure

James Ononiwu — Sun, 11 Jun 2023 16:47:11 +0000

Issue Summary:
Duration: The outage occurred from 10:00 AM to 11:30 AM (UTC-5) on June 9, 2023.
Impact: The primary service provided by the Nginx server was completely unavailable during the outage. Users experienced a complete loss of functionality and were unable to access the website. Approximately 80% of the users were affected.

Timeline:

10:00 AM: The issue was detected when monitoring alerts indicated a sudden increase in server response time.
10:05 AM: An engineer on the team noticed a spike in CPU utilization on the Nginx server and suspected it might be the root cause.
10:10 AM: Investigation began by analyzing server logs and network traffic to identify any unusual patterns.
10:20 AM: Initial assumption made that a sudden surge in incoming requests was overwhelming the server, leading to high CPU usage.
10:30 AM: Scaling up the server infrastructure was considered as a potential solution to handle the increased load.
10:40 AM: However, further analysis revealed that the increased load was caused by a DDoS attack targeting the Nginx server.
10:50 AM: The incident was escalated to the security team to handle the ongoing DDoS attack.
11:00 AM: Mitigation measures were put in place to filter and block malicious traffic, reducing the impact of the attack.
11:30 AM: The incident was resolved as the Nginx server regained stability and resumed normal operations.

Root Cause and Resolution:
The root cause of the issue was a DDoS attack on the Nginx server, which led to an overwhelming increase in incoming requests. The attack aimed to exhaust server resources and disrupt the service. The attack was mitigated by implementing traffic filtering and blocking mechanisms at the network level, effectively blocking malicious traffic and reducing the server load. This allowed the Nginx server to recover and restore normal operations.

Corrective and Preventative Measures:
To address the issue and prevent similar incidents in the future, the following measures will be implemented:

Improve DDoS protection: Enhance the existing DDoS mitigation strategies by implementing more robust traffic filtering and rate limiting mechanisms. Consider employing a specialized DDoS protection service.
Scaling and redundancy: Evaluate the server infrastructure's scalability and redundancy to handle sudden increases in traffic and mitigate the impact of DDoS attacks. Implement an auto-scaling solution to dynamically adjust resources based on demand.
Enhanced monitoring and alerting: Implement comprehensive monitoring and alerting systems to quickly detect anomalies, such as unusual spikes in traffic or CPU utilization. Set up proactive alerts to notify the team in real-time.
Incident response plan: Develop a detailed incident response plan specifically for DDoS attacks. Define roles and responsibilities, establish communication channels, and document step-by-step procedures for efficient incident handling.
Regular security audits: Conduct regular security audits to identify and address any vulnerabilities in the server infrastructure. This includes patching and updating server software, ensuring secure configurations, and performing penetration testing.
Employee training and awareness: Provide training sessions and awareness programs to educate employees about DDoS attacks, their impact, and the necessary actions to take during such incidents.

Tasks to Address the Issue:

Patch Nginx server software to the latest version to mitigate known vulnerabilities.
Implement traffic filtering and rate limiting rules to mitigate DDoS attacks.
Set up automated scaling mechanisms to handle sudden traffic spikes.
Enhance monitoring system to include CPU and network traffic metrics.
Develop an incident response plan specifically for DDoS attacks.
Conduct regular security audits and penetration testing to identify vulnerabilities.

By implementing these measures and addressing the outlined tasks, we can improve the overall resilience and security of our Nginx server infrastructure. This will help us mitigate the impact of future incidents and ensure uninterrupted service for our users.

In conclusion, the Nginx server failure was caused by a DDoS attack that overwhelmed the server with a sudden increase in incoming requests. The incident lasted for approximately 1.5 hours, during which the service was completely unavailable, affecting around 80% of the users. The attack was mitigated by implementing traffic filtering and blocking mechanisms, allowing the server to recover and resume normal operations.

To prevent similar incidents in the future, we will enhance our DDoS protection measures, improve the scalability and redundancy of our server infrastructure, and implement comprehensive monitoring and alerting systems. Additionally, we will develop a detailed incident response plan, conduct regular security audits, and provide employee training to enhance our overall security posture.

By learning from this incident and implementing the necessary measures, we aim to minimize the impact of potential future incidents and provide a reliable and secure experience for our users.

What happens when you type in a web address in a browser and press enter. (example: www.google.com)

James Ononiwu — Sun, 14 May 2023 20:37:11 +0000

The internet as we know it, has come a long way and is here to stay. the Internet was originally developed In the 1960s as a means for government researchers to exchange information. However, at that time, computers were bulky and stationary, and to access data stored on a particular computer, one had to either physically travel to its location or rely on magnetic tapes sent via traditional mail delivery systems.
Today one can access the internet from the comfort of their home, in fact every device nowadays is getting smart. from smart TVs to smart locks that gives you access to the doors of your home from anywhere around the world you might be.
But how does this work?.
In this article I will explain in simple terms what happens under the hood, the moment you typed in a web address and hit enter on your keyboard.
The fact is that web pages you see displayed on your browser when are just files called HTML (Hypertext Markup Language) stored in a computer sitting somewhere with access to the internet. this type of computer that stores and serves files is called a web server. HTML files are formatted in a certain way so that your browser understands it.
The web browser makes request to this servers asking for a file.
The web server after validating the request, performs some actions which might include retrieving data from a storage called a database and sends the required files back to the web browser. Some websites use an application server to handle more complex tasks, such as running business logic or processing payments.
The web browser parses the files and displays them on the screen.
But before the three steps are completed, a lot happened within some seconds taken to send out the GET request ( in this case www.google.com) from the browser and receiving a response data from the server. that process is what I'm about to explain.

DNS
A domain name system (DNS) is a record that holds the IP addresses of domain names, it translates human readable domain names to machine readable IP addresses. Your browser first needs to know the IP address of the domain name (ie. www.google.com) which it does by sending a request to the Domain Name System (DNS). The DNS server looks up the IP address associated with the domain name and returns it to your browser. The IP address is a unique set of numbers that identifies every device on the internet, it ranges from 0 to 255 and separated by periods (ie. 192.0. 2.44) . The browser sends the request to the IP address which is the identifier of a computer connected to the internet.
This is like looking up a phone number in a phonebook.

TCP/UDP
Once your browser has the IP address, it establishes a connection using the Transmission Control Protocol (TCP) or User Datagram Protocol (UDP).
TCP breaks the data into packets (smaller pieces) and sends them across the internet, this method ensures that all packets are delivered. for an instance when downloading a file from the internet and there's a network error, when the network reconnects, the download resumes from the exact spot. this is because the server knows the amount of packets sent and how many remaining. on the other hand UDP is mostly used for video streaming and playing online games, this is because it is faster, the speed is due to the fact that it cares less about delivering all packets, that is why if a live streaming freezes, you will be moved to the current stream when the network resumes, or see the screen skip to the current time in the case of video games. These protocols are used to connect and send data between a browser and a server while IP makes sure they are delivered to the right destination.

FIREWALL
As the packets travel through the internet, they may pass through several firewalls. These are security measures that monitor and filter incoming and outgoing traffic based on pre-defined rules set on the devices. Think of firewalls as a security personnel at a very important event who checks IDs and only allows access to people who meet certain criteria.

HTTPS/SSL
Most times when you send a request to a web server, you will see a key icon beside the domain name you entered, this means that the server is serving the file via HTTPS (Hyper Text Transfer Protocol Secure), which also means that the data is encrypted using SSL (Secure Sockets Layer) or its successor, TLS (Transport Layer Security). This ensures that any sensitive information, such as login credentials, is protected from people that might maliciously intercept it using a method called man-in-the-middle attack, and use it to perform actions that might be harmful to the owner, this set of people are called hackers. when you try to access a web page that does not have this security measures, your web browser usually gives a warning to inform you that the site you want to access might pose as a security risk.

LOAD BALANCER
If the website receives a lot of traffic, it may use a load-balancer. a load balancer is a server that distributes the incoming requests across other multiple web servers, so that no single server gets overloaded. using a single server might result in failures in delivering requests when there's too many visits in short amount of time (traffic). using a load balancer ensures that the requests is distributed among multiple servers, if one has an issue, there will be others to respond thereby delivering the expected result.
This is like a popular restaurant using multiple waiters to serve customers, so that no single waiter gets overwhelmed.

CONCLUSION
In conclusion, when you type www.google.com in your browser and hit Enter, a lot of complex processes happen behind the scenes to deliver the requested web page to your browser. Understanding these processes helps to appreciate the complexity of the internet and how it has revolutionized the way we communicate and access information.

Understanding loops

James Ononiwu — Sat, 01 Aug 2020 23:56:45 +0000

Loops are what makes programming such a powerful art. in fact without iteration a lot of problems coding solves today wouldn't have been possibly solved, it would have been a tedious task to write repetitive instructions for the number of times needed. this is why understanding what's happening under the hood when an iteration is going on is very important in making efficient use of loops to become a better programmer.

The simple part

simple use of a loop is the use of a single loop to print values contained in a list, or print characters that make up a string etc.

#print values in a list
countries = ['Germany','USA','Spain']
for country in countries: #iterate over countries list
    print(country)

#print characters that make up a string
country = 'Germany'
for letter in country: #iterate over countries list
    print(letter)

the above code is for simple cases and easy for a beginner to pick up.

Challenge most beginners face.

the real challenge comes when we have nested loops - that is a loop inside another loop. unless you are building a time machine, two nested loops are enough to solve most iteration problems. the number of nested loops is directly proportional to the efficiency of the code as the deeper the nesting the less efficient and more time required to run. let's combine the two examples above to form a nested loop.

#nested for loops
countries = ['Germany','USA','Spain']
for country in countries: #iterate over countries list
    print('beginning of outer loop') 
    print('{} is at index {} of {}'.format(country,countries.index(country), countries),'\n')
    print('beginning of inner loop for {}'.format(country))

    for letter in country: #iterate over each country in countries list
        print('{} is at index {} of {}'.format(letter,country.index(letter),country))
    print('end of inner loop for {}'.format(country),'\n')

print('end of outer loop')

running the code above will give the output below.

beginning of outer loop
Germany is at index 0 of ['Germany', 'USA', 'Spain'] 

beginning of inner loop for Germany
G is at index 0 of Germany
e is at index 1 of Germany
r is at index 2 of Germany
m is at index 3 of Germany
a is at index 4 of Germany
n is at index 5 of Germany
y is at index 6 of Germany
end of inner loop for Germany 

beginning of outer loop
USA is at index 1 of ['Germany', 'USA', 'Spain'] 

beginning of inner loop for USA
U is at index 0 of USA
S is at index 1 of USA
A is at index 2 of USA
end of inner loop for USA 

beginning of outer loop
Spain is at index 2 of ['Germany', 'USA', 'Spain'] 

beginning of inner loop for Spain
S is at index 0 of Spain
p is at index 1 of Spain
a is at index 2 of Spain
i is at index 3 of Spain
n is at index 4 of Spain
end of inner loop for Spain 

end of outer loop

The formating and explanation outputs are for a better understanding of what is going on.
the objective of the code is to print each country contained in the countries list with their index, and also print the characters that spell them. understanding loops have more to do with knowing how index changes with each iteration. from the example above each index range starts at 0 and ends at the length of the iterable -1, for instance, the count of values in countries list is 3, therefore iteration is of the range 0,1,2. for a nested loop, execution of the outer loop once, is followed by complete execution of the inner loop and then control is handed back to the outer loop to continue executing.


beginning of outer loop
Germany is at index 0 of ['Germany', 'USA', 'Spain'] 

beginning of inner loop for Germany
G is at index 0 of Germany
e is at index 1 of Germany
r is at index 2 of Germany
m is at index 3 of Germany
a is at index 4 of Germany
n is at index 5 of Germany
y is at index 6 of Germany
end of inner loop for Germany 

beginning of outer loop
USA is at index 1 of ['Germany', 'USA', 'Spain']

this is continued until the outer loop iteration reaches the end of the list in this case at the value Spain being the last value in the list.
Let's see another example of a nested loop where we need to check for duplicates in a sequence of numbers.

 def is_unique(S):
    """Return True if there are no duplicate elements in sequence S."""
    for j in range(len(S)):
        for k in range(j+1, len(S)):
            if S[j] == S[k]:
                print('S[{}] {}, S[{}] {}'.format(j,S[j],k,S[k]),'duplicate found')
                return False #found duplicate
            else:
                print('S[{}] {}, S[{}] {}'.format(j,S[j],k,S[k]))
    return True #if we reach this, elements were unique

Calling the function with the values..


result = is_unique([77,55,33,44,33])
print(result)

we will get the output below

S[0] 77, S[1] 55
S[0] 77, S[2] 33
S[0] 77, S[3] 44
S[0] 77, S[4] 33
S[1] 55, S[2] 33
S[1] 55, S[3] 44
S[1] 55, S[4] 33
S[2] 33, S[3] 44
S[2] 33, S[4] 33 duplicate found

False

Again, the output formatting is for you to better understand what is going on, but we did receive False showing that the list contains duplicate values which happen to be the number 33. as you can see from the output each iteration of the outer loop i.e S[0] is followed by a complete iteration of the inner loop.

S[0] 77, S[1] 55
S[0] 77, S[2] 33
S[0] 77, S[3] 44
S[0] 77, S[4] 33

S[1] 55, S[2] 33
S[1] 55, S[3] 44
S[1] 55, S[4] 33


S[2] 33, S[3] 44
S[2] 33, S[4] 33 duplicate found

But there's something new in the code, and that is the comparison of values inside the loop using the current iteration index.

  for j in range(len(S)):
        for k in range(j+1, len(S)):
            if S[j] == S[k]:

from the code above, you will understand that if the current index j of the outer loop is 0 then the first index k of the inner loop will be one because k starts at j+1, and that's why we have...

S[0] 77, S[1] 55

S[1] 55, S[2] 33

S[2] 33, S[3] 44

which helped us to compare value at index j S[j] to value at index j+1 S[j+1] or Sk to see if they are same.

Index out of range

Consider the code below that print each country contained in a list of countries

countries = ['Germany','USA','Spain']
country_index = 0
while country_index < len(countries):
    print(countries[country_index])
    country_index += 1 #increment index

Germany
USA
Spain

Notice that using while loops we have to specify a counter (in this case country_index) which is used as a checkpoint to terminate the loop if the condition no longer holds true, else we will have the loop running indefinitely.

Now we want to print the values of the same list in reverse order by trying the code below.

countries = ['Germany','USA','Spain']
countries_len = len(countries)
while countries_len >= 0:
    print(countries[countries_len])
    countries_len -= 1 #decrement index

after running the code, instead of getting what we want. we ended up with the output below.

 2 countries_len = len(countries)
      3 while countries_len >= 0:
----> 4     print(countries[countries_len])
      5     countries_len -= 1 #decrement index

IndexError: list index out of range

list index out of range error.
What causes this error is simply the fact that indexing in python starts at 0. we might say we specified that by doing..

while countries_len >= 0:

But that's not where the exception occurred. it happened at line 4.


----> 4     print(countries[countries_len])

Because we are trying to get a value with an index that is greater than the maximum index of the list. len(countries) will give you the value 3 which is the count of elements in the list starting at 1, but indexing in python starts at 0. so the range of index of countries list is 0,1,2 having maximum value as 2.

print(countries_len)

Gives the output.

while printing the indexes


for i in range(len(countries)):
    print(i)

gives.

0
1
2

to correct the error we simply subtract 1 from the length of the list.

countries = ['Germany','USA','Spain']
countries_len = len(countries)-1 #subtract 1 from length
while countries_len >= 0:
    print(countries[countries_len])
    countries_len -= 1 #decrement index

and we get the correct output.

Spain
USA
Germany

Another example of this error is gotten when we re-write the previous is_unique function this way.

def is_unique(S):
    """Return True if ther are no duplicate elements in sequence S."""
    sorted_S = sorted(S) # sort the sequence S.
    for j in range(1, len(sorted_S)):
        if sorted_S[j] == sorted_S[j+1]:
            return False
    return True

calling the function with the sequence below as argument..

numbers = [33,44,33,35,54,77]
result = is_unique(numbers)
print(result)

We will get the error output.

4     for j in range(1, len(sorted_S)):
----> 5         if sorted_S[j] == sorted_S[j+1]:
      6             return False
      7     return True

IndexError: list index out of range

Because we are trying to access values from an index j+1 which will not exist if we reach the maximum index j of the list.

To correct this error we simply change j+1 to j-1, this will have the list compare it's current value S[j] with it's previous value S[j-1],

def is_unique(S):
    """Return True if ther are no duplicate elements in sequence S."""
    sorted_S = sorted(S)
    for j in range(1, len(sorted_S)):
        if sorted_S[j] == sorted_S[j-1]: # changed j+1 to j-1
            return False
    return True

Executing

numbers = [33,44,33,35,54,77]
result = is_unique(numbers)
print(result)

returned the correct output.

False

we've seen what happens under the hood during an iteration, and also errors you might encounter and how to fix them. i hope this have helped you in understanding a loop better. Thanks for reading.

Custom model Evaluation Metric

James Ononiwu — Fri, 31 Jul 2020 01:48:53 +0000

In building machine learning models, There comes a time you might want to evaluate your model to know it's a performance using a metric you created. while scikit-learn with all it's beauty provides the evaluation metrics we need, there comes a time we might need to create our own metrics. scikit-learn makes this easy by providing a function make_scorer.
to create a custom metric, all you need to do is create a function containing the metric definition and convert it to a scorer function using scikit-learn make_scorer. we will use the make_regression dataset to show an example of creating a custom metric below.

let's begin by importing the neccessary libraries

# Load libraries
from sklearn.metrics import make_scorer, r2_score
from sklearn.model_selection import train_test_split
from sklearn.linear_model import Ridge
from sklearn.datasets import make_regression

Next we generate features and the target matrix.

features, target = make_regression(n_samples = 100,n_features = 4,
random_state = 1)

random_state parameter ensures we get the same distribution when we use the same random_state value.

let's continue by creating the training and testing set using train_test_split

X_train, X_test, y_train, y_test = train_test_split(features, target, test_size=0.10, random_state=1)

then create our custom metric function.

def custom_metric(X_test, target_predicted):
    # Calculate r-squared score
    r_squared = r2_score(X_test, target_predicted)
    # Return r-squared score
    return r_squared

next convert the custom metric into a scorer metric

# Make scorer and define that higher scores are better
score = make_scorer(custom_metric, greater_is_better=True)

greater_is_better signifies that higher score is better

Create the regression and train.

classifier = Ridge()
# Train ridge regression model
model = classifier.fit(X_train, y_train)

Finally apply the custom metric to check model performance.

score(model, X_test, y_test)

Python Convienences

James Ononiwu — Thu, 30 Jul 2020 19:35:25 +0000

There are several features of python which makes it more convienent for writng clean concise code. each of the features can be achieved with other python functionalities, however most times the features account for clear expression of the logic. let's look at them one after the other.

Conditional Expressions

expr1 if condition else expr2

The above syntax is equivalent to the statement:

condition ? expr1 : expr2

for those familiar with Java or C++. the compound expression evaluates to expr1, if the condition is True, else it evaluates to expr2.example usage are outlined below.
consider a goal of sending the maximum number as an argument to a function:

a,b = 4,5
if a > b:
   param = a
else:
   param = b
result = foo(param)

the objective of the code above can be achieved with the one below.

a,b = 4,5
param = a if a > b else b
result = foo(param)

in fact, we can even make it shorter by not assigning the result to any variable rather passing the expression directly as an argument.

a,b = 4,5
result = foo(a if a > b else b)

sometimes, shorting of source code makes it easy to read and understand a cumbersome code. however, it is advised that conditional expression is used only when it improves the readability of the source code.

Comprehension Syntax

most times the task might be to produce one series of values based on the processing of another series. this can be achieved easily with comprehension syntax. there are different types of comprehension syntax according to the datatype involved. let's use the code below as an example of computing values from one list to another list.

list1 = [1,2,3,4]
list2 = []
for num in list1:
    list2.append(num*num)

we can achieve the same result above in a shorter, cleaner way by making use of comprehension syntax for different conatainer types.

list2 = [ num*num for num in list1 ] #list comprehension
{ num*num for num in list1 } #set comprehension
( num*num for num in list1 ) #generator comprehension
{ num :num*num for num in list1 } #dictionary comprehension

The generator syntax is particularly attractive when results do not need to be stored
in memory.

Packing and Unpacking of Sequences

packing and unpacking is used in tuples and other data types.
for example in the code below:

values = 4,5,6,7

results in the identifier values being automatically assigned to the tuple (4,5,6,7). this behaviour is known as automatic packing of the tuple.
one of the uses is in returning multiple values from a function.

return x,y,z

returns the tuple (x,y,z)

python also offers a way to unpack a tuple for instance if the values returned from a function foo is a tuple (x,y,z). this can be unpacked by assigning each value in the tuple to an identifier.

x,y,z = foo()

a,b,c = (1,2,3) # a = 1, b = 2, c = 3

for x, y in [ (7, 2), (5, 8), (6, 4) ]: # x = 7, y = 2 | x = 5, y = 8 |... and so on

Simultaneous Assignments

a simultaneous assignment is a combination of packing and unpacking done simultaneously.

a,b,c = 1, 2, 3

good use of simultaneous assignment is in swapping values contained in variables. usually, to swap a value between two variables, a third variable temp is required.

a = 5
b = 6
temp = a # temp = 5
a = b # a is now 6
b = temp # b is now 5

the same code above can be achieved in a cleaner shorter way with just two lines of code as shown below.

a,b = 5, 6
a,b = b, a

We have seen the different ways to write better code while achieving the same goal. I hope this adds more value to your skill, thanks for reading.