DEV Community: Kedar Kodgire

How Generative AI Learns: From Data to Imagination

Kedar Kodgire — Sun, 06 Jul 2025 03:37:34 +0000

Generative AI feels like magic. You type in a few words, and it writes a story, creates an image, or answers your question like it has a brain of its own.

But under the hood, there's no magic — just math, data, and a whole lot of pattern recognition.

In this post, let’s explore how Gen AI learns, in simple terms, using a real-world example: how an AI learns to write restaurant reviews.

Imagine This: Teaching AI to Write Restaurant Reviews

Let’s say we want to train an AI to write reviews for restaurants.

But first — what do we actually give it?

Step 1: Feed it Data

We start by giving it tons of real reviews. Thousands, maybe millions.

Example input:

“This pizza was amazing! The crust was crispy, the sauce was rich, and the staff was super friendly.”

“I waited 40 minutes for cold spaghetti. Will never come back.”

The AI doesn’t understand what pizza or spaghetti mean — not yet. But it can start noticing patterns.

Step 2: Breaking Everything Into Tiny Pieces

AI doesn’t read like us. It doesn’t see sentences — it sees tokens.

Tokens are chunks of text. For example:

"This pizza was amazing!"

becomes → [This, pizza, was, amazing, !]

These tokens are turned into numbers using a technique called embedding. Why? Because math is what AI understands.

So now, the AI is looking at rows and rows of numbers that represent words, their meanings, and how they relate to each other.

Step 3: Spotting Patterns in the Chaos

Now comes the interesting part: training.

The AI goes through billions of examples like:

“The ___ was delicious.” → Tries to guess “pizza,” “pasta,” “soup”
“The staff was ___.” → Guesses “kind,” “rude,” “friendly”

Over and over again, it tries to predict the next word. Each time it gets it wrong, it adjusts its internal settings a little. This is how it learns — by failing millions of times and getting better with every try.

This process is done using massive models like transformers (more on that in future posts).

🤖 Real-World Analogy: Autocomplete on Steroids

Think about when you type a message and your phone suggests the next word.

Generative AI is doing something similar — but instead of just a few words, it can generate entire paragraphs, essays, poems, or code.

The core idea is still the same: predict what comes next based on everything it has learned before.

So What’s a “Model”?

You’ll hear this word a lot.

A model is what we get after training.

It’s the result — the thing that now “knows” how to respond based on the patterns it has seen. Just like a student who studied a thousand flashcards now knows all the answers.

Popular models include:

GPT (OpenAI)
Gemini (Google)
Claude (Anthropic)
LLaMA (Meta)

Each model is trained on huge amounts of data — books, articles, websites — and each learns slightly differently depending on how it was built.

Memory Without Memory

One mind-blowing thing: most AI models don’t have real memory.

They don’t remember your last conversation (unless specifically built to do so). They respond based only on the input you give right now, plus what they learned during training.

So if you ask it to write a pizza review, it doesn’t remember writing one earlier. It just sees the prompt and thinks:

“Based on what I know… what words should come next?”

🗺️ Zooming Out: What AI Isn't

Let’s clear up a common myth.

AI doesn’t think. It doesn’t feel. It doesn’t know truth.

It’s just really, really good at pattern prediction. That’s it.

So if it writes:

“This hidden sushi bar in Chicago has the best miso soup in the world.”

It may be totally made up — or based on something it once saw. That’s why accuracy is a big challenge in Gen AI.

Quick Recap

Here’s how Gen AI learns, using our restaurant review example:

Data In → Millions of real reviews
Tokenization → Break text into tiny, meaningful chunks
Embedding → Convert words to numbers
Training → Guess what comes next, adjust, repeat
Model Output → An AI that can now generate its own reviews

Pretty cool, right?

Bonus: Want to See It in Action?

Try writing:

“Write a review of a pizza place that only serves dessert pizza.”

into ChatGPT or Veo by DeepMind

Watch how it predicts a story based on everything it's ever seen.

What’s Next?

In the next post, we’ll explore embeddings and vector data — how AI turns words into numbers and how it finds the “closest match” when answering your questions.

We’ll explore it using a fun example: How an AI finds your favorite superhero based on personality. 🦸‍♂️

Got questions or want to see a specific example? Drop it in the comments!

Follow for the next post — we’re just getting started. 🚀

How Generative AI Learns: From Data to Imagination

Kedar Kodgire — Sun, 06 Jul 2025 03:37:34 +0000

Generative AI feels like magic. You type in a few words, and it writes a story, creates an image, or answers your question like it has a brain of its own.

But under the hood, there's no magic — just math, data, and a whole lot of pattern recognition.

In this post, let’s explore how Gen AI learns, in simple terms, using a real-world example: how an AI learns to write restaurant reviews.

Imagine This: Teaching AI to Write Restaurant Reviews

Let’s say we want to train an AI to write reviews for restaurants.

But first — what do we actually give it?

Step 1: Feed it Data

We start by giving it tons of real reviews. Thousands, maybe millions.

Example input:

“This pizza was amazing! The crust was crispy, the sauce was rich, and the staff was super friendly.”

“I waited 40 minutes for cold spaghetti. Will never come back.”

The AI doesn’t understand what pizza or spaghetti mean — not yet. But it can start noticing patterns.

Step 2: Breaking Everything Into Tiny Pieces

AI doesn’t read like us. It doesn’t see sentences — it sees tokens.

Tokens are chunks of text. For example:

"This pizza was amazing!"

becomes → [This, pizza, was, amazing, !]

These tokens are turned into numbers using a technique called embedding. Why? Because math is what AI understands.

So now, the AI is looking at rows and rows of numbers that represent words, their meanings, and how they relate to each other.

Step 3: Spotting Patterns in the Chaos

Now comes the interesting part: training.

The AI goes through billions of examples like:

“The ___ was delicious.” → Tries to guess “pizza,” “pasta,” “soup”
“The staff was ___.” → Guesses “kind,” “rude,” “friendly”

This process is done using massive models like transformers (more on that in future posts).

🤖 Real-World Analogy: Autocomplete on Steroids

Think about when you type a message and your phone suggests the next word.

Generative AI is doing something similar — but instead of just a few words, it can generate entire paragraphs, essays, poems, or code.

The core idea is still the same: predict what comes next based on everything it has learned before.

So What’s a “Model”?

You’ll hear this word a lot.

A model is what we get after training.

It’s the result — the thing that now “knows” how to respond based on the patterns it has seen. Just like a student who studied a thousand flashcards now knows all the answers.

Popular models include:

GPT (OpenAI)
Gemini (Google)
Claude (Anthropic)
LLaMA (Meta)

Each model is trained on huge amounts of data — books, articles, websites — and each learns slightly differently depending on how it was built.

Memory Without Memory

One mind-blowing thing: most AI models don’t have real memory.

They don’t remember your last conversation (unless specifically built to do so). They respond based only on the input you give right now, plus what they learned during training.

So if you ask it to write a pizza review, it doesn’t remember writing one earlier. It just sees the prompt and thinks:

“Based on what I know… what words should come next?”

🗺️ Zooming Out: What AI Isn't

Let’s clear up a common myth.

AI doesn’t think. It doesn’t feel. It doesn’t know truth.

It’s just really, really good at pattern prediction. That’s it.

So if it writes:

“This hidden sushi bar in Chicago has the best miso soup in the world.”

It may be totally made up — or based on something it once saw. That’s why accuracy is a big challenge in Gen AI.

Quick Recap

Here’s how Gen AI learns, using our restaurant review example:

Data In → Millions of real reviews
Tokenization → Break text into tiny, meaningful chunks
Embedding → Convert words to numbers
Training → Guess what comes next, adjust, repeat
Model Output → An AI that can now generate its own reviews

Pretty cool, right?

Bonus: Want to See It in Action?

Try writing:

“Write a review of a pizza place that only serves dessert pizza.”

into ChatGPT or Veo by DeepMind

Watch how it predicts a story based on everything it's ever seen.

What’s Next?

In the next post, we’ll explore embeddings and vector data — how AI turns words into numbers and how it finds the “closest match” when answering your questions.

We’ll explore it using a fun example: How an AI finds your favorite superhero based on personality. 🦸‍♂️

Got questions or want to see a specific example? Drop it in the comments!

Follow for the next post — we’re just getting started. 🚀

What is Generative AI? Like I’m 10!

Kedar Kodgire — Fri, 06 Jun 2025 02:21:50 +0000

Imagine you had a magic crayon that could draw anything you wanted — castles, dragons, or even a taco spaceship. Pretty cool, right? Now imagine you could just tell the crayon what to draw, and it would create it for you. That’s kind of what Generative AI does. It listens, learns, and then creates — just like magic, but powered by computers.
Let’s break it down and explore the wonder of Generative AI in a way that’s fun and super easy to understand:

What is Generative AI?

Generative AI (short for Generative Artificial Intelligence) is like a super-smart robot that can create things. Think of it as your creative buddy that can write wild stories, paint colorful masterpieces, compose music that sounds like it's from a movie, build fun little games, or even help you brainstorm amazing ideas for a science fair or school project — all in just seconds. And the best part? You can quickly try it out for yourself at
https://deepmind.google/models/veo/

But it doesn’t come up with these things out of thin air. It learns from a lot of examples — like reading a mountain of books, looking at millions of pictures, or listening to entire libraries of songs — and then it uses that knowledge to make something brand new.

How Does it Learn?

Think about how you learn new things. When you were smaller, you looked at picture books, listened to stories from your parents, and watched cartoons. That helped you understand shapes, colors, and how stories work.
Generative AI is kind of like that. It reads tons of information — way more than any person can — and it learns patterns. If it reads 1,000 bedtime stories, it figures out that most stories start with “Once upon a time” and end with “happily ever after.” This learning process is called training.
For example: if you show an AI 10,000 pictures of cats, it learns what a “cat” usually looks like. Then, if you ask it to draw a cat flying a plane, it can do that because it learned what cats look like and what planes look like.

What Can It Do?

Here are just a few of the amazing things Gen AI can do:
    •    Write you a bedtime story about a ninja potato who saves a village
    •    Create a painting of a dinosaur riding a skateboard through space
    •    Compose a silly song about your pet hamster’s birthday
    •    Translate your voice into another language in real-time
    •    Help you with homework, projects, or even write code (yep!)
It’s like having an extra-smart friend who’s always ready to help you create something fun.

Is It Like a Human 🧠?

Not quite. It doesn’t have feelings, dreams, or ideas of its own. It doesn’t get bored or excited. It just looks at patterns in data and tries to guess what comes next. But it’s super fast and can be surprisingly clever!
Let’s say you start a sentence with “The penguin wore a top hat and…” — Gen AI might guess you want to say “…danced on the moon.” That’s how it works: guessing the next word, picture, or sound based on everything it learned.

Why Should You Care?

Because Generative AI is becoming a part of everything! From the video games you play and YouTube videos you watch, to the chatbots you talk to and learning apps you use — AI is already quietly working behind the scenes. It helps make apps smarter, answers quicker, and creativity even more fun.
Knowing how it works helps you:
    •    Use it wisely and responsibly (like knowing when something might be fake)
    •    Stay safe online by understanding how AI might be used in ads or messages
    •    Be creative in exciting new ways, like making your own storybook or comic
    •    Maybe even build your own AI project one day — a game, an app, or a music generator!

Understanding this tech is like learning a new kind of superpower — one that mixes creativity and science. For example, imagine creating your own AI that draws a dragon eating spaghetti on the moon — you just describe it, and boom, it appears. Cool, right?
Of course, this power can be used for good or not-so-good. AI can help doctors spot diseases faster (good!), or it could be used to make fake videos that trick people (not good!). That’s why it's important to understand it — so you can use it for awesome things and spot when it’s being used in the wrong way.

In Simple Words:

    •    Generative AI = A robot that learns and makes cool stuff
    •    It reads and watches tons of data to get smart
    •    It uses what it learns to create art, stories, music, and more
    •    You can talk to it like you’d talk to a person, and it responds

Coming Up Next:
In the next post, we’ll explore how AI learns — step by step. Like teaching a robot to read, write, and draw using examples. We'll even walk through a fun example of how to teach an AI what a “sandwich” is using cartoons!

Got questions? Ask them in the comments. This blog is for curious minds like yours!

✨ About This Series: This blog series is all about making Generative AI easy to understand, even if you’ve never heard of it before. Whether you're 10 or just curious like a 10-year-old, I’ll walk through the big ideas using fun examples and simple words. You’ll go from “What is this stuff?” to “Whoa, I get it!” — one post at a time.

The Art of Semantic Software Versioning: A Tale of Numbers and Meaning

Kedar Kodgire — Thu, 27 Feb 2025 04:31:14 +0000

Picture this: You’re a weary traveler in the wild lands of software development. Your map? A tangled mess of code. Your compass? A version number like “2.1.7.” It doesn’t sound like much—just a string of digits separated by dots—but in the world of semantic versioning (or SemVer, as the cool kids call it), those numbers are a lifeline. They whisper tales of stability, chaos, or breaking changes lurking just around the corner. Today, I’m here to unravel this cryptic script, sprinkle in some storytelling flair, and make sense of the versioning saga that keeps developers sane (or at least, saner).

The Birth of SemVer

Imagine a bustling medieval marketplace—vendors shouting, wares clashing, chaos reigning supreme. That’s what software versioning looked like before SemVer rode into town. Developers slapped random labels on their releases: “v1-awesome,” “Release_Chaos_Edition,” or the dreaded “FinalFinalFINAL.”

No one knew what worked with what, and downstream users (poor souls) were left guessing whether an update would fix their bugs or torch their entire project.

Enter semantic versioning, stage left, with a simple yet elegant script: MAJOR.MINOR.PATCH. Three acts, one clear narrative. Introduced by Tom Preston-Werner (yes, the GitHub co-founder), SemVer brought order to the madness in 2013 with its official spec. It’s not just a naming convention—it’s a contract between developers and users, a promise etched in digits.

The Characters of the Version Number

Let’s break down our starring trio:

MAJOR (The Plot Twist): This is the big one. When the MAJOR number ticks up—say, from 2.3.1 to 3.0.0—you’re in for a seismic shift. Think of it as the moment in a script where the hero betrays the village, or the API you’ve been relying on gets a total rewrite. Backward compatibility? Out the window. Buckle up, because this update demands your attention and probably some frantic code refactoring.
MINOR (The Subplot): Here’s where the story gets richer without breaking everything. A bump from 2.3.1 to 2.4.0 signals new features—fresh subplots woven into the tale—but the old stuff still works. It’s like adding a quirky sidekick to the cast; they bring something new to the table, but the main arc holds steady. Developers love this part because it’s exciting yet safe.
PATCH (The Quick Fix): The unsung hero. From 2.3.1 to 2.3.2, this is the typo correction, the bug squash, the “oops, we forgot to close that door” moment. No new features, no grand upheavals—just a tighter, cleaner script. Users barely notice, but the story flows better for it.

The Plot Thickens with Pre-Releases and Build Metadata

But wait—the script has twists! SemVer doesn’t stop at three numbers. Ever seen something like “1.0.0-alpha” or “2.3.1+build.123”? These are the pre-release tags and build metadata, the director’s cut of versioning.

Pre-Releases (-alpha, -beta, -rc): These are the rehearsals before the big premiere. A version like 1.0.0-beta tells you it’s not fully baked yet—use it if you’re brave, but don’t expect a polished performance. Developers drop these to test the waters, gather feedback, and dodge pitchforks from users expecting perfection.
Build Metadata (+build.123): This is the behind-the-scenes trivia—stuff like commit hashes or timestamps. It doesn’t affect the version’s precedence (2.3.1+build.123 is equal to 2.3.1+build.xyz), but it’s handy for tracking exactly which 2.3.1 you’re dealing with. Think of it as the script’s production notes.

Why It Matters

So why should you care about this numerical drama? Because SemVer is the glue holding modern software ecosystems together. Take a library like React or a tool like Node.js—millions of projects depend on them. Without a clear versioning system, one rogue update could cascade into a blockbuster disaster, crashing apps worldwide. SemVer gives developers a shared language to signal intent: “This is safe,” “This adds goodies,” or “Heads up, rewrite incoming!”

It’s not just technical—it’s emotional. Ever felt that pang of dread when you type npm update or pip install --upgrade? SemVer tames that beast. A quick glance at the version number, and you know whether to sip your coffee calmly or brace for a late-night debugging session.

Semantic versioning isn’t perfect—humans still misjudge a “minor” bump or forget to tag a breaking change—but it’s a damn good script. As developers, we’re the playwrights, and our users are the audience. Stick to the SemVer plot, and you’ll keep the boos at bay. Ignore it, and you’re begging for a rotten tomato barrage.

So next time you bump that version number, channel your inner bard. Ask yourself: Is this a patch to polish the prose? A minor arc to dazzle the crowd? Or a major rewrite to burn the stage down and build anew? Write your version with purpose, and let the numbers tell a story worth reading.

Now, go forth and version wisely, dear developer. The world’s waiting for your next release!

The Hidden Language of the Web: A Deep Dive into HTTP Status Codes

Kedar Kodgire — Sat, 22 Feb 2025 04:09:17 +0000

Ever wondered what happens behind the scenes every time you load a webpage, click a link, or submit a form? Your browser and the web server engage in a silent conversation using HTTP status codes, three-digit messages that indicate whether a request was successful, redirected, or failed.

You’ve probably encountered a 404 Not Found error before, but there’s a lot more to these status codes than meets the eye. They help developers debug issues, improve website performance, and even affect SEO rankings.

Let’s break down the most important HTTP status codes, focusing particularly on client-side (4xx) and server-side (5xx) errors—and why they matter.

1xx: Informational – The Conversation Has Started

These codes are rarely seen by users but indicate that the request has been received and is still being processed.

100 Continue – The server acknowledges the request and is waiting for more information.
101 Switching Protocols – The client has requested a change in protocol (e.g., switching from HTTP to WebSockets), and the server has agreed.

2xx: Success – Everything’s Good

When you see a 2xx response, it means your request was successful, and the server has returned the requested content.

200 OK – The request was successful, and the response contains the requested resource.
201 Created – A new resource was successfully created (commonly used in APIs for new database entries).
204 No Content – The request was successful, but there is no content to return (used when an operation, like deleting a record, doesn’t require a response body).

3xx: Redirection – You’re Being Sent Somewhere Else

These codes indicate that the requested resource has moved, and the browser should redirect to a new URL.

301 Moved Permanently – The resource has been permanently moved to a new URL (important for SEO).
302 Found – The resource has been temporarily moved to another location.
304 Not Modified – The requested resource hasn’t changed since the last request, so the browser should use its cached version.

Understanding Client-Side (4xx) and Server-Side (5xx) Errors

When things go wrong on the web, the problem usually falls into one of two categories:

Client-Side Errors (4xx) → Issues caused by the user or their request.
Server-Side Errors (5xx) → Issues caused by the web server itself.

4xx: Client Errors – The User Messed Up

Client-side errors happen when there’s something wrong with the request. This could be due to a typo in the URL, missing authentication, or an unauthorized request.

Common Client Errors and Their Causes:

400 Bad Request → The request is malformed or contains invalid syntax.
- Example: Sending an API request with missing or incorrectly formatted data.
- Fix: Double-check the request syntax before submitting it.
401 Unauthorized → Authentication is required, but missing or incorrect.
- Example: Trying to access a restricted page without logging in.
- Fix: Provide the correct username and password or an API key if required.
403 Forbidden → The request was valid, but the server is refusing access.
- Example: Trying to view a file you don’t have permission to access.
- Fix: Ensure you have the right credentials or contact the website admin.
404 Not Found → The requested resource doesn’t exist.
- Example: Clicking on a broken link.
- Fix: Check the URL for typos or see if the page has been moved.
405 Method Not Allowed → The request method (GET, POST, PUT, DELETE) is not allowed for the resource.
- Example: Trying to use a POST request on a URL that only accepts GET requests.
- Fix: Use the correct request method as specified in the API documentation.
429 Too Many Requests → The user has sent too many requests in a short time (rate limiting).
- Example: Refreshing a webpage too frequently or making excessive API calls.
- Fix: Wait for a while before trying again or follow the API’s rate-limiting rules.

5xx: Server Errors – The Server Messed Up

Server-side errors indicate that something went wrong on the web server. Unlike 4xx errors, which are caused by the user, 5xx errors mean the problem is out of your control.

Common Server Errors and Their Causes:

500 Internal Server Error → A generic error message when something goes wrong on the server.
- Example: A coding bug in a website’s backend crashes the server.
- Fix: Website admins need to check error logs and debug their application.
502 Bad Gateway → A server acting as a proxy or gateway received an invalid response from an upstream server.
- Example: A CDN (like Cloudflare) fails to connect to the origin server.
- Fix: Check if the upstream server is down or misconfigured.
503 Service Unavailable → The server is temporarily down or overloaded.
- Example: A website crashes due to high traffic (like during a Black Friday sale).
- Fix: Wait and try again later. Web admins may need to optimize server resources.
504 Gateway Timeout → The server didn’t receive a response from an upstream server within the expected time.
- Example: A slow database query causes a timeout.
- Fix: Optimize queries, increase timeout limits, or scale the server infrastructure.

Why HTTP Status Codes Matter

Understanding HTTP status codes isn’t just for developers. They impact SEO, user experience, and website performance.

✅ For Developers → Helps debug issues and optimize API responses.

✅ For SEO → Google penalizes excessive 404 Not Found errors and rewards well-implemented 301 Redirects.

✅ For Users → Knowing why a 403 Forbidden error appears can save frustration when troubleshooting access issues.

Next time you encounter a 404, 500, or 503, you’ll know exactly what’s happening behind the scenes!

Want more tech insights? Stay tuned for more deep dives into the hidden mechanics of the web.

Understanding SLI, SLO and SLA.

Kedar Kodgire — Thu, 04 Jan 2024 02:48:53 +0000

SLI, SLO, and SLA are terms commonly used in service-level management to define and measure the performance and reliability of a system. These terms are often associated with key performance indicators (KPIs) that help organizations ensure their services meet specific standards. Let's break down each term and explore their connection:

SLI (Service Level Indicator)

Definition: SLI is a quantitative measure representing a specific aspect of a service's performance or dependability.

Purpose: It measures how well a service functions based on specific metrics, such as response time, availability, or error rate.

Example:

A web service's SLI might be the percentage of successful HTTP requests.
Request Latency - how long it takes to return a response upon request
Failure rate: Unsuccessful requests/ all requests

SLO (Service Level Objective)

Definition: SLO is a target or goal set for an SLI, specifying the acceptable level of performance or reliability for a service.

Purpose: SLOs define the expected level of service and aid teams and organizations in understanding what is considered a satisfactory performance benchmark.

Example

If the SLI is the percentage of successful HTTP requests, the SLO might be set at 99.9% availability over a given period.

Shooting for 100% reliability is not a good idea. The closer you get to 100%, the more difficult and expensive it is. Users usually don't need it to be 100% acceptable. Get a number where the customer is satisfied enough.

SLA (Service Level Agreement):

Definition: SLA is a standardized contract between a service provider and a customer summarising the desired level of service, including the agreed-upon SLOs and consequences for not meeting them.

Purpose: SLAs are contractual obligations that help manage expectations and provide a basis for accountability between service providers and consumers.

Example

An SLA might include commitments like "99.9% uptime" or "response time under 100 milliseconds."

Summary

In summary, SLI, SLO, and SLA are interconnected components in managing and assuring service performance and reliability. SLIs measure specific aspects, SLOs set performance targets, and SLAs formalize commitments and consequences, creating a structured approach to service-level management.

What is Master Data Management (MDM)

Kedar Kodgire — Mon, 01 Jan 2024 05:51:02 +0000

Master Data Management (MDM) typically involves creating a master record for core business entities - customers, products, employees, suppliers, and other foundational data. This record is also known as a golden record representing the best version of truth to ensure consistency, accuracy, and coherence across the enterprise.

Let's look at an example where we see the XYZ store, which has stores in different countries.

As you see, the customer Jon Doe has been registered in both stores. The customer has a different address, cell number and has_royalty_card status.

The marketing team may only refer to the Canada store where has_royalty_card status is set to N and email the customer to promote the card even if the customer is already a member. Similar issues can arise when the supply chain team refers to the products.

After the implementation, we will only have one golden record for each customer. In this case, the marketing team would have known that this customer was already a royalty card member, and they could have offered other new services via email in which the customer could be interested.

Benefits

Here are the high-level benefits of implementing MDM in the above example

Data Consistency
Ensures that all departments use the same, consistent customer data.
Elimination of Duplicates
MDM implementation Identifies and removes duplicate customer records.
Improved Decision Making
It provides accurate and reliable information for better decision-making; in other words, it improves the efficiency of models.
Compliance
Helps in meeting regulatory compliance by maintaining accurate and auditable customer data.

In conclusion, Master Data Management is the backbone of organizational success, ensuring data integrity, consistency, and reliability across diverse functions.

Cover Photo by Taylor Vick on Unsplash

What is Semantic Software Versioning?

Kedar Kodgire — Sun, 15 Oct 2023 04:08:18 +0000

Versioning is one of the essential elements to consider when publishing/using production-ready software. It helps us understand and maintain different varieties of software produced. It is also useful when you want to roll back to the previous software version in case of any problems.

Semantic versioning has a particular syntax. Let's look at the syntax first.

[major].[minor].[patch]

Here,

The Major increments when significant changes in your application break the backward compatibility.
The Minor increments when new features are introduced, supporting backward compatibility.
And the Patch adds up if there are any backward-compatible bug fixes.

While developing any software, we might have specific dependencies. For example, we might use react-bootstrap, Axios, react-scripts, etc. Keeping track of stable versions/new features of individual software can be challenging, especially when the number of dependencies is growing. On this occasion, we can take advantage of the software version scopes so that if new software is released, you can use it only if it's available in the scope. This way, you can ensure that your software doesn't break due to dependencies.

Let's look at how to use these ranges.

Comparators

A version range is a set of comparators that specify versions that satisfy the degree. A comparator consists of an operator and version. The collection of primitive operators is as follows:

< Less than
<= Less than or equal to
> Greater than
>= Greater than or equal to
= Equal. This operator is the default, so it is optional, but We can include it.

Examples:

The comparator >=1.2.7 matches the versions 1.2.7, 1.2.8, and 1.3.9, but not the versions 1.2.5 or 1.1.1.
The range >=1.2.7 <1.3.0 would match the versions 1.2.7 and 1.2.99, but not1.2.6, 1.3.0, or 1.1.1.

We can also join by the || (or) operator.

The range 1.2.7 || >=1.2.9 <2.0.0 would match 1.2.7, 1.2.9, and 1.4.6, but not the versions 1.2.8 or 2.0.0.

Exact Version

You can specify an exact version number, such as 1.2.3. This means your project will only use version 1.2.3 of the dependency.

Hyphen Ranges

Hyphen ranges specify the inclusive set. For example, If we define the range 3.4.5 - 5.6.7, it will be equivalent to >=3.4.5 <=5.6.7.

For the partial versions, 0 is considered for the missing place.

For example, 2.3 - 5 means >=2.3.0 <=5.0.0

X-Ranges

X-Ranges are similar to the partial versions, but we use X or x or * instead of blank spaces.

For example,

1.x will be >=1.0.0 <2.0.0
1.2.x will be >=1.2.0 <1.3.0
Similarly, * will be >=0.0.0

Tilde ranges

Tilde ranges allow the patch level changes if minor and major are present. For example, if you provide ~1.2.3 will allow this range >=1.2.3 <1.3.0.

If the minor is missing, then minor level changes are allowed. For example, if you specify ~1, it will enable releases ranging >=1.0.0 <2.0.0.

Caret Ranges

Caret will update the package to all later minor/patch versions without incrementing the major version. For example ^2.3.4 will use releases ranging >=2.3.4 <3.0.0

You can also combine x-ranges in this. For example, ^0.x will allow >=0.0.0 <1.0.0

These version range operators allow developers to express their requirements for dependencies more flexibly and ensure that their projects can adapt to updates in the dependencies while still adhering to the principles of Semantic Versioning. This flexibility is essential because it helps maintain compatibility while allowing for updates and bug fixes in the dependencies.

What are the best practices while using BigQuery?

Kedar Kodgire — Sat, 19 Feb 2022 11:35:37 +0000

BigQuery is affordable and fast, but it can quickly consume a lot of processing power, increasing the bills if not used properly. In BigQuery, slots decide the amount of processing power that directly impacts cost. First, let's understand slots.

Slots

In BigQuery, slots are the computational capacity required to execute a SQL query. Slots play a crucial role in pricing and resource allocation. BigQuery manages the slot allocation, and the number of slots allocated to a job depends on

Query size: The amount of data processed
Complexity: The amount of information shuffled

In supplement to consumption-based pricing, BigQuery also provides flat-rate pricing, that is, you can buy a fixed number of slots over some period which provides more granularity in planning capital.

BigQuery is fast because it uses heavy parallel processing by converting the declarative SQL into stages.

Without further delay, let's see the best practices in BigQuery for:

Cost controls
Query performance
Storage optimization

Cost controls

BigQuery is a columnar database. In other words, all columns from a table are stored separately at the physical level. Hence, we should always avoid using select * in our queries. This practice will limit the data processed.
For looking at the sample data, we should use the preview option instead of the query. The preview feature in BigQuery is free.
We should use a pricing calculator to check the costs before pushing the query to production. On-demand queries are charged based on the number of bytes read, and we can calculate the price based on bytes read.
We should use dashboard for viewing costs and logs. The dashboard can provide us with good insights on our usage of BigQuery.
Partitioning in BigQuery allows us to scan only the required data and reduce costs.

Query performance

We should avoid querying non-required partitions.
We should use external data sources suitably. There's no guarantee of good performance when using external sources.
We should avoid excessive use of wildcard tables, for example, FROM bigquery-public-data.noaa_gsod.good*.
JavaScript user-defined functions reduce overall performance. We should avoid them.
Certain joins, such as a cartesian product, can create more output than input, and should be avoided.
We should avoid DML statements with updating or inserting single rows.

Storage optimization

The expiration time for datasets/tables/partitions should be set. It allows us to control storage expenditures and optimize the usage of BigQuery storage.
Using BigQuery to process the data, we can also use it to store in tables and partitions for an extended period. It has lower charges for the data, which is not modified in the last 90 days.
Lastly, we can use the pricing calculator to check storage costs.

That's all for this post folks.
Don't forget to ❤ or 📑 this article if you enjoyed it.

Have a great day.
Happy Learning!!

What is horizontal and vertical scaling?

Kedar Kodgire — Thu, 30 Sep 2021 18:13:25 +0000

Scaling implies adding or removing resources from your machine/application to perform the underlying tasks with optimal cost and processing capability. Scalability is a challenge that every engineering team needs to go through.

Various scenarios require an application to:

Grow/shrink in requests
Increase/decrease in data
Reduce processing power

At this point, you will have two scaling options:

Horizontal scaling
Vertical scaling

Horizontal scaling (scaling out)

Horizontal scaling implies adding more machines to the existing system. The data is scattered across multiple machines, and each of them has its own capacity. As we are not modifying existing machines, this process involves less downtime.

This method allows us to perform distributed programming, which entails distributing jobs across devices. Horizontal scaling can increase the complexity as the address space increases. Therefore, updating and sharing data across machines can be more expensive.

Vertical scaling (scaling up)

Vertical scaling implies attaching more resources to the existing machine. Let’s consider a server rack, as before. We add more things like RAM, for example, to the same server rack in this method. The data resides on the same machine and is not distributed as in horizontal scaling. Usually, the activities performed on these machines use multi-threading and in-process data passing methods. Vertical scaling will have limited capacity within the existing machine. Scaling beyond this capacity will cause downtime.

Let’s look at the key differences between these methods:

	Horizontal Scaling	Vertical Scaling
Maintainance	Maintainance is complex as you will need to manage a lot of machines.	Maintenance is cheaper and it is less complex because of the number of nodes you will need to manage.
Costs	Initial costs are high but buying a new machine with low processing power is more affordable.	Adding a new machine is far more expensive than upgrading old ones.
Fault Tolerance	In case of failure in a machine, others can still provide the service.	Failures will lead to loss of service.
Communication Complexity	Having multiple machines requires complex protocols for exchanging data between them.	Data exchange becomes relatively straightforward as we only have one machine.
Load Balancing	Traffic/programming tasks can be distributed between the machines.	Since we have one device, tasks can't be spread. Some level of parallel processing is achievable using a multi-threading programming model, but it's limited to the machine's capacity.
Examples	Cassandra, Google Cloud Spanner	MySQL

That's all for the post guys. Hope you enjoyed it, don't forget to leave a like if you did :).

Happy Learning.

What is overfitting in machine learning?

Kedar Kodgire — Sat, 11 Sep 2021 18:25:19 +0000

Overfitting is a common challenge to overcome while training machine learning models.

To understand this concept, let’s consider a plane with an X and Y-axis. The X-axis represents the complexity of your model, and the Y-axis represents the loss function.

Explanation

In the diagram above, the blue line represents the training set and the orange line represents the validation set.

The training set

Let’s consider the blue line.

As the complexity of the model increases, the loss decreases, and vice versa.

The validation set

Now, let’s consider the orange line.

As we keep moving from left to right, the loss falls until a certain point. If you keep moving right from that point, the loss increases.

The point from which the loss increases is the minimal loss for the validation set.

Balanced model

Before this point, a model is considered under-fitted, and after this point, the model is deemed to be over-fitted. This level of complexity represents the balanced model.

To better understand the balanced model, let’s consider some data points to represent these three fittings.

Explanation

In the diagrams above, the red data points are the ones we use to train the model. This means they were already available.

The green data point is newly introduced, and we test it against our model.

If the green data point is introduced in the balanced machine learning model, it will be close to the plotted line, indicating greater accuracy.

Underfit model

In Fig. A, our ML model has plotted a straight line against the data points. The line crosses through a few data points, and other data points are further from the plotted line.

When a new data point is introduced, it is far from the line. This indicates that the model is not accurate.

Overfit model

The model is fitted exceptionally well, as seen in Fig. B because the plotted line passes through all the training data points.

This model is entirely accurate for the training data. However, introducing a new data point shows that the model may not work well for new data, as it is far away from the plotted line.

Balanced model

In Fig. C, which is our balanced model, the plotted curve passes through some of the data points. If we calculate the loss, this model would have the most minor loss out of all the models.

The introduction of a new point is near the plotted curve. Hence, when the model is used with actual/production data, our predictions will be reasonably accurate.

That's all for this post guys,
If you enjoyed it, don't forget to leave a like.
Happy Learning.

What is nohup in Linux?

Kedar Kodgire — Sun, 15 Aug 2021 19:23:29 +0000

When we execute a command in Linux, it creates our process in the background and assigns a unique process id. This process can end in three ways.

The task assigned is completed
You kill the process explicitly
You log off, or in the case of SSH, the connection drops, and the session is terminated

Imagine you are running a critical process, and you might need to log off urgently or the connection drops, the process stops immediately, and you might lose your work.

To avoid this, we can use the nohup command. nohup in Linux executes other commands specified in the arguments. This command ignores all SIGHUP (hangup) signals. SIGHUP is sent to a process when its controlling terminal is closed.

To understand better, let's see the syntax.

nohup COMMAND [ARGS]

Running command in the foreground

The nohup will run in the foreground, and the output of the command will be stored in the nohup.out file. nohup command will create this file in the current directory. If the user doesn't have sufficient permissions to write a file in the current directory, nohup will generate the output file in the home directory.

Let's look at example output.

root@kedar:/# MYCOMMAND="echo 'My name is Kedar.'"

root@kedar:/# nohup $MYCOMMAND
nohup: ignoring input and appending output to 'nohup.out'

When we cat the contents of the output file, we see this.

root@kedar:/# cat nohup.out 
'My name is Kedar'

We have used a small command, so it executes immediately. For an extensive process, you can log out at this point, and the process will still run.

Running command in the background.

Well, we can also run this command background. To run it in the background, we use the & operator.

If we run the same command in the background, we get the following output.

root@kedar:/# MYCOMMAND="echo 'My name is Kedar.'"
root@kedar:/# nohup $MYCOMMAND &
[1] 24
root@kedar:/# nohup: ignoring input and appending output to 'nohup.out'

The integer you see in the output is a process id.

You can also kill this background process using process id.

kill -9 24

That's all for this post, guys. Hopefully, now you have an idea about the nohup command.

if you enjoyed it, don't forget to ❤ or 🔖.

Happy learning.