DEV Community: Michellebuchiokonicha

I have been struggling to write…

Michellebuchiokonicha — Sat, 28 Feb 2026 08:22:51 +0000

I have been writing technical articles since 2022. My passion for technology has been built on writing. I have loved it even before I realized I wanted to go into software development. I used to write poems that got published while in school, so writing technical articles when I started coding was not a big deal for me. I have always written.

In fact, my parents used to say I would be like Chimamanda Adichie. Because not only did I know how to coin words, I knew how to express them in speech, and I also loved to read, which helped me greatly in writing with ease. At some point, while in secondary school, I considered becoming a writer and studying Literature just so I could write and speak. It always gave me joy.

But recently, I find myself struggling to do one of the things I would normally term ‘easiest’ to do.

This struggle started creeping in gradually in 2024. When ChatGPT came on board.
I still continued writing, and I had my articles featured on Dev. to amongst the 7 most read articles.

Some of my articles climbed to 40,000 reads, numerous comments, and likes. And in 2025, my new article also got featured on hasnode but in the heat of AI and arguments on AI being able to do everything, I wrote only two articles, and I paused.

I was confused, I was in doubt, I had questions: “Will people need to read articles on asynchronous functions, supervised learning, JavaScript, Python, and more, if AI can simply provide all the information?”

Secondly, I was confused because I have been a web developer all this time. Mostly writing and building frontend applications, hence most of my articles covered the web and JavaScript. But with this new excitement, I didn’t just want to stick only to the web; I wanted to branch into machine learning. This also made me pause to reflect. What started as a short-term pause in February 2025 turned into a full year of not writing any technical articles. I feel like crying as I type this. It hurts me more than I ever thought it would.

I have many excuses. Another would definitely be that I want to start a YouTube channel and be active on social media instead of writing. But I have asked so many questions. Can I edit videos? Do I really want to keep talking and explaining technology concepts instead of simply penning them down as articles? And right now, it has dawned on me. I don’t know how consistent I can be with explaining concepts via speech as opposed to writing them down, which is easier and smoother for me.
I have been stalling, waiting, postponing this, telling myself I am too busy, telling myself AI can explain everything, telling myself YouTube is better now, as people don’t read anymore. But it feels like something is missing. Like, I am not doing one of the things I am meant to be doing, and it hurts, it breaks my heart so much.

This is my first piece in one year, and truly, this morning, 7:50 am, 22nd February, 2026, thinking deeply about what could have been if I had not stopped writing, I decided to just write.

I have no will for any aspire to perspire, but I guess we will all figure it out. The questions, the doubts. If you are like me, then write.

JUST WRITE.

Infrastructure as Code: Getting Started with Terraform for Cloud Deployment

Michellebuchiokonicha — Fri, 14 Feb 2025 06:55:29 +0000

Introduction to Cloud Services

Cloud computing has changed the way we deploy and manage applications. It offers various service models to meet different needs.

Infrastructure as a Service (IaaS): Provides on-demand access to computing resources like servers, storage, and networking.

Platform as a Service (PaaS): Offers hardware and software resources for cloud application development, eliminating the need to manage the underlying infrastructure.

Software as a Service (SaaS): Delivers full application stacks as a cloud service, including maintenance and management of both infrastructure and software.

Benefits of Cloud Deployment

Using the cloud for computing and deployment comes with several advantages:

Faster Time to Market: Servers can be deployed with just a few clicks, significantly accelerating the development process.

Cost Savings: By choosing the right set of services, infrastructure costs can be drastically reduced.

Data Loss Prevention: Data is stored across multiple data centers in different availability zones, ensuring redundancy and reliability.

Challenges of Cloud Deployment

While the cloud offers many benefits, there are some challenges to consider:

Vendor Lock-In: Switching cloud providers can be costly and complex, often leaving customers tied to their original vendor.

Lack of Expertise: Finding skilled cloud professionals can be difficult, as the demand for cloud expertise often outstrips supply.

Major Cloud Providers

The three largest cloud providers dominate the market:

AWS (Amazon Web Services): Holds the largest market share in the US and offers a wide range of services.

Azure: Known for its seamless integration with Microsoft products and a strong presence in Germany.

GCP (Google Cloud Platform): This has seen rapid growth over the past three years and integrates well with Google products like Google Analytics.

Deploying to the Cloud

Deploying applications in the cloud involves using a variety of services. While you can manage these services manually through the cloud console, this approach has limitations:

Complexity: Dropping or modifying services can be tedious, requiring you to navigate through multiple menus.

Lack of Replicability: Manual configurations are hard to replicate, making it difficult for others to understand or reproduce your setup.

This is where Infrastructure as Code (IaC) comes in. IaC allows you to define and manage your infrastructure using code, making it replicable, understandable, and easier to maintain.

What is Infrastructure as Code (IaC)?

Infrastructure as Code (IaC) is the practice of managing and provisioning infrastructure through code instead of manual processes. This approach has become increasingly popular, with IaC skills now appearing in job postings for roles like Data Analysts.
Popular IaC tools include:

Terraform
AWS CloudFormation
Azure Resource Manager
Google Cloud Deployment Manager

Among these, Terraform stands out due to its ability to work across multiple cloud providers, making it a versatile choice for organizations that may migrate between clouds.

Why Terraform?

Terraform is an open-source IaC tool that allows you to build, change, and version infrastructure safely and efficiently. Here’s why it’s a great choice:

Multi-Cloud Support: Terraform supports multiple cloud providers, including AWS, Azure, and GCP.

Rollback Capabilities: Infrastructure changes can be easily rolled back, minimizing downtime and risks.
Ease of Use: Terraform uses a declarative language, making it easy to learn and use. You only need a few commands to get started.

Cost Efficiency: You can stop and resume your infrastructure as needed, saving on cloud credits and usage.

Setting Up Terraform

To get started with Terraform, you’ll need:

A Cloud Provider Account: For this guide, we’ll use AWS.

Terraform Installed: Download and install Terraform from the website: https://www.terraform.io/

Example: Setting Up an S3 Bucket in AWS

Here’s an example of how to set up an S3 bucket using Terraform:

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "example_bucket" {
  bucket = "my-unique-bucket-name"
  acl    = "private"

  versioning {
    enabled = true
  }

  tags = {
    Name        = "Example Bucket"
    Environment = "Dev"
  }
}

This code defines an S3 bucket with versioning enabled and tags for identification. You can customize it further based on your needs.
Conclusion

you need to further set up:

provider
aws_s3_bucket
aw3_s3_bucket_public_access_block
aws_s3_bucket_ownership_controls
aws_s3_bucket_acl
aws_s3_object
aws_s3_bucket_website_configuration

Here is a simple setup

provider "aws" {
    region = "us-east-1"
}

resource "aws_s3_bucket" "s3" {
    bucket = "your-bucket"
}

resource "aws_s3_bucket_public_access_block" "s3-public-block" {
    bucket = aws_s3_bucket.s3.id

    block_public_acls       = false
    block_public_policy     = false
    ignore_public_acls      = true
    restrict_public_buckets = false
}

resource "aws_s3_bucket_ownership_controls" "s3-ownership" {
    bucket = aws_s3_bucket.s3.id

    rule {
        object_ownership = "BucketOwnerPreferred"
    }
}

resource "aws_s3_bucket_acl" "s3-acl" {
    bucket = aws_s3_bucket.s3.id
    acl    = "public-read"
}

resource "aws_s3_object" "s3-object" {
    bucket       = aws_s3_bucket.s3.id
    key          = "index.html"
    source       = "index.html"
    content_type = "text/html"
    acl          = "public-read"
}

resource "aws_s3_bucket_website_configuration" "s3-website-configuration" {
    bucket = aws_s3_bucket.s3.id

    index_document {
        suffix = "index.html"
    }
}

with this simple setup, you can deploy your first web application.
I have further included a sample index.html file

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Welcome to My Website</title>
    <style>
        body {
            font-family: Arial, sans-serif;
            margin: 0;
            padding: 0;
            background-color: #f4f4f4;
            color: #333;
            text-align: center;
        }
        header {
            background-color: #007bff;
            color: white;
            padding: 20px 0;
        }
        h1 {
            margin: 0;
        }
        main {
            padding: 20px;
        }
        footer {
            background-color: #333;
            color: white;
            padding: 10px 0;
            position: fixed;
            bottom: 0;
            width: 100%;
        }
    </style>
</head>
<body>
    <header>
        <h1>Welcome to My Website</h1>
    </header>
    <main>
        <p>Hello, world! This is a simple webpage hosted on AWS S3.</p>
        <p>Enjoy browsing!</p>
    </main>
    <footer>
        <p>&copy; 2025 My Website</p>
    </footer>
</body>
</html>

To install terraform, use:

terraform -v

then open your IDE and run terraform init

To format and validate your terraform code, run terraform fmt and
terraform validate

Next, you would need to plan and make changes with
terraform plan

To apply the changes made, run terraform apply -auto-approve

Lastly, in order to save bandwidth and when you are not using the infrastructure at a certain time, you can destroy it with

terraform destroy -auto-approve

You can further rerun/rebuild by running
terraform apply

The process is quite seamless and easy to use and understand. it saves you time, provides flexibility, and also allows other developers to see what you have done.

Infrastructure as Code, particularly with tools like Terraform, simplifies cloud deployment and management. It ensures that your infrastructure is replicable, understandable, and easy to maintain. Whether you’re working with AWS, Azure, or GCP, Terraform provides a consistent and efficient way to manage your cloud resources.
Start your IaC journey today and experience the benefits of deploying infrastructure as code!

Python OOP Made Easy: Master Object-Oriented Programming with Ease

Michellebuchiokonicha — Tue, 04 Feb 2025 07:38:22 +0000

Object-Oriented Programming (OOP) is a programming paradigm that uses objects and classes to structure software. Unlike procedural programming, which focuses on functions and procedures, OOP emphasizes using objects that encapsulate data and behavior. Python, a versatile language, fully supports OOP, making it a powerful tool for building modular, reusable, and maintainable code.

In this article, we’ll explore the basics of OOP in Python, including classes, objects, attributes, methods, inheritance, polymorphism, encapsulation, and abstraction. We’ll also dive into functional programming concepts and how they complement OOP in Python.

Python OOP Basics

What is OOP?

OOP is a programming paradigm that organizes code into classes and objects. A class is a blueprint for creating objects, while an object is an instance of a class. OOP promotes concepts like modularity, reusability, and abstraction, making it easier to manage complex systems.

Pros of Object-Oriented Programming

Modularity: Code is organized into reusable components.

Abstraction: Hide complex implementation details.

Maintenance: Easier to update and debug.

Reusability: Classes can be reused across projects.

Cons of Object-Oriented Programming

Complexity: This can be overkill for simple programs.

Performance: Slightly slower due to overhead.

Overuse: Not every problem requires an OOP solution.

Classes and Objects in Python

What are Classes and Objects?

A class is a blueprint for creating objects. It defines the attributes (data) and methods (functions) that the objects will have. An object is an instance of a class.

Example:

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

    def say_hi(self):
        print(f"Hi, my name is {self.name} and I’m {self.age} years old.")

# Creating objects
person1 = Person("Alice", 25)
person2 = Person("Bob", 35)

person1.say_hi()  # Output: Hi, my name is Alice and I’m 25 years old.
person2.say_hi()  # Output: Hi, my name is Bob and I’m 35 years old.

What Can Go Wrong?

Python is flexible, but this can lead to mistakes. For example:

person3 = Person(name="Charlie", age=person1)
person3.say_hi()  # Output: Hi, my name is Charlie and I’m <__main__.Person object at 0x109f301c0> years old.

Here, age is mistakenly assigned an object instead of a number. Always validate inputs!

Attributes and Methods

Attributes

Attributes define the state of an object. They store data associated with the object.

Methods
Methods define the behavior of an object. They are functions attached to a class.

class Car:
    wheels = 4  # Class attribute

    def __init__(self, make, model, year=None):
        self.make = make  # Instance attribute
        self.model = model
        self.year = year

    def get_make_model(self):  # Instance method
        return f"{self.make} {self.model}"

car1 = Car("Toyota", "Camry", 2021)
print(car1.get_make_model())  # Output: Toyota Camry

Inheritance in Python

Inheritance allows a class (child) to inherit attributes and methods from another class (parent). This promotes code reuse and hierarchy.

Example:

class Animal:
    def __init__(self, name):
        self.name = name

    def speak(self):
        print("This animal speaks.")

class Dog(Animal):
    def speak(self):
        print("Woof!")

dog = Dog("Pelle")
dog.speak()  # Output: Woof!

Polymorphism, Encapsulation, and Abstraction

Polymorphism

Polymorphism allows objects of different classes to be treated as objects of a common superclass. For example:

def make_animal_speak(animal):
    animal.speak()

dog = Dog("Fido")
cat = Cat("Whiskers")

make_animal_speak(dog)  # Output: Woof!
make_animal_speak(cat)  # Output: Meow!

Encapsulation

Encapsulation hides the internal state of an object and restricts direct access. Use private attributes (__) to enforce this:

class BankAccount:
    def __init__(self, account_number, balance):
        self.__account_number = account_number
        self.__balance = balance

    def deposit(self, amount):
        self.__balance += amount

    def get_balance(self):
        return self.__balance

Abstraction

Abstraction focuses on hiding non-essential details and exposing only the necessary features. Use abstract classes to define interfaces:

from abc import ABC, abstractmethod

class Shape(ABC):
    @abstractmethod
    def area(self):
        pass

class Rectangle(Shape):
    def __init__(self, length, width):
        self.length = length
        self.width = width

    def area(self):
        return self.length * self.width

Functional Programming in Python

Python also supports functional programming (FP), which emphasizes immutability and pure functions. Key concepts include:

Pure Functions: No side effects, deterministic.

Higher-order functions: Functions that take other functions as arguments.

Immutability: Data cannot be changed after creation.

Example:

# Higher-order functions
numbers = [1, 2, 3, 4, 5]
squared_numbers = list(map(lambda x: x**2, numbers))  # Output: [1, 4, 9, 16, 25]

Conclusion

OOP and FP are powerful paradigms that can be used together to write clean, modular, and maintainable code in Python. By understanding classes, objects, inheritance, polymorphism, encapsulation, and abstraction, you can build robust applications. Additionally, functional programming concepts like immutability and higher-order functions can complement OOP to make your code even more expressive.

Whether you’re building a small script or a large-scale application, mastering these concepts will help you write better Python code.

Are Web APIs the same as REST APIs? How to Improve your Web Apps with Web APIs

Michellebuchiokonicha — Tue, 14 Jan 2025 08:36:11 +0000

To answer the question simply: No, Web APIs are not the same as REST APIs.

Web APIs refer to interfaces that allow applications to interact with the web or devices. This term is broad and can refer to Browser APIs (like WebRTC or Geolocation) or Server APIs (like REST or GraphQL).

On the other hand, REST APIs are a specific type of Server API that adhere to REST principles, focusing on statelessness, resource-based communication, and using HTTP methods like GET, POST, and DELETE.

The Evolution of the Web

Originally, the web was designed as a static site for providing information and connecting individuals to relevant resources, regardless of location. However, it has evolved to become interactive and progressive, with certain aspects also turning decentralized.
This article focuses on the web being progressive and more user-friendly.

The Role of Web APIs in Modern Development

Web APIs serve as the backbone of modern web development. With APIs like WebRTC, Web Bluetooth, and Service Workers, developers can create highly interactive, native-like web applications without requiring external plugins. These capabilities make Progressive Web Apps (PWAs) possible, bridging the gap between web and native apps.
Check out my extensive article on progressive web apps.
https://dev.to/michellebuchiokonicha/how-to-get-started-with-progressive-web-apps-3f44

Some Benefits of Progressive Web Apps

Works on any browser (cross-platform)
Responsive to any device
Functions without network connectivity
Performs like a mobile app
Highly secure
Discoverable via search engines
Boosts app engagement
No installation required
Can be shared through a link
Optionally available on app stores (e.g., Google Play, Apple Store)

Web APIs and Project Fugu

Web APIs here, referring to browser APIs, act as connectors, improvers, and builders for the browser's capabilities. These APIs are inherent to the web and are not third-party APIs purchased from libraries or sites. Instead, they are broadly available and designed to improve the web and enhance products created by developers, reducing boilerplate code and simplifying web interactions.
The significance of web APIs is underscored by the fact that over 70% of activities conducted online occur through the web, highlighting the need for improvements and scalability.

In summary, web APIs are crucial for the success of Progressive Web Apps and the future of web development. They act as enablers of advanced functionalities.

Project Fugu

Project Fugu derives its name from the Japanese word for pufferfish, "fugu."
Initially, I was curious about why this project was named after a fish until I learned more about it. The fugu is a dangerous fish that can be delicious when prepared correctly; however, if not handled properly, it can be harmful. This metaphor aptly reflects the project's goal of enhancing the web platform's capabilities to match those of native applications.
The correlation here is that the Project Fugu APIs aim to enrich the web (the ‘delicious’ aspect), and if they are not built correctly (the ‘preparation’), they can introduce significant security vulnerabilities. Imagine creating an API accessible to billions without adequate security measures; it would expose individuals to various security risks.

You can find more about Project Fugu and its APIs at https://developer.chrome.com/docs/capabilities/fugu-showcase. Here, you will discover interesting and exciting projects built with web APIs, as well as a comprehensive list of available APIs. Many native app features that were previously unavailable on the web are now accessible, thanks to web APIs.

Why Web APIs?

Web APIs are crucial for several reasons, and their importance should not be overlooked. They empower the browser, transforming the web from a simple content viewer into a powerful application platform. With APIs like WebRTC, WebSockets, Service Workers, Web Bluetooth, and Web XR, you can now accomplish tasks on the web that were once limited to native applications.

Why Should You Start Using Web APIs?

Web APIs offer numerous benefits, including:

Access to Native Device Features: They allow you to access device hardware such as cameras, microphones, sensors, and Bluetooth devices. This means you can make video calls, control robots, and experience augmented reality directly in your browser, without needing to download any apps.

Offline and Background Functionality: Service Workers enable features like push notifications and caching, enhancing the user experience even when offline.

Simplified App Development for Cross-Platforms: Web APIs facilitate easier development across different platforms.

How Web APIs Improve the Web

Web APIs bring the capabilities of native apps to the web, granting access to powerful features without requiring installations or downloads. They also help future-proof the web; as browsers evolve, more APIs will be developed for seamless integration. Additionally, web APIs are accessible across platforms.

Is it Easy to Get Started with Web APIs?

Yes, it is straightforward. Most web APIs are already integrated into browsers, and supported by robust documentation. Since many are native to the browser, there's no need for third-party libraries or installations. This makes getting started seamless, consistent, and accessible across platforms.

Some Web APIs

Web APIs are the cornerstone of the Progressive Web App (PWA) revolution. They form the backbone of improved web functionality and represent the future of web development. Numerous Web APIs are available, and you can find a comprehensive list in the MDN documentation or on web.dev. Below are about 10 Web APIs to help you understand how they work. You can find many more in the MDN docs,web.dev, or via the project fugu documentation.

WebRTC API: Enables real-time audio, video, and data communication directly between browsers without the need for external plugins. It is commonly used for video conferencing and peer-to-peer file sharing.

Web Bluetooth API: Allows websites to communicate with Bluetooth-enabled devices, such as fitness trackers, medical devices, and IoT gadgets.

WebXR API: Provides support for immersive technologies like Virtual Reality (VR) and Augmented Reality (AR), allowing developers to create engaging 3D experiences directly in the browser.

WebSocket API: Facilitates full-duplex communication between the client and server, making it ideal for real-time applications like chat systems, live streaming, and collaborative editing.

Push Notification API: Enables websites to send timely and relevant notifications to users even when the website is not open in the browser, enhancing user engagement.

Web Share API: Provides a standardized way for web apps to share content using native sharing mechanisms, making it easier to share links, files, or text with other applications.

Fetch API: Replaces the older XMLHttpRequest and simplifies asynchronous data fetching.

Web Payment API: Simplifies the online payment process by offering a consistent interface for users to make payments using various methods, such as credit cards or digital wallets.

Geolocation API: Provides access to the geographical location of the user, enabling location-aware functionality like maps, location-based recommendations, or geofencing.

File System Access API: Enables web applications to read from and write to the user’s local file system, creating a native-like experience for file manipulation.

WebRTC Demo

WebRTC enables real-time communication, including audio, video, and data-sharing capabilities. For a simple WebRTC demo, you will need:

index.html

app.js

server.js

WebSocket or HTTP REST API
Inbuilt WebRTC API
Local and remote video elements
Buttons to manage peer connections
WebRTC configurations

How It Works

Access User's Camera and Microphone: Create a function to access the user's camera and microphone, displaying the local stream on the local video element.

Peer Connection: Implement a function to handle media exchange.

Signaling: Simulate sending and receiving data through the signaling server.

Remote Stream Handling: Display the remote track in the remote video element.
ICE Candidates: Exchange these candidates with peers to establish a connection.

Web APIs are the cornerstone of modern web development, enabling developers to build powerful, scalable, and engaging applications. Whether you're exploring WebRTC for real-time communication or leveraging the File System Access API for native-like experiences, the possibilities are vast. Start experimenting today and contribute to the web's progression.

How to Automate Excel Files from APIs with Python and Openpyxl.

Michellebuchiokonicha — Fri, 16 Aug 2024 06:30:41 +0000

Automation in programming is the use of programming languages like Python, VBA, and other technology tools to create programs, scripts, or various tools that perform automatic tasks with no manual intervention.
The sole purpose of automating tasks is to avoid manual input and to ensure various systems run by themselves.

For this article, I will provide a step-by-step guide on how I automated an Excel file, and different sheets on a MacBook, without visual basic for applications, using Python in this case.

First of all, to get started, you don't need to be a Python dev as I will paste a code snippet here.

Tools Required

VScode of course
Python installed/updated
A virtual environment to run any new installation or updates for your Python code.
The virtual environment is the .venv. You will see it in your vscode.
Install openpyxyl
Install any other necessary dependency.
Get started.

The Different Aspects we will be considering:

Creating a new Excel file with python
Updating an existing Excel file with python Updating a specific Excel file sheet only with Python
Using APIs to update Excel files and Excel file sheets.
Creating a button that allows users to update on click.
Adding dynamic dates and time in your code
An alternative to the Excel button is cron or Windows shell
Instead of VBA, what else is possible?
Issues faced with writing VBA in a MacBook
Issues I faced while creating the button
Why I opted for cron
Creating this for both Windows and Mac users
Other tools that can be used for the automation of Excel
Power query from web feature
Power automate
Visual Basic in Excel

Creating a new Excel file with python

Creating an Excel sheet in Python with openpyxl is easy.
All you need to do is install openpyxl, pandas, and requests if you are getting data from an API.
Go to the openpyxl documentation to learn how to import it into your application and the packages you want to use.

import pandas
import requests
from openpyxl import Workbook, load_workbook
from openpyxl.utils import get_column_letter

Next up,
you create a new workbook
Set it as the active workbook
Add your title and header and populate the data
Save the new workbook with your preferred Excel name and tada!
you have created your first Excel file.

# create a new workbook
wb = Workbook()
ws = wb.active
ws.title = "Data"

ws.append(['Tim', 'Is', 'Great', '!'])
ws.append(['Sam', 'Is', 'Great', '!'])
ws.append(['John', 'Is', 'Great', '!'])
ws.append(['Mimi', 'Is', 'Great', '!'])
wb.save('mimi.xlsx')

Creating a new sheet in an Excel file.

Creating a specific sheet in your Excel file is a similar process. however, you need to specify the sheet to be created with a sheetname.

# create sheet
wb.create_sheet('Test')
print(wb.sheetnames)

Modifying an Excel sheet.

To modify an Excel sheet and not the full file,

Load the workbook you want to modify
They specify the particular sheet to modify using its name or index. It is safer to use the index in case the name eventually changes.
In the code snippet below, I used the Sheet label

# wb = load_workbook('mimi.xlsx')

# modify sheet
ws = wb.active
ws['A1'].value = "Test"
print(ws['A1'].value)
wb.save('mimi.xlsx')

Accessing multiple cells

To access multiple cells,
Load the workbook
Make it the active workbook
loop through its rows and columns

# Accessing multiple cells
 wb = load_workbook('mimi.xlsx')
 ws = wb.active

 for row in range(1, 11):
     for col in range(1, 5):
         char = get_column_letter(col)
         ws[char + str(row)] = char + str(row)
         print(ws[char + str(row)].value)

 wb.save('mimi.xlsx')

Merging Excel cells

To merge different cells in Excel using Python,
Load the workbook
Indicate the active workbook
indicate the cells you want to merge

# Merging excel cells
wb = load_workbook('mimi.xlsx')
ws = wb.active

ws.merge_cells("A1:D2")
wb.save("mimi.xlsx")

Unmerging cells

To unmerge different cells in Excel using python,
Load the workbook
Indicate the active workbook
indicate the cells you want to unmerge

# merging excel cells
wb = load_workbook('mimi.xlsx')
ws = wb.active

ws.unmerge_cells("A1:D1")
wb.save("mimi.xlsx")

Inserting new excel cells

To insert new cells

Load the workbook
Indicate the active workbook
use the insert_rows and insert_columns to insert new rows or new columns based on preference.

# inserting cells
wb = load_workbook('mimi.xlsx')
ws = wb. is active

ws.insert_rows(7)
ws.insert_rows(7)

ws.move_range("C1:D11", rows=2, cols=2)
wb.save("mimi.xlsx")

Updating an existing Excel file with internal Data
Add your arrays and objects and take in the information needed

from openpyxl import Workbook, load_workbook
from openpyxl.utils import get_column_letter
from openpyxl.styles import Font

data = {
    "Pam" : {
        "math":65,
        "science": 78,
        "english": 98,
        "gym": 89
    },
    "Mimi" : {
        "math":55,
        "science": 72,
        "english": 88,
        "gym": 77
    },
    "Sid" : {
        "math":100,
        "science": 66,
        "english": 93,
        "gym": 74
    },
    "Love" : {
        "math":77,
        "science": 83,
        "english": 59,
        "gym": 91
    },
}

wb = Workbook()
ws = wb.active
ws.title = "Mock"
headings = ['Name'] + list(data['Joe'].keys())
ws.append(headings)

for a person in data:
    grades = list(data[person].values())
    ws.append([person] + grades)

for col in range(2, len(data['Pam']) + 2):
    char = get_column_letter(col)
    ws[char + '7'] = f"=SUM({char + '2'}:{char + '6'})/{len(data)}"

for col in range(1, 6):
    ws[get_column_letter(col) + '1'].font = Font(bold=True, color="0099CCFF")


wb.save("NewMock.xlsx")

Updating an existing Excel file with Python and APIs

To update an Excel file using Python and APIs, you need to call the APIs into your file using a Get request.
Set the active Excel file as described above and then you run your script.
Here is an example of this:

from openpyxl import Workbook, load_workbook
import requests
from datetime import datetime, timedelta

import schedule
import time

api_url = "https://yourapi"
excel_file = "yourfilename.xlsx"

def fetch_energy_data(offset=0):
    response = requests.get(api_url + f"&offset={offset}")
    data = response.json()

    if response.status_code == 200:
        data = response.json()
        return data["results"], data["total_count"] 
    else:
        print(f"Error fetching data: {response.status_code}")
        return [], 0

def update_excel_data(data):
    try:
        wb = load_workbook(excel_file)
        ws = wb.worksheets[0]  

        for row in range(5, ws.max_row + 1):  
            for col in range(1, 9):  
                ws.cell(row=row, column=col).value = None  

                now = datetime.now()
                current_year = now.year
                current_month = now.month

        start_date = datetime(current_year,current_month, 1) 
        end_date = datetime(current_year, current_month, 24) 

        filtered_data = [
            result
            for result in data
            if start_date <= datetime.fromisoformat(result["datetime"]).replace(tzinfo=None) <= end_date]


        for i, result in enumerate(filtered_data):  
            row = i + 5  
            ws[f"A{row}"] = result["datetime"]
            ws[f"B{row}"] = result["yourinfo"]
            ws[f"C{row}"] = result["yourinfo"]
            ws[f"D{row}"] = result["yourinfo"]
            ws[f"E{row}"] = result["yourinfo"]
            ws[f"F{row}"] = result["yourinfo"]  
            ws[f"G{row}"] = result["yourinfo"]
            ws[f"H{row}"] = result["yourinfo"]

        for row in range(5, ws.max_row + 1):
            ws[f"I{row}"] = ws[f"I{row}"].value  
            ws[f"J{row}"] = ws[f"J{row}"].value  
            ws[f"K{row}"] = ws[f"K{row}"].value  
            ws[f"L{row}"] = ws[f"L{row}"].value  

        wb.save(excel_file)
        print(f"Excel file updated: {excel_file}")
    except FileNotFoundError:
        print(f"Excel file not found: {excel_file}")
    except KeyError:
        print(f"Sheet 'Forecast PV' not found in the Excel file.")
    schedule.every().hour.do(update_excel_data)

    while True:
             schedule.run_pending()

if __name__ == "__main__":
    all_data = []
    offset = 0
    total_count = 0
    while True:
        data, total_count = fetch_energy_data(offset)  
        if not data:
            break
        all_data.extend(data)
        offset += 100  
        if offset >= total_count:  
            break


    update_excel_data(all_data)


To update a particular sheet, use the method mentioned above. best practices are done with the excel sheets index number from 0 till n-1.
as sheet names can change but sheet positions can not change.

 wb = load_workbook(excel_file)
        ws = wb.worksheets[0]

Creating a button that allows users to update on click. To achieve a button to automatically run your Python script, you need to create a button in your Excel file and write a program using the inbuilt programming language, Visual Basic for applications. Next, you write a program similar to this. An example of a VBA script is below.

Sub RunPythonScript()
    Dim shell As Object
    Dim pythonExe As String
    Dim scriptPath As String
    Dim command As String

     Path to your Python executable
    pythonExe = "C:\Path\To\Python\python.exe"

     Path to your Python script
    scriptPath = "C:\Path\To\Your\Script\script.py"

     Command to run the Python script
    command = pythonExe & " " & scriptPath

     Create a Shell object and run the command
    Set shell = CreateObject("WScript.Shell")
    shell.Run command, 1, True

     Clean up
    Set shell = Nothing
End Sub

the issue with this is some functions do not run in non-windows applications seeing that Excel and VBA are built and managed by Microsoft, there are inbuilt Windows functions for this that can only work on Windows.

However, if you are not writing a very complicated program, it will run properly.

Adding dynamic dates and time in your code

To achieve dynamic dates and times, you can use the date.now function built into Python.

now = datetime.now()
 current_year = now.year
current_month = now.month

An alternative to the Excel button is cron or Windows shell

For MacBook users, an alternative to the VBA and button feature, you can use a corn for MacBook and a Windows shell for Windows. to automate your task.

You can also make use of Google Clouds's scheduler. that allows you to automate tasks.

Instead of VBA, what else is possible?

Instead of VBA, direct Python codes can suffice. you can also use the script and run it as required.

Issues faced while writing VBA in a MacBook

The major issue lies in the fact that VBA is a Windows language and hence, has limited functions in a non-windows device.

Issues I faced while creating the button

The same issues are related to the VBA code.

Why I opted for cron
I opted for corn because it is available and easy to use to achieve the goals.
Other tools that can be used for the automation of Excel

Other tools include:

Power query from web feature
Power automate
Visual Basic in Excel

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

Tips for Fixing Bugs and Resolving Code Issues.

Michellebuchiokonicha — Tue, 30 Jul 2024 01:53:52 +0000

Hey there!

These tips I have listed have worked for me and helped me resolve issues in my code. I also learn a lot while on it.

Understand the issue.
Write down the issue.
Go through the code given twice or more times.
Understand the code and its logic.
Analyze the issue based on the current code.
Device two or more ways to fix it on paper.
Write pseudocode.
Implement pseudocode in your file.
Use Debugging tools
Use Browser tools for developers like inspect, developer tools, console, etc.
Include error handling at every point
If still difficult, discuss this problem with another developer
regardless of the language used.
Sleep on the issue.
Think about the issue deeply.
Go back and fix the issue.

Sidenotes:

Research as much as you can. Someone might have fixed this issue previously. So search the internet to help you work seamlessly.

How to Improve Your Development Workflow with Gemini Code Assist

Michellebuchiokonicha — Sat, 20 Jul 2024 08:27:05 +0000

Have you ever dreamt of building and deploying applications three times faster? Look no further than Gemini Code Assist, an AI assistant from Google that empowers developers and cloud specialists throughout the development lifecycle.

What is Gemini Code Assist?

Gemini Code Assist is an AI-powered tool that streamlines the development process for software developers and cloud engineers. It integrates seamlessly with the Google Cloud platform and popular IDEs like Visual Studio Code.

Gemini Code Assist boasts a variety of features to enhance your coding experience, including:

Code Completion: It Generates code snippets and functions tailored
to your specific needs.
Code Explanation: You can Gain a deeper understanding of existing
code with clear explanations.
Code Refactoring: It Improves code readability, maintainability,
and performance with suggested optimizations.
Test Case Generation: It Automatically creates unit tests to
ensure code quality.

Get Started with Gemini Code Assist

A codelab session is available to guide you through the intricacies of using Gemini Code Assist. Here's a glimpse into what you can achieve:

https://codelabs.developers.google.com/codelabs/cloud-code-assist-sdlc#0

API Design: Describe your application's requirements, and Gemini - Code Assist will generate an appropriate architecture, complete with API routes, ports, and data types. You can even test the generated API to verify its functionality.

Full-Stack Development: Craft full-fledged applications in any programming language using natural language prompts. Gemini Code Assist can replace existing code, create new files, and cite sources for the generated code.

Automated Testing: Generate unit tests tailored to your specific needs and leverage Gemini Code Assist to help refactor code for passing tests.

Streamlined Deployment: Gemini Code Assist integrates with Google Cloud Platform for effortless application deployment, eliminating the need for manual configuration.

Benefits of Using Gemini Code Assist

Increased Development Speed: Build applications significantly
faster with code generation and completion features.
Improved Code Quality: Write cleaner, more maintainable code with
refactoring suggestions and automated testing.
Reduced Costs: Save time and resources by streamlining
development tasks.
Enhanced Developer Productivity: Focus on core development
activities while Gemini Code Assist handles repetitive tasks.

Using Gemini Code Assist

There are two primary ways to leverage Gemini Code Assist:

Google Cloud Platform: Access Gemini Code Assist directly within
the Google Cloud console for a seamless cloud-based development
experience.
Local IDE Integration: Install the Gemini Code Assist extension
for your IDE (like Visual Studio Code) to enjoy its features
within your familiar development environment.

Unique Features of Gemini Code Assist

Replace feature: Allows you to replace your existing code without copying and pasting
The update feature allows you to update and rewrite existing code
The citation feature: it cites code sources for further use and also allows the developer to know more about the code.
Explain feature: Explains highlighted code. You need to highlight it and it explains that aspect of code.
Specific use case: It creates specific use cases unique to the written code.

How To Use Gemin Code Assist In your IDE like VSCode

It is very easy and straightforward.

Go to your Visual Studio or the IDE you use
Go to Extensions
Search for Gemini Code Assist + Google Cloud Code.

Install and enable it
Click on the icon by the side

Log in or Sign up with your Chrome browser. If Chrome is not your default browser, copy it and paste it into your Chrome browser.
After which you can get started.

All the features and capabilities Listed above are fully applicable here.

Conclusion

Gemini Code Assist is a powerful and versatile tool that empowers developers and cloud specialists of all levels. It simplifies coding tasks, improves code quality, and accelerates the development process, allowing you to focus on creating innovative applications.

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

Data Science and the Cloud

Michellebuchiokonicha — Tue, 18 Jun 2024 16:15:47 +0000

There are many good reasons to move data science projects to the cloud as there are many advantages like highly clustered processes on many machines giving us enormous computing power.

Also, data science is nothing without data. In a distributed cloud system, our data can be stored and processed reliably even if they are enormous datasets that are also changing constantly. Drawbacks include:
Data science projects in the cloud tend to be more complex especially collaborating in large teams and using many services and technologies. We can however choose from a whole range of services supporting our calms in the cloud ranging from complete managed services to services giving us full control and flexibility of the environment.

Provider-Independent services and tools

MLFlow

A framework you will find implemented in most data cloud services independent of the cloud provider. It consists of 4 components, supporting and streamlining our ML projects and helping us conduct them systematically and collaboratively.

It lets us streamline our ML projects by systematically tracking experiment runs, and easy packaging using Anaconda and Docker.

Tracking: This can be used during model training to lock and track model parameters, and KPIs like model performance metrics, experimental models, code version numbers, etc.

Projects: can be used to package ML model training and trigger it remotely with varying parameters. Once we are satisfied with the performance of a trained model, we can proceed.

Model: register this model with ML flow.

Model registry: can be used as a centralized repository to train models and we can use these repositories to deploy models to the production environment easily.

An applied example of using an ML flow project to share our ML development easily and trigger runs with specific parameters over the network.
For example, someone optimized a Python script to train a machine-learning model. The new model looks very promising but you have to fill in that tweaking a parameter might make the ML model perform even better.

MLflow project can be used to do this without writing the code again.
Eg, this is the GitHub rep you used.
We have
Data, conda environment, readme file, trained pyscript, license.
The script imports models and defines evaluation metrics

Typical MLFLOW workflow:

Start MLFlow Server

Coduct and tract run: meaning to train models and lock the parameters and modern metrics.

Evaluate results using graphical user interphase or its SDK to query the best-performing models programmatically. Once satisfied with a model, we register it.

Register model

*Model registry *

Deploy models: we deploy a model for inference to a productive system typically equipped with a restful API that we can use to obtain predictions from the model.

Databricks: Processing massive amounts of data requires parallel computation on several cluster notes. Spark is a well-known and widely used solution for data processing and data-intensive projects. It is open source but the downside lies in its maintenance and operation overhead. It might not be that simple to set up a spark cluster and keep it running on custom machines.

If we lack the resources to do this, we can use Databricks.
Databricks is An overreaching technology that many cloud providers(CSP) can host; Databricks was founded in 2013 by some of the founders of Spark.
It is therefore not surprising that the core services offered by DB are managed by spark clusters.

It is active in open source and hosting the data and AI summit yearly. At its core, it offered managed spark clusters that can be hosted on Azure, AWS GCP, etc.

Different billing models depend on the cloud platform of choice. But in general, databricks units are used. Depending on the machine, processes, and storage that is used, it is calculated in DBUs. we can also use the community edition at no cost. The managed spark clusters are at the core of Databricks but additional features are designed for data-intensive projects.

Additional features are:

Lakehouses: aims to describe unified data storage that comes with data bricks designed for structured and unstructured data so it can be considered a mixture of a data lake and a data warehouse. It uses files to store the data in a delta lake.

Delta Lake: the data storage used in Databricks. It is a distributed datastore but using an elaborate metadata mechanism, delta lake is ACID-compliant.

Delta Live Tables(DLT): they automatically propagate underlined data updates and provide a GUI to design and manage DTL pipelines straightforwardly. We can also set up data quality checks in DLT that are automatically conducted regularly.

Delta Engine: A query optimizer that automatically and periodically optimizes data queries depending on the data access pattern.

Unity Catalog: With Unity Catalog, databricks offers an easy-to-use tool for data governance. We can the graphical user interface of the unity catalog or SQL queries to perform data governance tasks for example, we can set role-based access control for a specific dataset.

Structuring the data in Delta Lake by processing maturing, there are
Bronze, silver, and Gold tables.
Bronze tables: contains raw data as it was loaded from external sources.

Silver tables: contain processed data including established joints.

Gold tables: contain completely pre-processed data ready for specific use cases like machine learning.

We can use Python, R, SQL, and scala to develop the ML and data processing logic in scripts, and notebooks or use remote compute targets.

Analysis components

Databricks SQL: similar to spark SQL to use the SQL language to query our data.

Databricks data science and engineering environment: similar to ML leap in spark. It comes with a prepared environment for typical data science and data engineering use cases.

Databricks machine learning environment: it uses MLflow to track versions and deploy machine learning models.

Programming languages: python, R, SQL, Scala similar to Spark.
AutoML: to automatically perform feature selection, algorithm choice, and hyperparameter toning.

Google Data Science and ML services

Services give us maximum control and flexibility of the environment, leaving us with some responsibilities such as security patches and environment maintenance. Eg, we can use compute engines for our data science projects and Google’s cloud-based virtual machines.

Using containerization instead of VM, we can develop our containerized application for eg model training or inference on a local machine then deploy that container to

Cloud compute engine:

Kubernetes engine: the cloud-based container orchestration service on GCP.
Deep learning VM: Specialised VM coming with GPU support and pre-installed libraries that are typically used in data science projects.

All Cloud providers offer specialized ML services giving us more comfort and managing some tedious maintenance tasks for us but also taking a little bit of control and flexibility.

On the Google platform, the specialized ML service is called cloud AI/ vertex AI being the main component. They include:

Dataflow
Composer
Dataproc
Bigquery
Google Cloud AI for ML training etc
Google Cloud console
Vertex AI workbench
Vertex AI formerly datalab.
Auto ML
API endpoints
Visual reporting

Ready-to-use services on GCP: they can be used by calling the standardized restful APIs to obtain model predictions by pretraining machine learning models hosted on GCP. eg we can use the speech-tech API to send the sound file to that service and receive the transcribed text.

Natural language AI
Teachable machine
Dialogflow
Translations
Speech-to-text
Text-to-speech
Timeseries insights API
Vision AI
Video AI

Data science and ML services on AWS

Same as on GCP, there are services maximizing control and others suitable for more comfort and instant application. Example,
Cloud-based VM called EC2.

Deploying containers to elastic container service ECS/Elastic kubernetes service EKS
And Elastic map reduce EMR

Specialized ML Service

It is called *SageMaker: *
Sagemaker is a service family of an entire collection of sub-services dedicated to supporting typical DS and ML projects. eg, there are graphical user interfaces, auxiliary services for data wrangling, data labeling, prepared scripts and templates, and also auto ML features.

Notebook instances
Data labeling
Data wrangler
Feature Store
Clarify
Pipelines
Studio
Jumpstart
Canvas
Autopilot

Ready to use services on AWS:

Translate, transcribe, computer vision, and other services being pre-trained ML models that are callable for restful APIs for inference.
The computer vision API here includes:

Comprehend
Rekognition
Lookout for vision, panorama
Textract
A2I
Personalize
Translate, Transcribe
Polly, Lex
Forecast
Fraud Detector
Lookout for Metrics
Kendra
Auxillary services are DevOps Guru and Code Guru.

Data science and ML services on Microsoft Azure.

VM: For maximum control and flexibility, we can use Azure VMs to decide how to create and manage our development environment.

ACI/AKS: we can use Azure container instance ACI for containerized model training for testing purposes then use Azure Kubernetes service AKS for production settings.

HDInsight: A managed service for technology.

Databricks:

Synapse Analytics:

Specialized ML service.

This is called Azure Machine Learning. Similar to AWS sagemaker, Azure ML comprises several sub-services for example the GUI called designer and auxiliary services for data labeling and other features.

Azure machine learning
Studio
Workspace
Notebooks/RStudio
Data labelling
Designer

Ready-to-use services on Azure.

Giving us maximum comfort and usability of pre-trained models callable by restful APIs. These services are called cognitive services and they include:

Computer vision, Face
Azure cognitive service for language
Language understanding models,
Translators, and other services.
QnA Maker, Translator
Speech Service
Anomaly Detector
Content Moderator
Personalizer
Cognitive Services for Big Data

Note: This is a 4-fold series on cloud computing, virtualization, containerization, and data processing.
Check the remaining 3 articles on my blog.

This is the fourth. Here is the link to the third.

https://dev.to/michellebuchiokonicha/cloud-computing-platforms-4667

it focuses on cloud platforms.

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

Cloud Computing Platforms

Michellebuchiokonicha — Mon, 17 Jun 2024 16:40:01 +0000

Introduction

When moving to the cloud, we can use the services one of many CSPs offers. we can only choose the service optimal for our use case if we know what services are available.

Interapobaility has become increasingly important in today's inter-connected world, it is common for organizations to use multi-cloud strategies integrating cloud services by several cloud providers or with existing on-premise infrastructure.

Furthermore, to prevent vendor lock-in, we should be aware of the various offerings by competing cloud providers to stay flexible. There are many more reasons why we should learn what the different providers offer such as security and compliance, innovation and new services, ecosystem integration, performance optimization, scalability and flexibility, technical support, etc.
Here we will discuss some of the most frequently used services offered by the big three cloud providers to enable us to choose what is best for our situation. This applies to everyone using the cloud including data scientists, cloud architects and consultants, cloud developers and engineers and so many more.

Terminologies Defined

Cloud Platform: An abstraction layer that can be used to provision resources and as a starting point for cloud development.

Cloud service provider: A corporation running many data centers worldwide and offering services based on this service for rent.

There are many CSPs but three of them, due to their considerable market share are the most popular.

Amazon web services

It started in 2006, is one of the oldest CSPs, and has the biggest market share. Compute services include:

Elastic compute cloud(EC2)
Virtual Private Cloud (VPC)
Simple Storage Service (S3)
Relational Database Service(RDS)
Lambda
Kinesis
Elastic Map Reduce EMR

Others include:

Route 53,
Simple Notification Service(SQS),
Elastic Load Balancer
Network firewall.

Data security and protection

Considering some thoughts into which availability zone should be used to deploy a resource can contribute to data protection. Moreso, data being redundant in many zones can mitigate data loss and support data failover.

On the other hand, limiting the
geographical location to which data can be stored and processed can ensure that this falls under one jurisdiction or another.

As far as monitoring control and encryption services are concerned, there are:

CloudTrail, Macie, CLoudHSM, Key Management service(KMS), cloud Tower
GuardDuty, Nitro system
ISO compliance for cloud security, privacy information management, and cloud privacy.
VPC

Pricing and costs

Most services are offered on a pay-as-you-go basis.

There are 12 months when we can try services and some services are free.

Certifications

They run certification programs to become associates and then professionals.

Microsoft Azure

It started as the Windows Azure platform in 2010 then became Microsoft Azure in 2014. The virtual machines in Azure are called virtual machines and its services include:

Virtual machine
Kubernetes services
Container Instance
Blob storage: Used for object storage
Azure SQL: cloud version of Microsoft SQL Server, a relational database.
Azure Functions: the serverless compute service here is called Azure Functions.
DNS, notification hubs, load balancer, and firewall:
Azure machine learning: provides abstraction for data connection
Azure Databricks for big data processing
HDInsight
Data Factory

Data security and protection

Azure has ways of ensuring security and protection.
They include:

Access management in Azure Active Directory:
Data Encryption at rest and in transit
Virtual Private Network(VPN, VNet)
Security center

Pricing and Costs

Some services are free and it is for 12v months, pay as you go. You can also save costs by making long-term commitments and using either resource.

Certifications

There are certifications to become experts from associates to professionals.

Google Cloud Platform

It started in 2011 with the cloud computing engine which is Google Virtual Machines.
Its services include:

Cloud VPC: Virtual private cloud services
App Engine
Kubernetes Engine: containers as a service
Compute engine
BigQuery
Dataflow
Pub/Sub
Cloud storage: This is Google object storage
Cloud Spanner: A distributed relational database service providing transactional consistency
Cloud Functions: Google serverless computing service
Cloud DNS, Cloud Pub/Sub, cloud load balancing

Data security and protection

They use various ways to ensure security like:

Encryption at rest and in transit
ISO standards for cloud security, privacy, and regulatory compliance.
VPC - virtual private cloud
Security command center
Identity and access management
Firewalls

Pricing and costs

Similar to the other providers, 12 months and pay as you go. There are also free tier offerings.

The largest cloud players are:

AWS
Azure
Google
Alibaba Cloud
IBM
Salesforce

Gartner Magic Quadrant

It divides the market into Challengers, Leaders, Niche Players, Visionaries

Leaders: AWS, Azure, Google
Niche players: IBM
Visionaries: Alibaba cloud

Criteria to consider when choosing a suitable cloud provider

Company strategy
Regulatory Compliance
Regional presence
Technologies and service roadmap
Performance and reliability
Availability and SLA
Billing models and costs
Flexibility
Support

Cloud reference architectures

Azure architectures
AWS reference architecture
Google Cloud reference architecture

Note: This is a 4-fold series on cloud computing, virtualization, containerization, and data processing.
Check the remaining 3 articles on my blog.

This is the third. Here is the link to the second.

https://dev.to/michellebuchiokonicha/virtualization-containerization-with-docker-storage-and-network-services-2bjf/edit
it focuses on docker, containerization, virtualization, storage technologies, and network services.

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

Virtualization & Containerization with Docker.

Michellebuchiokonicha — Sat, 15 Jun 2024 01:23:16 +0000

Everyone relies on data for work and decision-making. There are various cloud-enabling technologies but we will concentrate on. Virtualization is responsible for the abstraction, isolation, and hoarding of applications. There are different levels of virtualization and different approaches.

Virtualization and containerization: Making applications easily portable to new environments eg in the cloud. This will allow us to easily spin up additional instances of the application eg when the load increases.

Storage technologies: Applications rely on some sort of data storage. AWS, GCP, and Azure.

Networks and restful services: They allow for abstraction, hiding inner complexities and easily connecting system components to a more extensive cloud system.

After reading this article, you should be able to:

Describe key features of virtualization and containerization technologies
Apply the key features of Docker.
Explain the benefits of characteristics of different cloud storage technologies
Explain how restful APIs work.
How different technologies and services can be used to manage
restful APIs.

Virtualization and Containerization: Containerization with docker.

Containerization is central to cloud computing because it allows us to wrap applications in self-contained capsules and thereby make them easily portable to the environment.
In cloud computing, we do that all the time for example, we might develop an application on our local machine, but for better reliability and scalability, we then want to execute the application on a more extensive cluster of devices in the cloud. Also working with containerized applications, we can easily add additional machines to a cluster running that container and thereby scale horizontally when the cloud increases.

Main components of containerization with docker.

- Dockerfile: A recipe containing the instructions to be executed when the container starts. Based on that docker file, we create. It describes how a containerized application is built in the form of a text file. Eg we specify the operating system which data is available within the container and which programs are to be executed.

- Docker Image: a blueprint or design to create instances of this container. These images are usually registered and distributed in the registry.

- Docker Image Registry: We can then store that image in a centralized image registry. there are private and public registries. The best-known open docker registry is the docker hub. Before we build a container from scratch, you can look at the docker hub to see if there is not a freely available image that we can adapt to our needs if necessary.

- Container Instance: create an instance from that container based on that image from everywhere we like. To launch an instance of an image, docker checks if the image already exists locally. If it doesn’t, the image is loaded from a registry. Then starting the containerized application is easy and it is done with a simple command. One of the advantages of this approach is that a container can be started independently of the host environment. it does not matter if we start a container on a local machine, a data center server, or the cloud.

- Docker Compose: To launch multiple containers that together form a larger application, we can use docker-compose. Docker compose is used to store an application containing several applications to work together. In a simple YAML document, we specify which containers should be launched with which configurations. Then we can use a simple command to spine up all the containers at once.

- Docker swarm: Used to run one or many containers on a cluster of several machines. A concept central to cloud computing. Here, we can run containers not on individual machines but distributed across multiple compute units in parallel. Docker swarm provides better scalability while abstracting internal complexity. From the outside, we see one logical unit running one or more containers.

Getting started with Docker: Hands-on docker

In your Integrated development environment/terminal,

Install docker.
Create a dockerfile. We write the instructions to be executed in the container with many commands.
.from: to start the container not from scratch but from an already existing image in an image registry.
.run: to execute a terminal command
.copy- to copy data into the container.

After creating a docker file, docker commands can be used to do several things.

Docker Build - will create a docker image based on the docker file.
Docker run- will create and run a container based on an image.
Docker images- gives us the list of all the images available in a running machine.
Docker ps: Shows us the running containers.
Rm: Remove a container from the machine.
start: Start a container.
stop: Stop a container.
commit: Commit changes to an image.
Tag: Add a name to an image.

Cloud deployment

Once done with development, we need to deploy the container to the cloud. We use the deployment manifest. It is a JSON file specifying which container to deploy, how ports of a container should be marched to the port of the hosting machine, and so on.

Storage Technologies.

There are so many storage solutions available. However, we will concentrate on the major three. AWS, Azure, and GCP.

AWS

- Simple Storage Service(S3): Object data store that can be used for unstructured data. It is highly reliable and its service level agreement concerning durability is about 99.999%. It is very reliable. It has an S3 access point and also has different tiering classes. S3 intelligent, S3 standard, S3 standard-infrequent access(S3-IA).

- Elastic Block Store (EBS) and Elastic File Store (EFS): They are cloud-based block storage and file systems that can be attached to EC2 instances. Amazon cloud virtual machines. It is a managed file storage for EC2. it has two classes standard and infrequent access (IA).

- RDS: Relational database services: It is used to host relational databases like aurora in the cloud.

- DynamoDB: Multimodal no SQL database on AWS and we can use a document-oriented API to work with it.

There are so many more solutions on AWS cloud storage. You can check their documentation for more.

FLASK in Practice

In many cases like cloud-related scenarios, we want to implement a program logic as a callable service. For example, imagine we trained a machine model and now is the time to provide this model as a service that we can call from another application or web page to obtain predictions by the model.

Flask is a popular lightweight web application framework for Python to achieve exactly this. here, we learn to set up and run a simple flask app with a rest API.

We want to offer the predictions made by this model as a service callable by our rest API. we need to import the model and flask to provide the rest API. we have to also tell the application which route to choose if we send data to that API.

You can use a graphical interface like Postman and paste the URL to which you deployed your API. then you choose the route and send the data you want to make predictions for.
You can use the model to make predictions of types, for example, what can of bag is this, etc.
It is easy to use Flask to provide a restful API for a machine learning model and other programming/business logic and provide this program with a restful API to be callable by other applications and web pages.

Google Cloud Platform(GCP)

Cloud storage: Object data store that can be attached to the app engine. Google’s cloud-based virtual machines. It is the cloud storage in GCP for unstructured object data. It is highly reliable and comes in 4 different tiers. Standard, Nearline tiering, coldline tiering, archive tiering.

Cloud SQL: A cloud-based relational database engine. it is fully managed and there is support for MySQL, PostgreSQL, and standard SQL
Firestore: no SQL data stores. This is also a document-oriented one. It is organized into collections of documents and sub-collections.
Bigtable: Also a no SQL data storage service. It is however column oriented one that is table-like data.

Microsoft Azure

Storage Account: used in many ways. A family of storage services consisting of object files, column-oriented NoSQL, and huge storage services. The four different storage types here are:
Blob storage: It is intended for storing unstructured binary data. It is organized in containers of which there are three types depending on the intended data usage.

File Storage:

Table Storage:

Queues:

Data Lake: An extension of storage accounts and can be used for data storage for an entire organization.

SQL Database: A cloud-based version of Microsoft SQL server. A relational database. It is a managed platform as a service. It comes with two resource usage models
Single database and elastic database pool.

Synapse: Azure’s cloud-based data warehouse solution.
Cosmos DB: globally distributed multi-modal NoSQL database. It allows data to geographically reside close to its intended usage it is also a multi-model NoSQL database. It has gradual consistency models.

Networks and Restful Services.

Rest APIs:

They are essential to cloud computing because they allow us to connect individual services in a standardized way without knowing about the inner complexities of each.
This allows for microservice architectures and makes it comfortable for us to develop applications on the cloud.

What are restful APIs? They are intermediate between the cloud and the server.
The client can be an application that needs data from a service such as a storage solution.

Instead of learning to communicate with each of the hundreds of available storage services, we can rely on a standardized intermediate. we will talk to the API in the language we know because they are standardized. The intermediate will speak to the servers to retrieve the resources we are asking for and this hides the servers/serves inner complexity leaving us with APIs that we can work with. Imagine we want to talk to a database service in the cloud and we use restful APIs for this. We can send an HTTP request. This request will start with a method.

Get: to retrieve a resource
Post: to create a resource and send data
Put: to update a resource
Delete: to delete a resource

Endpoint: A unique URL in which the API is reachable.
It contains

Header: Used for authentication as access for most APIs is managed by role-based access control and we have to ensure that all the authorized accesses are granted.

Body: Used to send additional data; the API will then process the request and respond to us. The response can come in various formats like JSON, XML, YAML, CSV, and HTML but JSON is the most popular format.
APIs make communication between services straightforward. This is done by several restful API characteristics.

Restful Paradigms

Client-server autonomy: means the client and the server are independent of each other. here, our application does not depend on how the database works and vice versa.
Uniform Interface between the services
Layered architecture: An intermediate between client and server can be interpreted as an additional architectural layer.
Caching: this can be disabled and enabled depending on the API. when enabled, we can allow clients to store resources on their machines thereby taking load from the server.

Stateless interactions: restful APIs are stateless. This means no request and no resources are stored and they do not influence each other.

Code on Demand: the requested resources can also be coded. This is called code on demand. example, we can request a short javascript code snippet to be executed on our client machine. This takes load from the servers and supports flexibility in developing and using applications.

Cloud Networks.

Transfer between cloud services: Cloud computing highly depends on interconnected services. This means that there is a considerable transfer between cloud services.

Low Latency: To ensure applications run smoothly, we depend on low latency as well as

Security: Secure communication. All cloud providers offer solutions for this. Eg, Azure virtual network(VNet) to establish a virtual network where our services can communicate securely and with low latency.

Cloud computing is all about distributing storage and processing across many machines while virtual networks can be used for secure communication between services.

Load Balancers: They distribute the load between servers. Every cloud provider offers load-balancing services in one way or another.
Managed Identities: As far as authentication and authorization are concerned, many different solutions are depending on the cloud provider and service. For example, in Azure, using managed identity, we can sign privileges to people, groups, applications, or services. For the latter, we can use a service principle and manage its privileges in the Azure active directory.

Note: This is a 4-fold series on cloud computing.

To read the first, here is the link:

https://dev.to/michellebuchiokonicha/cloud-computing-for-data-and-software-developers-43ce

it focuses on cloud services, models and roles.

Cloud Computing for Data and Software Engineers

Michellebuchiokonicha — Fri, 14 Jun 2024 10:01:34 +0000

Cloud computing has become a major buzzword in the last decade. The application of technology is enabled and made possible by cloud computing.
A simple instance is the alarm on your phone. This alarm, after being created was made available to users through cloud computing. The applications we use daily and the recently hyped innovations of ChatGPT and Gemini.AI are made available and accessible to users because of cloud computing.
It enables apps to synchronize data and manage services.

Commuting to work with maps relies on cloud computing. Your Microsoft 365 and Google Workspace are made possible because of cloud computing.

Here, we will understand cloud computing and how it helps distribute data and processes across multiple machines.
Here, we will describe the concept of cloud computing, explain the cloud delivery and service models, describe cloud computing for your purpose, and more.

We will also understand the fundamentals of cloud computing, cloud service models, infrastructure as a service, serverless computing, risks and benefits of cloud computing.

Cloud Delivery Models

Cloud Delivery models are also called cloud sourcing models and there are varying computing models.

Local: Traditionally, we would have our applications on a local machine like your laptop, containing all the data and software necessary for a particular task on that same machine with or without an internet connection.
The advantage is that the solution is cheap, fast, and readily available. And when there is no internet connection, it is pretty safe from outside attackers. The downside is that it is not easy to work collaboratively and the resources of a local machine might not be sufficient for extensive datasets. This is why many organizations store their data and install software not on local machines but on servers to which organizations can connect.

On-premise: On-premise means an organization owns hardware like servers located in the organization's rooms or geographically close by to the organization. However, if the organization decides to move to the cloud, the organization will no longer own the hardware but render it from a cloud service provider. These servers were physically located within the rooms owned by the organization hence the name, on-premise or on-prem computing. The advantages are that the organizations have complete control of the software and infrastructure. On the downside, this setup has limited scalability and high investment costs. All resources must be purchased by the organization and installed on the servers entering the cloud.

Private Cloud: This means infrastructure is reserved for a particular organization and we can connect to the service via a network as opposed to the public which requires the internet.
We can also use virtual private networks to establish a connection.

This is dependent on whether the hardware is reserved for organizations in the data centers run by the cloud service provider reserved for the organization only.

Public Cloud: In contrast to private cloud, if the organization decides to use the public cloud, the organization would share the resources with other cloud users. Another difference is that the public uses the internet while the private cloud uses a non-public network.

Hybrid cloud: This model uses a mixture of on-premise and cloud services/machines. This way, we can choose which data should be stored, processed, and analyzed in the cloud and what should remain in the organization's machines.

Multi-Cloud: This means that the organization uses cloud services provided by other cloud providers.
Using different services offered by more than one cloud provider. Eg, we can use storage services provided by both providers A and B but only processing services by cloud A and analytical services by provider B. This way we might avoid vendor-locking and select from a wide range of services the ones that are useful for our particular use case. One drawback is that these setups are more complex to manage.

Poly-Cloud: This means that we use one type of service by one cloud provider only eg, we would use a database by provider A and analytical services by provider B. Here, we use only the services by one cloud provider for one specific domain of the project. Eg, we would store all the data in Cloud A, process the data in Cloud B, and analyze the data in Cloud C.
Cloud-native is not a delivery model but implies that an application is developed with the cloud in mind from the very beginning. Eg, it is designed to be easily scalable, reliable, and available by deploying it to a cloud environment eg as microservices and containers, or as a serverless function.

Fundamental Roles in Cloud Computing

Cloud Solution Architect: This is also called cloud enterprise architect is responsible for designing a high-level concept aligned with a business strategy. Coming up with a cloud architecture to scale the organization. It deals with business understanding and technologies.

Cloud Architect: This is similar to the cloud solution Architect but is more focused on the implementation unlike the Cloud solution which is focused on the high-level concept of the architecture, the cloud architect is more focused on the hands-on implementation of that concept into concrete services that work together. They are specialized in the offerings of one or a few cloud providers.

Cloud engineers: they are the administrators and operators of the cloud systems. They maintain the cloud system and its services. They manage servers and clusters. They are not only responsible for the administration of the entire system but also individual services. This includes access control. They understand operating systems and server maintenance.

Cloud developers: They are programmers using cloud services in the applications they develop. This includes application developers because they do not come up with new cloud services but use the provided services and configure them in their applications. They use and orchestrate cloud services in their applications. They need knowledge of how the cloud systems are set up. However, their core is programming, API setup, and networking.

Cloud consultant: this is similar to the cloud solution architect. They offer advice, and guidance and provide an objective overview of cloud providers. They understand the organization's business and cloud needs and come up with solutions and architecture. They are more focused on added value for business processes.
Sometimes, these roles overlap and they can function to aid the other.

Cloud Service Models

Cloud service models are different from cloud delivery models.
This tells how responsibilities are shared between the cloud provider and us as the users/developers.

On-prem: We have complete control of the system but it also brings in a lot of responsibility of taking care of each component.

Bare-metal-as-a-service(BAAS): we might choose to not own physical hardware but get it from a provider responsible for maintenance. But why not give the provider responsibility for hardware virtualization? This is the next one called IAAS.

Infrastructure-as-a-service(IAAS): the cloud provider is responsible for the physical hardware and virtualization while users are responsible for everything that runs on top of that virtualized hardware. This means flexibility but also maintenance responsibility. Eg for the application code that we develop for these virtualized machines. example AWS EC2, Azure virtual machines, and GCP compute engines.

Container-as-a-service(CAAS): when cloud providers offer services to run containers and are responsible for maintaining all underlined components, it is called CAAS. eg Azure container instance, Azure Kubernetes serves, AWS elastic container service, AWS elastic Kubernetes service, Google Cloud run, and Google kubernetes engine.

Platform as a service (PAAS): Here, in addition to hardware and virtualization, the cloud provider is also responsible for the operating system and runtime executing the code we develop. Here the provider is responsible for the maintenance responsibility and the runtime to the provider. Examples are storage services like AWS S3, Azure block storage, and GCP block storage. Managed database storage like AWS dynamoDB and Azure Cosmos DB and IOT services like AWS IOT core, Azure IOT hub, and GCP cloud IOT core. As the name suggests, PAAS is all about providing a platform.

Serverless also called Function and a service (FAAS): It is similar to the platform as a service, however, it also implies that we are not busy maintaining and configuring any hardware but focus entirely on code development. We simply push our code to a serverless service to be executed in the cloud and we do not think about the underlined infrastructure at all.
Here, it is not so much about the platform being provided but the framework on which we can execute our application code and not on setting up and maintaining the underlined component.
Examples are AWS Lambda, Azure functions, and GCP Cloud functions.

Software as a service(SAAS): In addition to the hardware virtualization, operating system, and runtime, the provider is also responsible for the application code in this service model. An example is Office 365 or any other similar cloud-based software. All we can change in this service are the application settings but not its code. Here we use functional software and can not change anything in it except the application settings.

Risk and Advantages of Cloud Computing

Advantages

Cost reduction: It is cheaper than local or on-premise solutions. The cost of purchasing hardware and hiring specialized staff better scale to the many cloud customers than having all this cost for one organization only. Also, many services are offered as pay-as-you-go billing models. This means that we can quickly adapt the cost of our system to the load it experiences.

Also, we can start with cheap hardware and quickly scale to bigger or more machines in the cloud. We can also save costs by running processes with lower priority when machines in data centers are idol because other customers do not use them at that time.

Scalability: Because cloud providers dispose of many machines, we can scale instantaneously and unlimitedly on the cloud by adding more or bigger machines to a running cluster.

Flexibility: As there are no upfront costs, we can try out new services and change components of our system easily.

Easy access: You only need a few clicks and buttons to spin up instances of the available technology in the cloud.

Reliability: Being distributed systems with high data replication and parallelization makes cloud computing highly reliable. Also, all cloud providers clearly define service level agreements and we are reimbursed if these agreements are not fulfilled.

Physical safety: it is an advantage as there is much stricter surveillance of physical access to the data centers than to the servers of a single organization.
Certificates: All cloud providers hold standardized certificates supporting us in evaluating the applied data security and protection standards. This also helps us choose appropriate providers for given use cases.

Risk

Expenses: there can be additional expenses eg when resources are not adapted to need or we over-provision our services.
Redundancy in the system ensuring its reliability means additional network transfer. When not set up correctly, this can lead to unnecessary expenses. The good thing is that all cloud providers offer monitoring tools and automated alerts to mitigate these risks.

Security: When not set up correctly, it can be insecure. Of course, there can be more points of failure in a complex cloud system but depending on the service model, we might also give responsibilities to specialized units of the cloud provider with more workforce and higher security budget than single organizations.

Share responsibility: although this is usually considered a good thing, it can pose a risk. Independent of the cloud service model we choose, we will always be responsible for the stored data, the endpoint to our system, and the access management.
Also, legally, we always need clarity on how things are done by the cloud provider when handling personally identifiable information.

Certifications and transparency: Certifications are usually a good thing but the number of different certificates sometimes makes it difficult to understand what they mean and if they are worth something and applicable to a particular use case.

Data Sovereignty: This is important when handling personal data in the cloud. here, we must keep control of the data all the time. Eg, when the data is about a certain group of people, we must ensure that it never leaves the group of people.

Note: This is a 4-fold series on cloud computing, virtualization, containerization and data processing.
Watch out for the remaining 3 articles.

Note: This is the first of a 4-fold series.
The second can be found here.

Follow me on Twitter Handle: https://twitter.com/mchelleOkonicha

Follow me on LinkedIn Handle: https://www.linkedin.com/in/buchi-michelle-okonicha-0a3b2b194/
Follow me on Instagram: https://www.instagram.com/michelle_okonicha/

History of Artificial Intelligence: How it all started.

Michellebuchiokonicha — Mon, 13 May 2024 16:57:52 +0000

Introduction

According to John McCarthy, artificial intelligence is the science and engineering of making intelligent machines especially intelligent computer programs. AI attempts to mimic human intelligence by applying a set of algorithms to a set of data.

Development

The historical development of AI goes as far back as the 4th century BC with philosophers like Aristotle who formalized human thinking to be imitated. This early philosophy laid the foundation for what we now call AI as opposed to the usual assumption that AI first came up in the 1950s. The development of AI as a scientific discipline began in the 1950s. During this time scientists and engineers began to explore the possibilities of automating thought processes. This led to the development of the first AI systems capable of completing simple tasks that would normally require human thinking. In the decades that followed, AI became more powerful and able to handle increasingly complex tasks. The focus of the first wave of AI was on logical reasoning. In contrast, the main focus of the second wave was driven by an attempt to solve the problem of knowledge representation. This knowledge-based view was the origin of expert systems. The main feature of this technology was that the domain knowledge was stored in the databases. AI has made significant progress since the 1990s.

Deep Blue Computer Systems

A milestone was the deep blue computer system which defeated the world chess champion in 1997. Gary Kasparov. This demonstrated the capabilities of AI systems.
From speech recognition to purchasing behaviors in AI shops, advances in AI have been fueled by computing capabilities. This massive data provides food for AI to grow.

Deep Learning

Another important milestone in the development of AI is the emergence of deep learning models and adversarial learning which also brought about machine learning and expanded the scope of intelligent systems.

AI winter

Describes times when interest, research, and funding for AI decline significantly. the first AI winter was in 1974 - 1980 and the second was in 1987 - 1993.
When expectations of AI systems to automate tasks becomes high and these needs are not met, it brings about an AI winter caused by pessimism in the AI community.

Expert Systems

Helps to support decision-making processes. Expert knowledge is stored in a knowledge base. An Inference engine uses knowledge from the knowledge base to make decisions. Interaction with a non-expert user is done through a user interface. Expert systems emulate decision-making by using domain-specific knowledge of an expert.

The Gartner hype cycle curve evaluates the potential of new technologies.

Why Expert systems

Expert systems are used in many different application domains. They can help humans solve complex problems based on a predefined set of rules especially when a person has to make decisions in an unknown domain. Expert systems can be of great help to support decision processes. Another to enable a non-expert user to make decisions is an expert system that uses knowledge from experts in specific domains where decisions have to be made. This expert knowledge is stored in the knowledge base.

Expert systems can be divided into:

Case-based systems: This is based on problems that have occurred in the past. Decisions are made in the same way as they have been made in the past.
Rule-based systems: This is Based on a huge knowledge base consisting of a large number of rules in the form of facts. The rules describe the relations between different facts and can be used to arrive at a decision.
Problem-based systems: This is used to categorize a situation as a certain decision problem to handle it in a similar way.

One major advantage of expert systems is the decisions made by the system can easily be understood by humans when looking at the rules that have been used for respective decisions. If human beings interact with business processes, they will gain knowledge over time in the same way, for meaningful decision support. The knowledge base of a decision system must be extended with data about new cases from time to time.

The Gartner Hype Cycle

This curve is a graphical presentation to illustrate the maturity and trends of emerging technologies. This cycle exists for several topics. One of those topics is artificial intelligence. The Gartner hype cycle is divided into different phases.

Innovation Trigger phase: In this phase, New technology is not found in the market.
Peak of Inflated Expectations: In this phase, early publicity produces several success stories. These are mostly accompanied by failures.
Trough of Disillusionment: In this phase, early prototypes fail to deliver.
Slope of Enlightenment: Here, technology becomes more widely understood and starts being used by more and more companies. This leads to the next phase.
Plateau of Productivity: Here, mainstream adoption starts to take off. If a technology qualifies for a broad market, it will continue to grow.

The Gartner hype cycle is used to evaluate technology and determine its fate in the ecosystem.

Artificial Intelligence is an evolving field and will continue to be in years to come.