DEV Community: Roy Rebello

Journey to Azure AI Fundamentals Certification

Roy Rebello — Tue, 06 Feb 2024 18:15:10 +0000

Embarking on the path to earning the AI-900: Microsoft Azure AI Fundamentals certification was a journey of discovery, challenge, and ultimately, achievement. As cloud computing continues to evolve, the importance of AI and machine learning in today's technology landscape cannot be overstated. The AI-900 certification is a testament to one's understanding of these pivotal technologies, especially within the Microsoft Azure ecosystem. This blog post outlines my journey, the preparation strategies I employed, and the resources that guided me to success.

Understanding the AI-900 Exam

The AI-900 exam is designed to validate foundational knowledge in artificial intelligence (AI) and machine learning (ML), focusing on the capabilities of Microsoft Azure services. It covers a broad spectrum of topics, including AI workloads and considerations, machine learning principles, computer vision, and natural language processing workloads on Azure. The exam format includes a variety of question types, such as multiple-choice, drag-and-drop, and active screen questions, over a span of 40-60 questions.

My Preparation Strategy

Diving into Official Study Materials
My journey began with the official Microsoft Learning Path for Azure AI Fundamentals. These modules provided a comprehensive overview of AI concepts, machine learning, computer vision, and NLP, laying a solid foundation for further study.
Leveraging Online Courses and Tutorials
To deepen my understanding, I turned to online platforms offering structured learning paths. Cloud Academy's AI-900 Exam Preparation learning path was particularly beneficial, combining instructional courses with hands-on labs and an exam prep guide that covered all necessary topics comprehensively.
Practice, Practice, Practice
Practice tests were an integral part of my preparation, helping me familiarize myself with the exam format and question types. Platforms like Whizlabs offered a plethora of practice questions and exams that were instrumental in my success.

Crafting a Study Plan

I devised a detailed study plan, allocating specific times for studying theoretical concepts, engaging in hands-on practice, and taking practice tests. This balanced approach ensured I was not only prepared for the types of questions I would encounter but also understood the practical application of the concepts within Azure.

Personal Tips for Success

One of the biggest challenges was maintaining motivation and efficiently managing study time. Setting small, achievable goals and rewarding myself upon completing them helped keep me motivated.

Exam Day and Reflections

On exam day, time management and a calm mindset were key. I carefully read each question and planned my time accordingly. After the exam, I reflected on the journey, feeling a sense of accomplishment (838 /1000) and pondering the next steps in my professional development.

Conclusion

Achieving the AI-900: Microsoft Azure AI Fundamentals certification was a fulfilling endeavor, enhancing my understanding of AI and ML within the Azure ecosystem. For those embarking on this journey, remember that determination, a solid study plan, and the right resources are your best allies. Whether you're a beginner or have some experience, this certification is a valuable step in your career in technology.

I hope my journey inspires and guides others looking to achieve their Azure AI Fundamentals certification. The path may be challenging, but with the right preparation and mindset, success is within reach.

Certified Enterprise Chaos Engineer

Roy Rebello — Tue, 06 Feb 2024 17:41:51 +0000

Introduction:

In a world where technology is intricately woven into the fabric of every business, the robustness of digital infrastructure is paramount. I recently embarked on a transformative journey, culminating in achieving the Gremlin Certified Enterprise Chaos Engineering Certification. This milestone is not merely a personal triumph but a beacon of the vital role that Chaos Engineering plays in the technological realm today.

Understanding Chaos Engineering:

Chaos Engineering is akin to a vaccine for complex software systems. It intentionally introduces disruptions to expose weaknesses in a controlled and monitored environment. The core principle is to anticipate and prepare for chaos rather than react to it, thereby ensuring that when the inevitable happens, the system remains unfazed and reliable.

The Certification Journey:

Gremlin's certification program offered an immersive experience into the world of Chaos Engineering. It provided a blend of theoretical knowledge and practical applications, covering the principles of controlled experimentation on software systems to build confidence in their capability to withstand turbulent conditions.

The Importance of Resilience:

As digital services become more sophisticated, they also become more vulnerable to a plethora of disruptions, from server outages to unexpected traffic surges. This vulnerability can have a domino effect, crippling services, eroding customer trust, and incurring financial losses. Chaos Engineering prepares systems to be resilient, to self-heal, and to continue operating, making it an indispensable strategy for any organization that relies on digital services.

Applying the Knowledge:

Armed with the certification, I am now poised to apply these principles to real-world systems. The certification has empowered me with the skills to identify potential points of failure and to design systems that can adapt and recover swiftly from disruptions. This proactive approach to system reliability is what will set apart the digital solutions of the future.

Conclusion:

My journey to becoming a Gremlin Certified Enterprise Chaos Engineer has been enlightening. It has reinforced my belief in the necessity of adopting proactive measures to safeguard our digital infrastructure. As I integrate this knowledge into my professional endeavors, I invite fellow tech enthusiasts and industry professionals to connect and explore the vast potential of Chaos Engineering. Together, we can steer our digital future towards a horizon where reliability is not just an afterthought, but a foundational component.

ChaosEngineering #GremlinCertified #Resilience #DigitalInfrastructure #sre #chaos

ChatGPT with Langchain: An Overview

Roy Rebello — Wed, 26 Apr 2023 03:33:58 +0000

Introduction

Imagine a world where AI-driven technologies enable us to communicate more effectively, analyze enormous amounts of textual data, and make informed decisions in just seconds. A world where chatbots comprehend our intentions and respond with human-like clarity. This world is no longer a far-off dream, but an approaching reality, due to the remarkable advancements in AI technologies such as ChatGPT and LangChain. In this article, we will dive into the groundbreaking innovations of ChatGPT and LangChain, examine their potential applications, and uncover how they are transforming the AI landscape.

Understanding ChatGPT and 🦜🔗LangChain

ChatGPT

ChatGPT is a cutting-edge language model developed by OpenAI, based on the GPT-4 architecture. It harnesses the power of advanced machine learning algorithms to generate human-like responses in a wide range of applications, from chatbots to content generation. By understanding context and maintaining coherence, ChatGPT enables seamless, intelligent conversations between humans and AI.

🦜🔗LangChain

LangChain is a powerful Python library designed for natural language processing (NLP). It enables developers to convert text into embeddings, which are vector representations of textual data. These embeddings can be used for advanced analysis and manipulation of text data, making it possible to perform tasks like text classification, sentiment analysis, and document similarity with ease.

The Synergy of ChatGPT and 🦜🔗LangChain

When combined, ChatGPT and LangChain form an incredibly powerful force in the AI space. ChatGPT's conversational capabilities, along with LangChain's language understanding expertise, open up new possibilities for AI-driven solutions across various industries. The synergy of these technologies enables the creation of applications that not only comprehend human language but can also intelligently respond to queries and process vast amounts of textual data in realtime.

Applications of ChatGPT and 🦜🔗LangChain

1)Intelligent Chatbots and Virtual Assistants

The combination of ChatGPT and LangChain allows for the development of intelligent chatbots and virtual assistants capable of understanding context, maintaining coherence, and generating human-like responses. These AI-powered conversational agents can be used in customer support, sales, and various other business domains, improving user experience and reducing the need for human intervention.

2)Content Generation and Summarization

ChatGPT can be used to generate content, such as articles, blog posts, and social media updates, with impressive coherence and creativity. Meanwhile, LangChain's text analysis capabilities can be employed to summarize large documents or extract key information from a text, enabling users to access the most relevant data without sifting through extensive materials.

3)Sentiment Analysis and Opinion Mining

Using LangChain's embeddings, developers can perform sentiment analysis to gauge the emotions and opinions expressed in textual data. This information can be invaluable to businesses and organizations looking to understand customer feedback, monitor brand reputation, or analyze public opinions on various topics.

4)Language Translation and Multilingual Support

ChatGPT and LangChain can be used together to build applications that support multiple languages. ChatGPT can generate translations, while LangChain can analyze and process text data in various languages, enabling seamless communication and information exchange across linguistic barriers.

5)Personalized Recommendations

By analyzing user-generated text data, such as product reviews or social media posts, LangChain can help create personalized recommendations based on user preferences and opinions. ChatGPT can then be used to generate user-friendly, engaging content to communicate these recommendations effectively.

6)Code Analysis and Documentation

Developers can harness the power of ChatGPT and LangChain to analyze codebases, identify patterns, and generate documentation. This can improve code understanding, enable faster onboarding of new team members, and streamline the development process. Additionally, AI-driven code review tools can be built to identify potential issues, suggest improvements, and ensure adherence to best practices.

7)E-learning and Tutoring Systems

ChatGPT and LangChain can revolutionize e-learning and tutoring systems by offering personalized, AI-driven educational experiences. These technologies can be used to develop interactive learning platforms that provide instant feedback, answer questions, and adapt content based on individual learning styles and progress.

8)Natural Language Interface for Software Applications

Developers can leverage ChatGPT and LangChain to create natural language interfaces for software applications, allowing users to interact with complex systems using simple, human-like language. This can make software more accessible and user-friendly, particularly for non-technical users or those with limited expertise in a specific domain.

9)Social Media Analysis and Engagement

These technologies can be used to analyze social media content, identify trends, and gauge public sentiment. Businesses and organizations can then use ChatGPT to craft engaging, relevant content that resonates with their target audience, improving brand visibility and user engagement.

10)Creative Writing and Storytelling

ChatGPT's ability to generate human-like, coherent text can be harnessed for creative writing and storytelling. Writers can use ChatGPT as a collaborative tool to generate ideas, develop storylines, and even create entire narratives. LangChain can be employed to analyze text for style, tone, and structure, offering insights and suggestions for improvement.

Conclusion

The remarkable advancements in AI technologies, such as ChatGPT and LangChain, have the potential to transform the way we communicate, analyze data, and make decisions. By combining the strengths of these two groundbreaking innovations, we can create a future where AI not only enhances our daily lives but also empowers us to achieve more. As ChatGPT and LangChain continue to evolve and improve, we can anticipate even more incredible applications and use cases to emerge. The AI revolution is well underway, and the synergy between ChatGPT and LangChain is poised to be a driving force in shaping the future of conversational AI and natural language processing. Now is the time to embrace these cutting-edge technologies and unlock their full potential.

Spot VM in Google Cloud

Roy Rebello — Thu, 14 Oct 2021 04:12:13 +0000

What are Spot VMs?

Spot VMs are virtual machine (VM) instances with the spot provisioning model. Spot VMs are available at much lower price—a 60-91% discount—compared to the price of standard VMs. However, Compute Engine might preempt Spot VMs if it needs to reclaim those resources for other tasks. At this uncertain preemption time, Compute Engine either stops (default) or deletes your Spot VMs depending on your specified termination action for each VM. Spot VMs are excess Compute Engine capacity, so their availability varies with usage. Spot VMs do not have a minimum or maximum runtime.

If your workloads are fault-tolerant and can withstand possible VM preemption, Spot VMs can reduce your Compute Engine costs significantly. For example, batch processing jobs can run on Spot VMs. If some of those VMs stop during processing, the job slows but does not completely stop. Spot VMs complete your batch processing tasks without placing additional load on your existing VMs and without requiring you to pay full price for additional standard VMs.

Spot VMs limitations

Spot VMs function like standard VMs but have the following limitations:

Compute Engine might stop Spot VMs at any time due to system events. The probability that Compute Engine stops Spot VMs for a system event is generally low, but might vary from day to day and from zone to zone depending on current conditions.
Spot VMs are finite Compute Engine resources, so they might not always be available.
Spot VMs can't use live migrate to become standard VMs while they are running or be set to automatically restart when there is a maintenance event.
Due to the preceding limitations, Spot VMs are not covered by any Service Level Agreement and are excluded from the Compute Engine SLA.
The Google Cloud Free Tier credits for Compute Engine do not apply to Spot VMs.

How to create a Spot VM in Google Cloud ?

To create a VM from the gcloud tool, use the *gcloud beta compute instances create * command. To create Spot VMs, you must include the --provisioning-model=SPOT flag. Optionally, you can also specify a termination action for Spot VMs by also including the --instance-termination-action flag.

gcloud beta compute instances create my-spot-vm \
    --provisioning-model=SPOT \
    --instance-termination-action=DELETE

For instance-termination-action , you can specify which action to take when Compute Engine preempts the VM, either STOP (default behavior) or DELETE.

Like any other VM, Spot VMs start upon creation. Likewise, if Spot VMs are stopped, you can restart the VMs to resume the RUNNING state. You can stop and restart preempted Spot VMs as many times as you would like, as long as there is capacity. For more information, see VM instance life cycle.

If Compute Engine stops one or more Spot VMs in an autoscaling managed instance group (MIG) or Google Kubernetes Engine (GKE) cluster, the group restarts the VMs when the resources become available again.

Identifying Spot VMs

To describe a VM from the gcloud tool, use the gcloud beta compute instances describe command:

gcloud beta compute instances describe my-spot-vm

The output for describing Spot VMs includes the provisioningModel: spot field, similar to the following.

...
scheduling:
  ...
  provisioningModel: spot
  instanceTerminationAction: TERMINATION_ACTION
...

where TERMINATION_ACTION indicates which action to take when Compute Engine preempts the VM, either STOP or DELETE. If the instanceTerminationAction field is missing, the default behavior is STOP.

Detect preemption of Spot VMs

Determine if Spot VMs were preempted by Compute Engine using the gcloud command

gcloud compute operations list \
    --filter="operationType=compute.instances.preempted"

Few things to consider while using Spot VMs

Pick smaller machine shapes. Resources for Spot VMs come out of excess and backup Google Cloud capacity. It's often easier to get lots of capacity for Spot VMs with smaller machine types than larger ones. You might also get more spare capacity by using a custom machine type that is in between the predefined types. For example, there's likely more capacity for a custom machine type with 48 vCPUs than there are n1-standard-64s.
Run large clusters of Spot VMs during off peak times. The load on Google Cloud data centers varies with location and time of day, but generally lowest on nights and weekends. As such, nights and weekends are the best times to run large clusters of Spot VMs.
Design your applications to be fault and preemption tolerant. It's important to be prepared for the fact that there are changes in preemption patterns at different points in time. For example, if a zone suffers a partial outage, large numbers of Spot VMs could be preempted to make room for standard VMs that need to be moved as part of the recovery. In that small window of time, the preemption rate would look very different than on any other day. If your application assumes that preemptions are always done in small groups, you might not be prepared for such an event. You can test your application's behavior under a preemption event by stopping the VM.
Retry creating Spot VMs that have been preempted. If your Spot VMs have been preempted, try creating new Spot VMs once or twice before falling back to standard VMs. Depending on your requirements, it might be a good idea to combine standard VMs and Spot VMs in your clusters to ensure that work proceeds at an adequate pace.
Use shutdown scripts. Manage shutdown and preemption notices with a shutdown script that can save a job's progress so that it can pick up where it left off, rather than start over from scratch.

These articles are fueled by coffee. So if you enjoy my work and found it useful, consider buying me a coffee! I would really appreciate it.

Thank you for taking the time to read this post. If you've found this useful, please give it some likes, share and comment.

Creating a route based Cloud VPN in GCP

Roy Rebello — Sun, 12 Sep 2021 15:35:41 +0000

Hello everyone. In this blog post we are going to look at the Google Cloud VPN offering and also learn how to setup a classic VPN to connect two Compute instances setup in different VPC networks.

What is Cloud VPN ?

Cloud VPN securely extends your peer network to Google's network through an IPsec VPN tunnel. Traffic is encrypted and travels between the two networks over the public internet. Cloud VPN is useful for low-volume data connections. You can also connect two instances of Cloud VPN to each other.

Types of Cloud VPN

Google Cloud offers two types of Cloud VPN gateways: HA VPN and Classic VPN. However, Classic VPN is deprecating certain functionality on October 31, 2021

HA VPN

HA VPN is a high-availability (HA) Cloud VPN solution that lets you securely connect your on-premises network to your VPC network through an IPsec VPN connection in a single region. HA VPN provides an SLA of 99.99% service availability.

When you create an HA VPN gateway, Google Cloud automatically chooses two external IP addresses, one for each of its fixed number of two interfaces. Each IP address is automatically chosen from a unique address pool to support high availability. Each of the HA VPN gateway interfaces supports multiple tunnels.

Classic VPN

Classic VPN gateways have a single interface, a single external IP address, and support tunnels that use dynamic (BGP) or static routing (policy-based or route-based). They provide an SLA of 99.9% service availability

Setting up a Classic VPN

Let us implement the below scenario

I will be using gcloud commands to create the infrastructure as shown above.

Create first GCP project

gcloud projects create gcp-project-12s1

Create second GCP project

gcloud projects create gcp-project-12s2

Link both projects to an active billing account.

To find out your active billing account , use this command

gcloud beta billing accounts list --filter=open=true

Link the billing account to newly created projects

gcloud beta billing projects link gcp-project-12s1 --billing-account=0X0X0X-0X0X0X-0X0X0X

gcloud beta billing projects link gcp-project-12s2 --billing-account=0X0X0X-0X0X0X-0X0X0X

Create a VPC network in first GCP project with a subnet 10.5.1.0/24 in us-central1

gcloud compute networks create vpc-network-1 --project=gcp-project-12s1 --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional

gcloud compute networks subnets create subnet-a --project=gcp-project-12s1 --range=10.5.1.0/24 --network=vpc-network-1 --region=us-central1

Create a VPC network in second GCP project with a subnet 10.6.1.0/24 in europe-central2

gcloud compute networks create vpc-network-2 --project=gcp-project-12s2 --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional

gcloud compute networks subnets create subnet-b --project=gcp-project-12s2 --range=10.6.1.0/24 --network=vpc-network-2 --region=europe-central2

Create basic firewall rules

We will create firewall rules in both VPC for ICMP and SSH protocol to allow incoming traffic.

gcloud compute firewall-rules create network-1-allow-icmp --direction=INGRESS --priority=1000 --network=vpc-network-1 --action=ALLOW --rules=icmp --source-ranges=0.0.0.0/0 --project=gcp-project-12s1

gcloud compute firewall-rules create network-1-allow-ssh --direction=INGRESS --priority=1000 --network=vpc-network-1 --action=ALLOW --rules=tcp:22 --source-ranges=0.0.0.0/0 --project=gcp-project-12s1

gcloud compute firewall-rules create network-2-allow-icmp --direction=INGRESS --priority=1000 --network=vpc-network-2 --action=ALLOW --rules=icmp --source-ranges=0.0.0.0/0 --project=gcp-project-12s2

gcloud compute firewall-rules create network-2-allow-ssh --direction=INGRESS --priority=1000 --network=vpc-network-2 --action=ALLOW --rules=tcp:22 --source-ranges=0.0.0.0/0 --project=gcp-project-12s2

Create compute instances

Create VM server-1 in subnet-a

gcloud beta compute instances create server-1 --zone=us-central1-a --machine-type=e2-micro --subnet=subnet-a --network-tier=PREMIUM --maintenance-policy=MIGRATE --image=debian-10-buster-v20210817 --image-project=debian-cloud --boot-disk-size=10GB --boot-disk-type=pd-balanced --boot-disk-device-name=server-1 --project=gcp-project-12s1

Create VM server-2 in subnet-b

gcloud beta compute instances create server-2 --zone=europe-central2-a --machine-type=e2-micro --subnet=subnet-b --network-tier=PREMIUM --maintenance-policy=MIGRATE --image=debian-10-buster-v20210817 --image-project=debian-cloud --boot-disk-size=10GB --boot-disk-type=pd-balanced --boot-disk-device-name=server-2 --project=gcp-project-12s2

Now we will attempt to do a ping test from server-1 to server-2 and vice versa.

As you can see the results above we see 100% data packet loss because both the VPC networks cannot communicate with each other.

In order to make them communicate with each other we would need a static IP address which will be binded to a VPN network on both sides.

So lets create a static IP address for both the VPC networks

gcloud compute addresses create vpn-1-static-ip --project=gcp-project-12s1 --region=us-central1

gcloud compute addresses create vpn-2-static-ip --project=gcp-project-12s2 --region=europe-central2

The static IP's are now created but not in use.

So now lets create a VPN connection and a VPN tunnel for vpc-network-1

// Creating a classic VPN
gcloud compute target-vpn-gateways create vpn-1 --project=gcp-project-12s1 --region=us-central1 --network=vpc-network-1

// Creating the forwarding rules
gcloud compute forwarding-rules create vpn-1-rule-esp --project=gcp-project-12s1 --region=us-central1 --address=104.198.144.53 --ip-protocol=ESP --target-vpn-gateway=vpn-1

gcloud compute forwarding-rules create vpn-1-rule-udp500 --project=gcp-project-12s1 --region=us-central1 --address=35.225.21.155 --ip-protocol=UDP --ports=500 --target-vpn-gateway=vpn-1

gcloud compute forwarding-rules create vpn-1-rule-udp4500 --project=gcp-project-12s1 --region=us-central1 --address=35.225.21.155 --ip-protocol=UDP --ports=4500 --target-vpn-gateway=vpn-1

// Creating a VPN tunnel to destination IP 34.118.8.129 
gcloud compute vpn-tunnels create tunnel1to2 --project=gcp-project-12s1 --region=us-central1 --peer-address=34.118.8.129 --shared-secret=gcprocks --ike-version=2 --local-traffic-selector=0.0.0.0/0 --remote-traffic-selector=0.0.0.0/0 --target-vpn-gateway=vpn-1

// Creating a route to route traffic from VM server-1 to VM server-2 via VPN tunnel
gcloud compute routes create tunnel1to2-route-1 --project=gcp-project-12s1 --network=vpc-network-1 --priority=1000 --destination-range=10.6.1.0/24 --next-hop-vpn-tunnel=tunnel1to2 --next-hop-vpn-tunnel-region=us-central1

Similarly lets create a VPN connection and a VPN tunnel for vpc-network-2

// Creating a classic VPN
gcloud compute target-vpn-gateways create vpn-2 --project=gcp-project-12s2 --region=europe-central2 --network=vpc-network-2

// Creating the forwarding rules
gcloud compute forwarding-rules create vpn-2-rule-esp --project=gcp-project-12s2 --region=europe-central2 --address=34.118.8.129 --ip-protocol=ESP --target-vpn-gateway=vpn-2

gcloud compute forwarding-rules create vpn-2-rule-udp500 --project=gcp-project-12s2 --region=europe-central2 --address=34.118.8.129  --ip-protocol=UDP --ports=500 --target-vpn-gateway=vpn-2

gcloud compute forwarding-rules create vpn-2-rule-udp4500 --project=gcp-project-12s2 --region=europe-central2 --address=34.118.8.129 --ip-protocol=UDP --ports=4500 --target-vpn-gateway=vpn-2

// Creating a VPN tunnel to destination IP 35.225.21.155 
gcloud compute vpn-tunnels create tunnel2to1 --project=gcp-project-12s2 --region=europe-central2 --peer-address=35.225.21.155 --shared-secret=gcprocks --ike-version=2 --local-traffic-selector=0.0.0.0/0 --remote-traffic-selector=0.0.0.0/0 --target-vpn-gateway=vpn-2

// Creating a route to route traffic from VM server-2 to VM server-1 via VPN tunnel
gcloud compute routes create tunnel2to1-route-1 —project=gcp-project-12s2 --network=vpc-network-2 --priority=1000 --destination-range=10.5.1.0/24 --next-hop-vpn-tunnel=tunnel2to1 --next-hop-vpn-tunnel-region=europe-central2

Now we can see the status of the VPN tunnel as Established .

We can now attempt to perform a ping test from server-1 to server-2 and vice versa.

We have successfully established connectivity between the two VPC networks over the internet securely using encryption via VPN.

This completes the tutorial on how to setup a Classic VPN on GCP. The scenario is also similar for establishing VPN connectivity from an on-premise network to a Google Cloud network.

These articles are fueled by coffee. So if you enjoy my work and found it useful, consider buying me a coffee! I would really appreciate it.

Thank you for taking the time to read this post. If you've found this useful, please give it some likes, share and comment.