DEV Community: Nevenka Lukic

Building my online resume on Azure — “Cloud resume challenge”

Nevenka Lukic — Tue, 21 Nov 2023 14:29:52 +0000

After switching careers in 2022, I spent the past year working as a Data Engineer. This job provided me with lots of chances to learn new skills. Looking for a project to improve my abilities, I found the "Cloud Resume Challenge Guidebook-Azure" by Forrest Brazeal. This challenge seemed perfect for two main reasons: it would help me brush up on my Web Development skills and also let me learn more about Azure and Python.

Taking on this project was quite challenging, especially because of the complex steps involved in installing, configuring, and fixing compatibility issues. I'll talk more about these challenges in a bit.

In the end, I successfully completed the project, and now I have My Resume hosted on Azure, along with its corresponding GitHub repository.

Project Objectives

This project consists of series of challenges:

Development of a Website: Utilizing HTML, CSS, and JavaScript to showcase resume details.
Integration of a Visitor Counter: Implementing a JavaScript-based visitor counter interacting with a database via an Azure Function in Python.
Python Code Testing: Inclusion of unit tests for the Python code.
GitHub Repository: Storage of all code within a GitHub repository.
Automated Deployment: Configuration of GitHub Actions for automated deployment of code changes to the production environment upon each commit.
Testing Process Integration: Establishing a testing process for API code with automated testing integrated into GitHub Actions.

Preparation

Before starting the project, some tools needs to be installed and configured:

Visual Studio Code with Azure Extensions
The Azure CLI
Configuration of the local Python environment for Azure
Azure Functions Core Tools for local development of serverless functions
An active Azure Account

This guiding video from A Cloud Guru helped me a lot, despite the project being done in C#.

Web Development

If you want to make web pages quickly or you're not too familiar with HTML and CSS, Bootstrap provides free templates that are handy to start with. You can then customize them to suit your own style.

Next step was making a Storage Account on Azure and enabling the Static Website feature. This allowed me to create a special container called $web. Putting my resume online as an Azure Storage Static Website was a new and interesting experience, made easy with the help of the Azure Storage extension in VS Code.

Getting a domain name

You can search for a free domain name and buy it from any of multiple domain providers such as Namecheap, Godaddy etc.

Custom Domain Configuration

To set up a custom domain and enable HTTPS, I used Azure CDN. The first thing was getting the custom domain set up through the DNS provider. This step was needed, but it might take up to 48 hours for everything to get set up.

After that, I connected the custom domain to the CDN endpoint. The last step was turning on HTTPS by choosing and validating a certificate.

Building the API

To make a JavaScript-based visitor counter work, I had to create a Cosmos DB on Azure with a special container to keep track of visitor numbers. The bridge between the website and the database was an Azure Function written in Python. However, I ran into problems because of the specifications of my system, which is a Mac Pro with an M1 chip and Python 3.11. It took a lot of effort to overcome these issues, and I wrote about the whole process in a blog.

After that, I improved the function to meet the project's needs, counting visitors and showing the count on the web page. Making sure the Azure function could connect with Cosmos DB was done using a feature called binding, made easy with the Azure Functions extension in VS Code.

Front-end / Back-end Integration:

The ultimate goal is to achieve seamless updates for both the back-end API and the front-end website, involving the concept of continuous integration and deployment (CI/CD). To accomplish this, GitHub Actions were utilized. GitHub provides Linux, Windows, and macOS virtual machines to execute workflows, which are defined by a YAML file in the .github/workflows directory of a repository. A repository can have multiple workflows, and in this case, two were created: frontend.main.yml and backend.main.yml.

The frontend workflow is designed for the deployment of the static website in Azure storage. Microsoft offers comprehensive documentation and even a template for this workflow. However, a peculiar issue arose during the workflow execution. Although the build was successful, the "Upload to blob storage" section displayed multiple errors, all indicating "BlobAlreadyExists." The suggested solution was to add "--overwrite" to the command, which resolved the issue.

The backend workflow, tailored for Python code and testing, also benefited from Microsoft's documentation and template. Two adjustments were necessary to make it functional.

1. An error occurred: "No module named 'azure'." This was perplexing because the module was specified in the requirement.txt file.

After some troubleshooting and experimentation, the error was resolved by removing his underline part from my file.

2. The Microsoft template lacked a section for testing, requiring manual implementation. The crucial aspect here was ensuring the correct path, which took the following form:

Conclusion

Doing the "Cloud Resume Challenge Guidebook-Azure" was a big journey for me as a Data Engineer. This project helped me refresh my web development skills and learn more about Azure and Python, which was great for my job.

From figuring out how to install and set things up to making a working online resume and API, each step taught me a lot. Even though I faced some problems, like my computer not liking certain things, I didn't give up and found solutions.

While I know there's still more I can improve and automate, what I have now shows how much I've learned and how far I've come. As I keep working on making updates happen smoothly through GitHub Actions, I'm sure this experience will be a good base for future projects and my career.

In the end, this project isn't just a resume online. It's a reminder that facing challenges is a chance to get better, and I'm excited about what comes next.

Solving Azure Functions Compatibility Challenges on Mac M1: A Step-by-Step Guide

Nevenka Lukic — Sun, 29 Oct 2023 21:36:55 +0000

Introduction

While working on a new project that involves Azure Functions and Python, I faced an unexpected roadblock. I was surprised to discover that Azure Functions didn't seamlessly support my Macbook Pro with an M1 CPU. Two significant constraints became apparent:

1. Incompatibility with Apple M1/2 CPUs

Azure Functions, as of my initial research, did not have native support for Apple's M1/2 CPUs, causing a hiccup in my development journey.

2. Dependency on Older Python 3.9

Another challenge arose when I realized that Azure Functions required an older Python version, specifically Python 3.9, instead of the more recent Python 3.11.

Overcoming these roadblocks demanded substantial time and research. Today, I want to share the solution I found, hoping to assist anyone else fighting with this issue.

The Path to Compatibility: A Three-Part Journey

My quest to make Azure Functions work on my Mac M1 led me through a series of steps, which I've broken down into three parts for your convenience. By following these steps in order, you can ensure a smooth setup and overcome the compatibility obstacles.

Part 1: Setting Up x86 Emulation on ARM64 with Rosetta

The first phase of our journey involves setting up x86 emulation on ARM64 architecture with Rosetta. This step is crucial for ensuring compatibility between your Mac M1 and Azure Functions.

Part 2: Installing Required Packages and Tools

In the second part, we'll delve into the installation of necessary packages and tools that will enable Azure Functions to work seamlessly with Python 3.9. This step bridges the gap between the platform's requirements and the tools you need to build your project.

Part 3: Configuring .zshrc and Visual Studio Code Settings

The final leg of our journey focuses on configuring your .zshrc file and fine-tuning Visual Studio Code settings to ensure a smooth development experience. This step will help you make the most of your development environment and optimize your workflow.

In conclusion, solving issues with Azure Functions on a Mac M1 might seem hard at first, but with the right guidance, you can overcome the compatibility challenges. By following these three parts in order you should be able to solve this very quickly.

STEPS:

Install Rosetta
- To run older non-native Intel x86 apps on your Apple Silicon Mac (M1), you'll need to install Rosetta. Use the command: /usr/sbin/softwareupdate --install-rosetta --agree-to-license This step is specific to Apple Silicon ARM Macs and is not required for Intel Macs.
Duplicate iTerm for Rosetta
- Open Finder and duplicate your iTerm Terminal application. Rename the copied application to something like "iTerm rosetta."
Enable Rosetta in Terminal
- Right-click on your iTerm Rosetta copy and select "Get Info."
- In the Get Info window, select "Open using Rosetta."
- Open this new terminal and type arch. It should now show i386 or x86_64.
Install Homebrew in an Emulated Environment
- If you've previously installed Homebrew on your computer, you'll need to do it again within the Rosetta terminal.
- The following blog post was very useful.
  - Create a ~/Downloads/homebrew directory and download the Homebrew tarball, then extract it to the ~/Downloads/Homebrew directory.
  - Run these commands to move the Homebrew directory to /usr/local/Homebrew and add the path to your .zshrc file:
```
   sudo mv homebrew /usr/local/Homebrew
   export PATH=$HOME/bin:/usr/local/Homebrew/bin:$PATH
```

You can now use arch -x86_64 /usr/local/Homebrew/bin/brew to install apps to the /usr/local/Homebrew/Cellar directory.

Create an Alias for New Homebrew

Add an alias and path to your ~/.zshrc file to make working with this new Homebrew environment easier. Here's an example:

 # need this for x86_64 brew
 export PATH=$HOME/bin:/usr/local/Homebrew/bin:$PATH
 # for Intel x86_64 brew
 alias axbrew='arch -x86_64 /usr/local/Homebrew/bin/brew'

Install Azure Functions under Alias

You can use the alias you created to install Azure Functions:

 axbrew tap azure/functions
 axbrew install azure-functions-core-tools@4
 # if upgrading on a machine that has 2.x or 3.x installed:
 axbrew link --overwrite azure-functions-core-tools@4

Install Python under Alias
- To use Azure Functions, you need to install Python version 3.9 using pyenv:
```
 axbrew install pyenv
 pyenv install 3.9
 pyenv global 3.9
```

Add the following to your .zshrc to make Python 3.9 the default in this environment:

 if command -v pyenv 1>/dev/null 2>&1; then
   eval "$(pyenv init -)"
 fi

Set Up Visual Studio Code Terminal Profile to Use Rosetta
- In Visual Studio Code, open settings.json:
  - Click on "Code" at the top left.
  - Then select "Settings" and "Settings" again.
  - On the top right side, you'll find the "settings.json" icon. Click to open and add the following:
```
 "terminal.external.osxExec": "iTerm rosetta.app",
 "terminal.integrated.profiles.osx": {
     "x86 zsh": {
         "path": "/usr/bin/arch",
         "args": ["-arch", "x86_64", "/bin/zsh"]
     }
 }
```

Conclusion

By following these steps, you can successfully configure your Mac M1 to work with Azure Functions and Python, and overcome the hardware compatibility issues. And now when you have the right setup, you can continue working on your Azure development projects with your favorite Macbook Pro.

Unveiling the Azure Data Lake for Bike Share Data Analytics

Nevenka Lukic — Wed, 11 Oct 2023 10:02:14 +0000

This is the second big project from the course Data Engineering with Azure specialization on Udacity that I've recently completed. If you want to read about my first project, you can find it in this blog.

Data Engineering with Azure could be your gateway to unlocking thrilling projects like building a comprehensive data lake solution. In this blog post, we will unravel the great journey of developing a data lake solution for Divvy bikeshare, a bike sharing program based in Chicago, Illinois, USA.

You can find the code related to this project in my GitHub repository.

Project Overview

Divvy bike share is a popular program that allows riders in Chicago to access bikes via kiosks or a mobile application. The anonymized bike trip data from Divvy is made publicly available by the City of Chicago for analysis, forming the foundation of this data engineering project.

The goal of this project was to develop a data lake solution using Azure Databricks, using a lake house architecture. The objectives included designing a star schema, importing data into Azure Databricks using Delta Lake for creating bronze and gold data stores, and transforming the data into the star schema for the gold data store.

If you are interested and want to read the whole story, go to this link

Unlocking Insights: Azure Synapse Analytics Bike Share Data Warehouse Project

Nevenka Lukic — Sat, 07 Oct 2023 14:13:06 +0000

I recently completed Data Engineering with Azure specialization on Udacity. It took me three months to finish it. It was very challenging but also a very rewarding experience. I learned a lot and upgraded my Data engineering skills and Cloud knowledge.

In this blog I would like to share my experiences with anyone who might be interested in this specialization. I sincerely recommend this course and hope that this blog will be valuable to other people.

You can find the code related to this project in my GitHub repository.

Project introduction

This was the first big project in this course: Azure Synapse Analytics Bike Share Data Warehouse Project

The project offers a great chance to work with real-world data and build a complete data warehouse using Azure Synapse Analytics.

We'll dive into the interesting process of designing a star schema and organizing Divvy's bike share data into a structured, usable format.

If you are interested and want to read the whole story, go to this link