DEV Community: Aditya Raman

Seasons of Serverless, Lovely Ladoos

Aditya Raman — Tue, 01 Dec 2020 00:25:11 +0000

This article is part of #SeasonsOfServerless.

Each week we will publish a challenge created by Azure Advocates with amazing Student Ambassadors around the world. Discover popular festive recipes and learn how Microsoft Azure empowers you to do more with Serverless Functions! 🍽 😍.
Explore our serverless Resources here and learn how you can contribute solutions here.

Check original repository at, https://github.com/microsoft/Seasons-of-Serverless
Serverless Solution for Lovely Ladoos: https://github.com/ramanaditya/Seasons-of-Serverless-Solution-Lovely-Ladoos

Challenge 2: Lovely Ladoos

Featured Region: India
Your Chefs: Jasmine Greenaway, Cloud Advocate (Microsoft) with Soumya Narapaju and Aditya Raman, Microsoft Student Ambassadors

It's Diwali season in India! A very popular delicacy that Indians eat during Diwali are ladoos, balls of flour dipped in sugar syrup. Can you create a machine learning model to detect a proper ladoo? Learn more about this challenge.

Teaser

Solution

The Solution creates an image classifier using Custom Vision API
The solution also creates a Serverless Endpoint which is used to
- List all the images stored on the Azure Blob Storage
- Upload a new image on the Azure Blob Storage
- Predict the already uploaded image using Custom Vision API

Serverless Compute

Serverless computing enables developers to build applications faster by eliminating the need for them to manage infrastructure. With serverless applications, the cloud service provider automatically provisions, scales and manages the infrastructure required to run the code.

Altogether it can be summarized as

Function as a Service: Modular way of writing business logics as a different application(s) which is/are triggered with the event(s). eg., Azure Functions
Stateless Compute Containers: Service which does not require to read/store their previous states after each start.
Event-triggered and Ephemeral: Triggered only by events, and they exist until the time it is triggered. If not triggered for a period, the service gets down allowing other services to use their resources.
Dynamically Allocated: Services gets down when not in use so their resources might be used by other services, that is why they have to be redeployed(behind the scene) thus dynamically allocated.
Pay as you go: Pay for the services that have actually been used.

Advantages and Disadvantages of using Serverless

Advantages	Disadvantages
Lower Operational Cost	Complex Architecture
Easily Scalable	Execution Time
Billing is based upon Usage	Execution Frequency
Easy Deployment
Low Cost

Azure CLI

Azure CLI: The Azure command-line interface (Azure CLI) is a set of commands used to create and manage Azure resources. The Azure CLI is available across Azure services and is designed to get you working quickly with Azure, with an emphasis on automation.

Azure CLI Commands

Login to Azure: The command will open the tab in the browser letting you to log in to the portal.

az login

To check all the available locations

az account list-locations \
    --query "[].{Region:name}" \
    --out table

Setting up the subscription

If this step is performed you can skip the subscription argument while creating group, storage and applications. This set as the default subscription on your local system.

# To see all the available subscriptions
az account list 
# To set the subscription
az account set --subscription <Subscription-ID or Subscription-Name>

Create Azure Resource Group

Azure Resource Group is the container to store the resources as the directory in your system. It helps in managing the resources associated with a similar type of solutions or the related one.

az group create --name <Your-Resource-Group-Name> --location eastus

Blob Storage

Azure Blob storage is Microsoft's object storage solution for the cloud. Blob storage is optimized for storing massive amounts of unstructured data. Unstructured data is data that doesn't adhere to a particular data model or definition, such as text or binary data.

az storage account create \
    --name <Blob Storage Name> \
    --resource-group <Resource Group Name> \
    --location eastus \
    --sku Standard_ZRS \
    --encryption-services blob

Create Container

A container organizes a set of blobs, similar to a directory in a file system. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.

az ad signed-in-user show --query objectId -o tsv | az role assignment create \
    --role "Storage Blob Data Contributor" \
    --assignee @- \
    --scope "/subscriptions/<Subscription>/resourceGroups/<Resource Group Name>/providers/Microsoft.Storage/storageAccounts/<Blob Storage Name>"

az storage container create \
    --account-name <Storage Account> \
    --name <Container Name> \
    --auth-mode login

Azure Functions

Azure Functions is a serverless compute service that lets you run event-triggered code without having to explicitly provision or manage infrastructure.

Azure Functions Core Tools: To develop and test the functions locally using a terminal or command prompt. This is required and can be downloaded from here.

Creating Functions Project

The command initializes the project and creates a project directory - worker-runtime denotes the language you want to use in the application

# This will generate a folder containing two files host.json and local.settings.json
func init Seasons-of-Serverless-Solution-Lovely-Ladoos --worker-runtime node

cd Seasons-of-Serverless-Solution-Lovely-Ladoos

Note: The values of these files can be changed according to the connection strings which you had or will have for DB, storage etc.

Creating Functions Template

To Create a JavaScript HTTP Trigger, this is a sample template to create the HTTP Trigger.

# Here a pre-built template for the Http Trigger will be created
# with two files, function.json and index.js
# We are going to write our logics in the index.js
# To create blob endpoint
func new --template "Http Trigger" --name blobs

# To create prediction endpoint
func new --template "Http Trigger" --name predict

Initialize your nodeJs project with npm

npm init

Refer to the GitHub Repository for the complete code: ramanaditya/Seasons-of-Serverless-Solution-Lovely-Ladoos

List all the blobs in a Blob Storage Container

const { BlobServiceClient, StorageSharedKeyCredential, newPipeline } = require('@azure/storage-blob');

const sharedKeyCredential = new StorageSharedKeyCredential(
  process.env.AZURE_STORAGE_ACCOUNT_NAME,
  process.env.AZURE_STORAGE_ACCOUNT_ACCESS_KEY
);
const pipeline = newPipeline(sharedKeyCredential);
const containerName = process.env.CONTAINER_NAME;

const blobServiceClient = new BlobServiceClient(`https://${process.env.AZURE_STORAGE_ACCOUNT_NAME}.blob.core.windows.net`, pipeline );

const containerClient = blobServiceClient.getContainerClient(containerName);

const listBlobsResponse = await containerClient.listBlobFlatSegment();

Uploading the image to the Blob Storage

const { BlobServiceClient } = require('@azure/storage-blob');
const streamifier = require("streamifier");

const containerName = process.env.CONTAINER_NAME;

const blobServiceClient = BlobServiceClient.fromConnectionString(process.env.AzureWebJobsStorage);

const containerClient = blobServiceClient.getContainerClient(containerName);;

const blockBlobClient = containerClient.getBlockBlobClient(fileName);

const result = await blockBlobClient.uploadStream(streamifier.createReadStream(Buffer.from(fileData)), fileData.length);

Predict the image with image URL

// Import Dependencies
const msRest = require("@azure/ms-rest-js");
const { PredictionAPIClient } = require("@azure/cognitiveservices-customvision-prediction");

// Get the env variables
const projectId = process.env.PROJECT_ID
const publishedName = process.env.PUBLISHED_NAME
const predictionKey = process.env.PREDICTION_KEY
const endPoint = process.env.PREDICTION_ENDPOINT
const predictionResourceId = process.env.PREDICTION_RESOURCE_ID

const predictor_credentials = new msRest.ApiKeyCredentials({ inHeader: { "Prediction-key": predictionKey } });

const predictor = new PredictionAPIClient(predictor_credentials, endPoint);

const results = await predictor.classifyImageUrl(projectId, publishedName, { url: imageUrl });

Custom Vision API

Custom Vision lets you build, deploy, and improve your own image classifiers. An image classifier is an AI service that applies labels (which represent classes) to images, based on their visual characteristics.

Note: The solution here is not the only solution, one can use the Serverless Function or the Azure CLI for the same, we just went with the simpler method of using the Azure Portal.

Log in the Azure Portal, search for Cognitive Services → Custom Vision or search for Custom Vision, and click Create.
Select the subscription, resource group you created above.
Select the pricing tier best suited for your purpose, well here I would go with Free F0 Pricing Tier.
It takes a few minutes to deploy
Navigate to 'Go to prediction resource' ie., customvision.ai portal
Create a New Project

Name: <Any Name>
Description: <Any Description>
Resources: <The Resource just created>
Project Types: Classification
Classification Types: Multiclass (Single tag per image)
Domains: Food

On the Custom Vision Portal, click on 'Add Images', and upload the images of "ladoos" and tag them as "ladoo". At the end Upload all the files.

For your simplicity, I have attached the dataset: ramanaditya/Seasons-of-Serverless-Solution-Lovely-Ladoos/tree/main/datasets

Similarly, upload the images of doughnut and sesame buns.
Once all the images are uploaded click on Train, you can select any training options, here I have selected "Quick Training".
Once the training is completed you will find the beautiful statistics
Click on "Publish" to publish your prediction model to be used in the Azure Functions.
While publishing save the "Published Name" to your .env file
Navigate to settings and store Project ID, Prediction Endpoint, Prediction Key and Prediction Resource ID in the .env file.

Run locally

func start

Check the working of API Endpoints

GET: /api/blobs

List all the images stored with their image URL

POST: /api/blobs

To upload the image to the blob storage

GET: /api/predict?imageurl=<imageUrl>

To predict the image

Chefs for the Challenge

Jasmine Greenaway, Cloud Advocate (Microsoft)
Rama Soumya Naraparaju, Microsoft Student Ambassadors
Aditya Raman (me), Microsoft Student Ambassadors

Open Source Projects Thread

Aditya Raman — Thu, 01 Oct 2020 16:19:01 +0000

A little help will guide someone to eventually create some indistinguishable future.

Hacktoberfest 2020

A complete month of October dedicated to Open Source Development and learning more about different open-source software. We get a chance to contribute to the development of open-source software and communities.
Although it is common for the developers who continue to develop throughout the year. But the best thing is every year the freshers or the people who just started with the development gets an awesome timeline to learn and contribute.

Perks

You get a choice to pick a limited edition T-shirt or plant a tree. But that is just motivation, else you have a chance to step up in the world of innovation and development.

Why this thread

Generally, freshers have a great challenge in selecting the project. They try going with the listed projects on the Hacktoberfest website, which are really little difficult or they aren't resourceful or knowledgeable to get their PR accepted.

So We(You and me) are creating this thread and you can add the open-source projects which are accepting the PR from the people broadly classified into three parts as easy, medium and hard. You can add first-timer as well.

You can append your projects or the projects you know are accepting PRs for this Hacktoberfest.

Important Links:

Hacktoberfest Website

Request a comment

Add a link to the GitHub repository or the project website
Add label first-timer, easy, medium, hard
Post the comment
If you are a part of a community who supports open-source, do mention them or the link to their website so that people might like to contribute and connect for their development.
Yay! you just helped someone to make their first contribution.

You are free to share and repost with or without any permission(s). Solely used for Open Source Development.

Serverless with GraphQL #ServerlessSeptember

Aditya Raman — Wed, 09 Sep 2020 16:52:42 +0000

Everything on Azure. Scalable and cost-effective Serverless system leveraging GraphQL API hosted on Azure. Easy deployable and production-ready code with Azure CLI and Azure DevOps. Built a simple API to query your professional details. You can modify and create your own API, isn't it beautiful.

GraphQL API URL: https://ramanaditya-graphql.azurewebsites.net/graphql?code=mWybiYT9VGKSkNsZvuzMPajaNiMYvbEBhc6LErNr8hnXdeunrqOR7w==

About the Project

Medium Article: https://medium.com/@ramanaditya/serverless-with-graphql-c0bf59078303

This article is part of #ServerlessSeptember https://aka.ms/ServerlessSeptember2020. You'll find other helpful articles, detailed tutorials, and videos in this all-things-Serverless content collection. New articles from community members and cloud advocates are published every week from Monday to Thursday through September.

Find out more about how Microsoft Azure enables your Serverless functions at

https://docs.microsoft.com/azure/azure-functions/

Query

query {
    hello
}

query {
    name: {
        firstName
        lastName
    }
}

query {
    connect {
        professional {
            email
            website
            linkedin
            github
        }
        social {
            twitter
            instagram
        }
    }
}

query {
    skills {
        language
        database
        cloudService
        frameworks
        vcs
        api
        industryKnowledge
    }
}

Important Links:

Medium: https://medium.com/@ramanaditya/serverless-with-graphql-c0bf59078303
Azure DevOps: https://bellatrixdata@dev.azure.com/bellatrixdata/serverless-graphql/_git/serverless-graphql
GitHub: https://github.com/ramanaditya/serverless-graphql.git
GraphQL API URL: https://ramanaditya-graphql.azurewebsites.net/graphql?code=mWybiYT9VGKSkNsZvuzMPajaNiMYvbEBhc6LErNr8hnXdeunrqOR7w==

Reconfiguring your journey to Data Science

Aditya Raman — Thu, 28 May 2020 07:52:04 +0000

Nothing is better than writing what you have learnt, the journey was quite tough, but yes, YOU can do this.
I have written a blog on "Reconfiguring your journey to Data Science" . The blog will guide you when to jump on this technology and when to prepare more before starting with Data Science.

Do read this and let me know your feedback.

Reconfigure your journey to Data Science | by Aditya Raman | bellatrixdata | Medium

Aditya Raman ・ May 30, 2020 ・
Medium

I will be quite happy if my work helps you in any way. Let me know about this too.

If you want me to write on any special segment, comment below, I will look into it.

Hacktoberfest

Aditya Raman — Mon, 30 Sep 2019 22:48:18 +0000

hacktoberfest

This is the time when everyone come forward to contribute towards open source.

DEV Community: Aditya Raman

Seasons of Serverless, Lovely Ladoos

Challenge 2: Lovely Ladoos

Teaser

Solution

Serverless Compute

Altogether it can be summarized as

Advantages and Disadvantages of using Serverless

Azure CLI

Azure CLI Commands

To check all the available locations

Setting up the subscription

Create Azure Resource Group

Blob Storage

Create Container

Azure Functions

Creating Functions Project

Creating Functions Template

Initialize your nodeJs project with npm

List all the blobs in a Blob Storage Container

Uploading the image to the Blob Storage

Predict the image with image URL

Custom Vision API

Run locally

Check the working of API Endpoints

GET: /api/blobs

POST: /api/blobs

GET: /api/predict?imageurl=<imageUrl>

Chefs for the Challenge

Open Source Projects Thread

Hacktoberfest 2020

Perks

Why this thread

Important Links:

Request a comment

Serverless with GraphQL #ServerlessSeptember

About the Project

Query

Important Links:

Reconfiguring your journey to Data Science

Reconfigure your journey to Data Science | by Aditya Raman | bellatrixdata | Medium

Aditya Raman ・ May 30, 2020 ・ Medium

Hacktoberfest

hacktoberfest

Aditya Raman ・ May 30, 2020 ・
Medium