DEV Community: Syed Nashet Ali

Top 10 Cloud Security Startups to Watch in 2024

Syed Nashet Ali — Tue, 16 Jul 2024 20:00:05 +0000

As cloud environments continue to evolve, several startups are making significant strides in providing innovative cloud security solutions. These startups are not only leveraging cutting-edge technologies but also addressing the specific challenges that organizations face. In this blog, we'll explore the top 10 cloud security startups of 2024, diving into their unique offerings, behind-the-scenes technology, case studies, and client responses.

1. Aqua Security

Overview: Aqua Security specializes in securing cloud-native environments. Their recent updates include the Kubernetes Bill of Materials (KBOM) and AI-Guided Remediation capabilities.

Tech Behind the Scenes: Aqua Security utilizes machine learning to analyze data from various sources, providing automated remediation steps. The KBOM tool dissects Kubernetes clusters to identify all components, enhancing security monitoring.

Case Study:

Client: A leading e-commerce platform
Challenge: Securing a complex Kubernetes environment with multiple clusters.
Solution: Implemented Aqua Security’s KBOM and AI-Guided Remediation.
Results: Reduced vulnerability remediation time by 40%, improved visibility, and compliance.

Client Response: "Aqua Security has significantly streamlined our Kubernetes security processes. The automated remediation is a game-changer."

Code Snippet:

apiVersion: v1
kind: Pod
metadata:
  name: secure-pod
spec:
  containers:
  - name: secure-container
    image: secure-image:latest
    securityContext:
      readOnlyRootFilesystem: true
      runAsNonRoot: true

More Info: Aqua Security

2. Dazz

Overview: Dazz offers a platform focused on the prioritization and remediation of cloud vulnerabilities with its Unified Remediation Platform.

Tech Behind the Scenes: Dazz uses data correlation techniques to connect related security issues and trace them back to their root causes, enabling efficient remediation.

Case Study:

Client: A global financial services company
Challenge: High volume of security alerts with limited context.
Solution: Adopted Dazz’s Unified Remediation Platform.
Results: 50% reduction in false positives, quicker identification of critical vulnerabilities.

Client Response: "Dazz has transformed our vulnerability management process, making it more efficient and focused."

Code Snippet:

import dazz_sdk

client = dazz_sdk.Client(api_key='your_api_key')
vulnerabilities = client.get_vulnerabilities()
critical_vulns = [v for v in vulnerabilities if v.severity == 'Critical']
for vuln in critical_vulns:
    client.remediate_vulnerability(vuln.id)

More Info: Dazz

3. Orca Security

Overview: Orca Security provides AI-driven cloud asset search for fast visibility into cloud environments.

Tech Behind the Scenes: Orca leverages large language models (LLMs) to create a question-and-answer functionality, enabling users to query their cloud assets and receive detailed responses.

Case Study:

Client: A healthcare provider
Challenge: Lack of visibility into cloud assets and security posture.
Solution: Deployed Orca Security’s AI-driven search functionality.
Results: Achieved 100% visibility into cloud assets, improved incident response times.

Client Response: "Orca Security has given us unprecedented visibility and control over our cloud environment."

Code Snippet:

{
  "query": "List all EC2 instances with open security groups",
  "results": [
    {
      "instance_id": "i-1234567890abcdef",
      "security_group": "sg-12345",
      "open_ports": [22, 80]
    }
  ]
}

More Info: Orca Security

4. Cyble

Overview: Cyble uses AI to monitor the Darkweb, Deepweb, and Surface Web for cyber threats, providing real-time insights and threat intelligence.

Tech Behind the Scenes: Cyble's platform continuously scans and analyzes data from various sources, leveraging machine learning to identify potential threats and vulnerabilities.

Case Study:

Client: A major retail chain
Challenge: Frequent data breaches and lack of threat intelligence.
Solution: Implemented Cyble’s threat intelligence platform.
Results: Early detection of potential breaches, improved security posture.

Client Response: "Cyble’s threat intelligence capabilities have been instrumental in protecting our data."

Code Snippet:

import cyble_sdk

client = cyble_sdk.Client(api_key='your_api_key')
threats = client.get_threats()
for threat in threats:
    if threat.risk_level == 'High':
        client.alert_security_team(threat.id)

More Info: Cyble

5. Illumio

Overview: Illumio provides microsegmentation technology for zero-trust security, preventing the lateral movement of breaches within data centers and cloud environments.

Tech Behind the Scenes: Illumio's platform uses dynamic policies and real-time traffic analysis to enforce security boundaries within and across cloud environments.

Case Study:

Client: A large manufacturing firm
Challenge: Preventing lateral movement of attacks within the network.
Solution: Deployed Illumio’s microsegmentation solution.
Results: 60% reduction in breach impact, enhanced network security.

Client Response: "Illumio’s microsegmentation has fortified our internal security against breaches."

Code Snippet:

apiVersion: illumio.com/v1
kind: SecurityPolicy
metadata:
  name: zero-trust-policy
spec:
  rules:
  - action: Allow
    source: app-tier
    destination: db-tier
    ports: [5432]
  - action: Deny
    source: app-tier
    destination: any

More Info: Illumio

6. Safebreach

Overview: Safebreach offers continuous security assessment through breach and attack simulations, identifying vulnerabilities before hackers do.

Tech Behind the Scenes: Safebreach’s platform simulates various attack vectors, providing detailed reports on security weaknesses and remediation steps.

Case Study:

Client: A telecommunications company
Challenge: Identifying and mitigating security gaps.
Solution: Utilized Safebreach’s simulation tools.
Results: Identified critical vulnerabilities, improved overall security posture.

Client Response: "Safebreach’s simulations have been crucial in proactively addressing our security weaknesses."

Code Snippet:

import safebreach_sdk

client = safebreach_sdk.Client(api_key='your_api_key')
simulations = client.run_simulations()
for sim in simulations:
    if sim.status == 'Failed':
        client.remediate_issue(sim.id)

More Info: Safebreach

7. Beyond Identity

Overview: Beyond Identity focuses on passwordless authentication, binding identities to devices to enhance security.

Tech Behind the Scenes: The platform uses cryptographic keys stored on users’ devices, eliminating the need for passwords and reducing the risk of credential-based attacks.

Case Study:

Client: A financial technology startup
Challenge: Preventing credential-based attacks.
Solution: Implemented Beyond Identity’s passwordless authentication.
Results: Enhanced security, reduced phishing attacks by 70%.

Client Response: "Beyond Identity has made our authentication process more secure and user-friendly."

Code Snippet:

const beyondIdentity = require('beyond-identity-sdk');

beyondIdentity.authenticateUser(user_id, device_id)
  .then(response => {
    console.log('Authentication successful:', response);
  })
  .catch(error => {
    console.error('Authentication failed:', error);
  });

More Info: Beyond Identity

8. Confluera

Overview: Confluera helps organizations identify, track, and manage sophisticated security threats through its advanced cybersecurity platform.

Tech Behind the Scenes: Confluera uses real-time tracking and correlation of security events, providing actionable insights for threat management.

Case Study:

Client: A government agency
Challenge: Managing advanced persistent threats (APTs).
Solution: Adopted Confluera’s cybersecurity platform.
Results: Improved threat detection and response times, reduced APT impact.

Client Response: "Confluera’s platform has enhanced our ability to detect and respond to sophisticated threats."

Code Snippet:

import confluera_sdk

client = confluera_sdk.Client(api_key='your_api_key')
threats = client.get_advanced_threats()
for threat in threats:
    client.take_action(threat.id)

More Info: Confluera

9. Liongard

Overview: Liongard offers cybersecurity services for Managed Service Providers (MSPs), providing comprehensive visibility and protection for sensitive data.

Tech Behind the Scenes: Liongard’s platform integrates with various IT systems to automate security monitoring and compliance reporting.

Case Study:

Client: An MSP serving multiple small businesses
Challenge: Ensuring consistent security across diverse environments.
Solution: Implemented Liongard’s integrated security platform.
Results: Improved compliance, reduced manual security management efforts by 50%.

Client Response: "Liongard has simplified our security management, making it more efficient and reliable."

Code Snippet:

apiVersion: liongard.io/v1
kind: SecurityIntegration
metadata:
  name: msp-security
spec:
  rules:
  - action: Monitor
    source: all
    target: sensitive_data
    frequency: daily
  - action: Alert
    source: any
    condition: anomaly_detected

More Info: Liongard

10. Valtix

Overview: Valtix offers a cloud-native network security platform designed to provide visibility, control, and protection across multi-cloud environments.

Tech Behind the Scenes: Valtix's platform uses a single-pass architecture to inspect and enforce security policies, integrating seamlessly with AWS, Azure, GCP, and Oracle Cloud.

Case Study:

Client: A global technology company
Challenge: Securing multi-cloud environments with unified policies.
Solution: Deployed Valtix’s network security platform.
Results: Enhanced security visibility, streamlined policy management across clouds.

Client Response: "Valtix has revolutionized our multi-cloud security strategy, making it more efficient and comprehensive."

Code Snippet:

{
  "policy": {
    "name": "multi-cloud-policy",
    "description": "Unified security policy for multi-cloud environments",
    "rules": [
      {
        "action": "Allow",
        "source": "internal_network",
        "destination": "cloud_resources",
        "ports": [443, 80]
      },
      {
        "action": "Deny",
        "source": "external_network",
        "destination": "cloud_resources"
      }
    ]
  }
}

More Info: Valtix

Conclusion

The landscape of cloud security is rapidly evolving, with startups like Aqua Security, Dazz, Orca Security, Cyble, Illumio, Safebreach, Beyond Identity, Confluera, Liongard, and Valtix leading the way. These companies are leveraging cutting-edge technologies such as AI, machine learning, and advanced analytics to provide innovative solutions that address the complex challenges of securing cloud environments.

From automated remediation and threat intelligence to zero-trust security and breach simulations, these startups are setting new standards in cloud security. Their case studies demonstrate the tangible benefits of their solutions, highlighting significant improvements in security posture, compliance, and operational efficiency.

As cloud adoption continues to grow, keeping an eye on these innovative startups can provide valuable insights into the future of cloud security. Whether you're an enterprise looking to enhance your security measures or an MSP seeking to streamline your security management, these startups offer solutions that can meet your needs.

Author Note: For more in-depth articles on cloud security and the latest tech innovations, follow my profile:

Connect with me:

Stay secure and keep innovating!

Algorithmic Trading Architecture and Quants: A Deep Dive with Case Studies on BlackRock and Tower Research

Syed Nashet Ali — Sat, 06 Jul 2024 21:26:22 +0000

(https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t044ftmmhixpbqma1io7.png)

Algorithmic trading, or "algo trading," involves using computer programs and algorithms to trade securities. These algorithms execute pre-defined strategies at speeds and frequencies that a human trader cannot match. This article delves into the architecture of algorithmic trading systems, the role of quants, and explores case studies from industry giants BlackRock and Tower Research. We will also provide step-by-step implementation examples, code snippets, calculations, and real-world examples from the London Stock Exchange (LSE) and the Singapore Exchange (SGX) over the past 25 years.

Algorithmic Trading Architecture

1. Market Data Feed Handlers

Algo trading systems begin with market data feed handlers, which receive real-time data from various exchanges. These handlers process, filter, and normalize the data for the subsequent components.

2. Strategy Engine

The strategy engine is the core of the algo trading system. It runs the trading algorithms and makes trading decisions based on the incoming data. Strategies can range from simple rules to complex mathematical models.

3. Order Management System (OMS)

The OMS is responsible for managing and executing orders. It ensures that the orders are sent to the market, filled correctly, and that any necessary modifications or cancellations are handled.

4. Risk Management

Risk management systems monitor the trading activities in real-time to ensure compliance with predefined risk parameters. They can halt trading activities if the risk thresholds are breached.

5. Execution Management System (EMS)

The EMS optimizes the execution of orders. It determines the best possible way to execute a trade, taking into account factors like order size, market conditions, and transaction costs.

6. Backtesting and Simulation

Before deploying strategies, they are rigorously tested using historical data. This process, known as backtesting, helps in understanding the performance and potential pitfalls of the strategy.

7. Latency and Infrastructure

Latency is a critical factor in algorithmic trading. High-frequency trading (HFT) firms invest heavily in low-latency infrastructure, including direct market access (DMA), co-location of servers, and high-speed communication networks.

8. Compliance and Reporting

Algo trading systems must adhere to regulatory requirements. Compliance modules ensure that trading activities comply with legal standards, and reporting modules generate necessary reports for regulatory bodies.

The Role of Quants

Quantitative analysts, or quants, are the backbone of algorithmic trading. They use mathematical models, statistical techniques, and programming skills to develop trading strategies. Their work involves:

Data Analysis: Sifting through vast amounts of historical and real-time data to identify patterns and trends.
Model Development: Creating mathematical models to predict price movements and optimize trading strategies.
Strategy Implementation: Coding the strategies into algorithms and integrating them with the trading system.
Risk Assessment: Evaluating the risk associated with each strategy and ensuring it aligns with the firm’s risk appetite.

Step-by-Step Implementation of an Algorithmic Trading Strategy

Example Strategy: Mean Reversion

Mean reversion is a popular trading strategy that assumes prices will revert to their historical mean. Here’s a step-by-step implementation with code snippets in Python.

Step 1: Import Libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from datetime import datetime
import yfinance as yf

Step 2: Fetch Historical Data

We use the yfinance library to fetch historical stock data.

ticker = "AAPL"
start_date = "2020-01-01"
end_date = "2023-01-01"
data = yf.download(ticker, start=start_date, end=end_date)

Step 3: Calculate Moving Averages

We calculate the short-term and long-term moving averages.

short_window = 40
long_window = 100

data['short_mavg'] = data['Close'].rolling(window=short_window, min_periods=1).mean()
data['long_mavg'] = data['Close'].rolling(window=long_window, min_periods=1).mean()

Step 4: Generate Trading Signals

We generate buy and sell signals based on the moving averages.

data['signal'] = 0.0
data['signal'][short_window:] = np.where(data['short_mavg'][short_window:] > data['long_mavg'][short_window:], 1.0, 0.0)   
data['positions'] = data['signal'].diff()

Step 5: Backtest the Strategy

We backtest the strategy to evaluate its performance.

initial_capital = float(100000.0)
positions = pd.DataFrame(index=data.index).fillna(0.0)
positions[ticker] = 100*data['signal']   
portfolio = positions.multiply(data['Close'], axis=0)
pos_diff = positions.diff()

portfolio['holdings'] = (positions.multiply(data['Close'], axis=0)).sum(axis=1)
portfolio['cash'] = initial_capital - (pos_diff.multiply(data['Close'], axis=0)).sum(axis=1).cumsum()   
portfolio['total'] = portfolio['cash'] + portfolio['holdings']
portfolio['returns'] = portfolio['total'].pct_change()

Step 6: Plot the Results

We visualize the strategy's performance.

fig = plt.figure()
ax1 = fig.add_subplot(111, ylabel='Portfolio value in $')

portfolio['total'].plot(ax=ax1, lw=2.)
data['signal'].plot(ax=ax1, lw=2.)

ax1.plot(data.loc[data.positions == 1.0].index, 
         data.short_mavg[data.positions == 1.0],
         '^', markersize=10, color='m')

ax1.plot(data.loc[data.positions == -1.0].index, 
         data.short_mavg[data.positions == -1.0],
         'v', markersize=10, color='k')

plt.show()

Explanation and Conditions

Buy Signal: Generated when the short-term moving average crosses above the long-term moving average.
Sell Signal: Generated when the short-term moving average crosses below the long-term moving average.

Real-World Examples from LSE and SGX

London Stock Exchange (LSE)

Example 1: Mean Reversion on LSE

Let's consider a mean reversion strategy implemented on the FTSE 100 index.

Step 1: Fetch Historical Data

We fetch the historical data for the FTSE 100 index.

ticker = "^FTSE"
start_date = "2000-01-01"
end_date = "2023-01-01"
data = yf.download(ticker, start=start_date, end=end_date)

Step 2-6: Similar to the steps outlined above for mean reversion strategy

By applying the same steps to the FTSE 100 index data, we can observe how the strategy performs on the LSE.

Singapore Exchange (SGX)

Example 2: Momentum Trading on SGX

Momentum trading is another popular strategy. Let's implement a simple momentum strategy on the STI index.

Step 1: Fetch Historical Data

ticker = "^STI"
start_date = "2000-01-01"
end_date = "2023-01-01"
data = yf.download(ticker, start=start_date, end=end_date)

Step 2: Calculate Momentum

We calculate the momentum as the percentage change in price over a certain period.

momentum_window = 20
data['momentum'] = data['Close'].pct_change(momentum_window)

Step 3: Generate Trading Signals

We generate buy and sell signals based on momentum.

data['signal'] = 0.0
data['signal'] = np.where(data['momentum'] > 0, 1.0, 0.0)
data['positions'] = data['signal'].diff()

Step 4-6: Similar to the steps outlined above for backtesting and plotting results

By applying these steps, we can backtest and visualize the performance of a momentum trading strategy on the SGX.

Calculations and Analysis

Example 1: LSE Mean Reversion Strategy Performance

CAGR (Compound Annual Growth Rate) Calculation:

[ \text{CAGR} = \left( \frac{\text{Ending Value}}{\text{Beginning Value}} \right)^{\frac{1}{n}} - 1 ]

Where ( n ) is the number of years.

beginning_value = portfolio['total'].iloc[0]
ending_value = portfolio['total'].iloc[-1]
years = (data.index[-1] - data.index[0]).days / 365.25

CAGR = (ending_value / beginning_value) ** (1 / years) - 1
print(f"CAGR: {CAGR:.2%}")

Example 2: SGX Momentum Strategy Performance

Sharpe Ratio Calculation:

[ \text{Sharpe Ratio} = \frac{\text{Mean Portfolio Return} - \text{Risk-Free Rate}}{\text{Portfolio Standard Deviation}} ]

Assuming a risk-free rate of 2%.

risk_free_rate = 0.02
mean_return = portfolio['returns'].mean()
std_return = portfolio['returns'].std()

sharpe_ratio = (mean_return - risk_free_rate) / std_return
print(f"Sharpe Ratio: {sharpe_ratio:.2f}")

## Case Study: BlackRock

### Overview
BlackRock, the world’s largest asset manager, leverages algorithmic trading to manage its extensive portfolio. The firm employs sophisticated algorithms to execute trades, manage risks, and optimize portfolio performance.

### Trading Strategies
BlackRock’s algo trading strategies include:

- **Index Arbitrage:** Exploiting price differences between index futures and underlying stocks.
- **Mean Reversion:** Identifying stocks that have deviated from their historical price patterns and betting on their return to the mean.
- **Momentum Trading:** Capitalizing on stocks that show strong trends in a particular direction.

### Technology Stack
BlackRock uses a proprietary trading platform called Aladdin, which integrates risk management, trading, and portfolio management. Aladdin employs advanced data analytics, machine learning, and cloud computing to support algo trading activities.

### Impact
BlackRock’s algorithmic trading has significantly enhanced its trading efficiency, reduced transaction costs, and improved overall portfolio performance. The firm’s ability to execute large volumes of trades with minimal market impact has been a critical factor in its success.

### Real-World Example: Index Arbitrage on LSE

#### Index Arbitrage Strategy:
Assume a scenario where BlackRock is using index arbitrage on the FTSE 100 index. If the index futures price is trading above its fair value compared to the underlying stocks, BlackRock would sell the futures and buy the underlying stocks.

### Calculations:
- **Fair Value of Futures**: \( \text{Futures Price} = \text{Spot Price} \times (1 + \text{Risk-Free Rate} - \text{Dividend Yield})^{T} \)
- **Spot Price**: Current price of the underlying index.
- **Risk-Free Rate**: Assume a risk-free rate of 2%.
- **Dividend Yield**: Assume a dividend yield of 3%.
- **Time to Maturity (T)**: Assume 0.5 years.

python
spot_price = 7000 # Current spot price of FTSE 100
risk_free_rate = 0.02
dividend_yield = 0.03
T = 0.5

fair_value = spot_price * (1 + risk_free_rate - dividend_yield) ** T
print(f"Fair Value of Futures: {fair_value:.2f}")


### Impact:
If the actual futures price is significantly higher than the fair value, BlackRock can capitalize on this discrepancy by executing the arbitrage strategy.

## Case Study: Tower Research

### Overview
Tower Research Capital, a leading HFT firm, is renowned for its low-latency trading strategies. The firm employs cutting-edge technology and sophisticated algorithms to trade across various asset classes.

### Trading Strategies
Tower Research’s strategies include:

- **Statistical Arbitrage:** Using statistical models to identify and exploit price discrepancies between related financial instruments.
- **Market Making:** Providing liquidity by continuously quoting buy and sell prices and profiting from the bid-ask spread.
- **Event-Driven Trading:** Trading based on news events, earnings announcements, and economic data releases.

### Technology Stack
Tower Research invests heavily in low-latency infrastructure. The firm uses custom-built hardware, co-located servers, and high-speed communication networks to achieve ultra-fast trade execution.

### Impact
Tower Research’s focus on low latency has enabled it to gain a competitive edge in the market. The firm’s ability to execute trades within microseconds has resulted in consistent profitability and significant market share in the HFT space.

### Real-World Example: Statistical Arbitrage on SGX

#### Statistical Arbitrage Strategy:
Assume a scenario where Tower Research is using statistical arbitrage to trade pairs of stocks on the SGX. If stock A and stock B are typically correlated but have diverged, Tower Research can buy the underperforming stock and short the outperforming stock, expecting them to revert to their mean correlation.

### Calculations:
- **Z-Score Calculation**: \( Z = \frac{\text{Price Spread} - \mu}{\sigma} \)
- **Price Spread**: Difference between the prices of stock A and stock B.
- **Mean (\(\mu\)) and Standard Deviation (\(\sigma\)) of Price Spread**: Calculated based on historical data.

python
spread = data['stock_A'] - data['stock_B']
mean_spread = spread.mean()
std_spread = spread.std()
z_score = (spread - mean_spread) / std_spread

data['z_score'] = z_score


### Trading Signals:
- **Buy Signal**: If Z-Score < -1.0 (Buy stock A and short stock B).
- **Sell Signal**: If Z-Score > 1.0 (Sell stock A and buy stock B).

python
data['signal'] = 0.0
data['signal'][data['z_score'] < -1] = 1.0
data['signal'][data['z_score'] > 1] = -1.0




### Impact:
By implementing statistical arbitrage, Tower Research can profit from temporary deviations in the prices of correlated stocks.

## Conclusion

Algorithmic trading and quants have revolutionized the financial markets, enabling firms to execute trades with speed, precision, and efficiency. The cases of BlackRock and Tower Research highlight the diverse applications and impact of algo trading in the industry. Real-world examples from the LSE and SGX illustrate how these strategies can be applied and the calculations involved. As technology continues to evolve, the role of quants and the sophistication of algorithmic trading systems are expected to grow, further transforming the landscape of financial markets.

BharatGPT: The Next Generation AI Language Model

Syed Nashet Ali — Sat, 15 Jun 2024 20:50:44 +0000

Introduction

BharatGPT represents a monumental stride in the field of natural language processing (NLP) and AI, tailored specifically to understand and generate text in multiple Indian languages with contextual accuracy and cultural relevance. This blog delves into the technical intricacies, working mechanisms, deployment strategies, hardware design, and the collaborative efforts behind BharatGPT, including key contributions from Jio and the Indian Institutes of Technology (IITs).

Technical Foundation

Model Architecture

BharatGPT is built upon the GPT-4 architecture, leveraging transformer models which utilize self-attention mechanisms to process and generate human-like text. The model comprises:

Encoder-Decoder Layers: Multiple layers of encoders and decoders that process the input text, capturing intricate patterns and contextual information.
Attention Mechanisms: Self-attention and cross-attention mechanisms that help the model focus on relevant parts of the input sequence, enhancing its understanding of context and relationships between words.

The architecture can be broken down into:

Embedding Layer: Converts input tokens into dense vectors of fixed size.
Positional Encoding: Adds positional information to the embeddings to help the model understand the order of tokens.
Multi-Head Attention: Computes attention scores across different heads, allowing the model to focus on various parts of the input.
Feedforward Neural Networks: Processes the attention outputs, applying transformations to capture complex patterns.
Layer Normalization and Residual Connections: Stabilizes and accelerates training.

Multilingual Training

The model is trained on a diverse corpus containing text in Hindi, Tamil, Bengali, Telugu, Marathi, and other Indian languages. This multilingual training involves:

Tokenization: Utilizing a subword tokenization approach (Byte Pair Encoding or BPE) to handle the diverse scripts and linguistic structures.
Pre-training: Extensive pre-training on a vast dataset, including books, articles, social media content, and more, to capture linguistic nuances and cultural context.
Fine-tuning: Specific fine-tuning tasks to adapt the model for various applications such as translation, summarization, and question-answering.

Calculations and Parameters

BharatGPT involves a significant number of parameters to ensure its robustness:

Number of Layers (L): 48 layers
Embedding Dimension (d_model): 1600 dimensions
Number of Attention Heads (h): 20 heads
Feedforward Dimension (d_ff): 6400 dimensions
Total Parameters: Approximately 175 billion parameters

The calculations for the self-attention mechanism are given by:

[ \text{Attention}(Q, K, V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right) V ]

Where:

( Q ) (Query), ( K ) (Key), and ( V ) (Value) are derived from the input embeddings.
( d_k ) is the dimension of the key vectors.

The multi-head attention mechanism can be expressed as:

[ \text{MultiHead}(Q, K, V) = \text{Concat}(\text{head}_1, \ldots, \text{head}_h)W^O ]

Where each head is computed as:

[ \text{head}_i = \text{Attention}(QW_i^Q, KW_i^K, VW_i^V) ]

( W_i^Q ), ( W_i^K ), ( W_i^V ), and ( W^O ) are learned weight matrices.

Working Mechanisms

Text Generation

The core of BharatGPT's functionality lies in its ability to generate coherent and contextually relevant text. This involves:

Input Processing: The input text is tokenized and passed through the encoder layers, where self-attention mechanisms help in understanding the context.
Contextual Embeddings: The model generates contextual embeddings for each token, capturing its meaning in relation to surrounding words.
Decoding: Using these embeddings, the decoder generates the output sequence, one token at a time, while maintaining contextual coherence.

Conversational AI

BharatGPT excels in conversational AI, making it suitable for chatbots and virtual assistants. It handles:

Dialogue Management: Maintaining context across turns in a conversation, ensuring relevant and consistent responses.
Intent Recognition: Identifying user intents and providing appropriate responses or actions.

Behind the Scenes

Model Deployment

Deploying BharatGPT involves several critical steps:

Infrastructure Setup: Utilizing cloud platforms like AWS, Azure, or GCP to provide scalable computing resources.
Containerization: Using Docker to create portable and consistent environments for the model.
Orchestration: Employing Kubernetes for automating deployment, scaling, and managing containerized applications.

Hardware Design

BharatGPT's deployment demands high-performance hardware to ensure efficient processing and quick response times:

GPUs: Leveraging NVIDIA A100 GPUs for their parallel processing capabilities, essential for handling the large-scale computations involved in running transformer models.
TPUs: Google’s Tensor Processing Units (TPUs) are also used for accelerating machine learning workloads, providing an alternative to GPUs.
Custom Hardware: Exploring custom ASICs (Application-Specific Integrated Circuits) tailored for specific NLP tasks to further enhance performance.

Architecture Diagram

Below is a simplified architecture diagram for BharatGPT:

                          +----------------------+
                          |   Input Tokenizer    |
                          +----------+-----------+
                                     |
                                     v
                          +----------------------+
                          |    Embedding Layer   |
                          +----------+-----------+
                                     |
                                     v
                          +----------------------+
                          |  Positional Encoding |
                          +----------+-----------+
                                     |
                                     v
              +----------------------+---------------------+
              | Multi-Head Self-Attention (Multi-Layers)  |
              +----------------------+---------------------+
                                     |
                                     v
              +----------------------+---------------------+
              | Feedforward Neural Networks (Multi-Layers) |
              +----------------------+---------------------+
                                     |
                                     v
                          +----------------------+
                          |   Output Decoder     |
                          +----------+-----------+
                                     |
                                     v
                          +----------------------+
                          |    Output Tokens     |
                          +----------------------+

API Integration

To facilitate easy integration into various applications, BharatGPT offers robust APIs:

RESTful APIs: Providing endpoints for text generation, language translation, summarization, and more.
GraphQL APIs: Allowing more flexible and efficient queries, suitable for complex applications.
SDKs: Software Development Kits (SDKs) for popular programming languages like Python, JavaScript, and Java to simplify integration.

Collaborative Efforts

Development Team

BharatGPT is the result of a collaborative effort involving:

Data Scientists and NLP Researchers: Leading the research and development of the model, fine-tuning algorithms, and ensuring linguistic diversity.
Software Engineers: Handling the implementation, optimization, and deployment of the model.
Linguists and Cultural Experts: Providing insights into linguistic nuances and cultural contexts to enhance the model's relevance and accuracy.

Jio's Contribution

Reliance Jio, one of India's largest telecommunications companies, played a crucial role in the development and deployment of BharatGPT:

Data Infrastructure: Jio provided robust data infrastructure and cloud services, ensuring scalable and reliable computing resources for training and deploying the model.
Connectivity: Leveraging Jio's extensive network to enable widespread access to BharatGPT, particularly in rural and underserved areas.
Research Collaboration: Partnering with academic institutions and providing funding and resources for cutting-edge research in NLP and AI.

IITs' Involvement

The Indian Institutes of Technology (IITs) were instrumental in the research and development of BharatGPT:

Expertise: Leading researchers and professors from IITs contributed their expertise in machine learning, NLP, and data science.
Data Curation: Collaborating on the collection and curation of diverse linguistic datasets, ensuring comprehensive coverage of Indian languages.
Algorithm Development: Developing and refining algorithms to enhance the model's performance and accuracy, especially for complex linguistic structures unique to Indian languages.

Conclusion

BharatGPT stands as a testament to the advancements in AI and NLP, tailored specifically for the rich and diverse linguistic landscape of India. With cutting-edge technology, robust deployment strategies, and a dedicated team of experts, BharatGPT is poised to revolutionize how AI interacts with and understands Indian languages. Whether it's for conversational AI, content generation, or language translation, BharatGPT offers unparalleled capabilities, making it an invaluable tool in the digital transformation of India.

The collaboration between industry leaders like Jio and academic powerhouses like the IITs underscores the importance of synergy in technological innovation. Together, they have not only created a powerful AI model but also paved the way for future advancements that will continue to drive India's technological progress.

Adani One Super App: System Design Unlocked

Syed Nashet Ali — Thu, 06 Jun 2024 13:15:21 +0000

Adani One Super App

The digital age demands convenience and efficiency, and super apps are becoming the cornerstone of this transformation. Adani One is envisioned to be a super app that integrates multiple services, from e-commerce and travel bookings to utilities and customer support, into a single platform. In this blog, we'll unlock the system design of the Adani One super app, focusing on architecture, scalability, and security.

Understanding Super Apps

A super app is a platform that offers a wide range of services under one roof. Instead of switching between different apps for various needs, users can access everything from payments and travel bookings to customer support and social interactions in one place.

System Design Overview

Designing a super app involves multiple layers of complexity, including user management, service integration, real-time notifications, and security. Let’s dive into the details.

Architecture

The architecture of the Adani One super app is based on a microservices approach, ensuring modularity, scalability, and maintainability.

Microservices Architecture:

Authentication Service: Handles user registration, login, and token management.
Payment Service: Manages transactions, payment gateways, and wallet functionalities.
Service Discovery Service: Provides listings for various services like flights, hotels, etc.
Booking Service: Manages reservations and bookings.
Notification Service: Sends real-time notifications.
Search Service: Handles user search queries.
Customer Support Service: Manages chat and support interactions.
User Profile Service: Manages user data and preferences.
Review and Rating Service: Handles user reviews and ratings.

API Gateway:

Serves as a single entry point for all client requests, routing them to the appropriate microservices. It also provides functionalities like rate limiting, caching, and logging.

Database Layer:

Relational Databases: For transactional data.
NoSQL Databases: For flexible data storage.
In-Memory Databases: For caching and session management.

Messaging and Event Streaming:

Message Broker: For asynchronous communication between microservices.
Event Streaming: For real-time data processing.

Data Analytics and Reporting:

Data Warehouse: For storing and querying large datasets.
ETL Pipeline: For data extraction, transformation, and loading.
Analytics Tools: For generating reports and dashboards.

High-Level Architecture Diagram

Scalability

Scalability is crucial for handling millions of users and ensuring smooth performance. Here’s how we achieve it:

Horizontal Scaling:

Deploy each microservice independently, allowing specific components to scale based on demand.
Use Kubernetes to manage containerized microservices, enabling automatic scaling.

Load Balancing:

Implement load balancing at the API gateway and service levels to distribute traffic evenly.

Auto-scaling:

Use Kubernetes Horizontal Pod Autoscaler (HPA) and cloud provider auto-scaling features to adjust resources based on demand.

Caching:

Use in-memory databases like Redis for caching frequently accessed data.
Employ a CDN for caching static assets closer to users.

Event-Driven Architecture:

Use message brokers like RabbitMQ for decoupling services and handling asynchronous communication.

Database Replication:

Implement master-slave replication and multi-region deployment for high availability and disaster recovery.

Detailed Scalability Calculations

Example Scenario: Handling Peak Traffic

Assume the following:

Average user makes 5 requests per session.
Peak traffic: 1 million users per hour.

Calculations:

Total requests per hour: (1,000,000 \text{ users/hour} \times 5 \text{ requests/user} = 5,000,000 \text{ requests/hour})
Requests per second (RPS): (\frac{5,000,000}{3600} \approx 1389 \text{ RPS})

To handle this load:

Deploy API Gateway instances capable of handling 500 RPS each: ( \frac{1389}{500} \approx 3 \text{ instances} )
Scale microservices similarly based on their respective load profiles.

Kubernetes Configuration for Auto-Scaling:

Here's a sample configuration for setting up Horizontal Pod Autoscaling in Kubernetes:

apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
  name: my-microservice-hpa
  namespace: my-namespace
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-microservice
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 80

This configuration ensures that the deployment named my-microservice scales between 2 and 10 replicas based on CPU utilization.

Security

Security is paramount for protecting user data and maintaining trust. Here’s how we ensure robust security:

Authentication and Authorization:

Use OAuth 2.0 and OpenID Connect for secure authentication.
Implement JWT for secure, stateless token management.

Data Encryption:

Encrypt data in transit using TLS/SSL.
Encrypt data at rest in databases and storage.

Access Control:

Implement RBAC to manage permissions based on user roles.
Use ABAC for fine-grained access control.

API Security:

Enforce rate limiting, IP whitelisting/blacklisting, and request validation at the API gateway.
Validate and sanitize inputs to prevent injection attacks.

Monitoring and Logging:

Maintain comprehensive audit logs for critical operations.
Use SIEM tools for security monitoring and threat detection.

Compliance:

Ensure compliance with GDPR, CCPA, and other data protection regulations.

Incident Response:

Develop an incident response plan for quick mitigation of security incidents.
Implement regular backups and disaster recovery plans.

Code Snippets for Security

JWT Authentication Middleware in Node.js:

const jwt = require('jsonwebtoken');

function authenticateToken(req, res, next) {
  const token = req.header('Authorization').split(' ')[1];
  if (!token) return res.sendStatus(401);

  jwt.verify(token, process.env.ACCESS_TOKEN_SECRET, (err, user) => {
    if (err) return res.sendStatus(403);
    req.user = user;
    next();
  });
}

module.exports = authenticateToken;

Encryption in Transit (TLS/SSL) Configuration for Nginx:

server {
    listen 443 ssl;
    server_name example.com;

    ssl_certificate /etc/nginx/ssl/nginx.crt;
    ssl_certificate_key /etc/nginx/ssl/nginx.key;

    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers HIGH:!aNULL:!MD5;

    location / {
        proxy_pass http://localhost:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Conclusion

The Adani One super app is designed to be a robust, scalable, and secure platform, offering a seamless experience to users while integrating a multitude of services. By leveraging a microservices architecture, horizontal scaling, and stringent security measures, the app is well-equipped to handle high traffic and protect user data.

Building a super app is a complex but rewarding endeavor, requiring meticulous planning and execution. With the right design and architecture, Adani One is poised to become a cornerstone of digital convenience.

Feel free to share your thoughts and feedback in the comments. Let's discuss and unlock the potential of super apps together!

This enhanced blog post includes more details, code snippets, and calculations to provide a comprehensive guide to designing the Adani One super app.

Azure Cloud & Container Security Best Practices

Syed Nashet Ali — Sun, 26 Nov 2023 05:35:43 +0000

With a total market share that hovers north of 20 percent, the Microsoft Azure cloud has grown into a top contender in the public cloud computing market in recent years. At the same time, Azure cloud security has become a top contender for more and more businesses.

Although Azure security boils down to the same practices and principles as security on any major public cloud platform in most ways, there are some important distinctions and nuances to know about when devising an Azure security strategy. Keep reading for an overview of these details as we unpack the fundamentals of Azure cloud security, followed by a discussion of Azure security best practices.

Azure’s Shared Responsibility Model
Like all public clouds, Azure uses a shared responsibility model to define which security tasks fall to customers and which ones Azure handles. Understanding shared responsibility concepts in Azure is the first step in building an Azure cloud security strategy that allows you to manage the security responsibilities Azure doesn’t oversee within your cloud environments.

Azure spells out the details of its shared responsibility model here, but they can be summarized as follows:

Azure assumes most security responsibilities for SaaS offerings, with the exception of customer-created data or applications customers deploy on a SaaS service.
Customers share responsibility with Azure for securing workloads deployed on Azure PaaS services. In this context, Azure secures the underlying physical infrastructure, but most responsibilities related to networking and Identity and Access Management (IAM) fall to customers.
On Azure IaaS services, customers handle most security tasks. The exception is the underlying physical infrastructure, which Azure secures.
If customers integrate on-premises infrastructure, applications, or data with Azure via one of Azure’s hybrid cloud frameworks (such as Azure Stack or Azure Arc), customers bear full responsibility for securing on-premises resources.
If you’re familiar with cloud shared responsibility models in general, all the above should be unsurprising. Azure’s shared responsibility architecture is very consistent with the models embraced by other major public cloud providers.

Sysdig
Search Cloud Native...
OVERVIEW
Azure Cloud & Container Security Best Practices
SHARE:

CONTENT

Azure spells out the details of its shared responsibility model here, but they can be summarized as follows:

Azure Cloud Security Best Practices
Once you’ve identified the security responsibilities that fall to you based on which types of Azure cloud services you consume, you can plan an Azure security strategy that allows you to meet your security obligations.

Standard Cloud Security Best Practices
Many of the best practices you’ll want to follow in this respect are standard fare for securing any major public cloud environment. They include:

Use IAM: Identity and Access Management, or IAM, is a fundamental tool for securing workloads in any public cloud. If you use Azure, be sure to make full use of its IAM framework to manage access to your cloud resources based on the principle of least privilege. As we discuss below, Azure IAM works a little differently from other cloud IAMs because it is based on Active Directory, but you can nonetheless achieve the same granular access control configurations on Azure that you could enforce on any major public cloud. For example, this Azure IAM role example creates a role that can monitor and restart virtual machines:
{
"Name": "Virtual Machine Operator",
"Id": "88888888-8888-8888-8888-888888888888",
"IsCustom": true,
"Description": "Can monitor and restart virtual machines.",
"Actions": [
"Microsoft.Storage//read",
"Microsoft.Network//read",
"Microsoft.Compute//read",
"Microsoft.Compute/virtualMachines/start/action",
"Microsoft.Compute/virtualMachines/restart/action",
"Microsoft.Authorization//read",
"Microsoft.ResourceHealth/availabilityStatuses/read",
"Microsoft.Resources/subscriptions/resourceGroups/read",
"Microsoft.Insights/alertRules/",
"Microsoft.Insights/diagnosticSettings/",
"Microsoft.Support/*"
],
"NotActions": [],
"DataActions": [],
"NotDataActions": [],
"AssignableScopes": [
"/subscriptions/{subscriptionId1}",
"/subscriptions/{subscriptionId2}",
"/providers/Microsoft.Management/managementGroups/{groupId1}"
]
}
Code language: JSON / JSON with Comments (json)
Isolate cloud networks: Where possible, use Azure virtual networks and private clouds to isolate your cloud environments at the network level. These services aren’t available for every type of Azure workload, but most IaaS- and PaaS- based workloads (and some SaaS-based workloads as well) can be isolated in private networks or virtual private clouds.
Use cloud tags: Like most other clouds, Azure lets you label and organize resources using tags. Although cloud resources that lack tags are not necessarily insecure, tags are beneficial from a security perspective because they make it easier to keep track of which cloud resources you have running and where. In turn, they help ensure that you don’t overlook some workloads when configuring access controls, auditing your cloud environment, and so on.
Secure Azure data: In addition to using Azure IAM to manage access to data that you store in Azure Blob Storage or other storage services within Azure, you should encrypt your cloud data as well as the transport layers that you use to access data. Azure also offers a storage account “lock” feature, which is useful for preventing unauthorized tampering with cloud data.

Don’t miss our fireside chat with our CEO Suresh Vasudevan! Learn More
When Seconds Count: Expanding Real-Time Capabilities Across CNAPP Learn More
Sysdig
Search Cloud Native...
OVERVIEW
Azure Cloud & Container Security Best Practices
SHARE:

CONTENT

Azure spells out the details of its shared responsibility model here, but they can be summarized as follows:

Standard Cloud Security Best Practices
Many of the best practices you’ll want to follow in this respect are standard fare for securing any major public cloud environment. They include:

Understand Azure IAM: As we mentioned above, Azure’s IAM system is unique among the major public clouds because it is based on Microsoft Active Directory. You can configure and enforce very effective access policies using Azure IAM, but the policies are written and managed differently than those of clouds like AWS. If you haven’t worked with Active Directory previously, you’ll want to spend some time educating yourself about Azure Active Directory and its role in Azure cloud IAM.
Use hybrid cloud services wisely: Azure has invested significantly in its hybrid cloud offerings in recent years with the introduction of Azure Stack and Azure Arc, both of which allow customers to manage on-premises or colocated infrastructure through Azure. As noted above, Azure expects customers to manage most security risks on private infrastructure, so it’s important not to assume that just because Azure is managing your private infrastructure, it’s also securing it.
Azure security tools: Azure provides some cloud-specific security tools to help users secure and audit workloads. The two most important are Azure Security Center and Azure Defender (which is technically part of Azure Security Center, but operates as a distinct service). These services are useful for configuring and managing alerts related to security events and risks on both Azure itself and hybrid cloud environments that include Azure. Although you’ll typically want to use external security tools as well to manage risks that Azure’s native services can’t handle, you should familiarize yourself with the basic security services that Azure provides natively.
Azure Container Security Best Practices
Because container services have become one of the most important product offerings within Azure, it’s worth saying a few words about how to manage container security on Azure.

This is a complex topic because Azure provides several container services. The most notable include:

Azure Kubernetes Services (AKS), a managed Kubernetes service.
Azure Container Instances, a managed container service that doesn’t require users to work with Kubernetes or another orchestration service.
Azure Red Hat OpenShift, a managed OpenShift service. (OpenShift is based on Kubernetes, but is not identical to it.)
Azure Web App for Containers, which lets users deploy containers quickly without having to manage orchestration or infrastructure themselves.
Securing containers on Azure depends in large part on which of these services you choose to use. But in general, best practices for Azure container security include:

Scan container images, because Azure won’t automatically detect vulnerabilities or malware within your images for you.
Manage access to container registries, whether you use Azure’s native container registry service or a third-party registry.
If your container environment includes Kubernetes, use Kubernetes audit logs, RBAC, and security contexts to help secure the environment. This is important even if you use a managed Kubernetes service like AKS. On AKS, Azure mostly secures only the cluster infrastructure; responsibility for securing Kubernetes itself and containers deployed in it, falls to the customer.
Audit your container and Kubernetes configuration data, deployments, and other files to catch misconfigurations that could cause a weak security posture.

Building Strong Azure Cloud Security Posture
As one of the world’s largest cloud computing platforms, Azure offers dozens of cloud services that can be deployed as part of single-cloud, multi-cloud, or hybrid cloud models. All of this complexity means that there is no simple formula for securing Azure cloud environments.

However, by identifying the security risks that you need to manage based on the Azure services you consume, then deploying tools that can harden those services against attack and detect breaches when they do occur, you’ll set yourself up for the strongest possible Azure cloud security posture.

Follow for more such Content!!!!

Nashet Ali - Senior Software Development Engineer - LTI - Larsen & Toubro Infotech https://in.linkedin.com/in/nashet-ali-464a65115

(@nashetking) / X https://twitter.com/NashetKing

Azure ChatGPT

Syed Nashet Ali — Sat, 25 Nov 2023 14:36:12 +0000

Azure ChatGPT

Microsoft recently unveiled Azure ChatGPT, a cutting-edge chatbot built upon OpenAI’s framework. Notably, the code for Azure ChatGPT is open-sourced under the MIT Licence, offering developers transparency and adaptability. One of its standout features is its commitment to user privacy; the system is designed to be fully isolated from data managed by OpenAI, ensuring a secure user experience.

Moreover, organizations are given an unparalleled level of control, as the network traffic can be confined exclusively to their own network. This ensures a higher degree of security and adaptability. Further sweetening the deal, Azure ChatGPT is plug-n-play, which means that users can seamlessly incorporate their internal data sources or employ plug-ins for effortless integration with their internal services. This blend of privacy, control, and ease of use positions Azure ChatGPT as a remarkable tool for businesses and developers alike.

Prerequisites
Azure OpenAI: To deploy and run ChatGPT on Azure, you’ll need an Azure subscription with access to the Azure OpenAI service. Request access here. Once you have access, follow the instructions in this link to deploy the gpt-35-turbo or gpt-4 models.
Setup GitHub or Azure AD for Authentication: The add an identity provider section below shows how to configure authentication providers.💡Note: You can configure the authentication provider to your identity solution using NextAuth providers
Introduction
Azure ChatGPT is built with the following technologies.

Node.js 18: an open-source, cross-platform JavaScript runtime environment.

Next.js 13: enables you to create full-stack web applications by extending the latest React features

NextAuth.js: configurable authentication framework for Next.js 13

LangChain JS: AI orchestration layer to build intelligent apps

Tailwind CSS: is a utility-first CSS framework that provides a series of predefined classes that can be used to style each element by mixing and matching

shadcn/ui: re-usable components built using Radix UI and Tailwind CSS.

Azure Cosmos DB: fully managed platform-as-a-service (PaaS) NoSQL database to store chat history

Azure App Service: fully managed platform-as-a-service (PaaS) for hosting web applications, REST APIs, and mobile back ends.

Provision Azure resources
Click on the Deploy to Azure button and configure your settings in the Azure Portal as described in the Environment variables section.

⚠️ Note: This will only create azure resources. Follow the deploy to Azure section to build and deploy Azure ChatGPT.

Deploy to Azure
Run from your local machine
Clone this repository locally or fork to your Github account. Run all of the the steps below from the “src” directory:

https://github.com/microsoft/azurechatgpt/tree/main

Make sure you deploy an instance of Cosmos DB in your Azure Subscription
Create a new file named .env.local to store the environment variables add the following variables.
change the active directory to be src
Install npm packages by running npm install
Start the project by running npm run dev
You should now be prompted to login with your chosen OAuth provider. Once successfully logged in, you can start creating new conversations.
Please note:

Do not use double-quotes and do not delete any of the variables.
Make sure that NEXTAUTH_URL=http://localhost:3000 has no comments in the same line.

Azure OpenAI configuration

AZURE_OPENAI_API_KEY=
AZURE_OPENAI_API_INSTANCE_NAME=
AZURE_OPENAI_API_DEPLOYMENT_NAME=
AZURE_OPENAI_API_VERSION=

GitHub OAuth app configuration

AUTH_GITHUB_ID=
AUTH_GITHUB_SECRET=

Azure AD OAuth app configuration

AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=

When deploying to production,

set the NEXTAUTH_URL environment variable to the canonical URL of your site.

More information: https://next-auth.js.org/configuration/options

NEXTAUTH_SECRET=
NEXTAUTH_URL=http://localhost:3000

AZURE_COSMOSDB_URI=
AZURE_COSMOSDB_KEY=

Deploy to Azure – GitHub Actions
The following steps describes how Azure ChatGPT can be deployed to Azure App service using GitHub Actions.

Fork the repository
Fork this repository to your own organisation so that you can execute GitHub Actions against your own Azure Subscription.

Configure secrets in your GitHub repository

AZURE_CREDENTIALS The GitHub workflow requires a secret named AZURE_CREDENTIALS to authenticate with Azure. The secret contains the credentials for a service principal with the Contributor role on the resource group containing the container app and container registry.

Create a service principal with the Contributor role on the resource group that contains the Azure App Service.az ad sp create-for-rbac --name --role contributor --scopes /subscriptions//resourceGroups/ --sdk-auth --output json
Copy the JSON output from the command.
In the GitHub repository, navigate to Settings > Secrets > Actions and select New repository secret.
Enter AZURE_CREDENTIALS as the name and paste the contents of the JSON output as the value.
Select Add secret.

AZURE_APP_SERVICE_NAME Under the same repository secrets add a new variable AZURE_APP_SERVICE_NAME to deploy to your Azure Web app. The value of this secret is the name of your Azure Web app e.g. my-web-app-name from the domain https://my-web-app-name.azurewebsites.net/

Run GitHub Actions
Once the secrets are configured, the GitHub Actions will be triggered for every code push to the repository. Alternatively, you can manually run the workflow by clicking on the “Run Workflow” button in the Actions tab in GitHub.

Add an identity provider
Once the deployment is complete, you will need to add an identity provider to authenticate your app.

⚠️ Note: Only one of the identity provider is required below.

GitHub Authentication provider
We’ll create two GitHub apps: one for testing locally and another for production.

🟡 Development app setup
Navigate to GitHub OAuth Apps setup https://github.com/settings/developers
Create a New OAuth App https://github.com/settings/applications/new
Fill in the following details
Application name: Azure ChatGPT DEV Environment
Homepage URL: http://localhost:3000
Authorization callback URL: http://localhost:3000/api/auth/callback/github
🟢 Production app setup
Navigate to GitHub OAuth Apps setup https://github.com/settings/developers
Create a New OAuth App https://github.com/settings/applications/new
Fill in the following details
Application name: Azure ChatGPT Production
Homepage URL: https://YOUR-WEBSITE-NAME.azurewebsites.net
Authorization callback URL: https://YOUR-WEBSITE-NAME.azurewebsites.net/api/auth/callback/github
⚠️ After completing app setup, ensure your environment variables locally and on Azure App Service are up to date.

# GitHub OAuth app configuration
AUTH_GITHUB_ID=
AUTH_GITHUB_SECRET=
Azure AD Authentication provider
🟡 Development app setup
Navigate to Azure AD Apps setup
Create a New Registration
Fill in the following details
Application name: Azure ChatGPT DEV Environment
Supported account types: Accounts in this organizational directory only
Redirect URI Platform: Web
Redirect URI: http://localhost:3000/api/auth/callback/azure-ad
🟢 Production app setup
Navigate to Azure AD Apps setup
Create a New Registration
Fill in the following details
Application name: Azure ChatGPT Production
Supported account types: Accounts in this organizational directory only
Redirect URI Platform: Web
Redirect URI: https://YOUR-WEBSITE-NAME.azurewebsites.net/api/auth/callback/azure-ad
After completing app setup, ensure your environment variables locally and on Azure App Service are up to date.

Azure AD OAuth app configuration

AZURE_AD_CLIENT_ID=
AZURE_AD_CLIENT_SECRET=
AZURE_AD_TENANT_ID=
Chatting with your file
Users can utilise this functionality to upload their PDF files through the portal and engage in chat discussions related to the content of those files.

Chat with your data utilises the following Azure Services:

Azure Document Intelligence for extracting information from documents.
Azure Cognitive Search for indexing and retrieving information.
Azure OpenAI Embeddings for embed content extracted from files
Azure OpenAI Embeddings
We use Azure OpenAI Embeddings to convert text to vectors and index it in Azure Cognitive Search.

update the OpenAI environment variables with the following:

AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME=
When deploying to Azure, ensure to update the Azure App service app settings with AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME

Setup Azure Cognitive Search index and Document Intelligence
Create Azure Cognitive Search using the following link
Create an index on Azure Cognitive Search with the following schema. You can use the Azure portal to create the following indexes
{
"name": "azure-chatgpt",
"fields": [
{
"name": "id",
"type": "Edm.String",
"key": true,
"filterable": true
},
{
"name": "user",
"type": "Edm.String",
"searchable": true,
"filterable": true
},
{
"name": "chatThreadId",
"type": "Edm.String",
"searchable": true,
"filterable": true
},
{
"name": "pageContent",
"searchable": true,
"type": "Edm.String"
},
{
"name": "metadata",
"type": "Edm.String"
},
{
"name": "embedding",
"type": "Collection(Edm.Single)",
"searchable": true,
"filterable": false,
"sortable": false,
"facetable": false,
"retrievable": true,
"analyzer": "",
"dimensions": 1536,
"vectorSearchConfiguration": "vectorConfig"
}
],
"vectorSearch": {
"algorithmConfigurations": [
{
"name": "vectorConfig",
"kind": "hnsw"
}
]
}
}
After the index has been created, proceed to modify the env.local file with the appropriate Azure Cognitive Search environment variables.

Azure cognitive search is used for chat over your data

AZURE_SEARCH_API_KEY=
AZURE_SEARCH_NAME=
AZURE_SEARCH_INDEX_NAME=
AZURE_SEARCH_API_VERSION="2023-07-01-Preview"
Create an instance of Azure Form Recognizer (also known as Azure Document Intelligence) using the following link. Please be aware that this resource might be called Form recognizer in Azure Portal.
After the Form Recognizer (Document Intelligence) resource has been created, proceed to modify the env.local file with appropriate environment variables. You can find values for these variables in your Form Recognizer resource (Resource Management blade > Keys and Endpoint). Please make sure that you don’t copy the endpoint from there, but only replace the region in the example below. For example, if your Form Recognizer resource is located in East US Azure region, your AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT variable would be https://eastus.api.cognitive.microsoft.com/.Please note that the file is only preserved for each chat thread:

Azure AI Document Intelligence to extract content from your data

AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT="https://REGION.api.cognitive.microsoft.com/"
AZURE_DOCUMENT_INTELLIGENCE_KEY=
At this point, you should be able to start new chat sessions with the File option.

Once the File chat option is selected, click the Choose File button to select your document and then click the Upload button to upload your file. Please note that the Form Recognizer service supports PDF (text or scanned), JPG and PNG input documents.
Once you receive a notification about a successful file upload, you should be able to start chatting with chatting with a chatbot.
Things to consider:
Central place maintain uploaded files (e.g a storage account with blob storage)
A way to delete indexed documents on Azure Cognitive Search if the chat thread is deleted
Environment variables
Below are the required environment variables, to be added to the Azure Portal or in the .env.local file.

App Setting Value Note
AZURE_OPENAI_API_KEY API keys of your Azure OpenAI resource
AZURE_OPENAI_API_INSTANCE_NAME the name of your Azure OpenAI resource
AZURE_OPENAI_API_DEPLOYMENT_NAME The name of your model deployment
AZURE_OPENAI_API_VERSION 2023-03-15-preview API version when using gpt chat
AUTH_GITHUB_ID Client ID of your GitHub OAuth application
AUTH_GITHUB_SECRET Client Secret of your GitHub OAuth application
NEXTAUTH_SECRET Used to encrypt the NextAuth.js JWT, and to hash email verification tokens. This set by default as part of the deployment template
NEXTAUTH_URL Current webs hosting domain name with HTTP or HTTPS. This set by default as part of the deployment template
AZURE_COSMOSDB_URI URL of the Azure CosmosDB
AZURE_COSMOSDB_KEY API Key for Azure Cosmos DB
AZURE_AD_CLIENT_ID The client id specific to the application
AZURE_AD_CLIENT_SECRET The client secret specific to the application
AZURE_AD_TENANT_ID The organisation Tenant ID
Azure Cognitive Search is optional for Azure ChatGPT. This is only required for chat over file feature.

AZURE_SEARCH_API_KEY API Key of Azure Cognitive search
AZURE_SEARCH_NAME https://AZURE_SEARCH_NAME.search.windows.net The deployment name of your Azure Cognitive Search
AZURE_SEARCH_INDEX_NAME The index name with vector search enabled
AZURE_SEARCH_API_VERSION 2023-07-01-Preview API version which supports vector search 2023-07-01-Preview
AZURE_DOCUMENT_INTELLIGENCE_ENDPOINT https://REGION.api.cognitive.microsoft.com/ Endpoint url of the Azure document intelligence. The REGION is specific to your Azure resource location
AZURE_DOCUMENT_INTELLIGENCE_KEY API keys of your Azure Document intelligence resource
Azure ChatGPT Environment variables
Posted

Follow me for more such content!!!!

Image description

Nashet Ali - Senior Software Development Engineer - LTI - Larsen & Toubro Infotech https://in.linkedin.com/in/nashet-ali-464a65115
Cloudify with Nashet
Nashet Ali (@nashetking) / X https://twitter.com/NashetKing

Contributing to AI Shopping Cart - App Template for Java, Azure OpenAI and Azure Spring Apps

Syed Nashet Ali — Fri, 11 Aug 2023 12:56:51 +0000

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

AI Shopping Cart is a sample application that supercharges your shopping experience with the power of AI. It leverages Azure OpenAI and Azure Spring Apps to build a recommendation engine that is not only scalable, resilient, and secure, but also personalized to your needs. Taking advantage of Azure OpenAI, the application performs nutrition analysis on the items in your cart and generates the top 3 recipes using those ingredients. With Azure Developer CLI (azd), you’re just a few commands away from having this fully functional sample application up and running in Azure. Let's get started!

This sample application take inspiration on this original work: https://github.com/lopezleandro03/ai-assisted-groceries-cart

Refer to the App Templates repository Readme for more samples that are compatible with azd.

AI Shopping Cart

Pre-requisites
Install the Azure Developer CLI
An Azure account with an active subscription. Create one for free.
OpenJDK 17
Node.js 20.5.0+
Docker
Azure OpenAI with gpt-4 or gpt-35-turbo [Note]
Review the architecture diagram and the resources you'll deploy and the Azure OpenAI section.
Quickstart
To learn how to get started with any template, follow this quickstart. For this template Azure-Samples/app-templates-java-openai-springapps, you need to execute a few additional steps as described below.

This quickstart will show you how to authenticate on Azure, enable Spring Apps alpha feature for azd, initialize using a template, set the environment variables for Azure OpenAI, provision the infrastructure, and deploy the code to Azure:

Log in to azd if you haven't already

azd auth login

Enable Azure Spring Apps alpha feature for azd

azd config set alpha.springapp on

First-time project setup. Initialize a project in the current directory using this template

azd init --template Azure-Samples/app-templates-java-openai-springapps

Set the environment variables for Azure OpenAI

azd env set azureOpenAiApiKey
azd env set azureOpenAiEndpoint
azd env set azureOpenAiDeploymentId

To use GPT-3.5 Turbo model set this environment variable to false

azd env set isAzureOpenAiGpt4Model true

Provision and deploy to Azure

azd up
Notes

Replace the placeholders with the values from your Azure OpenAI resource.
If you are using gpt-35-turbo model, you need to set isAzureOpenAiGpt4Model to false before provisioning the resource and deploying the sample application to Azure:
azd env set isAzureOpenAiGpt4Model false
At the end of the deployment, you will see the URL of the front-end. Open the URL in a browser to see the application in action.

Application Architecture
This sample application uses the following Azure resources:

Azure Container Apps (Environment) to host the frontend as a Container App and Azure Spring Apps Standard comsumption and dedicated plan
Azure Spring Apps to host the AI Shopping Cart Service as a Spring App
Azure Container Registry to host the Docker image for the frontend
Azure Database for PostgreSQL (Flexible Server) to store the data for the AI Shopping Cart Service
Azure Monitor for monitoring and logging
Azure OpenAI to perform nutrition analysis and generate top 3 recipes. It is not deployed with the sample app[Note].
Here's a high level architecture diagram that illustrates these components. Excepted Azure OpenAI, all the other resources are provisioned in a single resource group that is created when you create your resources using azd up.

Architecture diagram

This template provisions resources to an Azure subscription that you will select upon provisioning them. Please refer to the Pricing calculator for Microsoft Azure and, if needed, update the included Azure resource definitions found in infra/main.bicep to suit your needs.

Azure OpenAI
This sample application uses Azure OpenAI. It is not part of the automated deployment process. You will need to create an Azure OpenAI resource and configure the application to use it. Please follow the instructions in the Azure OpenAI documentation to get access to Azure OpenAI. Do not forget to read the overview of the Responsible AI practices for Azure OpenAI models before you start using Azure OpenAI and request access.

The current version of the sample app requires a publicly accessible Azure OpenAI resource (i.e. Allow access from all networks). This sample is not intended to be used in production. To know more about networking and security for Azure OpenAI, please refer to the Azure OpenAI documentation.

This sample app was developed to be used with gpt-4 model. It also supports gpt-35-turbo. To use gpt-35-turbo, you need to set isAzureOpenAiGpt4Model to false (cf. Quickstart). By default, this parameter/environment variable is set to true. To complete the setup of the application, you need to set the following information from the Azure OpenAI resource:

azureOpenAiApiKey - Azure OpenAI API key
azureOpenAiEndpoint - Azure OpenAI endpoint
azureOpenAiDeploymentId - Azure OpenAI deployment ID of gpt-4 or gpt-3.5-turbo model
The API key and the endpoint can be found in the Azure Portal. You can follow these instructions: Retrieve key and enpoint. The deployment id corresponds to the deployment name in this guide.

Prompt engineering is important to get the best results from Azure OpenAI. Text prompts are how users interact with GPT models. As with all generative large language model (LLM), GPT models try to produce the next series of words that are the most likely to follow the previous text. It is a bit like asking to the AI model: What is the first thing that comes to mind when I say ?

With the Chat Completion API, there are distinct sections of the prompt that are sent to the API associated with a specific role: system, user and assitant. The system message is included at the begining of the prompt and is used to provides the initial instructions to the model: description of the assitant, personality traits, instructions/rules it will follow, etc.

AI Shopping Cart Service is using Azure OpenAI client library for Java. This libary is part of of Azure SDK for Java. It is implemented as a chat completion. In the service, we have 2 system messages in SystemMessageConstants.java: one for AI Nutrition Analysis and one to generate top 3 recipes. The system message is followed by a user message: The basket is: . The assistant message is the response from the model. The service is using the ShoppingCartAiRecommendations to interact with Azure OpenAI. In this class you will find the code that is responsible for generating the prompt and calling the Azure OpenAI API: getChatCompletion. To know more about temperature and topP used in this class, please refer to the documentation.

For gpt-35-turbo model, more context is added to the user message. This additional context is added at the end of the user message. It provides more information on the format of the JSON that OpenAI model needs to return and ask the model tor return only the JSON without additional text. This additional context is available in UserMessageConstants.java.

Pre-requisites ⤴️
Application Architecture ⤴️
Application Code
This template is structured to follow the Azure Developer CLI template convetions. You can learn more about azd architecture in the official documentation.

Next Steps
At this point, you have a complete application deployed on Azure.

Enterprise Scenarios
For enterprise needs, looking for polyglot applications deployment, Tanzu components support and SLA assurance, we recommend to use Azure Spring Apps Enterprise. Check the Azure Spring Apps landing zone accelerator that provides architectural guidance designed to streamline the production ready infrastructure provisioning and deployment of Spring Boot and Spring Cloud applications to Azure Spring Apps. As the workload owner, use architectural guidance provided in landing zone accelerator to achieve your target technical state with confidence.

Azure Developer CLI
You have deployed the sample application using Azure Developer CLI, however there is much more that the Azure Developer CLI can do. These next steps will introduce you to additional commands that will make creating applications on Azure much easier. Using the Azure Developer CLI, you can setup your pipelines, monitor your application, test and debug locally.

azd down - to delete all the Azure resources created with this template

azd pipeline config - to configure a CI/CD pipeline (using GitHub Actions or Azure DevOps) to deploy your application whenever code is pushed to the main branch.

Several environment variables / secrets need to be set for Azure OpenAI resource:
AZURE_OPENAI_API_KEY: API key for Azure OpenAI resource
For GitHub workflows, you should use GitHub Secrets
For Azure DevOps pipelines, you check 'Keep this value secret' when creating the variable
AZURE_OPENAI_ENDPOINT: Endpoint for Azure OpenAI resource
AZURE_OPENAI_DEPLOYMENT_ID: Deployment ID/name for Azure OpenAI resource
IS_AZURE_OPENAI_GPT4_MODEL: Set to true if you are using GPT-4 model and to false if you are using GPT-3.5 Turbo model
azd monitor - to monitor the application and quickly navigate to the various Application Insights dashboards (e.g. overview, live metrics, logs)

Run and Debug Locally - using Visual Studio Code and the Azure Developer CLI extension

Additional azd commands
The Azure Developer CLI includes many other commands to help with your Azure development experience. You can view these commands at the terminal by running azd help. You can also view the full list of commands on our Azure Developer CLI command page.

Resources
These are additional resources that you can use to learn more about the sample application and its underlying technologies.

Start from zero and scale to zero – Azure Spring Apps consumption plan
Azure Spring Apps Consumption - Networking and Security
https://learn.microsoft.com/en-us/azure/ai-services/openai/encrypt-data-at-rest
https://learn.microsoft.com/en-us/azure/ai-services/openai/encrypt-data-at-rest
How to configure Azure OpenAI Service with managed identities
Data Collection
The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkId=521839. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.

Telemetry Configuration
Telemetry collection is on by default.

To opt-out, set the variable enableTelemetry to false in infra/main.parameters.json or in bicep template infra/main.bicep. It can be set using the following command when the provisionning is done with Azure Developer CLI:

azd env set enableTelemetry false
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.

Code of Conduct
Issues and Bugs
Feature Requests
Submission Guidelines
Code of Conduct
Help us keep this project open and inclusive. Please read and follow our Code of Conduct.

Found an Issue?
If you find a bug in the source code or a mistake in the documentation, you can help us by submitting an issue to the GitHub Repository. Even better, you can submit a Pull Request with a fix.

Want a Feature?
You can request a new feature by submitting an issue to the GitHub Repository. If you would like to implement a new feature, please submit an issue with a proposal for your work first, to be sure that we can use it.

Small Features can be crafted and directly submitted as a Pull Request.
Submission Guidelines
Submitting an Issue
Before you submit an issue, search the archive, maybe your question was already answered.

If your issue appears to be a bug, and hasn't been reported, open a new issue. Help us to maximize the effort we can spend fixing issues and adding new features, by not reporting duplicate issues. Providing the following information will increase the chances of your issue being dealt with quickly:

Overview of the Issue - if an error is being thrown a non-minified stack trace helps
Version - what version is affected (e.g. 0.1.2)
Motivation for or Use Case - explain what are you trying to do and why the current behavior is a bug for you
Browsers and Operating System - is this a problem with all browsers?
Reproduce the Error - provide a live example or a unambiguous set of steps
Related Issues - has a similar issue been reported before?
Suggest a Fix - if you can't fix the bug yourself, perhaps you can point to what might be causing the problem (line of code or commit)
You can file new issues by providing the above information at the corresponding repository's issues link: https://github.com/Azure-Samples/app-templates-java-openai-springapps/issues/new].

Submitting a Pull Request (PR)
Before you submit your Pull Request (PR) consider the following guidelines:

Search the repository (https://github.com/Azure-Samples/app-templates-java-openai-springapps/pulls) for an open or closed PR that relates to your submission. You don't want to duplicate effort.

Make your changes in a new git fork:

Commit your changes using a descriptive commit message

page_type: sample
languages:

azdeveloper
java
typescript
bicep
html products:
azure
azure-container-apps
azure-spring-apps
azure-container-registry
azure-monitor
ms-build-openjdk
ai-services
azure-openai
azure-database-postgresql urlFragment: app-templates-java-openai-springapps name: AI Shopping Cart - App Template for Java, Azure OpenAI and Azure Spring Apps description: "AI Shopping Cart Sample Application with Azure OpenAI and Azure Spring Apps" ---

AI Shopping Cart - App Template for Java, Azure OpenAI and Azure Spring Apps

This sample application take inspiration on this original work: https://github.com/lopezleandro03/ai-assisted-groceries-cart

Refer to the App Templates repository Readme for more samples that are compatible with azd.

Pre-requisites

Install the Azure Developer CLI
An Azure account with an active subscription. Create one for free.
OpenJDK 17
Node.js 20.5.0+
Docker
Azure OpenAI with gpt-4 or gpt-35-turbo ^[Note]
Review the architecture diagram and the resources you'll deploy and the Azure OpenAI section.

Quickstart

To learn how to get started with any template, follow this quickstart. For this template Azure-Samples/app-templates-java-openai-springapps, you need to execute a few additional steps as described below.

# Log in to azd if you haven't already
azd auth login

# Enable Azure Spring Apps alpha feature for azd
azd config set alpha.springapp on

# First-time project setup. Initialize a project in the current directory using this template
azd init --template Azure-Samples/app-templates-java-openai-springapps

# Set the environment variables for Azure OpenAI
azd env set azureOpenAiApiKey <replace-with-Azure-OpenAi-API-key> 
azd env set azureOpenAiEndpoint <replace-with-Azure-OpenAi-endpoint>
azd env set azureOpenAiDeploymentId <replace-with-Azure-OpenAi-deployment-id/name>

# To use GPT-3.5 Turbo model set this environment variable to false
azd env set isAzureOpenAiGpt4Model true

# Provision and deploy to Azure
azd up

Notes

Replace the placeholders with the values from your Azure OpenAI resource.

If you are using gpt-35-turbo model, you need to set isAzureOpenAiGpt4Model to false before provisioning the resource and deploying the sample application to Azure:

bash azd env set isAzureOpenAiGpt4Model false

At the end of the deployment, you will see the URL of the front-end. Open the URL in a browser to see the application in action.

Application Architecture

This sample application uses the following Azure resources:

Azure Container Apps (Environment) to host the frontend as a Container App and Azure Spring Apps Standard comsumption and dedicated plan
Azure Spring Apps to host the AI Shopping Cart Service as a Spring App
Azure Container Registry to host the Docker image for the frontend
Azure Database for PostgreSQL (Flexible Server) to store the data for the AI Shopping Cart Service
Azure Monitor for monitoring and logging
Azure OpenAI to perform nutrition analysis and generate top 3 recipes. It is not deployed with the sample app^[Note].

Here's a high level architecture diagram that illustrates these components. Excepted Azure OpenAI, all the other resources are provisioned in a single resource group that is created when you create your resources using azd up.

This template provisions resources to an Azure subscription that you will select upon provisioning them. Please refer to the Pricing calculator for Microsoft Azure and, if needed, update the included Azure resource definitions found in infra/main.bicep to suit your needs.

Azure OpenAI

This sample application uses Azure OpenAI. It is not part of the automated deployment process. You will need to create an Azure OpenAI resource and configure the application to use it. Please follow the instructions in the Azure OpenAI documentation to get access to Azure OpenAI. Do not forget to read the overview of the Responsible AI practices for Azure OpenAI models before you start using Azure OpenAI and request access.

This sample app was developed to be used with gpt-4 model. It also supports gpt-35-turbo. To use gpt-35-turbo, you need to set isAzureOpenAiGpt4Model to false (cf. Quickstart). By default, this parameter/environment variable is set to true. To complete the setup of the application, you need to set the following information from the Azure OpenAI resource:

azureOpenAiApiKey - Azure OpenAI API key
azureOpenAiEndpoint - Azure OpenAI endpoint
azureOpenAiDeploymentId - Azure OpenAI deployment ID of gpt-4 or gpt-3.5-turbo model

The API key and the endpoint can be found in the Azure Portal. You can follow these instructions: Retrieve key and enpoint. The deployment id corresponds to the deployment name in this guide.

AI Shopping Cart Service is using Azure OpenAI client library for Java. This libary is part of of Azure SDK for Java. It is implemented as a chat completion. In the service, we have 2 system messages in SystemMessageConstants.java: one for AI Nutrition Analysis and one to generate top 3 recipes. The system message is followed by a user message: The basket is: <list of items in the basket separated by a comma>. The assistant message is the response from the model. The service is using the ShoppingCartAiRecommendations to interact with Azure OpenAI. In this class you will find the code that is responsible for generating the prompt and calling the Azure OpenAI API: getChatCompletion. To know more about temperature and topP used in this class, please refer to the documentation.

For gpt-35-turbo model, more context is added to the user message. This additional context is added at the end of the user message. It provides more information on the format of the JSON that OpenAI model needs to return and ask the model tor return only the JSON without additional text. This additional context is available in UserMessageConstants.java.

Pre-requisites ⤴️
Application Architecture ⤴️

Application Code

This template is structured to follow the Azure Developer CLI template convetions. You can learn more about azd architecture in the official documentation.

[Code](https://github.com/Azure-Samples/app-templates-java-openai-springapps.git)

Next Steps

At this point, you have a complete application deployed on Azure.

Enterprise Scenarios

For enterprise needs, looking for polyglot applications deployment, Tanzu components support and SLA assurance, we recommend to use Azure Spring Apps Enterprise. Check the Azure Spring Apps landing zone accelerator that provides architectural guidance designed to streamline the production ready infrastructure provisioning and deployment of Spring Boot and Spring Cloud applications to Azure Spring Apps. As the workload owner, use architectural guidance provided in landing zone accelerator to achieve your target technical state with confidence.

Azure Developer CLI

You have deployed the sample application using Azure Developer CLI, however there is much more that the Azure Developer CLI can do. These next steps will introduce you to additional commands that will make creating applications on Azure much easier. Using the Azure Developer CLI, you can setup your pipelines, monitor your application, test and debug locally.

azd down - to delete all the Azure resources created with this template
azd pipeline config - to configure a CI/CD pipeline (using GitHub Actions or Azure DevOps) to deploy your application whenever code is pushed to the main branch.
- Several environment variables / secrets need to be set for Azure OpenAI resource:
- AZURE_OPENAI_API_KEY: API key for Azure OpenAI resource
  - For GitHub workflows, you should use GitHub Secrets
  - For Azure DevOps pipelines, you check 'Keep this value secret' when creating the variable
- AZURE_OPENAI_ENDPOINT: Endpoint for Azure OpenAI resource
- AZURE_OPENAI_DEPLOYMENT_ID: Deployment ID/name for Azure OpenAI resource
- IS_AZURE_OPENAI_GPT4_MODEL: Set to true if you are using GPT-4 model and to false if you are using GPT-3.5 Turbo model
azd monitor - to monitor the application and quickly navigate to the various Application Insights dashboards (e.g. overview, live metrics, logs)
Run and Debug Locally - using Visual Studio Code and the Azure Developer CLI extension

Additional `azd` commands

The Azure Developer CLI includes many other commands to help with your Azure development experience. You can view these commands at the terminal by running azd help. You can also view the full list of commands on our Azure Developer CLI command page.

Resources

These are additional resources that you can use to learn more about the sample application and its underlying technologies.

Start from zero and scale to zero – Azure Spring Apps consumption plan

Azure Spring Apps Consumption - Networking and Security

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkId=521839. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.

Telemetry Configuration

Telemetry collection is on by default.

To opt-out, set the variable enableTelemetry to false in infra/main.parameters.json or in bicep template infra/main.bicep. It can be set using the following command when the provisioning is done with Azure Developer CLI:

azd env set enableTelemetry false

Push your fork to GitHub:

In GitHub, create a pull request

If we suggest changes then:

Make the required updates.

Rebase your fork and force push to your GitHub repository (this will update your Pull Request):

git rebase main -i
git push -f
That's it! Happy Contributing !!

For More such content and Open Source Project Tutorial Guide, Please follow, share and like.

https://www.linkedin.com/in/nashet-ali-464a65115/

Open Source LLM: Llama 2

Syed Nashet Ali — Tue, 08 Aug 2023 13:51:51 +0000

In this Blog, I will give you a description about Llama 2 from Meta' s prospective and also the steps on how you can contribute to Meta's Model:

Llama 2
Meta has unlocked the power of large language models. Meta's latest version of Llama is now accessible to individuals, creators, researchers and businesses of all sizes so that they can experiment, innovate and scale their ideas responsibly.

This release includes model weights and starting code for pretrained and fine-tuned Llama language models — ranging from 7B to 70B parameters.

This repository is intended as a minimal example to load Llama 2 models and run inference. For more detailed examples leveraging HuggingFace, see llama-recipes.

Updates post-launch
See UPDATES.md.

Download
⚠️ 7/18: We're aware of people encountering a number of download issues today. Anyone still encountering issues should remove all local files, re-clone the repository, and request a new download link. It's critical to do all of these in case you have local corrupt files. When you receive the email, copy only the link text - it should begin with https://download.llamameta.net and not with https://l.facebook.com, which will give errors.

In order to download the model weights and tokenizer, please visit the Meta AI website and accept our License.

Once your request is approved, you will receive a signed URL over email. Then run the download.sh script, passing the URL provided when prompted to start the download. Make sure that you copy the URL text itself, do not use the 'Copy link address' option when you right click the URL. If the copied URL text starts with: https://download.llamameta.net, you copied it correctly. If the copied URL text starts with: https://l.facebook.com, you copied it the wrong way.

Pre-requisites: make sure you have wget and md5sum installed. Then to run the script: ./download.sh.

Keep in mind that the links expire after 24 hours and a certain amount of downloads. If you start seeing errors such as 403: Forbidden, you can always re-request a link.

Access on Hugging Face
We are also providing downloads on Hugging Face. You must first request a download from the Meta AI website using the same email address as your Hugging Face account. After doing so, you can request access to any of the models on Hugging Face and within 1-2 days your account will be granted access to all versions.

Setup
In a conda env with PyTorch / CUDA available, clone the repo and run in the top-level directory:

pip install -e .
Inference
Different models require different model-parallel (MP) values:

Model MP
7B 1
13B 2
70B 8
All models support sequence length up to 4096 tokens, but we pre-allocate the cache according to max_seq_len and max_batch_size values. So set those according to your hardware.

Pretrained Models
These models are not finetuned for chat or Q&A. They should be prompted so that the expected answer is the natural continuation of the prompt.

See example_text_completion.py for some examples. To illustrate, see command below to run it with the llama-2-7b model (nproc_per_node needs to be set to the MP value):

torchrun --nproc_per_node 1 example_text_completion.py \
--ckpt_dir llama-2-7b/ \
--tokenizer_path tokenizer.model \
--max_seq_len 128 --max_batch_size 4
Fine-tuned Chat Models
The fine-tuned models were trained for dialogue applications. To get the expected features and performance for them, a specific formatting defined in chat_completion needs to be followed, including the INST and <> tags, BOS and EOS tokens, and the whitespaces and breaklines in between (we recommend calling strip() on inputs to avoid double-spaces).

You can also deploy additional classifiers for filtering out inputs and outputs that are deemed unsafe. See the llama-recipes repo for an example of how to add a safety checker to the inputs and outputs of your inference code.

Examples using llama-2-7b-chat:

torchrun --nproc_per_node 1 example_chat_completion.py \
--ckpt_dir llama-2-7b-chat/ \
--tokenizer_path tokenizer.model \
--max_seq_len 512 --max_batch_size 4
Llama 2 is a new technology that carries potential risks with use. Testing conducted to date has not — and could not — cover all scenarios. In order to help developers address these risks, we have created the Responsible Use Guide. More details can be found in our research paper as well.

Issues
Please report any software “bug,” or other problems with the models through one of the following means:

Reporting issues with the model: github.com/facebookresearch/llama
Reporting risky content generated by the model: developers.facebook.com/llama_output_feedback
Reporting bugs and security concerns: facebook.com/whitehat/info
Model Card
See MODEL_CARD.md.

License
Our model and weights are licensed for both researchers and commercial entities, upholding the principles of openness. Our mission is to empower individuals, and industry through this opportunity, while fostering an environment of discovery and ethical AI advancements.

See the LICENSE file, as well as our accompanying Acceptable Use Policy

References
Research Paper
Llama 2 technical overview
Open Innovation AI Research Community
Original LLaMA
The repo for the original llama release is in the llama_v1 branch.

Contributing to Llama

We want to make contributing to this project as easy and transparent as possible.

Pull Requests
We actively welcome your pull requests.

Fork the repo and create your branch from main.
If you've added code that should be tested, add tests.
If you've changed APIs, update the documentation.
Ensure the test suite passes.
Make sure your code lints.
If you haven't already, complete the Contributor License Agreement ("CLA").
Contributor License Agreement ("CLA")
In order to accept your pull request, we need you to submit a CLA. You only need to do this once to work on any of Meta's open source projects.

Complete your CLA here: https://code.facebook.com/cla

Issues
We use GitHub issues to track public bugs. Please ensure your description is clear and has sufficient instructions to be able to reproduce the issue.

Meta has a bounty program for the safe disclosure of security bugs. In those cases, please go through the process outlined on that page and do not file a public issue.

License
By contributing to Llama, you agree that your contributions will be licensed under the LICENSE file in the root directory of this source tree.

Please to like, share and follow for more such Content!!!
https://www.youtube.com/@Cloudify_with_Nashet/featured
https://www.linkedin.com/in/nashet-ali-464a65115/
https://twitter.com/NashetKing

New Azure for Operators solutions and services built for the future of telecommunications

Syed Nashet Ali — Thu, 02 Mar 2023 21:02:38 +0000

After years of laying the foundation for the rollout of 5G and network cloudification, we are now seeing tremendous possibilities for how consumers and organizations will interact with technology for work, play, and connecting with communities. Around the world and across industries, 5G, augmented and virtual reality, AI, IoT, and edge computing are creating opportunities for rapid digital transformation. This is compelling operators to move away from legacy systems to accelerate network transformation and begin monetizing their investments in 5G and edge technologies.

Imagine the benefits to communities and organizations that have access to improved bandwidth, reliability, and reduced latency, while leveraging the rich capabilities of cloud-to-edge technology without compromise to security, critical services, or key workloads.

With the most complete offerings for the telecommunications industry, Microsoft is the ideal cloud provider to help operators with their digital transformation journey and enable them to deliver these innovative services to their consumer, enterprise, and public sector customers.

Azure for Operators
With Azure for Operators, we’re empowering operators to unlock the power of 5G by bringing cloud and edge closer together to modernize their networks so that they can streamline and optimize their business operations and deliver new services faster with greater reach and lower cost. With solutions that run on-premises, at the edge, or in the cloud but are always managed and secured by Azure, Microsoft meets you where you are, offering flexibility to transform on your terms and timelines. We are committed to partnering with you, not competing against you. Your consumer, enterprise, and government customers will benefit from a cloud platform with industry-leading security and governance built-in, and with an unmatched partner and developer ecosystem to maximize the value of the cloud at the edge. We’re applying Microsoft technology and developer ecosystem capabilities to offer the next-generation Azure for Operators portfolio—carrier-grade hybrid cloud platform, voice core, mobile core, and multi-access edge compute.

Introducing the new Azure for Operators solutions and services
In September 2020, we unveiled the Azure for Operators initiative to help operators evolve their own networks, and for partners to assist enterprises to maximize their industry 4.0 opportunities. In June of 2021, we introduced a new category, Azure private multi-access edge compute (MEC), which empowers operators and system integrators to unlock the enterprise 5G opportunity. Today, we’re announcing the next wave of Azure for Operators solutions and services.

New Azure for Operators solutions and services

The hybrid cloud platform built for the future of telecommunications
Operators are continually looking toward digital transformation to improve competitiveness, deliver new services, and yield positive financial results through both innovation and growth. We believe telecom digital transformation starts with the modernization of network core systems and operations.

In June 2021, Microsoft acquired the AT&T Network Cloud 2.7 Software technology, an industry-first collaboration. The AT&T Network Cloud supports AT&T's 5G core and FirstNet today, representing seven years of experience developing on-premises cloud for network workloads. We’ve taken the acquisition of this technology, moved the engineering and program organization from AT&T’s Network Cloud organization into Azure for Operators, and directly integrated the intellectual property into a new Azure offering. We continue to make significant enhancements to the acquired technology itself, including security across AT&T’s entire Network Cloud deployment.

Today we are announcing the next-generation hybrid cloud platform for operators—Azure Operator Distributed Services—which combines the enhanced version of the acquired AT&T technology with the best of Azure, including our industry-leading security, monitoring, analytics, AI, machine learning, and so much more.

Built from the ground up and proven to run network-intensive workloads and mission-critical applications, Azure Operator Distributed Services meets the security, resiliency, observability, manageability, and performance needs required by operators to achieve meaningful results from digital transformation. It will enable operators to run all their workloads (such as core, RAN, mobile and voice core, OSS, and BSS) on a single carrier-grade hybrid platform. With Azure Operator Distributed Services, business operations can be streamlined, providing operators with simplified management, policy, and automation delivered through unified cloud management using Azure Services, Azure Arc management, Azure Security, and platform as a service with Azure. Azure Operator Distributed Services provides the flexibility and scalability to support customer deployments at the edge of the cloud, the edge of the network, or the enterprise edge. Now, operators can effectively create new services to monetize their network, all while gaining robust network and customer insights that are necessary to facilitate proactive decision-making, critical action, and the creation of real value.

With Azure Operator Distributed Services, we are delivering a carrier-grade hybrid cloud service to the market and AT&T where it can run at AT&T on-premises or on Azure public cloud. Azure Operator Distributed Services is designed to support the AT&T mobile core network that today spans more than 60 containerized network functions (CNFs) and virtual network functions (VNFs) from 15 different vendors, which currently are deployed and running on the AT&T Network Cloud platform. AT&T will continue to select and manage VNFs and CNFs and their configurations to deliver mobility services to AT&T customers. AT&T and Microsoft are closely collaborating on the deployment of Azure Operator Distributed Services with initial testing stages planned for later this year.

“AT&T's 5G network is built on an agile, future-ready mobile core that's designed to boost innovation, resiliency, and security for our customers. Our 5G mobility core is fully cloud-based, integrating Network Functions from multiple vendors that today are hosted on Network Cloud infrastructure based on a cloud software stack that Microsoft acquired from AT&T last year. We are pleased with Microsoft’s plan to evolve Network Cloud and integrate it with Azure technologies to create hybrid telco-grade Azure Operator Distributed Services. This will enable AT&T and other operators to host Network Functions on clouds spanning telco premises and public cloud and will help us realize the many benefits of the cloud-native approach and Azure innovation including additional speed, resiliency, security, cost, and operational improvements.“—Andre Fuetsch, Executive Vice President and CTO Network Services, AT&T

We designed the Azure Operator Distributed Services for use by all operators while maintaining security and without losing differentiation. In terms of security and privacy, we want to make clear that operators using Azure Operator Distributed Services continue to hold access to their customer data, Microsoft cannot access or see it. With this product, we want to enable operators to deliver new services faster and more flexibly across Azure public cloud and on-premises with common tooling and services, reducing time-to-market with a cloud-native approach. True to our mission of empowering everyone to achieve more, we are collaborating with the broader partner ecosystem, including Ericsson and Nokia, to build on our next-gen hybrid cloud platform for operators.

“Combined with cloud capabilities, 5G has the potential to accelerate the digital transformation of virtually any sector of industry or society. The combination of Azure Operator Distributed Services and Ericsson’s market-leading cloud-native network functions and orchestration suite promise significant benefits for customers. Microsoft and Ericsson are jointly exploring enterprise 5G use cases in conjunction with leading operators.”—Jan Karlsson, Senior Vice President and Head of Business Area Digital Services, Ericsson

“Open collaboration is key to the development of new and innovative high value 5G use cases that will equip our customers with the tools they need for digital transformation. This is part of Nokia’s continued commitment to leading an open mobile future, making it simple for our customers to take advantage of the 5G world, helping to drive it forward. We are enthusiastic to work closely with Microsoft to integrate our Cloud RAN technology with Microsoft Azure, offer Microsoft Azure IoT Edge Services allowing industries to realize new capabilities at the edge, and bring Nokia’s new SaaS-based NetGuard Cybersecurity Dome as well as other Cloud and Network Services software to Azure.”—Chris D. Jones, Vice President Strategic Partnerships, Strategy and Technology, Nokia
Azure Private 5G Core in 5G.MIL solutions will enable high performance in a small footprint and the ability to support, secure, and scale interconnection between 5G wireless and military networks. “Lockheed Martin’s strategic collaboration with Microsoft unlocks new capabilities that will provide a decisive edge for military personnel across all domains,” said Dan Rice, Vice President of 5G.MIL® Programs at Lockheed Martin. “Leveraging Microsoft’s expertise in cloud computing and distributed system orchestration, we are accelerating the scale and speed of critical military communications, which will be necessary in increasingly contested environments.”
HARMAN Digital Transformation Solutions (DTS) has employed Azure private MEC to successfully advance digital transformation at one of the largest airports in the United States. The solution improved operational efficiencies for the airport’s manual cargo handling process. "Advances in technologies like 5G yield significant opportunity to accelerate innovation across nearly every industry—from transportation and healthcare to manufacturing and education,” said David Owens, Senior Vice President and General Manager, Digital Transformation Solutions, HARMAN. “HARMAN's deep knowledge and experience in the communications domain and Azure private MEC offers a comprehensive solution to enterprises planning to deploy private networks and cloud computing applications. We are glad to join hands with Microsoft in re-shaping the future by elevating experiences for our customers and end consumers."
"Northrop Grumman’s advanced battle management technologies allow military forces to effectively communicate and securely share mission-critical data across all domains. With the integration of the Azure private MEC platform, we are leveraging Microsoft’s commercial-edge computing capabilities to inform tactical mobile ad hoc networks for the U.S. Navy’s Information Warfare Research Project.”—Tom Pieronek, Chief Technology Officer, Aeronautics Systems, Northrop Grumman
"As a pioneer in digital transformation, Fujitsu is committed to helping enterprises and telcos accelerate adoption of 5G. Our market-leading O-RAN solutions and managed services offerings, combined with the power of Microsoft’s Azure private MEC offering, will play an important role in delivering 5G solutions to market, enabling rich customer experiences at the convergence of 5G, digital, and the edge.”—Shingo Mizuno, Corporate Executive Officer, Fujitsu Limited
"We are entering a new era of enterprise digital transformation that will be fueled by private networks, edge computing, secure access service edge, and amplification of industry-specific applications. Along with Microsoft, we are making it easy for enterprises to order, provision, and consume full suite of infrastructure capabilities and applications—the cloud way"—Manish Mangal, Global Head of 5G and Network Services, Tech Mahindra

transformation journeys through adopting advanced technologies such as autonomous robots, vehicles, and drones, as well as Metaverse applications, into their enterprise operations through a single unified platform.”—Bill Chang, CEO, Group Enterprise, Singtel

Telstra announced customer trials of Australia’s first 5G-enabled edge compute solution for enterprises (Branch Offload) through a collaboration with Microsoft and Ericsson that will use technologies including Telstra’s 5G and fixed connectivity, Azure Stack Edge, Secure Edge, SD-WAN and service orchestration, and will be delivered as a managed service by Telstra Purple. Nikos Katinakis, Telstra’s Group Executive for Networks and IT said, “Telstra’s collaboration with our major strategic partners, Microsoft and Ericsson, continues to enable us to break new ground, leveraging new technologies and our smarter network to standardize solutions and that will help Australian businesses adapt for the digital future.”

“Technical trials demonstrated how Azure public MEC and AT&T’s 5G mobile network-enabled Summit Tech’s Odience 360 8K live streaming platform with built-in e-commerce to bring immersive and interactive experiences to use cases such as shopping, live concerts, sports stadiums, and eSports. MEC video processing yielded up to 80 percent bandwidth reduction, imperceptible motion-to-photon delay, and impressive glass-to-glass low latency for true two-way interactivity. Check out the demo video to learn more."—Doug Makishima, Chief Sales, and Marketing Officer (CSMO), Summit Tech

"As the requirements of modern applications continue to evolve, Couchbase is collaborating with Microsoft and Azure public MEC to bring the cloud and the edge closer together for customers. Azure public MEC and Couchbase’s modern database enables developers to build next-gen edge use cases."—Matt McDonough, SVP Business Development and Strategy, Couchbase

Watch this ISV testimonial to hear from Citrix, Spirent, Game Cloud, VMware, and Fortinet about how they are working with Microsoft to build and deliver public MEC solutions.

AWS is powering the continued advancement of Autonomous Vehicle (AV) development.

Syed Nashet Ali — Mon, 19 Dec 2022 17:53:10 +0000

Developing and deploying Advanced Driver-Assistance Systems (ADAS) and AV Systems requires a development platform with highly scalable compute, storage, networking, as well analytics and deep learning frameworks. This platform requires capabilities that allow for the collection, ingestion, storage, processing and analytics, labeling and annotation, map development, algorithm and model development, simulations, verification and validation, and workspace management functions (inclusive of MLOps and DevOps). Leading automotive customers turn to AWS as their ADAS/AV development platform for our breadth and depth of managed services, solutions, experience, and partner community to deliver the architecture and technology required for to develop safe, reliable, and cost-optimized autonomous and ADAS systems.
Benefits
Unmatched compute storage and network scale
AWS solves petabyte-scale data processing, storage, and management needs by delivering thousands of cores of compute for development and validation.
Accelerate time to market
Building on AWS helps customers optimize their software engineering to be more agile, reducing development and validation costs and supporting a faster time to market.
Multiple ways to control cost
AWS has the most classes of storage of any cloud resulting in increased cost efficiency because data can be configured and paid for based on the frequency of access, durability, and availability requirements.

Thanks & Regards,
Nashet Ali.
Cloud Expert.

Serverless Computing - AWS Lambda - Amazon Web Services

Syed Nashet Ali — Fri, 19 Aug 2022 17:58:42 +0000

What is Serverless?
Serverless is a term that generally refers to serverless applications. Serverless applications are ones that don’t need any server provision and do not require to manage servers.

What is AWS Lambda?
AWS Lambda is an event-driven, serverless computing platform provided by Amazon as a part of Amazon Web Services. Therefore you don’t need to worry about which AWS resources to launch, or how will you manage them. Instead, you need to put the code on Lambda, and it runs.

In AWS Lambda the code is executed based on the response of events in AWS services such as add/delete files in S3 bucket, HTTP request from Amazon API gateway, etc. However, Amazon Lambda can only be used to execute background tasks.

AWS Lambda function helps you to focus on your core product and business logic instead of managing operating system (OS) access control, OS patching, right-sizing, provisioning, scaling, etc.

In this AWS Lambda tutorial for beginners, you will learn:

How does AWS Lambda work?
Events that Trigger AWS Lambda
AWS Lambda Concepts
AWS Lambda VS AWS EC2
AWS Lambda VS AWS Elastic Beanstalk
Use Cases of AWS Lambda
Best practices of Lambda function
When not to use AWS Lambda
Advantages of using AWS Lambda
Limitations of AWS Lambda
How does AWS Lambda work?
The following AWS Lambda example with block diagram explains the working of AWS Lambda in a few easy steps:
Step 1: First upload your AWS Lambda code in any language supported by AWS Lambda. Java, Python, Go, and C# are some of the languages that are supported by AWS Lambda function.

Step 2: These are some AWS services which allow you to trigger AWS Lambda.

Step 3: AWS Lambda helps you to upload code and the event details on which it should be triggered.

Step 4: Executes AWS Lambda Code when it is triggered by AWS services:

Step 5: AWS charges only when the AWS lambda code executes, and not otherwise.

This will happen in the following scenarios:

Upload files in an S3 bucket
When HTTP get/post endpoint URL is hit
For adding/modifying and deleting Dynamo DB tables
In the process of data streams collection
Push notification
Hosting of website
Email sending
Note: You should remember that you will charge for AWS services only when the AWS Lambda code executes, else you don’t need to pay anything.
Events that Trigger AWS Lambda
Here, are Events which will be triggered when you use AWS Lambda.

Insert, updating and deleting data Dynamo DB table
To include push notifications in SNS
To search for log history in CloudTrail
Entry into an S3 object
DynamoDB can trigger AWS Lambda whenever there is data added, modified, and deleted in the table.
Helps you to schedule the event to carry out the task at regular time pattern.
Modifications to objects in S3 buckets
Notifications sent from Amazon SNS.
AWS Lambda can be used to process the CloudTrail logs
API Gateway allows you to trigger AWS Lambda on GET/POST methods.
AWS Lambda Concepts
Function:

A function is a program or a script which runs in AWS Lambda. Lambda passes invocation events into your function, which processes an event and returns its response.

Runtimes:

Runtime allows functions in various languages which runs on the same base execution environment. This helps you to configure your function in runtime. It also matches your selected programming language.

Event source:

An event source is an AWS service, such as Amazon SNS, or a custom service. This triggers function helps you to executes its logic.

Lambda Layers:

Lambda layers are an important distribution mechanism for libraries, custom runtimes, and other important function dependencies. This AWS component also helps you to manage your development function code separately from the unchanging code and resources that it uses.

Log streams:

Log stream allows you to annotate your function code with custom logging statements which helps you to analyse the execution flow and performance of your AWS Lambda functions.

How to use AWS Lambda
Now, we will learn how to use AWS Lambda with AWS Lambda example:

Step 1) Step 1) Open AWS Lambda URL
Goto https://aws.amazon.com/lambda/ and Get Started

Get Started with AWS Lambda
Step 2) Create an account
Next, Create an account or sign in with your existing account

Step 3) Edit the code & Click Run,
In the next Lambda page,

Edit the code
Click Run
AWS Lambda Run Code
Step 4) Check output
You will see output

AWS Lambda Output

AWS Lambda VS AWS EC2
Here, are some major differences between AWS Lambda and EC2.

Parameters AWS Lambda AWS EC2
Definition AWS Lambda is a Platform as a Service (PaaS). It helps you to run and execute your backend code. AWS EC2 Is an Infrastructure as a Service (laaS). It provides virtualized computing resources.
Flexibility Does not offers any flexibility to log in to compute instances. It allows you to choose a customized operating system or language runtime. Offers the flexibility to select the variety of instances, customoperating systems, security patches, and network, etc.
Installation process You need to select your environment where you want to runthe code and push the code into AWS Lambda. For the first time in EC2, you have to choose the OS and install all the software required and then push your code in EC2.
Environment restrictions It is restricted to fewlanguages. No environment restrictions.
AWS Lambda VS AWS Elastic Beanstalk
Here, are some major differences between AWS Lambda and Elastic Beanstalk.

Parameters AWS Elastic Beanstalk AWS Lambda
Main task Deploy and manage the apps on AWS Cloud without worrying about the infrastructure which runs those applications. AWS Lambda is used for running and executing your Back-end code. You can’t use it to deploy an application.
Selection of AWS resources It gives you a Freedom to select AWS resources; For example, you can choose EC2 instance which is optimal according to your application. You can’t select the AWS resources, like a type of EC2 instance, Lambda offers resources based on your workload.
Type of system It is a stateful system. It is a stateless system.
Use Cases of AWS Lambda
AWS Lambda used for a wide range of applications like:

Helps you for ETL process
Allows you to perform real-time file processing and real-time stream processing
Use for creating web applications
Use in Amazon products like Alexa Chatbots and Amazon Echo/Alexa
Data processing (real-time streaming analytics)
Automated Backups of everyday tasks
Scalable back ends (mobile apps, loT devices)
Helps you to execute server-side backend logic
Allows you to filter and Transform data
Best practices of Lambda function
Here are some best practices of AWS Lambda functions:

Use the right “timeout.”
Utilize the functions of local storage which is 500MB in size in the /temp folder
Minimizing the use of start-up code which is not directly related to processing the current event.
You should use built-in CloudWatch monitoring of your Lambda functions to view and optimize request latencies.
When not to use AWS Lambda
Following are the situation where Lambda is surely not an ideal option:

It is not appropriate to use AWS Lambda software packages or applications which rely on calling underlying Windows RPCs
If is used for custom software applications with licensing agreements like MS-Office document processing, Oracle databases, etc.
AWS Lambda should not be used for custom hardware process such as GPU acceleration, hardware affinity.
Advantages of using AWS Lambda
Here, are pros/benefits of using AWS lambda:

AWS Lambda is a highly flexible tool to use
It helps you to grant access to resources, including VPCs
Author directly with WYSIWYG editor in console.
You can use it as a plugin for Eclipse and Visual Studio.
As it is serverless architecture, you don’t need to worry about managing or provisioning servers.
You do not need to set up any Virtual Machine.
Helps developers to run and execute the code’s response to events without building any infrastructure.
You just need to for the compute time taken, only when your code runs.
You can monitor your code performance in real time through CloudWatch.
It allows you to run your code without provisioning or to manage any other server
Helps you to execute the code only when needed
You can scale it automatically to handle a few requests per day and even support more than thousands of requests per second.
AWS Lambda can be configured with the help of external event timers to perform scheduled tasks.
Lambda function in AWS should be configured with external event and timers so; it can be used for scheduling.
Lambda functions are stateless so that it can be scaled quickly.
AWS Lambda is fast so it will execute your code within milliseconds.
Limitations of AWS Lambda
Here are the cons/disadvantages of using AWS Lambda:

AWS Lambda tool is not suitable for small projects.
AWS Lambda entirely relies on AWS for the infrastructure, so you can’t install any additional software if your code demands it.
Concurrent execution is limited to 100
AWS Lambda completely depended on AWS for the infrastructure; you cannot install anything additional software if your code demands it.
Its memory volume can vary between 128 to 1536 MB.
Event request should not exceed 128 KB.
Lambda functions help you to write their logs only in CloudWatch. This is the only tool that allows you to monitor or troubleshoot your functions.
Its code execution timeout is just 5 minutes.
Summary
Serverless is a term that generally refers to serverless applications.
AWS Lambda is one such serverless compute service. Therefore, you don’t need to worry about which AWS resources to launch, or how will they manage them.
A function is a program or a script which runs in AWS serverless Lambda.
Runtime allows functions in various languages which runs on the same base execution environment.
An event source is an AWS service, such as Amazon SNS, or a custom service.
Lambda layers are an important distribution mechanism for libraries, custom runtimes, and other important function dependencies.
Log stream allows you to annotate your function code with custom logging statements which helps you to analyse the execution flow and performance of your Lambda functions.
AWS Lambda is a Platform as a Service (PaaS). It helps you to run and execute your backend code.
AWS EC2 Is an Infrastructure as a Service (laaS). It provides virtualized computing resources.
Deploy and manage the apps on AWS Cloud without worrying about the infrastructure which runs those applications.
AWS Lambda is used for running and executing your Back-end code. You can’t use it to deploy an application.
AWS Lambda helps you for the ETL process.
The best practice of Lambda function in AWS is to use the right “timeout.”
It is not appropriate to use AWS Lambda software packages or applications which rely on calling underlying Windows RPCs
AWS Lambda is a highly flexible tool.
AWS Lambda tool is not suitable for small projects.
A common event which will be triggered when you use AWS Lambda is Insert, updating and deleting data Dynamo DB table.
You Might Like:
How to Change Instance Type & Security Group of EC2 in AWS
How to Create EC2 Instance in AWS: Step by Step Tutorial
AWS (Amazon Web Services) Tutorial: Basics for Beginners
Azure vs. AWS: What is the Difference Between AWS and Azure
AWS Lamda Documentation Official