DEV Community: Ayyanar Jeyakrishnan

Building AI applications on AWS Bedrock with enhanced security measures

Ayyanar Jeyakrishnan — Tue, 20 Jun 2023 10:08:02 +0000

Great session with Andrew Kane and Mark Ryland on building AI applications on AWS Bedrock with enhanced security measures.

https://www.youtube.com/watch?v=5EDOTtYmkmI&list=PL2yQDdvlhXf_kZMl0XZYqWfysycSXCJx8&index=40

Here are my top key takeaways from this session:

1) Access to Bedrock is facilitated through APIs powered by foundation models from Amazon Titan, Anthropic, and Stability.ai.

2) Sagemaker training/pipeline can be utilized for fine-tuning the base model provided by the model provider.

3) Fine-tuned data resides in your account, while the fine-tuned model is stored in the Model Provider Escrow Account, owned and operated by AWS. However, you can securely use your fine-tuned model through a Single-tenant endpoint, ensuring exclusive consumption.

4) All data transmitted is secured using TLS1.2, and data at rest is protected by AWS KMS (Key Management Service).

5) Prompt data and its outputs are not used by AWS, and the prompt history is stored securely.

6) IAM provides role-based access control (RBAC) for managing model access permissions.

7) #AWS CodeWhisperer significantly improves developer productivity as an AI code companion and enables one-click security scans of your code.

8) Additionally, AWS Sagemaker MLOps, distributed training of LLM, and Foundation Model from Sagemaker Jumpstart empower experimentation before utilizing models from AI21 Labs, LightOn, Stability AI, Cohere, Amazon Titan, and facilitate easy single-click deployments.

TriviaGen_AI

Ayyanar Jeyakrishnan — Tue, 20 Jun 2023 09:25:51 +0000

As an active member of the AWS User Group Bangalore and Cloudnloud Tech Community, we regularly publish monthly projects focused on Machine Learning and Artificial Intelligence. These projects come with detailed architectures, allowing interested community members to participate, contribute, and receive credit for their involvement. It serves as a collaborative platform where individuals can join forces, learn, and showcase their skills in the exciting fields of ML/AI within our community.

Introduction

We are building a Quiz Platform to interact with target audience to test their knowledge, Involve them in Poll. We help the Clients to create Poll and Trivia using Generative AI - using a Large Language Model.

Technical quizzes and marketing polls are both important tools for different reasons. Technical quizzes serve as a way to test and assess knowledge on a specific subject or topic, which is essential for personal growth, skill development, and career advancement. On the other hand, marketing polls are used to collect information from potential customers to gain insights into their preferences, behaviors, and opinions, which is crucial for businesses to develop and market products effectively. By combining the insights from technical quizzes and marketing polls, businesses can gain a better understanding of their target audience and develop products that meet their needs and preferences. Overall, both technical quizzes and marketing polls are important tools for personal and professional development, as well as for businesses to succeed in the market.

The application supports the following use-cases for web users:

Register & login
Create a Custom Quiz, Poll Manually and Publish to target users.
Generate a Quiz or Poll Automatically using Generative AI and Human in Loop.
Custom Fine-Tune Generative AI based on your data - Text and Image.
Analytics over user Engagements.

Architecture Overview

Frontend-UI

The frontend of a quiz application is the user interface that users interact with to access and use the application. It is the visible part of the application that users see and interact with, and it is responsible for providing a seamless and intuitive user experience.

In our application using Amplify, the frontend built using React JS,. Amplify provides a set of pre-built UI components that can be easily integrated into the frontend to handle tasks such as user authentication, API calls, and data management. Using Amplify, we leverage pre-built UI components and frameworks to rapidly build and deploy a frontend that meets the needs of users.

User authentication: Users must be authenticated before they can access the quiz application. Amplify provides pre-built UI components for user authentication, including sign-in and sign-up using external IdP (Google, Linkedin, Github and Twitter) , and handles the secure storage and management of user credentials.
Quiz Creation Interface: This has three options.
a) Manual Trivia
b) GenAI-Trivia
c) Fine-GenAI-Trivia
Quiz interface: The quiz interface provides a clear and intuitive way for consumers to answer questions and navigate through the quiz. This may include features such as question numbering, progress tracking and a timer.
Results screen: Once the quiz is completed, the frontend should display the user's score and provide feedback on the questions answered incorrectly. This may include a breakdown of the user's performance and suggestions for improvement.

Data Processing and Serving.

Manual Trivia: Clients can create their quiz using the Dynamic Page. Here we use GraphQL API, AppSync, DynamoDB, and Lambda. The platform would have a GraphQL API for CRUD operations on quizzes, questions, and answers, with DynamoDB as a backend database. AppSync would be used for backend development and scaling, and Lambda functions would execute server-side logic. Additionally, the platform would allow real-time data synchronization using WebSockets, and support authentication and authorization for users.

GenAI-Trivia: Client Provide Input as String “AWS Lambda”. Here we create a quiz using Generative AI We add Amplify to use API Gateway service to create a new API and define the necessary routes for creating, reading, updating, and deleting quizzes, questions, and answers. Integrate with Sagemaker Model Endpoint with API Gateway to allow for natural language processing and generation of quiz questions. AWS Lambda to create serverless functions that will execute business logic such as authentication, authorization, and quiz scoring and update DynamoDB table to store quiz data and integrate it with the Lambda functions for data retrieval and storage. You can use AWS Foundation Model like AI21Labs Jurrasic Jumbo for this usecase or use AWS Bedrock API (still in beta).

AI-In-Loop : Here based on GenAI data, Clients will provide a Dynamic Page to approve/Reject the Generated Quiz and once Approved. It will be send to target Consumers.

Fine-GenAI-Trivia: Client Provide PDF file or Text, Eg new book and like to create a quiz based on new Data.

Here you can upload the PDF or text file, It will store in S3 and lambda will be invoked based on the S3 Put Event and launch the AWS Textract Job which extracts the Data from the PDF file and stores it again in S3. Create a Eventbridge and trigger a sagemaker pipeline and use embed model to convert the text in vector and store it in the Serverless Opensearch domain (Here Elasticsearch act as Vector Database) with the unique index it.
Now clients can use their “Prompt” to generate the quiz for their custom content uploaded as PDF. Here Prompt + Context retrieved from Elasticsearch provided to Sagemaker Endpoint as input for generate text.

AI-In-Loop : Here based on GenAI data, Clients will provide a Dynamic Page to approve/Reject the Generated Quiz and once Approved. It will be send to target Consumers.

Benefits of this solution

Scalability of quiz/Trivia Platform using AWS Amplify and Generative AI
Clients can easily publish the Trivia to target users to test their knowledge.
We can use the same logic and pivot the Usecase to Marketing, Campaign, User Analytics.
Support for multiple devices using adaptive bitrate streaming for playback in all networks.
Quickly building and deploying the solution using Amplify and CloudFormation.
Colleges, Universities, Companies can Publish the MCQ to different questions to consumers/Students.

GitHub Link
https://github.com/aws-data-usergroup-bangalore/triviagen-ai

awscommunitybuilders #awscommunitybuilder

Build a Generative AI on AWS using AWS BedRock and Titan

Ayyanar Jeyakrishnan — Thu, 13 Apr 2023 19:42:00 +0000

AWS Announced “Amazon Bed Rock” a new service that makes Foundation Model from AI21 Labs, Anthropic, Stability AI, and Amazon accessible via an API.

Bedrock helps us to access all the Foundation Models which are pre-trained with a large amount of dataset and Billions of Parameters.

Amazon announced the Amazon Titan Foundation Models.

Generate AI - Large Language Model LLM

Titan Text is a powerful large language model (LLM) that can automate natural language tasks such as summarization and text generation. It can also be used for classification, open-ended Q&A, and information extraction, making it a versatile tool for various language-related applications.

Embedding.

Embeddings are used to represent queries and documents in a vector space, where the similarity between vectors can be used to rank search results. By using embeddings, search engines can better understand the meaning of queries and documents, and return more relevant results.

For example, suppose a user searches for "best Pizza restaurant in New York City." By using embeddings, a search engine can represent this query as a vector in a high-dimensional space, where words like "Pizza," "restaurant," "best," and "New York City" are mapped to specific coordinates in the vector space. Similarly, each document in the search index can also be represented as a vector.

The search engine can then rank search results based on the similarity between the query vector and the document vectors. Documents that are more similar to the query vector will be ranked higher, and returned as the top search results. By using embeddings, search engines can better understand the meaning of queries and documents, and return more relevant results.

Responsible AI

AWS is Committed to advance the design and deployment of safe and trustworthy artificial intelligence which benefits all of humanity by becoming a member of https://www.responsible.ai/

Titan FMs are designed to promote responsible use of AI by identifying and mitigating inappropriate or harmful content. They are capable of detecting and eliminating harmful content in the data, preventing inappropriate content in user input, and filtering out model outputs that contain inappropriate content such as hate speech, profanity, or violent language.

For More details
https://aws.amazon.com/blogs/machine-learning/announcing-new-tools-for-building-with-generative-ai-on-aws/

Integrate OpenAI with AWS Lambda

Ayyanar Jeyakrishnan — Fri, 10 Mar 2023 19:08:03 +0000

Improve Productivity Using CodeCatalyst

Ayyanar Jeyakrishnan — Mon, 19 Dec 2022 17:58:11 +0000

What is CodeCatalyst

https://medium.com/@jayyanar/improve-productivity-using-codecatalyst-1adbae6d1e1f

Will ChatGPT replace Google Search Now

Ayyanar Jeyakrishnan — Sun, 18 Dec 2022 18:15:01 +0000

Hot Topic Decoded
Chatbot GPT - What is ChatBot GPT

Try - https://chat.openai.com/chat
GPT-3 (Generative Pre-training Transformer) can interpret natural language text, and generate natural language. This will learn a better understanding than like of Google or Siri.

This machine learning model has analyzed billions of pages of the internet to model out natural language — it can recognize and imitate patterns of language, which are customizable.

GPT-3 understands models of language. It can see which words are connected to other words, and why they’re connected in a particular order, based on what it takes from the internet.

GPT-3 is an extensive language model built to be used for a wide variety of natural language processing tasks. The model is incredibly huge at 175 billion parameters and is trained on 570 gigabytes of text. At a high level, the model works to predict the next word in a sentence, similar to what you see in text messaging or google docs. This idea of generating the next word is based on tokens and is used for more than just the next word, but entire sentences and articles.

The model can be fine-tuned similarly to how we train neural networks or can be used in a shot learning method. Few-shot learning is the process of showing GPT-3 a few examples of a task we want the model to accomplish and the correct result, such as sentiment analysis, and then running the model on a new example.

This learning method is incredibly useful and efficient as it allows us to start getting results for our task without long training and optimization. With just a few examples of tweets and their sentiment, we were able to achieve 73% accuracy with few-shot learning!

Example 1) My Question is : What is State of AI in 2035

Example 2 ) My Question is : GPT 3 AWS

How it works?

This is the third generation of GPT - The previous generation GPT2 has 1.5 Billion contains Parameters but GPT-3 is beast of 155 Billion parameters

Will it Replace Google Search ?

GPT 3 Training in AWS

Amazon FSx for Lustre high-performance Storage with Amazon EC2 Trn1 instances, powered by AWS Trainium accelerators, are purpose built for high-performance (DL) training while offering up to 50% cost-to-train savings over comparable GPU-based instances. Trn1 instance supports up to 800 Gbps of Elastic Fabric Adapter networking bandwidth. Each Trn1 instance also supports up to 80 Gbps of Amazon Elastic Block Store (EBS) bandwidth and up to 8 TB of local NVMe solid state drive (SSD) storage for fast workload access to large datasets.

Amazon EC2 Trn1 Instances for High-Performance Model Training are Now Available

Ref: https://aws.amazon.com/blogs/aws/amazon-ec2-trn1-instances-for-high-performance-model-training-are-now-available/

Join our Group: https://www.meetup.com/aws-data-user-group-bangalore

Join Cloudnloud Tech Community for Training, Re-Engineering, and career opportunities.

@bvijaycom

Follow Page 👉 - https://lnkd.in/dJNeuhYA
Follow Group 👉- https://lnkd.in/e4V7bkgP

What is new in AWS DataExchange

Ayyanar Jeyakrishnan — Tue, 23 Aug 2022 08:29:50 +0000

𝐀𝐖𝐒 𝐃𝐚𝐭𝐚 𝐄𝐱𝐜𝐡𝐚𝐧𝐠𝐞 is on a mission to increase speed to value for third-party data sets in the cloud. There is no other place where customers can find data files

𝙒𝙝𝙮 𝘼𝙒𝙎 𝘿𝙖𝙩𝙖 𝙀𝙭𝙘𝙝𝙖𝙣𝙜𝙚?

AWS Data Exchange is a service that makes it easy for AWS customers to find, subscribe to, and use third-party data in the AWS Cloud.

𝙒𝙝𝙖𝙩 𝙙𝙤 𝘼𝙒𝙎 𝙘𝙪𝙨𝙩𝙤𝙢𝙚𝙧𝙨 𝙜𝙚𝙩 𝙛𝙤𝙧 𝙙𝙖𝙩𝙖?

These data protection features include: Data encryption capabilities available in over 100 AWS services. Flexible key management options using AWS Key Management Service (KMS), allowing customers to choose whether to have AWS manage their encryption keys or enabling customers to keep complete control over their keys.

What is new in AWS QuickSights

Ayyanar Jeyakrishnan — Tue, 23 Aug 2022 08:27:04 +0000

Join our #cloudnloud Tech Community for more information on Cloud Training and Career Opportunity

Follow Page - https://lnkd.in/dJNeuhYA #
Follow Group - https://lnkd.in/e4V7bkgP