DEV Community: Newanga Wickramasinghe

Azure DP-900 Short Notes: Explore provisioning and deploying relational database services in Azure

Newanga Wickramasinghe — Fri, 19 Feb 2021 16:36:42 +0000

👉Learn Module: Explore provisioning and deploying relational database services in Azure

👉Provisioning is the act of running series of tasks that a service provider, such as Azure SQL Database, performs to create and configure a service.

👉A user is only able to define parameters that determine the size of the resources required.

👉The inner tasks done to provision a service are hidden from the end user.

👉several tools you can use to provision services.

The Azure portal
The Azure command-line interface (CLI)
Azure PowerShell
Azure Resource Manager templates

👉Pricing tiers of three relation DB services

Basic - workloads that require light compute and I/O performance.
General Purpose - business workloads that require balanced compute and memory with scalable I/O throughput.
Memory Optimized - high-performance database workloads that require in-memory performance for faster transaction processing and higher concurrency.

Describe configuring relational data services

Configure connectivity to virtual networks and on-premises computers

👉To connect to provisioned DB

1.configure Firewalls and virtual networks page.
2.Select networks.
3.Three further sections will appear, labeled Virtual network, Firewall, and Exceptions.

👉Azure SQL Database communicates over port 1433.

👉A firewall rule of 0.0.0.0 enables all Azure services to pass through the server-level firewall rule.
Configure connectivity from private endpoints.

👉Private endpoint is a Network interface that connects you privately and securely to a service powered by Azure Private Link.
Configure authentication

👉Azure Active Directory(AD) to manage database user identities.

👉Who can access the database?
Configure access control

👉Azure role-based access control (Azure RBAC) helps you manage who has access to Azure resources, and what they can do with access.

👉RBAC through role assignment consisting of three elements

1.A security principal
2.A role definition
3.A scope
Configure advanced data security

👉Apart from Authentication and Authorization, implements threat protection and assessment.

🌟⭐ Do not forget to use a free sandbox environment and try out all the above database provisioning services.🙂

Azure DP-900 Short Notes: Explore concepts of data analytics

Newanga Wickramasinghe — Fri, 19 Feb 2021 16:26:00 +0000

👉Learn Module: Explore Concepts of Data Analytics

👉Main objective of data analytics is to make efficient and successful decisions driving the business to success.

Describe data ingestion and processing

👉Two important stages in data analytics

Data ingestion
- Capturing data from various data sources and immediately processing or storing in a database.
Data processing
- Taking raw data and cleaning and converting into a meaningful format.
- The aim of data processing is to convert the raw data into one or more business models.
- There are two forms of Data processing
  1. ETL (Extract, Transform, and Load)
    - Data is directly extracted from source and processed and stored.
    - Suitable for systems that only require simple models, with little dependency between items.
    - Improve data privacy and compliance.
  2. ELT (Extract, Load, Transform)
    - Data is stored before processing.
    - Suitable for systems with complex data models.
    - Suitable for large volume of data.

Explore data visualization

👉The output of data processing is a business model .

👉There are different techniques to analyze and understand the information in a business model.

Reporting
- Process of organizing data into informational summaries to monitor how different areas of an organization are performing.
- Reporting = what has happened.
- Analysis = why something happened and what can be done about it.
Business intelligence
- Technologies, applications, and practices for the collection, integration, analysis, and presentation of business information.
- The purpose is to support better decision making.
Data visualization
- Graphical representation of information and data.
- Power BI connect to multiple different sources of data, and combine them into a data model.
- There are different forms visualizations including Bar and column charts, Line charts, Matrix, Key influencers, Treemap, Scatter and Filled map

Explore data analytics

👉Data analytics = Data analytics is concerned with examining, transforming, and arranging data in order to study and extract useful information.

👉There are different activities in data analytics.

Descriptive analytics = What has happened based on historical data.
Diagnostic analytics = Why things have happened.
Predictive analytics = What will happen in the future.
Prescriptive analytics = What actions should be taken to archive a goal or target.
Cognitive analytics = How to draw inferences from existing data and patterns.

Azure DP-900 Short Notes: Explore relational data services in Azure

Newanga Wickramasinghe — Wed, 17 Feb 2021 04:10:59 +0000

👉Learn Module: Explore relational data services in Azure

Introduction

👉Database(DB) is a collection of data.

👉Data can be structured, semi-structured or unstructured.

👉Two types of DB

Relational DB - Holds structured data.
Non-Relational DB - Holds semi-structured or unstructured data.

👉On-premise DB = High infrastructure cost | Solution = Migrate to cloud.

Explore relational Azure data services

👉Three common cloud offering models.

Infrastructure-as-a-Service(IaaS)

👉Creating Virtual infrastructure in the cloud just as on premise.

👉Same as on-premise except you do not have to buy and maintain the hardware.

Ex: Azure Virtual Network
Platform-as-a-service(PaaS)

👉 Azure creates virtual infrastructure in cloud

👉Customer responsible for installing and managing software.

👉Can easily scale up and down.

Ex: Azure SQL Databases
Software-as-a-Service(SaaS)

👉specific software packages that runs on the cloud

Ex: Office 365

👉Azure Data Services = PaaS based DBMS service manage by Microsoft.

Ex: Azure Database for MySQL servers , Azure Database for MariaDB servers, Azure Database for PostgreSQL

👉Advantages of Azure Data Services

Reduce DB administration workload.
SLA = 99.99%

👉Disadvantages of Azure Data Services

Custom DB scripts restricted (Ensure security)
Lack of full DBMS features (Ensure security)

👉 Base price = infrastructure charges+ licensing charges + administration charges.

👉 Designed to be always on.

👉 Capital Expenditure & Administrative Effort in running DB .

Physical(On-Premise DB) > IaaS(DB on Azure VM) > PaaS (Azure SQL database)

SQL Server on Azure virtual machines

👉IaaS offering to run SQL Server in cloud.

👉No on-premises hardware to manage.

👉lift-and-shift ready = Move a database directly from on-premise server to VM on the cloud and configure any app connecting to on-premise database server to connect to the VM hosting the database server in the cloud.

👉Advantages

Create hybrid environment where operation running are shared with both on-premise and the cloud.
lift-and-shift ready.
Create test environment without buying on-premise new hardware.
Scale up and down without reinstalling software.

Azure SQL Database

👉PaaS offering.

👉There are three differ options

Single Database
Elastic pool
Managed Database

1.Single Database

👉a single SQL Server database.

👉By default, resources are pre-allocated.

👉Charged per hour for the resources you've requested.

👉Optional server-less configuration.

2.Elastic pool

👉Multiple databases can share the same resources.

👉Pool = resources

👉Your DBs only access the pool.

👉Suitable for databases with resource requirements that vary over time.

👉Data Migration Assistant (DMA) helps to check on-premise DB environment compatibility with Azure SQL Database and reports issues and and recommendations.

Advantages

👉scalability

👉Automatic updates and patches.

👉High Availability (99.99% SLA)

👉Advanced threat protection

👉Auditing

👉Encryption

**Important —>linked servers used to perform distributed queries are not supported by single elastic pool options.

3.Managed instances

👉Same as Single Database or Elastic Pool with administrative features available to SQL Server.

👉A fully controllable instance of SQL Server in the cloud.

👉All advantages of option 1 and 2 and

Full control over security and resource allocation for your databases
Do not have to manage VMS(On-premise option).
100% compatibility with SQL Server Enterprise Edition, running on-premises.

👉Depend on other azure resources such as Azure Storage for backups to provide security and supportability features.

**Additional features (Not available in other 2 options) - > linked servers, Service Broker, DB Mail

Azure DP-900 Short Notes: Explore concepts of non-relational data

Newanga Wickramasinghe — Tue, 16 Feb 2021 04:09:39 +0000

👉Learn Module: Explore concepts of non-relational data

characteristics of non-relational data

👉Non relational data has no proper structure in them.

👉No scheme on Data.

👉Store the information for entities in collections or containers and not in relational tables.

👉Two entities in the same collection can have a different set of fields.

👉Data retrieval capabilities of a non-relational database vary based on stored data.

👉Advanced non-relational systems such as Azure Cosmos DB support indexing.

👉Use Cases include IoT and telematics, Retail and marketing, Gaming and Web and mobile applications.

Describe types of non-relational data

👉Two types of non-relational data

Semi-structured Data
- Data contains fields and two entities of same type can have different fields.
- There are different formats of semi-structured data.
  1. JSON =Document enclosed in curly brackets.
  2. Avro =Row-based format.
  3. ORC = organizes data into columns rather than rows.
  4. Parquet = columnar data format.
Non-structured Data
- Do not have fields.
- Ex: Audio and Video

Describe types of non-relational and NoSQL databases

👉Non-relational database also known as No-SQL database.

👉There are four types of non-relational database.

1.key-value store

Each data item in a key-value store has two elements, a key and a value.
Key uniquely identifies the item and valued hold the data.
value is opaque(DBMS just sees the value as an unstructured block).
Ex: Azure Table storage

2.Document database

Each document has a unique and fields in the documents are transparent to the DBMS.
Store data in JSON format or other formats such as XML, YAML, and BSON
Ex: Azure Cosmos DB

3.Column family database

Organizes data into rows and columns. Ex: ORC and Parquet files
Conceptually similar to relational database but different in implementation.
Ex: Apache Cassandra in Azure Cosmos DB supported through the Cassandra API

4.Graph database

Store entities with a main focus on the relationship between these entities.
stores two types of information. Nodes = Instances of entities and Edges= Relationships between nodes.
Nodes and edges can both have properties that provide information about that node or edge.

Azure DP-900 Short Notes: Describe concepts of relational data

Newanga Wickramasinghe — Mon, 15 Feb 2021 03:28:24 +0000

👉Learn Module: Explore roles and responsibilities in the world of data

Understand the characteristics of relational data

👉Relational data model collections of entities from the real world as tables.

👉A table consist of row and columns where each row represent a single entity.

👉A single table consist of a Primary key.

👉A foreign key in a table helps to build relationships with other tables.

👉Most relational databases support Structured Query Language (SQL).

👉Commonly used in ecommerce systems.

👉Also used in Online Transaction Processing (OLTP) such as Banking applications and Online retail applications.

Explore relational data structures

👉Relational database contains some other structures that help to optimize data organization.

Index
- Helps to search data in a table.
- Indexes consume additional storage space.
- Some relational database management systems also support clustered indexes. A clustered index physically reorganizes a table by the index key.
- A table can have multiple indexes, but only a single clustered index.
View
- A virtual table based on the result set of a query.

Choose the right platform for a relational workload

👉On-premise database hosting is expensive.

👉All tasks including including scaling, patching, updating and maintaining security should be handle by the organization which is a trivial task.

👉Cloud consists of two different models to host databases.

IaaS
- Database is hosted on a virtual machine.
- OS level control is provided over the system.
- All tasks such are patching and updating software should be handle by the organization.
PaaS
- The cloud provider handles hardware maintenance and software maintenance.
- Can easily scale the database based on demand.
- Ex: Azure SQL Database, Azure Database for PostgreSQL, Azure Database for MySQL, and Azure Database for MariaDB
- Some database features are missing from the respective database type due to security concerns.

Azure DP-900 Short Notes: Explore roles and responsibilities in the world of data

Newanga Wickramasinghe — Sun, 14 Feb 2021 12:18:59 +0000

👉Learn Module: Explore roles and responsibilities in the world of data

👉In real world, there are different roles in managing, controlling, and using data.

👉The three most common roles are

Azure Database Administrator role = Responsible for the design, implementation, maintenance, and operational aspects of on-premises and cloud-based database.
Data Engineer role = Collaborates with stakeholders to design and implement data-related assets that include data ingestion pipelines, cleansing and transformation activities, and data stores for analytical workloads.
Data Analyst role =Enables businesses to maximize the value of their data assets.

👉Azure Database Administrator role

Tasks

Backing up and restoring databases.
Enrolling users and maintaining system security.
Installing and upgrading the database server and application tools.

Tools

Azure Data Studio
SQL Server Management Studio
Azure portal

👉Data Engineer role

Tasks

Data acquisition.
Using programming languages and tools to examine the data.
Aligning the data architecture with business requirements.

Tools

Relational database management system(Ex: SSMS)
Azure Databricks
Azure HDInsight
Azure Cosmos DB

👉Data Analyst role

Tasks

Making large or complex data more accessible, understandable, and usable.
Creating charts and graphs, histograms, geographical maps.
Combining the data result sets across multiple sources.
Finding hidden patterns using data.

Tools

MS-Excel
Power BI

Azure DP-900 Short Notes: Explore Core Data Concepts

Newanga Wickramasinghe — Sat, 13 Feb 2021 03:17:01 +0000

👉 Learn Module: Explore Core Data Concepts

Identify the need for data solutions

👉Data is a collection of facts such as numbers, descriptions, and observations used in making of decision.

👉Three types of data

Structured data
- Tabular data that is represented by rows and columns in a database.
- Tables in this form are called relational databases.
Semi-structured data
- Information that doesn't reside in a relational database but still has some structure to it.
- Ex: JSON, key-value stores and graph databases
Unstructured data
- Data with no proper structure.
- Ex: Audio, Video , Binary dat files

👉Based on the type of data, there are multiple ways to store and access data in Azure cloud.

👉Stored data needs to be processed. There are two types of data processing solutions.

Transaction processing systems
- primary function of business computing.
- work performed by transactional systems is often referred to as Online Transactional Processing (OLTP).
- Data is divided into small pieces for faster processing.
- For example in a database tables are split out into separate groups of columns and this is called normalization.
Analytical systems
- Support business users who need to query data and gain a big picture view.
- Capturing raw data and generate insights to make future business decisions.
- Common tasks of a analysis system
  1. Data Ingestion - Capturing the raw data.
  2. Data Processing - Converting captured data into a common format to be processed.
  3. Data Querying - Querying data to analyze.
  4. Data Visualization - Generating charts such as bar charts, line charts out of queried data in order.

Identify types of data and data storage

👉Relational Data and Non-relational Data have different characteristics.

Relational Data
- Most well-understood model for holding data.
- Data normalization helps to reduce any data redundancy within the database.
Non-relational Data
- Store data in a format that more closely matches the original structure.
- Data duplication present which increases the storage required.
- Due to data duplication, any data modification may cause to update data present at multiple locations.

👉Two different types of workloads.

Transactional workloads
- Transaction is a sequence of operations that are atomic.
- Mostly commonly use relational databases.
- A transactional database must adhere to the ACID.
  1. Atomicity = A transaction is treated as a single unit, which either succeeds completely, or fails.
  2. Consistency = A transaction can only take the data in the database from one valid state to another.
  3. Isolation = Concurrent execution of transactions leaves the database in the same state that would have been obtained if the transactions were executed sequentially.
  4. Durability = Once a transaction has been committed, it will remain committed even if there's a system failure.
Analytical workloads
- Read-only systems that store vast volumes of historical data or business metrics.
- Used for data analysis and decision making.

Describe the difference between batch and streaming data

👉Data processing is converting data into meaningful information.

👉There are two types of data processing.

Batch Processing
- New data elements are collected into a group and the whole group is then processed at a future time as a batch.
- Data Scope = Process all the data in the dataset.
- Data Size = large datasets.
- Performance = latency is a few hours.
- Analysis = performing complex analytics.
Streaming and real-time data
- In stream processing, each new piece of data is processed when it arrives.
- Beneficial for dynamic data.
- Ideal for time-critical operations that require an instant real-time response.
- Data Scope = Access to the most recent data received.
- Data Size = Individual records or micro batches.
- Performance = latency in the order of seconds or milliseconds.
- Analysis = simple response functions.

Registering for an online proctored Microsoft Certification Exam🎓

Newanga Wickramasinghe — Fri, 29 May 2020 14:12:55 +0000

This will be a guide on registering and taking an online proctored Microsoft Certification Exam. (This guide will specifically follow the registration process for the AZ-900 Microsoft Azure certification exam)

1. Create a personal Microsoft account if you do not already have one. ( All Microsoft certificates earned are linked to this account. Do not loose access to this account)

2. Visit the Microsoft Certifications Exam Page and select the exam you wish to book from all certification exams.

3. Next you will be forwarded to the respective exam details page.

This page includes all the important details related to your exam. Make sure that you are familiar with these details.
You may sometimes see about future exam revision dates related to your exam. Pay special attention to these dates.

Before continuing make sure you have disabled all adblockers, allowed popups and third party cookies and adobe flash from your browser. The Microsoft Dashboard webpages and Pearson exam schedule Webpages will not load unless you do so and still if you face difficulties, clear your browser cache and cookies.

4. Select schedule with Pearson Vue and you will forward to your Microsoft Certification dashboard Profile Page.

Complete the details of this page.
Make sure to use your full name as in one of your Identification cards. On the exam day, Pearson requests you to upload a picture of one of your Identification cards. If the names of your profile and identification card are non-identical, you are not allowed to take the exam. You can read more details here.
I personally had to contact Microsoft support to change my name due to a mismatch before the exam.
Please note that your middle name will not appear in your badge or your Microsoft Certificate. If you wish to include your middle name, the trick is to include you middle name with a space after your first name in the First name field of the form.
The other important field is your telephone number. Pearson will use this number to contact you on your exam date in case of any technical problems.

*Optional exam discount for students*

If you are a student, you may be eligible for a student discount. Validate your eligibility through following the 'Get verified now' link.

Verified Academic status will show a Green tick as above.

5. After you have completed the details, Accept the terms and conditions and select save and continue.

6. Next, Select Go to Pearson vue.

7. Select option "At my home or office" and select Next.

*You can run a system check to check compatibility of your device now or you can do this later. More details can be found here.*

8. In the following page, Read and accept all terms and conditions and click Next.

9. Select exam language and select Next.

If you wish to do the exam in another language displayed above other than English, Check with the exam details pages whether the exam is available in that language.

10. The following Page will give details about the exam you have selected and price of the exam. Select Next.

If you are unfamiliar with the the exam testing policies and requirements, read them through the links displayed in the above page.

11. Choose your preferred language and select Next.

12. In the following page choose the date and time you wish to take the exam.

Once you select a date, you are able to see the time zone. Check whether you are in the right time zone or change it.
Check whether you have selected the right time since exam time slots are available for both daytime and nighttime.

13. Once again re-check details of your exam date and time and select Proceed to checkout.

14. In the following page enter your payments details.

*If you have a voucher for your exam, you can use it here to make the payment.*

15. In the Next following page, you will be able to re-check all details one last time before placing your order. If any detail is incorrect, Cancel the order and make the necessary changes and retry.

After successfully registering for an exam, you will receive an email from Pearson vue about your exam registration and the steps to follow on the exam date. Microsoft Learn Dashboard Appointments section will update respectively as below with your booked exam details.

If you have any questions, Let me know in the comments section :)