DEV Community: Avelyn Hyunjeong Choi

How to create a rule to filter out emails received by BCC

Avelyn Hyunjeong Choi — Wed, 13 Mar 2024 00:21:38 +0000

1.Open an BCC email
2.Select File > Properties

3.Look for Internet headers and find and save BCC identifier which looks something like this: X-MS-Exchange-Organization-Recipient-P2-Type: Bcc

4.Access the outlook on the web (OWA)
5.Select Settings > Mail > Rules

6.Create a rule and choose “Message header includes”
7.Copy in the value you found on step 3

8.Save rule

Logic behind this

With Outlook rules, some are “client side” and some are “server side”. When you create them in OWA, they are automatically server side and will trigger regardless of Outlook client being open or not.
If the rule is “client side”, then Outlook must be open for them to trigger. The specific predicate of the subject is available both client and server side.
Creating the rule in OWA ensures it is completely server side, and that’s why it works when the rule is added to OWA, not to Outlook Desktop.

Add Parameters for Workflow Input in Azure Logic Apps

Avelyn Hyunjeong Choi — Wed, 13 Mar 2024 00:17:32 +0000

1.Open Azure logic app
2.Select Workflows > choose workflow you want to edit

3.Select Developer > Designer on the left side menu

4.Select the action box
5.Select the desired parameter and add value for the selected parameter

System assigned vs User assigned identity in Azure

Avelyn Hyunjeong Choi — Fri, 16 Feb 2024 01:40:51 +0000

User assigned identity

let the function app to read/manage azure resources
can be assigned to multiple resources and we have more flexibility when using user-assigned identity

System assigned identity

let the function app to connect to sql db using its own identity instead of username + password, which is less secure
solely used by one resource
with this, SQL_CONNECTION_STRING in configuration doesn't require to contain username or password. Instead, it sets Authentication to Active Directory Managed Identity
to be able to use this, you should

1.enable function app system assigned identity by clicking Identity> Status On

2.add user assigned identity

***NOTE: make sure to change IdentityId in configuration to the client (applicaton) Id of the user assigned identity

3.update SQL_CONNECTION_STRING in configuration to use Active Directory Managed Identity

4.In sql db, create a function app as sql user and make it as db_owner

CREATE USER [function app name] 
FROM EXTERNAL PROVIDER 
WITH DEFAULT_SCHEMA = dbo;  

ALTER ROLE db_owner ADD MEMBER [function app name];

5.you can test your function code by clicking your function app > Code+Test > Test/Run > Run

6.you can see the result by clicking Monitor

How to add hyperlink to a table in Power BI

Avelyn Hyunjeong Choi — Thu, 15 Feb 2024 01:59:48 +0000

1.Go to Table view > Select the column you want to change > Change Data category to Web URL

2.Select the table > Visualization > Visual > URL icon > click On for Values

3.Now you have the hyperlink icon only on your table

Microsoft Copilot Studio

Avelyn Hyunjeong Choi — Fri, 09 Feb 2024 16:01:39 +0000

Building conversational AI is evolving....

Custom Solutions - Domains of specialized vendors and data scientists
Pro Code - Built in-house by professional developers
Low Code + Pro Code - Low code users can build bots with no-code, no model-building experiences
Generative AI + Authored Content - AI helps you build and respond from existing knowledge and services

Create copilot with E2E SaaS and single pane of glass

Chat over knowledge with Generative AI
Create specific topics
Actions and Plugins: create actions, plugins, or use pre-build Power Platform and OpenAI connectors to call your backends and APIs
Publish the copilot to multiple channels, live instantly on the SaaS service
Monitor and improve with rich-out-of-the box telemetry and analytics
Conversational services: integrate with Azure AI Studio, Azure Cognitive Services, Bot Framework and more -> Circle back to step 1

Azure Function Apps

Avelyn Hyunjeong Choi — Thu, 08 Feb 2024 16:32:01 +0000

What is Azure Functions?

Event-driven, serverless compute platform that allows developers to run event-triggered code without having to explicitly provision or manage infrastructure
Helps developers focus on writing the codes without having to take care of scaling and managing the underlying infrastructure
Stateless, meaning they do not maintain state between executions, which makes them well-suited for handling individual tasks or responding to specific events

Key Features

Pay-per-use pricing model (very cheap)
Bring your own dependencies (NPM or NuGet)
Integrated security
Simplified integration
Flexible development
Open-source

How Function Apps work
Trigger(action) -> (Input) -> Code -> Output

Triggers and bindings

Blob storage, Cosmos DB, Event Grid, Event Hubs, HTTP & webhooks, MS Graph Excel tables, Mobile Apps, Table storage., etc.

Host Configuration

host.json metadata files contains global configuration options that affect all functions for the function app.

How to add a new data to the existing Power BI Report

Avelyn Hyunjeong Choi — Sat, 03 Feb 2024 00:48:39 +0000

1.Open the existing Power BI report
2.Click Model view > Transform data > Transform data

3.Click New Source > SQL Server

4.Enter name of your server and database

Select Import under Data Connectivity mode

5.In the Navigator popup window, select the table you want to add > click Ok

6.Click Close & Apply

7.You will see the new table appeared under Data

Azure Databricks - Unity Catalog

Avelyn Hyunjeong Choi — Tue, 30 Jan 2024 17:22:07 +0000

What is Unity Catalog?

Centralized security and governance service for your Lakehouse
Unity Catalog allows admins and data stewards manage users and access to data centrally across all of the workspace in an Azure Databricks account
Users in different workspace can share access to the same data
Offers a single place to administer data access policies that apply across all workspaces
Automatically captures user-level audit logs recording access to the data

Unity Catalog Object Model

Metastore: the top-level container for metadata by exposing a three-level namespace (catalog.schema.table)
Catalog: first layer of object hierarchy to organize data assets
Schema: database. Second layer of object to contain tables and views
Table: the lowest level in the object hierarchy. Includes: storage credentials and external table, managed table or views

https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/
Unity Catalog API 2.1 Specification
https://api-docs.databricks.com/rest/latest/unity-catalog-api-specification-2-1.html

Databricks API

Avelyn Hyunjeong Choi — Tue, 30 Jan 2024 17:04:39 +0000

NOTE
In the example, whenever the user creates a new entity, it creates a new PAT. In order to create/list/delete the PAT, it needs two following tokens for the service principal: Azure Active Directory(AD) token and Azure Active Directory management endpoint token.

Requirement - The service principal requires the Contributor or Owner role on the Databricks workspace in Azure.

Step 1 - Get Azure AD token for the service principal

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/aad/service-prin-aad-token#--get-an-azure-ad-access-token-with-the-azure-cli

**NOTE - You can get tenant_id, client_id, and client_secrect from your service principal as below.

POST https://login.microsoftonline.com//oauth2/v2.0/token

Step 2 - Get Azure AD management endpoint token

POST https://login.microsoftonline.com//oauth2/token

Now you have two Azure AD tokens (one with Databricks service principal from step1 and another with AD management endpoint from step2)

Step 3 - Create Personal Access Token based on two AD tokens from previous steps

POST /api/2.0/token/create \
-H 'Authorization: Bearer ' \
-H 'X-Databricks-Azure-SP-Management-Token: ' \
-H 'X-Databricks-Azure-Workspace-Resource-Id: /subscriptions//resourceGroups//providers/Microsoft.Databricks/workspaces/'

You can now retrieve token_value as well as token_id from above API call. These values will be used to create an external location, catalog and schema and delete PAT that has been already used.

Optionally, you can put request body as below.

https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/tokens

You can run any Databricks REST API call. Below link provides the REST API endpoint available with Azure Databricks
https://learn.microsoft.com/en-us/azure/databricks/dev-tools/api/latest/

Purview - Data estate insights

Avelyn Hyunjeong Choi — Tue, 30 Jan 2024 16:57:40 +0000

The Data Estate Insights application is purpose-built for governance stakeholders, primarily for roles focused on data management, compliance, and data use: like a Chief Data Officer. The application provides actionable insights into the organization's data estate, catalog usage, adoption, and processes.

The dashboards and reports available within Microsoft Purview Data Estate Insights are categorized in three sections:

Health
Inventory and Ownership
Curation and Governance

Health

Data, governance, and quality focused users like chief data officers and data stewards can start at the health dashboards to understand the current health status of their data estate, current return on investment on their catalog, and begin to address any outstanding issues.

Data stewardship

The data stewardship dashboard highlights key performing indicators that the governance stakeholders need to focus on, to attain a clean and governance-ready data estate. Information like asset curation rates, data ownership rates, and classification rates are calculated out of the box and trended over time.

Management-focused users, like a Chief Data Officer, can also get a high-level understanding of weekly and monthly active users of their catalog, and information about how the catalog is being used.

Inventory and ownership

This area focuses on summarizing data estate inventory for data quality and management focused users, like data stewards and data curators. These dashboards provide key metrics and overviews to give users the ability to find and resolve gaps in their assets, all from within the data estate insights application.

Assets

This report provides a summary of your data estate and its distribution by collection and source type. You can also view new assets, deleted assets, updated assets, and stale assets from the last 30 days.

Explore your data by classification, investigate why assets didn't get classified, and see how many assets exist without a data owner assigned. To take action, the report provides a “View Detail” button to view and edit the specific assets that need treatment.

You can also view data asset trends by asset count and data size, as we record this metadata during the data map scanning process.

Curation and governance

This area focuses on giving a summary of how curated your assets are by several curation contexts. Currently we focus on showcasing assets with glossary, classification, and sensitivity labels.

Purview - Data catalog

Avelyn Hyunjeong Choi — Tue, 30 Jan 2024 16:56:49 +0000

Data catalog in purview provides the information about your organizations data assets in a searchable format based on lenses, such as glossary terms, classifications, sensitivity labels and more.
Depending on which level of Data Catalog you choose, you can also access a business glossary, lineage visualization, catalog insights, and sensitive data identification insights.

AI Builder

Avelyn Hyunjeong Choi — Tue, 30 Jan 2024 16:53:17 +0000

What is AI Builder?

build, train, and publish AI models without a single line of code or the help of a data scientist
build AI models in minutes with guided "point and click" wizards
customizable for broad range of industries
part of the Power Platform and Dynamic 365, so you can leverage your AL models anywhere and build robust, dynamic automations

Wide Range of AI Scenarios

Documents: form processing, text processing, invoice processing, receipt processing, identify document reader, business card reader
Vision: object detection, classify images
Prediction
Language: category classification, entity extraction, key phrase extraction, language detection, sentiment analysis, translation

Custom vs. Prebuilt AI Models
Custom AI Models

requires training to be able to understand company specific information
before using them, makers need to train the model for a specific need by submitting historical data and teaching the model how to understand it
no-code wizard guides the maker during this process
example) understand a specific company template to process purchase orders

Pre-built AI Models

ready to use out of the box and don't require any training
can be invoked directly from Power Automate or Power apps to build intelligent apps and workflows
example) process an ID document