DEV Community

Cover image for Fetch selective secrets with no access to key vault using Devops pipelines
Arindam Mitra
Arindam Mitra

Posted on • Edited on

Fetch selective secrets with no access to key vault using Devops pipelines

Greetings my fellow Technology Advocates and Specialists!!!

This Blog post is a follow-up to my previous post - Fetch Secrets with no access to key vault

MY SUGGESTION:-
Please Read my previous blog first to better understand the Use Case
USE CASES:-
Fetch Secrets only with Tags (any)
Fetch Secrets only with Specific defined Tags
LIVE RECORDED SESSION:-
LIVE DEMO was Recorded as part of my Presentation in WELSH AZURE GROUP Forum/Platform
Duration of My Demo = 32 Mins 38 Secs
Start and End Time = 00:39:22 to 01:12:00

REQUIREMENTS:-
  1. Azure Key Vault
  2. Four Sample Secrets in Azure Key Vault
  3. Add Tags to One or More Secrets
  4. Azure Resource Manager Service Connection
  5. Azure DevOps Pipeline (YAML)
NOTE:-

The Service Principal (which is required to Create Service Connection) should at minimum have GET and LIST Access Policy Permissions in Azure Key Vault.

BELOW DISPLAYS THE SAMPLE SECRETS IN KEY VAULT:-
Image description
WHERE DO WE APPLY TAGS IN KEY VAULT SECRET:-
Image description
Image description
BELOW DISPLAYS THE SECRET TAGS IN KEY VAULT:-
NAME OF THE SECRET TAGS ASSOCIATED WITH SECRET
AMSecret005 Image description
AMSecret006 Image description
AMSecret007 No Tags Configured
AMSecret008 Image description
# PIPELINE TASKS
1. AZURE KEY VAULT TASKS
2. FETCH ALL SECRETS WITH TAGS OR ALL SECRETS WITH SPECIFIC TAG. EXPORT IT THEN IN A TEXT FILE
3. COPY THE SECRETS TEXT FILE TO ARTIFACTS STAGING DIRECTORY
4. PUBLISH THE ARTIFACTS
CODE REPOSITORY:-

GitHub logo arindam0310018 / 21-Apr-2022-DevOps__Fetch-Selective-Secrets-With-No-Access-To-KeyVault

FETCH SELECTIVE SECRETS WITH NO ACCESS TO KEY VAULT USING DEVOPS PIPELINES

FETCH SELECTIVE SECRETS WITH NO ACCESS TO KEY VAULT USING DEVOPS PIPELINES

Greetings my fellow Technology Advocates and Specialists!!!

This Blog post is a follow-up to my previous post - [Fetch Secrets with no access to key vault] (https://dev.to/arindam0310018/fetch-secrets-with-no-access-to-key-vault-using-devops-pipelines-54h3)

MY SUGGESTION:-
Please Read my previous blog first to better understand the Use Case
USE CASES:-
Fetch Secrets only with Tags (any)
Fetch Secrets only with Specific defined Tags
LIVE RECORDED SESSION:-
LIVE DEMO was Recorded as part of my Presentation in WELSH AZURE GROUP Forum/Platform
Duration of My Demo = 32 Mins 38 Secs
Start and End Time = 00:39:22 to 01:12:00
IMAGE ALT TEXT HERE
REQUIREMENTS:-
  1. Azure Key Vault
  2. Four Sample Secrets in Azure Key Vault
  3. Add Tags to One or More Secrets
  4. Azure Resource Manager Service Connection
  5. Azure DevOps Pipeline (YAML)
NOTE:-

The Service Principal (which is required to Create Service Connection) should at minimum have GET and LIST Access…

Below follows the contents of the YAML File (Azure DevOps):-
trigger:
  none

######################
#DECLARE VARIABLES:-
######################
variables:
  SubscriptionID: 210e66cb-55cf-424e-8daa-6cad804ab604
  ServiceConnection: amcloud-cicd-service-connection
  KVName: ampockv
  Artifact: AM
  KVTag: "RequiredBy: CloudProvider" 
  KVNoTag: "{}"

#########################
# Declare Build Agents:-
#########################
pool:
  vmImage: windows-latest

###################
# Declare Stages:-
###################
stages:

- stage: USECASE_DISPLAY_ALL_SECRETS_AND_VALUES
  jobs:
  - job: DISPLAY_SECRETS_AND_VALUES
    displayName: DISPLAY SECRETS AND VALUES
    steps:

########################################################################
# Azure Key Vault Task.
# Display the name of Key Vault.
# Display the No. of Secrets found in Key Vault.
# Display the No. of enabled and unexpired Secrets found in Key Vault.
# Downloads values of Each Secret in Key Vault.
########################################################################

    - task: AzureKeyVault@2
      displayName: AZ KEYVAULT TASK
      inputs:
        azureSubscription: '$(ServiceConnection)'
        KeyVaultName: '$(KVName)'
        SecretsFilter: '*'
        RunAsPreJob: false

#######################################################
# Integers can be compared with these operators:
#   -eq # Equal
#   -ne # Not equal
#   -lt # Less than
#   -le # Less than or equal
#   -gt # Greater than
#   -ge # Greater than or equal
#######################################################

###############################################################
# Export the Secret with Tags only to Secrets Text File
# Copy the Secrets Text file to Artifacts Staging Directory.
###############################################################

    # - task: AzureCLI@2
    #   displayName: FETCH ALL SECRETS WITH TAGS
    #   inputs:
    #     azureSubscription: '$(ServiceConnection)'
    #     scriptType: ps
    #     scriptLocation: inlineScript
    #     inlineScript: |
    #       az --version
    #       az account set --subscription $(SubscriptionID)
    #       az account show
    #       $count = az keyvault secret list --vault-name $(KVName) --query "[] | length(@)"
    #       For ($i=0; $i -lt $count; $i++) {
    #       $sectag = az keyvault secret list --vault-name $(KVName) --query [$i].tags -o yaml
    #       if($sectag -eq "$(KVNoTag)") {
    #       $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
    #       echo "THERE IS NO TAGS FOR THE $secretname"
    #         }
    #       else {
    #       $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
    #       $secretvalue = az keyvault secret show --vault-name $(KVName) --name $secretname --query value -o tsv
    #       echo ($secretname + ':' + $secretvalue) >> Secrets.txt
    #       echo "VALUE of $secretname IS EXPORTED TO SECRETS TEXT FILE"
    #       }
    #       }

#################################################################################################
# Export the Secret with Specific Tag "RequiredBy: CloudProvider" only to Secrets Text File
# Copy the Secrets Text file to Artifacts Staging Directory.
#################################################################################################

    - task: AzureCLI@2
      displayName: FETCH ALL SECRETS WITH SPECIFIC TAGS
      inputs:
        azureSubscription: '$(ServiceConnection)'
        scriptType: ps
        scriptLocation: inlineScript
        inlineScript: |
          az --version
          az account set --subscription $(SubscriptionID)
          az account show
          $count = az keyvault secret list --vault-name $(KVName) --query "[] | length(@)"
          For ($i=0; $i -lt $count; $i++) {
          $sectag = az keyvault secret list --vault-name $(KVName) --query [$i].tags -o yaml
          if($sectag -eq "$(KVNoTag)") {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          echo "THERE IS NO TAGS FOR THE $secretname"
            }
          elseif ($sectag -eq "$(KVTag)") {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          $secretvalue = az keyvault secret show --vault-name $(KVName) --name $secretname --query value -o tsv
          echo ($secretname + ':' + $secretvalue) >> Secrets.txt
          echo "VALUE of $secretname IS EXPORTED TO SECRETS TEXT FILE"
            }
          else {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          echo "There are other TAG(S) configured apart from Tag "$(KVTag)" for $secretname"
          }
          }  

###############################################################
# Copy the Secrets text file to Artifacts Staging Directory:-
###############################################################

    - task: CopyFiles@2
      displayName: COPY TO ARTIFACTS STAGING DIRECTORY
      inputs:
        Contents: Secrets.txt
        targetFolder: '$(Build.ArtifactStagingDirectory)'

###########################
# Publish the Artifacts:-
###########################

    - task: PublishBuildArtifacts@1
      displayName: PUBLISH ARTIFACTS
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
        ArtifactName: '$(Artifact)'
        publishLocation: 'Container'
Enter fullscreen mode Exit fullscreen mode
Values of the VARIABLES incase if you wish to change:-
######################
#DECLARE VARIABLES:-
######################
variables:
  SubscriptionID: 210e66cb-55cf-424e-8daa-6cad804ab604
  ServiceConnection: amcloud-cicd-service-connection
  KVName: ampockv
  Artifact: AM
  KVTag: "RequiredBy: CloudProvider" 
  KVNoTag: "{}"

Enter fullscreen mode Exit fullscreen mode
Purpose of the VARIABLE "KVNoTag" :-
If the Key Vault Secret Version does not have any Tag configured, it still returns value = { }. Hence we defined it as a variable and during runtime, we compare the output to the value of the variable. If the Value matches, we then know that the Secret Version does not have any Tag Configured.
Image description
Image description
Change the Agent Pool from MICROSOFT HOSTED BUILD AGENT to SELF HOSTED BUILD AGENT otherwise, below error is encountered:-
Image description
Below follows the code snippet for Use Case #1 (Fetch Secrets only with Tags [any]):-
###############################################################
# Export the Secret with Tags only to Secrets Text File
# Copy the Secrets Text file to Artifacts Staging Directory.
###############################################################

    # - task: AzureCLI@2
    #   displayName: FETCH ALL SECRETS WITH TAGS
    #   inputs:
    #     azureSubscription: '$(ServiceConnection)'
    #     scriptType: ps
    #     scriptLocation: inlineScript
    #     inlineScript: |
    #       az --version
    #       az account set --subscription $(SubscriptionID)
    #       az account show
    #       $count = az keyvault secret list --vault-name $(KVName) --query "[] | length(@)"
    #       For ($i=0; $i -lt $count; $i++) {
    #       $sectag = az keyvault secret list --vault-name $(KVName) --query [$i].tags -o yaml
    #       if($sectag -eq "$(KVNoTag)") {
    #       $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
    #       echo "THERE IS NO TAGS FOR THE $secretname"
    #         }
    #       else {
    #       $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
    #       $secretvalue = az keyvault secret show --vault-name $(KVName) --name $secretname --query value -o tsv
    #       echo ($secretname + ':' + $secretvalue) >> Secrets.txt
    #       echo "VALUE of $secretname IS EXPORTED TO SECRETS TEXT FILE"
    #       }
    #       }

Enter fullscreen mode Exit fullscreen mode
Below follows the code snippet for Use Case #2 (Fetch Secrets only with Specific defined Tags [RequiredBy: CloudProvider]):-
#################################################################################################
# Export the Secret with Specific Tag "RequiredBy: CloudProvider" only to Secrets Text File
# Copy the Secrets Text file to Artifacts Staging Directory.
#################################################################################################

    - task: AzureCLI@2
      displayName: FETCH ALL SECRETS WITH SPECIFIC TAGS
      inputs:
        azureSubscription: '$(ServiceConnection)'
        scriptType: ps
        scriptLocation: inlineScript
        inlineScript: |
          az --version
          az account set --subscription $(SubscriptionID)
          az account show
          $count = az keyvault secret list --vault-name $(KVName) --query "[] | length(@)"
          For ($i=0; $i -lt $count; $i++) {
          $sectag = az keyvault secret list --vault-name $(KVName) --query [$i].tags -o yaml
          if($sectag -eq "$(KVNoTag)") {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          echo "THERE IS NO TAGS FOR THE $secretname"
            }
          elseif ($sectag -eq "$(KVTag)") {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          $secretvalue = az keyvault secret show --vault-name $(KVName) --name $secretname --query value -o tsv
          echo ($secretname + ':' + $secretvalue) >> Secrets.txt
          echo "VALUE of $secretname IS EXPORTED TO SECRETS TEXT FILE"
            }
          else {
          $secretname = az keyvault secret list --vault-name $(KVName) --query [$i].name -o tsv
          echo "There are other TAG(S) configured apart from Tag "$(KVTag)" for $secretname"
          }
          } 
Enter fullscreen mode Exit fullscreen mode
IMPORTANT NOTE:-
You can comment and uncomment the Task based on your requirement (Use Case #1 or Use Case #2). For this Blog, I am focusing on Exporting Secret with Specific Defined Tags [RequiredBy: CloudProvider] which is Use Case #2
PIPELINE RESULTS:-
Image description
SECRETS TEXT FILE PUBLISHED IN ARTIFACTS:-
Image description
Image description
DOWNLOAD AND VIEW THE SECRETS TEXT FILE:-
Image description
The Output Format in Secrets Text file is [NAME OF THE SECRET]:[VALUE OF THE SECRET]
As the Secrets AMSecret005 and AMSecret008 has similar tags [RequiredBy: CloudProvider], only those secrets with values got exported to Secrets.txt file. Rest of the Secrets AMSecret005 and AMSecret008 got ignored

Top comments (0)