A Teacher's Learning Log for learning Azure DevOps and Terraform

Monday, 28 November 2022

Problem:

Today I could not log 'git push' to Azure private repo. It prompted me for username and password. I entered my normal username/password for Azure DevOps, but it gave the error:

fatal: Authentication failed

Solution:

1. Login to Azure DevOps
2. Go to the User Settings menu in the top-right

3. Click Personal Access Tokens
   

4. Click + New Token

5. Give it a name and an expiry (I set a custom expiry and chose a date 12 months away).

6. Copy the token string to a safe place you store your secrets.
   

7. Attempt to git push again

8. Login with your usual Azure DevOps username but use the new token string saved at Step 5 as the password.

---

Wednesday, 30 November 2022

Problem:

Using variable secrets (from a variable group linked to an Azure Key Vault) in a bash command in an Azure Pipeline (Azure DevOps).

Solution:

1. Go to Pipeline --> Library to setup your variable group.
   
   Image 1: I have two variable groups

2. Declare the variable group (which contains secrets) in the Azure Pipeline.

3. Use the secret variable in tasks (not just directly into 'bash').

A variable secret cannot be used as a global environmental variable. It can only be passed in as an environmental variable local to a task. It cannot be used in 'bash', but only a 'Bash@3' task.

Incorrect way

Be careful, if you try to output a secret local environment variable in a 'bash' using echo $(MYSECRET), it will show as ***. However, when you use it has input into a bash command, it will pass in an empty value.

Correct way

If you try to output a secret local environment variable using a 'Bash@3' task, it will show as a blank after giving the following error:

line xxx: MYSECRET: command not found

However, when you use it as input into a bash command (within the 'Bash@3' task), it will pass in the correct secret value.

Example of using a secret variable correctly

variables:
- group: Terraform-ARM-variables

steps:
- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      echo "MYSQL_SERVER_PASSWORD_SECRET=$(MYSQL_SERVER_PASSWORD_SECRET)"
      helm upgrade --install wp1 bitnami/wordpress --wait --namespace $NAMESPACE \
        --set *.ingress.enabled=true \
        --set *.ingress.hostname=xxx \
        --set mariadb.enabled=false \
        --set externalDatabase.host=xxx \
        --set externalDatabase.user=xxx \
        --set externalDatabase.password=$MYSQL_SERVER_PASSWORD_SECRET \
        --set externalDatabase.database=xxx
        --set externalDatabase.port='3306'
  displayName: Helm install wordpress to AKS cluster
  env:
    MYSQL_SERVER_PASSWORD_SECRET: $(MYSQLSERVERPASSWORD)

Example explained

My variable group Terraform-ARM-variables was declared in the variable section. After this, each secret variable in it can be accessed in the pipeline. One of my secret variables in my variable group is MYSQLSERVERPASSWORD.

To use MYSQLSERVERPASSWORD in my Bash@3 task, I must set it has a local environment variable, in this case MYSQL_SERVER_PASSWORD_SECRET.

When the MYSQL_SERVER_PASSWORD_SECRET environment variable is echoed, it gives an error and displays as blank. (See screenshot below)

However, when the MYSQL_SERVER_PASSWORD_SECRET environment variable is used as an input in the bash Helm command that follows, it passes the secret across successfully.

Turning your variable group's variables into secrets

There are two ways of making your variables 'secret' (and be masked in logs and encrypted rather than stored as just plain text).

Method 1

Link the variable group to an Azure Key Vault (the option to do this is highlighted in the image below). EVERY variable retrieved from the Key Vault is classified as 'secret'. There is no option to choose which ones to make a secret.

Method 2

Do not link the variable to an Azure Key Vault. You need to manually create each variable. In this case, you can choose which variables are classed as 'secrets' (see image below).

Additional resources
Add & use variable groups
Bash@3 - Bash v3 task
Define variables