Greetings to my fellow Technology Advocates and Specialists.
In this Session, I will demonstrate SFTP with Azure DevOps.
I had the Privilege to talk on this topic in ONE Azure Communities:-
NAME OF THE AZURE COMMUNITY
TYPE OF SPEAKER SESSION
Festive Tech Calendar 2022
Virtual
LIVE RECORDED SESSIONS:-
LIVE DEMO was Recorded as part of my Presentation in FESTIVE TECH CALENDAR 2022 Forum/Platform
Duration of My Demo = 1 Hour 05 Mins 08 Secs
THIS IS HOW IT LOOKS:-
AUTOMATION OBJECTIVE:-
Validate if provided Resource Group exists. If Not, Pipeline will FAIL.
Validate if Storage Account exists inside the specified Resource Group. If Not, Pipeline will FAIL.
Validate if Hierarchical Namespace is Enabled in the specified Storage Account. If Not, Pipeline will FAIL.
Validate if Key Vault exists inside the specified Resource Group. If Not, Pipeline will FAIL.
Validate if SFTP is enabled in the specified Storage Account. If No, it will enable SFTP and Proceed to Next Validation. If Yes, It will skip and and Proceed to Next Validation.
Validate if SFTP Local User Home Directory Container exists. If Yes, Pipeline will FAIL.
Validate If SFTP Local User Exists. If Yes, Pipeline will FAIL.
If all of the above validation is SUCCESSFUL, SFTP will be Enabled or Skipped in the Storage Account (Depending upon the Status at the time), Local SSH User will be created and Password will be Generated. Finally, Local SSH Username, Password and Connection String will be stored in Key Vault.
IMPORTANT NOTE:-
The YAML Pipeline is tested on WINDOWS BUILD AGENT Only!!!
REQUIREMENTS:-
Azure Subscription.
Azure DevOps Organisation and Project.
Service Principal with Required RBAC ( Contributor) applied on Subscription or Resource Group(s).
Azure Resource Manager Service Connection in Azure DevOps.
In this Stage, Pipeline will validate Resource Group, Storage Account (With Hierarchal Namespace), and Key Vault. If any one of the Azure Resource is Not Available, Pipeline will FAIL and the Next Stage will get SKIPPED.
- stage: VALIDATE_RG_STORAGE_ACCOUNT_HIERARCHICAL_NAMESPACE_AND_KV
jobs:
- job: VALIDATE_RG_STORAGE_ACCOUNT_HIERARCHICAL_NAMESPACE_AND_KV
displayName: VALIDATE RG STORAGE ACCOUNT HIERARCHICAL_NAMESPACE & KV
steps:
- task: AzureCLI@2
displayName: SET AZURE ACCOUNT
inputs:
azureSubscription: $(ServiceConnection)
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
az --version
az account set --subscription ${{ parameters.SUBSCRIPTIONID }}
az account show
- task: AzureCLI@2
displayName: VALIDATE RG STORAGE ACCOUNT HIERARCHICAL_NAMESPACE & RG
inputs:
azureSubscription: $(ServiceConnection)
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
$i = az group exists -n ${{ parameters.RGNAME }}
if ($i -eq "true") {
echo "#####################################################"
echo "Resource Group ${{ parameters.RGNAME }} exists!!!"
echo "#####################################################"
$j = az storage account check-name --name ${{ parameters.STORAGEACCOUNTNAME }} --query "reason" --out tsv
if ($j -eq "AlreadyExists") {
echo "###################################################################"
echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} exists!!!"
echo "###################################################################"
$k = az storage account show -n ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} --query [isHnsEnabled] --output tsv
if ($k -eq "true") {
echo "###################################################################"
echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} has Hierarchical Namespace Enabled!!!"
echo "###################################################################"
$l = az keyvault list --resource-group ${{ parameters.RGNAME }} --query [].name -o tsv
if ($l -eq "${{ parameters.KVNAME }}") {
echo "###################################################################"
echo "Key Vault ${{ parameters.KVNAME }} exists!!!"
echo "###################################################################"
}
else {
echo "###################################################################################################"
echo "Key Vault ${{ parameters.KVNAME }} DOES NOT EXISTS in Resource Group ${{ parameters.RGNAME }}!!!"
echo "###################################################################################################"
exit 1
}
}
else {
echo "#######################################################################################################################"
echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} DOES NOT have Hierarchical Namespace Enabled!!!!!!"
echo "#######################################################################################################################"
exit 1
}
}
else {
echo "#######################################################################################################################"
echo "Storage Account ${{ parameters.STORAGEACCOUNTNAME }} DOES NOT EXISTS in Resource Group ${{ parameters.RGNAME }}!!!"
echo "#######################################################################################################################"
exit 1
}
}
else {
echo "#############################################################"
echo "Resource Group ${{ parameters.RGNAME }} DOES NOT EXISTS!!!"
echo "#############################################################"
exit 1
}
STAGE #2 - SFTP_ENABLE:-
In this Stage, Pipeline has Conditions in Place.
Condition #1: The Previous Stage has to be Successful.
Condition #2: The User should Select option "Enable".
BELOW FOLLOWS THE LOGIC DEFINED TO ENABLE SFTP IN STORAGE ACCOUNT AND STORE CREDENTIALS IN THE MENTIONED KEYVAULT:-
Validate if SFTP is enabled in the specified Storage Account. If No, it will enable SFTP and Proceed to Next Validation. If Yes, It will skip and and Proceed to Next Validation.
Validate if SFTP Local User Home Directory Container exists. If Yes, Pipeline will FAIL.
Validate If SFTP Local User Exists. If Yes, Pipeline will FAIL.
If all of the above validation is SUCCESSFUL, SFTP will be Enabled or Skipped in the Storage Account (Depending upon the Status at the time), Local SSH User will be created and Password will be Generated. Finally, Local SSH Username, Password and Connection String will be stored in Key Vault.
jobs:
- job: SFTP_ENABLE
displayName: ENABLE SFTP & STORE CREDENTIALS IN KV
steps:
- task: AzureCLI@2
displayName: ENABLE SFTP & STORE CREDENTIALS IN KV
inputs:
azureSubscription: $(ServiceConnection)
scriptType: ps
scriptLocation: inlineScript
inlineScript: |
$i = az storage account show -n ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} --query [isSftpEnabled] --output tsv
if ($i -eq "false") {
az storage account update -n ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} --enable-sftp=true
echo "#####################################################"
echo "SFTP Enabled for Storage Account ${{ parameters.STORAGEACCOUNTNAME }} in the Resource Group ${{ parameters.RGNAME }}!!!"
echo "#####################################################"
echo "Validating if SFTP Local User Home Directory Exists!!!"
echo "#####################################################"
$j = az storage container exists --account-name ${{ parameters.STORAGEACCOUNTNAME }} --account-key $(az storage account keys list -g ${{ parameters.RGNAME }} -n ${{ parameters.STORAGEACCOUNTNAME }} --query [0].value -o tsv) --name ${{ parameters.SFTPUSER }}-dir --query "exists" --out tsv
if ($j -ne "true") {
az storage container create --name ${{ parameters.SFTPUSER }}-dir --account-key $(az storage account keys list -n ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} --query [0].value -o tsv) --account-name ${{ parameters.STORAGEACCOUNTNAME }}
echo "#####################################################"
echo "SFTP User Home Directory Container ${{ parameters.SFTPUSER }}-dir created successfully!!!"
echo "#####################################################"
echo "Validating if SFTP Local User Exists!!!"
echo "#####################################################"
$k = az storage account local-user show --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --query [name] --output tsv
if ($k -ne "${{ parameters.SFTPUSER }}") {
az storage account local-user create --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --home-directory ${{ parameters.SFTPUSER }}-dir --permission-scope permissions=$(Permissions) service=$(Service) resource-name=${{ parameters.SFTPUSER }}-dir --has-ssh-password $(SSHPasswd)
$sftppasswd = az storage account local-user regenerate-password --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --query [sshPassword] -o tsv
az keyvault secret set --name ${{ parameters.SFTPUSER }}-username --vault-name ${{ parameters.KVNAME }} --value ${{ parameters.SFTPUSER }}
az keyvault secret set --name ${{ parameters.SFTPUSER }}-passwd --vault-name ${{ parameters.KVNAME }} --value $sftppasswd
az keyvault secret set --name ${{ parameters.SFTPUSER }}-connection-string --vault-name ${{ parameters.KVNAME }} --value "${{ parameters.STORAGEACCOUNTNAME }}.${{ parameters.SFTPUSER }}@${{ parameters.STORAGEACCOUNTNAME }}.blob.core.windows.net"
echo "#####################################################"
echo "SFTP Local User ${{ parameters.SFTPUSER }} created successfully and Credentials Stored in ${{ parameters.KVNAME }}!!!"
echo "#####################################################"
}
else {
echo "#####################################################"
echo "SFTP Local User ${{ parameters.SFTPUSER }} already Exists!!!"
echo "#####################################################"
exit 1
}
}
else {
echo "#####################################################"
echo "SFTP User Home Directory Container ${{ parameters.SFTPUSER }}-dir already Exists!!!"
echo "#####################################################"
exit 1
}
}
else {
echo "#####################################################"
echo "SFTP is already Enabled for Storage Account ${{ parameters.STORAGEACCOUNTNAME }} in the Resource Group ${{ parameters.RGNAME }}!!!"
echo "#####################################################"
$l = az storage container exists --account-name ${{ parameters.STORAGEACCOUNTNAME }} --account-key $(az storage account keys list -g ${{ parameters.RGNAME }} -n ${{ parameters.STORAGEACCOUNTNAME }} --query [0].value -o tsv) --name ${{ parameters.SFTPUSER }}-dir --query "exists" --out tsv
if ($l -ne "true") {
az storage container create --name ${{ parameters.SFTPUSER }}-dir --account-key $(az storage account keys list -n ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} --query [0].value -o tsv) --account-name ${{ parameters.STORAGEACCOUNTNAME }}
echo "#####################################################"
echo "SFTP User Home Directory Container ${{ parameters.SFTPUSER }}-dir created successfully!!!"
echo "#####################################################"
echo "Validating if SFTP Local User Exists!!!"
echo "#####################################################"
$m = az storage account local-user show --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --query [name] --output tsv
if ($m -ne "${{ parameters.SFTPUSER }}") {
az storage account local-user create --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --home-directory ${{ parameters.SFTPUSER }}-dir --permission-scope permissions=$(Permissions) service=$(Service) resource-name=${{ parameters.SFTPUSER }}-dir --has-ssh-password $(SSHPasswd)
$sftppasswd = az storage account local-user regenerate-password --account-name ${{ parameters.STORAGEACCOUNTNAME }} -g ${{ parameters.RGNAME }} -n ${{ parameters.SFTPUSER }} --query [sshPassword] -o tsv
az keyvault secret set --name ${{ parameters.SFTPUSER }}-username --vault-name ${{ parameters.KVNAME }} --value ${{ parameters.SFTPUSER }}
az keyvault secret set --name ${{ parameters.SFTPUSER }}-passwd --vault-name ${{ parameters.KVNAME }} --value $sftppasswd
az keyvault secret set --name ${{ parameters.SFTPUSER }}-connection-string --vault-name ${{ parameters.KVNAME }} --value "${{ parameters.STORAGEACCOUNTNAME }}.${{ parameters.SFTPUSER }}@${{ parameters.STORAGEACCOUNTNAME }}.blob.core.windows.net"
echo "#####################################################"
echo "SFTP Local User ${{ parameters.SFTPUSER }} created successfully and Credentials Stored in ${{ parameters.KVNAME }}!!!"
echo "#####################################################"
}
else {
echo "#####################################################"
echo "SFTP Local User ${{ parameters.SFTPUSER }} already Exists!!!"
echo "#####################################################"
exit 1
}
}
else {
echo "#####################################################"
echo "SFTP User Home Directory Container ${{ parameters.SFTPUSER }}-dir already Exists!!!"
echo "#####################################################"
exit 1
}
}
NOW ITS TIME TO TEST !!!...
TEST CASES:-
TEST CASE #1: VALIDATE RESOURCE GROUP, STORAGE ACCOUNT (WITH HIERARCHICAL NAMESPACE) AND KEY VAULT:-
DESIRED OUTPUT: PIPELINE FAILS WHEN RESOURCE GROUP DOES NOT EXISTS.
DESIRED OUTPUT: PIPELINE FAILS WHEN STORAGE ACCOUNT DOES NOT EXISTS.
DESIRED OUTPUT: PIPELINE FAILS WHEN STORAGE ACCOUNT DOES NOT HAVE HIERARCHICAL NAMESPACE ENABLED.
DESIRED OUTPUT: PIPELINE FAILS WHEN KEY VAULT DOES NOT EXISTS.
TEST CASE #2: SFTP NOT ENABLED, LOCAL SSH USER AND HOME DIRECTORY CONTAINER DOES NOT EXISTS:-
DESIRED OUTPUT: SFTP IS ENABLED. LOCAL USER IS CREATED WITH HOME DIRECTORY CONTAINER. PASSWORD GENERATED. ALL CREDENTIALS STORED IN KEY VAULT.
TEST CASE #3: SFTP ALREADY ENABLED. LOCAL SSH USER DOES NOT EXISTS BUT PREVIOUSLY CREATED LOCAL SSH USER`S HOME DIRECTORY CONTAINER ALREADY EXISTS:-
DESIRED OUTPUT: PIPELINE FAILS. LOCAL SSH USER CREATED PREVIOUSLY WAS DELETED BUT HOME DIRECTORY STILL EXISTS
TEST CASE #4: SFTP ENABLED, LOCAL SSH USER ALREADY EXISTS:-
DESIRED OUTPUT: PIPELINE FAILS SINCE WE ARE TRYING TO CREATE LOCAL SSH USER WITH SAME NAME WHICH ALREADY EXISTS
TEST CASE #5: SFTP ENABLED, CREATE NEW ADDITIONAL LOCAL SSH USER AND HOME DIRECTORY CONTAINER:-
DESIRED OUTPUT: ADDITIONAL NEW LOCAL USER IS CREATED WITH HOME DIRECTORY CONTAINER. PASSWORD GENERATED. ALL CREDENTIALS STORED IN KEY VAULT.
Hope You Enjoyed the Session!!!
Stay Safe | Keep Learning | Spread Knowledge
Top comments (0)
Subscribe
For further actions, you may consider blocking this person and/or reporting abuse
Top comments (0)