DEV Community

Cover image for Deploying a highly available Vault cluster on Amazon EKS using Terraform

Deploying a highly available Vault cluster on Amazon EKS using Terraform

Chabane R. on April 15, 2021

Many companies moving to the cloud want to continue working with legacy tools to: avoid vendor lock-in, use the existing skill and process, take ...
Read full post
Collapse
 
ultralite profile image
Ultralite •

Hello, nice terraform template.
I've got an issue with the job: vault-server/certificate-vault which is in failed state.

Error: job: vault-server/certificate-vault is in failed state
│
│   with kubernetes_job.vault-certificate,
│   on k8s.tf line 155, in resource "kubernetes_job" "vault-certificate":
│  155: resource "kubernetes_job" "vault-certificate" {
Enter fullscreen mode Exit fullscreen mode

I've provided all the requirements but i'm not sure about the type of certificate requested.
First I was using an vault.subdomain.domain.com without success.
After I've tried with a wildcard certfiticate *.subdomain.domain.com
Both certificate were issued without error.

Do you have an idea ?

Collapse
 
chabane profile image
Chabane R. •

you can delete the vault resources and run terraform apply again

Collapse
 
chabane profile image
Chabane R. •

kubectl delete secret vault-server-tls -n vault-server
kubectl delete CertificateSigningRequest vault-csr -n vault-server
terraform destroy -target=kubernetes_job.vault-certificate

Thread Thread
 
gitaccrosh profile image
gitaccrosh •

Deploying on AWS EKS 1.21 ,
I got stuck on CSR approval , as I got the approval but not getting singined . Suspecting missing clusterrole binding for the user boot-vault . Need some help

Thread Thread
 
chabane profile image
Chabane R. •

hello

thanks for your contribution

yes, I tested with the 1.17 version. If I remember I got the same issue with the 1.18 version. As I see with your comment, it's still not working with the new versions.

If you resolve the CSR issue, do not hesitate to share :-)

Thread Thread
 
lupunita profile image
Lupunita •

There are few steps to get over it.

  1. Add spec.signerName into the CSR manifest in certificate.sh e.g 
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: ${CSR_NAME}
spec:
  groups:
  - system:authenticated
  request: $(cat ${TMPDIR}/server.csr | base64 | tr -d '\n')
  signerName: example.certificates.k8s.io/vault-signer
Enter fullscreen mode Exit fullscreen mode
  1. Add new rule with signe permisssions to kubernetes_cluster_role" "boot_vault" resource : 
  rule {
    api_groups = ["certificates.k8s.io"]
    resources  = ["signers"]
    resource_names = ["example.certificates.k8s.io/vault-signer"]
    verbs      = ["approve"]
  }
Enter fullscreen mode Exit fullscreen mode

Hope I did not skip anothing. :-)

Collapse
 
tbadmus profile image
tbadmus •

Hello
I followed your steps in detailed in the README for the code and ran into an issue with kubernetes version (1.17), it failed on that. After reading through AWS EKS documentation and saw end of support for that version i moved to the next version (1.18), deployment was running fine but later failed with an error "Unauthorized".
A further research showed it was a terraform provider (2.8.0) issue.
My question is, how do i get around the reported issue or have you been able to deploy the code as-is lately.
Thanks
BTW: Loved the article, very detailed .... keep up the good work sharing your knowledge!

Collapse
 
chabane profile image
Chabane R. • • Edited

Thank you for your feedback

I forgot to fix the terraform provider version in the version.tf. You should fix the terraform provider version to be sure your terraform resources will continue to work with the AWS APIs.

Collapse
 
tbadmus profile image
tbadmus •

So what version of terraform do you advice i stick to for this?

Thread Thread
 
chabane profile image
Chabane R. • • Edited

As the article has been written last year (at least 3.37.0), you should fix the terraform version with this one

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 3.0"
    }
  }
}
Enter fullscreen mode Exit fullscreen mode
Collapse
 
tbadmus profile image
tbadmus •

Thanks for the quick response, i will do as advised.

Collapse
 
clfhhc profile image
David Chen •

For people that get stuck at CSR step in "kubernetes_job.vault-certificate" after using eks/kubernetes >= v1.22, follow this step docs.aws.amazon.com/eks/latest/use... to have the csr signed.

Collapse
 
klauswoolhouse profile image
klauswoolhouse • • Edited

This is a great tool for amazon. thanks for sharing. I am currently looking into this platform to build my business. I heard a lot of people are successful here so i am looking for a better strategy to be successful. I have already found myrealprofit.com/blog/amazon-fba-f... and it was a huge furore for me because here I found everything I needed to sell properly. I think it's an indispensable thing.