Our organization has enabled aws role based access across the organization, long story short, no aws_access_key_id and aws_secret_access_key could be used any more, instead we must use role based and aws STS to gain aws resource access

aws-role-based-access

there are 2 categories of Role based access, the user and the aws compute resources.

Developer Access

we've enabled azure sso with aws , so anyone who want's to access aws resources must go through azure sso and gain access to aws console by assuming relevant role. if we want to access aws resources through aws cli locally, I'm using a tool called aws-azure-login, use npm install -g aws-azure-login to install, then config ~/.aws/config with content below

[profile dev-profile]
output=json
region=us-east-1
azure_tenant_id=11111111-1111-1111-1111-111111111111
azure_app_id_uri=11111111-1111-1111-1111-111111111111
azure_default_username=ivan@vimmind.me
azure_default_role_arn=
azure_default_duration_hours=8
azure_default_remember_me=true

if we leave azure_default_role_arn as blank, when issue command aws-azure-login --profile dev-profile it will allow us to select which role to assume after successful go through azure sso process.

then we could get sample content below in ~/.aws/credentails

[dev-profile]
aws_access_key_id=xxxxxxxxxxxxxxxxx
aws_secret_access_key=xxxxxxxxxxxxxxxxx
aws_session_token="xxxxxxxxxxxxxxxxxxxxxxxxxx"

as a temp aws access key it will be valid for 8 hours (normal working hours), then we could use aws cli to communicate with aws resources normally

aws compute resource access

we mainly use 3 types of aws compute resources

  • the ec2
  • the lambda functions
  • the eks pod

ec2 and lambda functions

we need to create a role to and attach related permissions to the role which allow the compute resources to perform, then we need to trust ec2.amazonaws.com and lambda.amazonaws.com to perform sts:AssumeRole action, further more, for EC2 resources, we need to attach an iam instance profile to the role, so that the ec2 resources could use them, a sample terraform scripts is like below

resource "aws_iam_role" "runtime-role" {
  name = "${var.env-name}-runtime-role"
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
  ]
  # Terraform's "jsonencode" function converts a
  # Terraform expression result to valid JSON syntax.
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17"
    "Statement" : [
      {
        "Action" : "sts:AssumeRole",
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "ec2.amazonaws.com"
        }
      },
      {
        "Action" : "sts:AssumeRole",
        "Effect" : "Allow",
        "Principal" : {
          "Service" : "lambda.amazonaws.com"
        }
      }
    ]
  })

  tags = var.resource_tags
}

resource "aws_iam_instance_profile" "runtime-profile" {
  name = "${var.env-name}-runtime-profile"
  role = aws_iam_role.runtime-role.name
}

eks pod access

it's a little bit complicated when we want to grant aws access to pod based on role. here is a detailed explanation on how to enabled role based access for service account, https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html

  1. need to add the eks oidc issuer to aws IAM OICD provider, this could be done via aws console
  2. need to trust eks oidc and eks's designed service account
  3. create an eks service account which has iam role as an annotation

for step 2, below is the sample terraform scripts, this will allowed my eks in ap-southeast-2 in namespace dev could assume aws role.

resource "aws_iam_role" "runtime-role" {
  name = "${var.env-name}-runtime-role"
  managed_policy_arns = [
    "arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole"
  ]
  # Terraform's "jsonencode" function converts a
  # Terraform expression result to valid JSON syntax.
  assume_role_policy = jsonencode({
    "Version" : "2012-10-17"
    "Statement" : [
      {
        "Effect" : "Allow",
        "Principal" : {
          "Federated" : "arn:aws:iam::111111111:oidc-provider/oidc.eks.ap-southeast-2.amazonaws.com/id/xxxxxxxx"
        },
        "Action" : "sts:AssumeRoleWithWebIdentity",
        "Condition" : {
          "StringEquals" : {
            "oidc.eks.ap-southeast-2.amazonaws.com/id/xxxxxxxx:sub" : "system:serviceaccount:dev:runtime-serviceaccount"
          }
        }
      }
    ]
  })

  tags = var.resource_tags
}

for step 3, here is a sample service creation yaml file

apiVersion: v1
kind: ServiceAccount
metadata:
  name: runtime-serviceaccount
  namespace: dev
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::1111111:role/runtime-role
    eks.amazonaws.com/sts-regional-endpoints: "true"

when create a pod with service account, it will have 2 ENV populated in the pod envs AWS_WEB_IDENTITY_TOKEN_FILE , and AWS_ROLE_ARN, then we could use aws cli inside pod to communicated with AWS resources based on permission allowed in role arn:aws:iam::1111111:role/runtime-role