Fixed AK/SKSave to a Secret or an external configuration system.Inject into the Pod.The SDK reads the AK/SK.Access TencentCloud APIs directly.
The ServiceAccount used by the PodTKE issues a short-lived OIDC Token.STS validates the Token and the role trust relationship.Return temporary credentials for the CAM role.The SDK accesses TencentCloud APIs.
TmpSecretId, TmpSecretKey, and Token).Comparison Item | Fixed AK/SK | Pod OIDC |
Credentials held in the Pod | Long-term AK/SK | Short-lived OIDC tokens and CAM temporary credentials obtained on demand |
Identity source | Cloud user or cloud identity associated with the AK/SK | Kubernetes namespace + ServiceAccount |
Permission binding | Permissions are bound to the identity associated with the AK/SK. | A ServiceAccount is mapped to a CAM role through an OIDC identity. |
Permission isolation | Multiple Pods may share the same set of credentials. | Different roles can be bound by business or ServiceAccount. |
Credential storage | Requires storing and distributing AK/SK. | Long-term AK/SK is not stored in the Pod; the Token is mounted through a projected Volume. |
Lifecycle | Usually long-term valid and requires proactive rotation. | Tokens and temporary credentials are short-lived and can be automatically refreshed. |
Impact of credential leakage | May remain valid until the AK/SK is disabled or rotated. | Restricted by the Token validity period, temporary credential validity period, and role permissions. |
Rotation cost | Requires updating storage and notifying users to reload. | The cluster and SDK automatically refresh credentials, and applications do not need to perceive the refresh process. |
Audit locating | First locate the identity associated with the AK/SK, and then correlate it with the specific workload. | Workloads can be located by combining role sessions, ServiceAccounts, and namespaces. |
Component | Main Responsibility |
TKE OIDC Provider | Issues OIDC tokens for ServiceAccounts and provides Discovery and JWKS endpoints. |
CAM OIDC identity provider and CAM role | Verifies OIDC identities, maps ServiceAccounts to CAM roles, and grants permissions based on role policies. |
pod-identity-webhook | Injects projected tokens, role ARNs, and related environment variables into Pods based on ServiceAccount annotations. |
pod-identity-webhook does not issue CAM temporary credentials or proxy applications to access cloud resources.ServiceAccountIssuerDiscovery, the TKE cluster can serve as an OIDC Provider and provide the following capabilities:jwks_uri through the OIDC Discovery endpoint, then obtains the public key from the JWKS endpoint and verifies the JWT signature.serviceAccountName. When the cluster issues a Token, it writes the ServiceAccount identity into the standard claims of the JWT. The following example lists only the primary fields related to authentication:{"iss": "<OIDC Issuer URL of the TKE cluster>","sub": "system:serviceaccount:prod:cos-reader","aud": ["sts.cloud.tencent.com"],"iat": 1720000000,"exp": 1720003600}
Field | Description | Function |
iss | Token issuer | Identifies the OIDC Provider corresponding to the TKE cluster. |
sub | Token subject | Format: system:serviceaccount:<namespace>:<serviceaccount> |
aud | Token audience | Must be consistent with the Audience in the cloud identity provider and role trust policy. |
iat / exp | Issued-at and expiration time | Limits the validity period of the Token. |
sub. For example:system:serviceaccount:prod:cos-readersystem:serviceaccount:prod:log-writersystem:serviceaccount:test:cos-reader/.well-known/openid-configuration based on the iss in the Token.jwks_uri from the Discovery document.kid in the JWT.iss, and aud.Policy | Purpose | Typical Content |
Role trust policy | Which identities can assume this role | OIDC Provider, oidc:aud,oidc:sub,sts:AssumeRoleWithWebIdentity |
Role permission policy | Which operations can this role perform | Cloud service APIs, resource scopes, and access conditions |
oidc:sub. For example:TKE OIDC Provider+ aud = sts.cloud.tencent.com+ sub = system:serviceaccount:prod:cos-reader└──> CAM role cos-reader-role└──> Read-only access to the specified COS Bucket
cos-reader ServiceAccount in the prod namespace can assume the cos-reader-role. ServiceAccounts with the same name in other namespaces do not meet this condition.AssumeRoleWithWebIdentity, CAM and STS perform the following checks:iss corresponds to a registered CAM OIDC Provider.aud matches the configuration.aud, and sub.pod-identity-Webhook is a mutating admission Webhook. When a Pod is created, this component reads the annotations of the ServiceAccount used by the Pod and injects the following content:TKE_ROLE_ARN: The ARN of the target cloud role.TKE_WEB_IDENTITY_TOKEN_FILE: The path of the projected Token file.TKE_REGION and TKE_PROVIDER_ID based on the configuration.Action | Actual Executor |
Decide whether to inject identity configuration for the Pod. | pod-identity-webhook |
Issue ServiceAccount tokens. | Kubernetes API Server |
Project the Token to the Pod and refresh it before expiration. | kubelet |
Verify the Token and issue CAM temporary credentials. | CAM / STS |
Cache and refresh CAM temporary credentials. | Cloud SDK used by the application |
Credential | How to Obtain | Purpose |
OIDC ServiceAccount Token | Issued by the TKE cluster based on the ServiceAccount used by the Pod and mounted into the Pod. | Prove the Kubernetes identity of the Pod to CAM. |
CAM temporary credentials | The application or cloud SDK uses the OIDC Token and target role information to request temporary credentials from STS. | Call TencentCloud APIs within the scope of role permissions. |
Business credentials (optional) | The application uses CAM temporary credentials to read from services such as SSM. | Connect to services that require independent business credentials, such as databases. |
SecretId, SecretKey, and Token after the verification succeeds. The application then uses these temporary credentials to access the cloud resources authorized for the role.ServiceAccountIssuerDiscovery and select the following options:pod-identity-webhook component.sts.cloud.tencent.com.QcloudAccessForTKERoleInOIDCConfig.service-account-issuer and service-account-jwks-uri are managed by TKE and cannot be modified by users.pod-identity-webhook is installed and maintained by the TKE managed plane. You can view the component status on the Component Management page of the cluster.oidc:aud = the client ID from step 1.oidc:sub = system:serviceaccount:<namespace>:<serviceaccount-name> to restrict access to a specific ServiceAccount.*ReadOnlyAccess.qcs::cam::uin/xxx:roleName/xxx).cos-reader ServiceAccount in the prod namespace to assume this role:{"version": "2.0","statement": [{"effect": "allow","principal": {"federated": ["qcs::cam::uin/xxx:oidcProvider/tke-<cluster-id>"]},"action": ["sts:AssumeRoleWithWebIdentity"],"condition": {"string_equal": {"oidc:aud": ["sts.cloud.tencent.com"],"oidc:sub": ["system:serviceaccount:prod:cos-reader"]}}}]}
{"version": "2.0","statement": [{"effect": "allow","action": ["cos:GetObject", "cos:HeadObject", "cos:ListBucket"],"resource": ["qcs::cos:ap-guangzhou:uid/xxx:my-bucket-xxx/*"]}]}
apiVersion: v1kind: ServiceAccountmetadata:name: cos-readernamespace: prodannotations:tke.cloud.tencent.com/role-arn: qcs::cam::uin/xxx:roleName/cos-reader-roletke.cloud.tencent.com/audience: sts.cloud.tencent.comtke.cloud.tencent.com/token-expiration: "3600"
serviceAccountName:apiVersion: apps/v1kind: Deploymentmetadata:name: my-appnamespace: prodspec:template:spec:serviceAccountName: cos-readercontainers:- name: appimage: my-app:latest
AssumeRoleWithWebIdentity, and then use the returned temporary credentials to initialize the cloud product client.tke.cloud.tencent.com/role-arn annotation, the Webhook injects the following content into the Pod:Type | Key / Path | Description |
Environment Variable | TKE_ROLE_ARN | Target CAM role ARN |
Environment Variable | TKE_WEB_IDENTITY_TOKEN_FILE | Projected token file path. Default: /var/run/secrets/cloud.tencent.com/serviceaccount/token |
Environment Variable | TKE_REGION (if configured) | Default region |
Volume | tke-oidc-token | Projected ServiceAccount Token volume |
VolumeMount | The Token file path mentioned above | Read-only mount |
env | grep TKE_, and check the /var/run/secrets/cloud.tencent.com/serviceaccount/ directory to confirm whether the environment variables and Token file have been injected correctly.common.DefaultTkeOIDCRoleArnProvider() or a custom credential provider.AssumeRoleWithWebIdentity, and use the returned temporary credentials to initialize the client.DefaultTkeOIDCRoleArnProvider():Explicitly pass in the SecretId/Key.Environment variables TENCENTCLOUD_SECRET_ID/KEYOIDC Web Identity Token (Pod OIDC scenario)CVM instance role (node role)Throw an error.
DefaultProviderChain and DefaultTkeOIDCRoleArnProvider in the Go SDK common package.provider, err := common.DefaultTkeOIDCRoleArnProvider()if err != nil { /* ... */ }cred, err := provider.GetCredential()if err != nil { /* ... */ }client, _ := cos.NewClient(&cos.BaseURL{...}, &http.Client{Transport: &cos.AuthorizationTransport{SecretID: cred.GetSecretId(),SecretKey: cred.GetSecretKey(),SessionToken: cred.GetToken(),},})
Annotation | Required | Default Value | Description |
tke.cloud.tencent.com/role-arn | Yes | None | Target CAM role ARN. |
tke.cloud.tencent.com/audience | No | sts.cloud.tencent.com | The JWT Audience must match the oidc:aud in the CAM role trust policy. If multiple audiences are configured in the trust policy, you can specify any one of them. |
tke.cloud.tencent.com/token-expiration | No | 3600 | Validity period of the projected Token, in seconds. Value range: 600 to 86400. |
Item | Limit |
Supported cluster type | TKE managed clusters; for the support status of standalone clusters / Serverless clusters / edge clusters, check the console or submit a ticket for confirmation. |
Cluster version requirement | Greater than or equal to v1.20.6-tke.27 or greater than or equal to v1.22.5-tke.1. Later versions are supported by default. |
Pod network requirement | Must be able to access the Tencent Cloud STS endpoint. For VPC clusters without internet access, use the private STS endpoint or configure PrivateLink. |
Token validity period | 600 to 86400 seconds. |
Number of roles that can be bound to a single ServiceAccount | 1; if multiple roles are needed, create multiple ServiceAccounts. |
AssumeRoleWithWebIdentity rate limiting | Follows CAM global quotas. It is recommended that the SDK cache temporary credentials and refresh them when they are close to expiration. |
Super nodes / virtual nodes | Supported, and the annotation usage is the same. |
Private cluster / no public network egress | Configure the network path from the Pod to the STS private endpoint. |
kubectl describe pod and check whether environment variables such as TKE_ROLE_ARN exist.serviceAccountName, and the status of the pod-identity-webhook component.kubectl exec to enter the Pod, and then run cat $TKE_WEB_IDENTITY_TOKEN_FILE.AssumeRoleWithWebIdentity Returns 401 or Access Deniedoidc:aud in the CAM role trust policy matches the Audience in the ServiceAccount annotation.oidc:sub exactly matches the ServiceAccount used by the Pod, in the format system:serviceaccount:<namespace>:<serviceaccount>.ServiceAccountIssuerDiscovery indicates that the identity provider does not exist, recreate the Provider.ServiceAccountIssuerDiscovery Cannot Be EditedQcloudAccessForTKERoleInOIDCConfig has been authorized.role-arn annotation from the ServiceAccount and recreate the Pod so that the new Pod no longer has identity configuration injected.oidc:sub.tke.cloud.tencent.com/role-arn annotation, this component injects the projected ServiceAccount Token, environment variables, and volumes. The component is installed and maintained by the TKE managed plane, and you can view its running status on the Component Management page in the cluster.rules:- apiGroups: [""]resources: ["serviceaccounts"]verbs: ["get", "watch", "list"]- apiGroups: [""]resources: ["events"]verbs: ["patch", "update"]- apiGroups: ["admissionregistration.k8s.io"]resources: ["mutatingwebhookconfigurations"]verbs: ["get"]
update permission to mutatingwebhookconfigurations to inject self-signed certificates. The actual required permissions depend on the component version.TKE ServiceAccount└──> TKE OIDC Token└──> AWS STS AssumeRoleWithWebIdentity└──> Temporary credentials for the AWS IAM Role└──> S3 / DynamoDB / SQS / Other AWS services
AWS Object | Function | Key Content to Configure |
IAM OIDC Provider | Register and validate tokens issued by the TKE cluster. | TKE Issuer URL, Discovery/JWKS, Audience, and certificate fingerprint used when necessary |
IAM Role trust policy | Define OIDC identities that are allowed to assume the role. | Provider ARN, sts:AssumeRoleWithWebIdentity,aud,sub |
IAM Role permission policy | Define AWS APIs that the role can call. | Action,Resource,Condition |
ServiceAccountIssuerDiscovery for the TKE cluster and obtain the cluster's OIDC Issuer URL.<Issuer URL>/.well-known/openid-configurationjwks_uri in the Discovery documentProvider URL, enter the Issuer URL of the TKE cluster.Audience, enter sts.amazonaws.com. The aud field of the ServiceAccount projected Token must match this value.issuer and jwks_uri in the Discovery document. If the root CA used by the OIDC endpoint is not trusted by AWS, or if AWS cannot verify the TLS certificate chain, you also need to configure the certificate fingerprint according to AWS requirements. For specific requirements, see AWS: Creating an IAM OIDC identity provider.{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Action": ["s3:ListBucket"],"Resource": ["arn:aws:s3:::example-bucket"]},{"Effect": "Allow","Action": ["s3:GetObject"],"Resource": ["arn:aws:s3:::example-bucket/*"]}]}
s3-reader ServiceAccount in the prod namespace to this role:{"Version": "2012-10-17","Statement": [{"Effect": "Allow","Principal": {"Federated": "arn:aws:iam::<AWS_ACCOUNT_ID>:oidc-provider/<ISSUER_HOST_AND_PATH>"},"Action": "sts:AssumeRoleWithWebIdentity","Condition": {"StringEquals": {"<ISSUER_HOST_AND_PATH>:aud": "sts.amazonaws.com","<ISSUER_HOST_AND_PATH>:sub": "system:serviceaccount:prod:s3-reader"}}}]}
<AWS_ACCOUNT_ID>: The AWS account ID.<ISSUER_HOST_AND_PATH>: The complete Issuer hostname and path with https:// removed. This value must match the value in the IAM OIDC Provider ARN.sub: The TKE namespace and ServiceAccount that are allowed to assume the role. In this example, only prod/s3-reader is allowed.aud: Must match the Audience of the IAM OIDC Provider and the ServiceAccount annotation.sub wildcards. Otherwise, other ServiceAccounts in the same cluster may also assume this role. For the trust policy requirements of Web Identity roles, see AWS: Creating a role for OIDC federation.pod-identity-webhook writes the value of the role-arn annotation to TKE_ROLE_ARN. When integrating with AWS, you can specify the AWS IAM Role ARN in this annotation:apiVersion: v1kind: ServiceAccountmetadata:name: s3-readernamespace: prodannotations:tke.cloud.tencent.com/role-arn: arn:aws:iam::<AWS_ACCOUNT_ID>:role/tke-s3-readertke.cloud.tencent.com/audience: sts.amazonaws.comtke.cloud.tencent.com/token-expiration: "3600"
serviceAccountName: s3-reader. The Webhook injects the following environment variables:TKE_ROLE_ARN=arn:aws:iam::<AWS_ACCOUNT_ID>:role/tke-s3-readerTKE_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/cloud.tencent.com/serviceaccount/token
AWS_ROLE_ARNAWS_WEB_IDENTITY_TOKEN_FILEAWS_ROLE_SESSION_NAME (optional)TKE_ROLE_ARN and TKE_WEB_IDENTITY_TOKEN_FILE. The AWS SDK default credential chain does not read these variables. You can use the following method for adaptation. The following example explicitly reads the TKE environment variables and initializes an S3 client by using the AWS SDK for Go v2:package awsauthimport ("context""fmt""os""github.com/aws/aws-sdk-go-v2/config""github.com/aws/aws-sdk-go-v2/credentials/stscreds""github.com/aws/aws-sdk-go-v2/service/s3""github.com/aws/aws-sdk-go-v2/service/sts")func NewS3Client(ctx context.Context, awsRegion string) (*s3.Client, error) {roleARN := os.Getenv("TKE_ROLE_ARN")tokenFile := os.Getenv("TKE_WEB_IDENTITY_TOKEN_FILE")if roleARN == "" || tokenFile == "" {return nil, fmt.Errorf("missing TKE_ROLE_ARN or TKE_WEB_IDENTITY_TOKEN_FILE")}baseCfg, err := config.LoadDefaultConfig(ctx, config.WithRegion(awsRegion))if err != nil {return nil, fmt.Errorf("load AWS base config: %w", err)}stsClient := sts.NewFromConfig(baseCfg)roleSessionName := os.Getenv("HOSTNAME")if roleSessionName == "" {roleSessionName = "tke-pod"}provider := stscreds.NewWebIdentityRoleProvider(stsClient,roleARN,stscreds.IdentityTokenFile(tokenFile),func(options *stscreds.WebIdentityRoleOptions) {options.RoleSessionName = roleSessionName},)cfg, err := config.LoadDefaultConfig(ctx,config.WithRegion(awsRegion),config.WithCredentialsProvider(provider),)if err != nil {return nil, fmt.Errorf("load AWS config with web identity: %w", err)}return s3.NewFromConfig(cfg), nil}
TKE_ROLE_ARN and TKE_WEB_IDENTITY_TOKEN_FILE as the role ARN and Token file path, respectively. For AWS standard configuration items, see AWS SDK: Assume role credential provider. For Go v2 Provider usage, see AWS SDK for Go v2: AssumeRoleWithWebIdentity.TKE_REGION indicates a Tencent Cloud region and cannot be used as an AWS Region. When initializing the AWS client, you need to configure an AWS Region such as ap-southeast-1 separately.InvalidIdentityToken: Confirm that AWS can access the TKE Discovery and JWKS endpoints, and check the Issuer URL and TLS certificate chain.sts.amazonaws.com.AccessDenied (unable to assume the role): Confirm that the Provider ARN, sub, and target Role ARN match exactly.TKE_* environment variables, or that they have been mapped to the AWS_* environment variables that the AWS SDK reads by default.pod-identity-webhook related issues on the Component Management page of the cluster.Esta página foi útil?
Você também pode entrar em contato com a Equipe de vendas ou Enviar um tíquete em caso de ajuda.
comentários