Configuring tcr-assistant-oidc for Cross-Account Dynamic Password-Free Image Pull
Download
Mode fokus
Ukuran font
Scenarios
In enterprises with multi-account governance, image assets (TCR Enterprise Edition instances) are typically centrally hosted in a single account, while business applications (TKE clusters) are distributed across multiple business accounts. In this scenario, TKE clusters in business accounts need to pull container images from the image account. The conventional approach involves distributing the image account's long-term access credentials (username/password) as ImagePullSecrets to each business account cluster, which presents the following pain points:
Credentials are difficult to manage: Long-term credentials require manual synchronization and rotation across multiple accounts, resulting in high Ops costs.
High security risk: If compromised, long-term credentials remain valid indefinitely, and it is difficult to trace them to specific users.
The tcr-assistant-oidc plugin enables cross-account password-free image pulls by leveraging the OIDC authentication mechanism, CAM roles, and STS dynamic temporary credentials. It is applicable to the following typical business scenarios:
Unified Multi-Account Image Governance: TCR Enterprise Edition instances are centrally hosted in the image account, while TKE clusters in business accounts can pull images across accounts without passwords.
Automatic Rotation of Temporary Credentials: This feature must meet compliance requirements and prohibits scenarios where long-term image repository credentials are persistently stored within a cluster.
Fine-Grained Authorization by Repository/Namespace: Business parties can only pull images from authorized image repositories or namespaces.
Image Pull via Custom Domain: The TCR instance is bound to a custom corporate domain (for example,
docker.mycompany.com). Images must be pulled using this custom domain.Compared to the original tcr-assistant plugin, tcr-assistant-oidc offers the following enhanced capabilities:
Capability | tcr-assistant | tcr-assistant-oidc |
Credential type | Static long-term credentials (username + password) | Dynamic temporary keys, automatically rotated |
Cross-account pull | Only supports pulling from the same account. | Supports pulling from the same account and across accounts |
Note:
If you only need to perform same-account password-free image pulls (where the TKE cluster and the TCR instance belong to the same Tencent Cloud account), see Configuring tcr-assistant-oidc for Same-Account Dynamic Password-Free Image Pull.
Prerequisites
A TKE cluster (Account B) has been created, and its version is greater than or equal to 1.20. (Kubernetes version 1.20 upgraded the ServiceAccountIssuerDiscovery feature to General Availability.)
A TCR Enterprise Edition instance (Account A) has been created and is in the "Running" state.
Implementation Principles
In a cross-account scenario, the TKE cluster resides in Account B (the image consumer), and the TCR instance resides in Account A (the image provider). The trust boundary expands from "within an account" to "cross-account". The end-to-end call flow can be abstracted as a five-stage collaboration: ① OIDC identity injection → ② cross-account CAM authorization to obtain temporary credentials → ③ obtaining the docker login password → ④ Secret distribution → ⑤ cross-VPC password-free image pull.
Link Interpretation:
Case No. | Phase | Key Action |
① | OIDC Identity Injection | pod-identity-webhook mounts the OIDC JWT issued by the TKE cluster of Account B to the Controller Pod, and the iss of the JWT points to Cluster B. |
② | Cross-account CAM Authorization | The Controller calls the STS across accounts with the JWT from Cluster B. The STS verifies the signature using the JWKS of Cluster B registered under Account A, matches the federated OIDC Provider(B) in the Role trust policy, and returns a 2-hour temporary credential for Account A. |
③ | Retrieve docker Password | The Controller calls the TCR of Account A with the temporary credential of Account A to obtain the docker login username/password. |
④ | Secret Distribution | The Controller renders the dockerconfigjson (which includes public network, VPC, and custom domain names) and distributes it to the target namespaces/service accounts in the cluster of Account B according to the CR rules. |
⑤ | Cross-VPC Passwordless Pull | The business Pod completes the docker pull via the imagePullSecrets on the SA, relying on VPC peering/CCN + Private DNS to ensure network reachability. |
Note:
Key Cross-Account Features: The CAM of Account A must manually register the OIDC Issuer and JWKS of the TKE cluster in Account B and explicitly declare "allow the OIDC identity of Account B to assume this role" in the Role trust policy. Additionally, business Pods require cross-VPC network connectivity when pulling images.
Account Role Descriptions
This document involves operations across two Tencent Cloud accounts. For clarity, they are uniformly referred to as Account A / B.
Role | Description | Example UIN |
Account A | The account where the sub-account resides (image provider) is responsible for creating the CAM OIDC identity provider, CAM policies, and CAM roles. | 100001113387 |
Account B | The account where the TKE cluster resides (image consumer) is responsible for enabling OIDC and installing the tcr-assistant-oidc plugin. | 100002223233 |
The core mechanism of cross-account password-free image pull is: The TKE cluster in Account B requests to assume a CAM role in Account A via OIDC. After STS returns temporary credentials, the plugin writes them into an ImagePullSecret, enabling cross-account image pulls.
Operation Steps
Step 1: Enable the OIDC Feature for the TKE Cluster in Account B
1. Log in to the TKE console, and select Cluster Management from the left sidebar.
2. Select the target cluster, go to its details page, and choose Basic Information > API Server Information.
3. Click the Edit button to the right of ServiceAccountIssuerDiscovery, as shown below:
4. Go to the page for modifying ServiceAccountIssuerDiscovery-related parameters. If the system prompts you that the parameters cannot be modified, perform service authorization first.
5. On the Role Management page, locate the authorization policy QcloudAccessForTKERoleInOIDCConfig and click Authorize.
6. After authorization is completed, select Create CAM OIDC Provider and Create webhook Component, enter the Client ID (which is fixed as
sts.cloud.tencent.com), and click OK, as shown below:
7. Return to the Basic Information > API Server Information page. When ServiceAccountIssuerDiscovery becomes editable again, it indicates that the process of enabling OIDC resource access control is complete.
Note:
The values of the
service-account-issuer and service-account-jwks-uri parameters cannot be edited and use the default rules.Step 2: Verify the Successful Creation of the OIDC Identity Provider and Webhook Component in Account B
After the OIDC feature is enabled, you need to confirm that the CAM OIDC identity provider has been created and the
pod-identity-webhook component has been successfully deployed:1. On the cluster details page, switch to the API Server Information option and click the Edit button to the right of ServiceAccountIssuerDiscovery.
2. Go to the page for modifying ServiceAccountIssuerDiscovery-related parameters. The system will display the message "The identity provider you created already exists. Go to view." Click Go to view to confirm that the CAM OIDC identity provider has been created successfully.
3. View the details of the newly created CAM OIDC identity provider and confirm that the Client ID is configured correctly, as shown below:
4. In Cluster Information > Component Management, confirm that the status of the
pod-identity-webhook component in the list is "Success", which indicates that the component has been installed successfully.5. On the Cluster Details > Workloads > Deployment page, confirm that the status of
pod-identity-webhook in the kube-system namespace is Running, which indicates that the installation is successful.Step 3: Create an OIDC Identity Provider in Account A
When the OIDC feature is enabled for a TKE cluster in Account B, an identity provider is created by default in Account B. To access TCR resources in Account A across accounts, you must manually create an identity provider in Account A as well. This identity provider explicitly allows Account B to access TCR resources via OIDC through a role bearer mechanism.
1. Log in to the CAM Console - Identity Provider using Account B, select the corresponding cluster to view the OIDC identity provider information for Account B. All the following information required for creating an identity provider for Account A can be obtained from the details, as shown below:
2. Log in to the CAM Console - Identity Provider using Account A and create a new provider. You need to fill in the following information:
Identity Provider Name: Enter the ClusterID of the TKE cluster in Account B (for example,
cls-xxxxxxxx).Identity Provider URL: Enter the value of
service-account-issuer from the APIServer information of the TKE cluster in Account B.Client ID: Enter
sts.cloud.tencent.com as a fixed value.Signature Public Key: Enter the public key of the identity provider for the TKE cluster in Account B.
3. After preparing the above information, click Create to complete the identity provider creation, as shown below:
Step 4: Create a CAM Custom Policy for the TCR Instance in Account A
Create custom policies based on your business requirements to control access permissions for TCR instances. These policies can precisely control access permissions at the namespace level.
Attention:
Do not add region or uin information to the six-segment resource descriptor. For example, in
qcs::tcr:::instance/tcr-5jm4cabc, you only need to replace the final TCR instance ID part.Policy Example:
{"version": "2.0","statement": [{"action": ["tcr:DescribeInstances"],"effect": "allow","resource": ["qcs::tcr:::instance/tcr-5jm4cabc"]},{"action": ["tcr:PullRepository","tcr:PushRepository","tcr:CreateRepository"],"effect": "allow","resource": ["qcs::tcr:::repository/tcr-5jm4cabc/production-backend/*"]}]}
Policy Description:
tcr:DescribeInstances: The permission required for login.tcr:PullRepository: The policy permission required for pulling images.tcr:PushRepository: This permission is granted on demand and does not affect image pulling.tcr:CreateRepository: This permission is granted on demand and is required when a push is performed if the repository does not exist.In the resource,
production-backend/* indicates that only namespace-level granularity is controlled.Step 5: Create a Role and Bind the Custom Policy in Account A
1. Log in to the CAM Console - Roles page.
2. On the Roles page, click Create Role > Identity Provider.
3. On the Create Custom Role page, configure the settings by referring to the following information, as shown in the figure below.
Note:
The value of
oidc:aud must match the Client ID value of the CAM OIDC provider.Set the value of
oidc:aud to $my_pod_audience. When oidc:aud has multiple values, you can select any one of them.
4. Bind the custom policy created in Step 4 to complete the role creation, as shown in the figure below.
Step 6: Establish VPC Network Connectivity Between Different Accounts
The VPC where the TKE cluster in Account B resides and the VPC where the TCR instance in Account A resides must have network connectivity. You can refer to the Peering Connection documentation to establish the VPC network connection.
Step 7: Install the tcr-assistant-oidc Plugin in the TKE Cluster in Account B
Install the tcr-assistant-oidc plugin in the TKE cluster of Account B. The plugin dynamically generates temporary docker login credentials based on OIDC identity for accessing TCR resources in Account A.
Select one of the following installation methods based on your actual scenario:
Method 1: Installing via TencentCloud API (Latest Version 1.0.2)
Install the plugin by calling the InstallAddon API via TencentCloud API. As shown in the following figure:
Field description:
The role ARN for Account A can be obtained on the CAM Console Role Details page. As shown in the figure below:
The TCR instance name, registries, can be obtained from the TCR Console Instance List. As shown in the figure below:
If a custom domain (for example,
docker.mycompany.com) is bound to the TCR instance in Account A, you need to additionally configure customDomains.Note: The Values parameter is deprecated. The Values parameter is deprecated. If you need to use it, pass it via RawValues, where the parameter value is the content of a JSON string after base64 encoding. Example:
echo -n '{"roleArn": "qcs::cam::uin/100000000000:roleName/your-role-name","registries": ["your-tcr-registry"]}' | base64
Complete example of RawValues JSON:
{"roleArn": "qcs::cam::uin/100001113387:roleName/tcr-assistant-odic-read-harbor","registries": ["your-tcr-registry"],"customDomains": ["docker.mycompany.com"],"tokenExpiration": 3600,"namespaces": "*","serviceAccounts": "*"}
Method 2: Installing via Terraform
resource "tencentcloud_kubernetes_addon" "tcr_assistant_oidc" {cluster_id = "cls-xxxxxxxx" # Replace with your TKE cluster IDaddon_name = "tcr-assistant-oidc"raw_values = jsonencode({# CAM Role ARN (Required)# Format: qcs::cam::uin/{UIN}:roleName/{ROLE_NAME}roleArn = "qcs::cam::uin/100000000000:roleName/your-role-name"# TCR Image Repository Name List (Required)registries = ["your-tcr-registry"]# Target namespace ("*" indicates all, or "ns1,ns2,ns3")namespaces = "*"# Target ServiceAccount ("*" indicates all, or "sa1,sa2")serviceAccounts = "*"# Custom Domain (Optional)customDomains = ["docker.mycompany.com"]})}
Verifying the Plugin Installation Status in Account B
After installation, run the following command in the TKE cluster of Account B to verify that the add-on is running properly:
# Verify that the Controller Pod is running properlykubectl get pods -n tcr-assistant-system# Expected: Pod is in the Running state# Check the CR status to confirm that the temporary key has been successfully obtained from Account A.kubectl get oips tcr-oidc-public -o yaml# Ensure that status.credentialStatus is Ready.# Verify that the imagePullSecret has been distributed to the target namespace.kubectl get secret tcr.ips.tcr-oidc-public -n default \\-o jsonpath='{.data.\\.dockerconfigjson}' | base64 -d | jq '.auths | keys'
The expected output should include all domain names of the TCR instance in Account A:
["your-tcr-registry.tencentcloudcr.com","docker.mycompany.com"]
Step 8: Add Private DNS Resolution for the TCR Instance in Account B
Add private domain resolution for the TCR instance of Account A on the Account B side, ensuring that the cluster can access the TCR instance via the private network domain name. As shown in the following figure:
Typical Use Cases
Scenario 1: Multiple Accounts (B, C) Sharing Access to TCR Images in Account A
The TCR instance in Account A needs to be accessed by TKE clusters from multiple different accounts:
1. When creating a role in Account A, configure multiple principals:
qcs::cam::uin/222:roleName/a-policy-for-tcr # Account Bqcs::cam::uin/333:roleName/a-policy-for-tcr # Account C
2. Install the plugin separately for each account cluster, using the same
roleArn.Scenario 2: Pulling Images Across Accounts via a Custom Domain Name
The custom domain
docker.mycompany.com is bound to the TCR instance in Account A. Account B needs to pull images using this domain. During component installation, specify customDomains=["docker.mycompany.com"]. After installation, the cluster in Account B can then pull images from Account A via the custom domain:kubectl run nginx --image=docker.mycompany.com/my-namespace/nginx:latest
Scenario 3: Limiting the Effective Scope (Specifying Namespace and ServiceAccount)
It takes effect only within specific namespaces and ServiceAccounts in the Account B cluster:
helm install tcr-assistant-oidc ./tcr-assistant-oidc \\--namespace tcr-assistant-system --create-namespace \\--set roleArn="qcs::cam::uin/100000000000:roleName/your-role-name" \\--set "registries={your-tcr-registry}" \\--set namespaces="production,staging" \\--set serviceAccounts="default,app-sa"
Upgrades and Changes
Upgrading Component Versions
Use UpdateAddon to upgrade the tcr-assistant-oidc component version. You do not need to respecify parameters during the upgrade; only update the component version and the parameter configurations you intend to modify. For the list of target versions for the tcr-assistant-oidc component, refer to Component Version Maintenance Notes.
Note:
During the upgrade, existing Secrets are not interrupted. After the Controller restarts, it automatically resumes reconciliation.
Verifying the Upgrade Result
After the upgrade is complete, verify that the Controller Pod has been updated and is running normally:
# Check the Pod status and image versionkubectl get pods -n tcr-assistant-system -o wide# Check the CR statuskubectl get oips tcr-oidc-public -o jsonpath='{.status.credentialStatus}'# Expected output: Ready
Troubleshooting
Pod Image Pull Failures (ImagePullBackOff)
Troubleshooting Steps:
1. Check whether the Secret exists:
kubectl get secret tcr.ips.tcr-oidc-public -n <pod-namespace>
2. Check whether the ServiceAccount has been injected with imagePullSecrets:
kubectl get sa default -n <pod-namespace> -o jsonpath='{.imagePullSecrets}'
3. Check the Controller logs:
kubectl logs -n tcr-assistant-system deployment/tcr-assistant-oidc-controller
Temporary Key Expiration
The validity period of dynamically generated temporary credentials is fixed at 2 hours (7200 seconds).
tokenExpiration controls the rotation interval. If it is set to a value greater than 5400 (>5400), the credentials may expire.Solution: Ensure that
tokenExpiration is within the range of 3600 to 5400.OIDC Environment Variables Not Injected
If the Controller Pod lacks OIDC-related environment variables (such as
TKE_WEB_IDENTITY_TOKEN_FILE), it indicates that the pod-identity-webhook in the B account cluster is not functioning properly.Solutions:
1. Confirm that the TKE cluster in the B account has the OIDC feature enabled.
2. Confirm that the
pod-identity-webhook Pod is running normally.3. Recreate the Controller Pod.
AssumeRole Failures (Insufficient Permissions)
If a failure of
AssumeRoleWithWebIdentity is recorded in the Controller logs, it indicates that the role trust configuration for Account A or the policy configuration for Account B is incorrect.Troubleshooting Steps:
1. Confirm that the trust policy for the role in Account A includes the OIDC Provider for the TKE cluster in Account B.
2. Confirm that the role in Account A has been granted TCR access permissions.
3. Confirm that the
resource field in the policy for Account B exactly matches the ARN of the role in Account A.Manually Verifying Temporary Keys
Extract credentials from the Secret and manually log in for verification:
# Obtain credentials.SECRET_JSON=$(kubectl get secret tcr.ips.tcr-oidc-public -n default \\-o jsonpath='{.data.\\.dockerconfigjson}' | base64 -d)# Extract the username and password, and replace your-tcr-registry.tencentcloudcr.com with the actual access domain.USERNAME=$(echo $SECRET_JSON | jq -r '.auths["your-tcr-registry.tencentcloudcr.com"].username')PASSWORD=$(echo $SECRET_JSON | jq -r '.auths["your-tcr-registry.tencentcloudcr.com"].password')# Manually log in to the TCR instance for Account A, replacing your-tcr-registry.tencentcloudcr.com with the actual access domain. Ensure that the client machine used for access can reach the TCR domain normally.docker login your-tcr-registry.tencentcloudcr.com --username "$USERNAME" --password "$PASSWORD"
Bantuan dan Dukungan
Apakah halaman ini membantu?
Anda juga dapat Menghubungi Penjualan atau Mengirimkan Tiket untuk meminta bantuan.
masukan