This document describes how to use the extended SCAN command to iterate over cluster data in the cluster architecture of a cloud distributed cache database (Redis-compatible), and how to handle cursor invalidation scenarios to prevent scans from entering an infinite loop.
Business Scenarios
In Ops scenarios such as batch cleanup of expired keys, key distribution statistics, data inspection, or pre-migration verification, businesses need to iterate over all keys in an instance. In native Redis Cluster, data is distributed across multiple shard nodes, and global SCAN is not supported, making direct cross-node scanning impossible and causing inconvenience for the aforementioned Ops operations. The cluster architecture of the cloud distributed cache database extends SCAN capabilities through the Proxy architecture, supporting cross-shard iteration in a single command and simplifying full key traversal operations in cluster mode.
Background Information
The cluster architecture of the cloud distributed cache database extends the SCAN command through the Proxy architecture, supporting two scan modes:
Targeted scan: Add the NODEID parameter to the end of the command to scan only the specified shard node.
Global scan: Starting from Proxy version 5.8.9, global SCAN without node limits is supported, allowing complete traversal of data across all shards in the cluster.
Prerequisites
You have created a Distributed Cache instance with cluster architecture and obtained the connection address and access password of the instance.
The Proxy version of the instance must be 5.8.9 or later, which is required for global scan capability.
Use Limits
Note:
When a node switch (primary-secondary role change) or shard reduction (cluster scale-in) occurs in the cluster, it may rarely trigger SCAN cursor invalidation, causing the mapping between the shard index corresponding to the cursor and the actual node to break. The following error is reported: -ERR invalid cursor(master node idx out of range)
If this error is ignored and iteration continues, valid data cannot be obtained. In addition, the cursor and shard logic may become out of sync, causing the scan to enter an infinite loop and continuously consume system resources. Therefore, the scan logic must handle cursor invalidation as an exception.
Operation Steps
Step 1: Scanning a Specific Shard
Append the NODEID parameter to the end of the SCAN command to scan only the data of a specified shard node, which is suitable for shard-specific troubleshooting or shard-level processing scenarios.
Example: When a hot Key in a shard of the cluster causes increased latency, you can perform a targeted scan on that shard to quickly identify the distribution of big keys or abnormal keys without traversing the entire cluster. If the node ID of the shard is f2f3c387b9fab0e67af02039845c60278b13bed0, run the following command:
scan 0 MATCH * COUNT 100 f2f3c387b9fab0e67af02039****************
A node ID is a 40-bit hexadecimal string. You can view it on the Node Management page in the console or obtain it by running the cluster nodes command.
Step 2: Performing a Global Scan
Without the NODEID parameter, SCAN traverses data across all shards in the cluster, which requires Proxy 5.8.9 or later. Iteration starts from cursor 0 and ends when the cursor returns to 0, indicating that the traversal is complete.
Step 3: Handling Cursor Invalidations and Traversing Safely
To avoid an infinite loop caused by cursor invalidation, the scan logic should catch exceptions, reset the cursor to the initial position 0, restart the scan process, and set a maximum number of retries. The complete reference code (Java + Jedis) is as follows:
package com.example.service.impl;
public class RedisServiceImpl implements RedisService {
private static final int MAX_RETRIES = 3;
private final RedisConnectionFactory connectionFactory;
@Override
public void scanKeys(String pattern, int count) {
try (Jedis jedis = connectionFactory.getConnection()) {
int retryCount = 0;
boolean scanCompleted = false;
while (!scanCompleted && retryCount <= MAX_RETRIES) {
String cursor = ScanParams.SCAN_POINTER_START;
ScanParams scanParams = new ScanParams();
scanParams.count(count);
scanParams.match(pattern);
try {
while (true) {
ScanResult<String> scanResult = jedis.scan(cursor, scanParams);
List<String> keys = scanResult.getResult();
if (!keys.isEmpty()) {
processKeysWithBusinessLogic(keys);
}
cursor = scanResult.getCursor();
if (cursor.equals(ScanParams.SCAN_POINTER_START)) {
scanCompleted = true;
break;
}
}
} catch (Exception e) {
retryCount++;
if (retryCount > MAX_RETRIES) {
throw new RuntimeException("Maximum retry count reached, scan failed", e);
}
}
}
}
}
}
The key parameters and logic are described as follows:
|
MAX_RETRIES
| Maximum number of SCAN retries. Example: 3. Adjust this value based on business requirements. |
cursor
| The cursor. Each scan round starts from the initial position 0 (ScanParams.SCAN_POINTER_START), and the cursor returning to 0 indicates that the traversal is complete. |
scanParams.match(pattern)
| Matches keys by pattern. |
scanParams.count(count)
| Hint on the number of elements returned in each iteration. |
Exception catching | After exceptions such as cursor invalidation are caught, increment the retry count by 1 and rescan from cursor 0; if MAX_RETRIES is exceeded, throw an exception to terminate and avoid an infinite loop. |
Verification Methods
Observe the scan process: the cursor eventually returns to 0 and scanCompleted is true, indicating that the traversal is complete without entering an infinite loop.
Summarize the scan results: record the total number of processed batches, the total number of keys, and the elapsed time, and verify them against the expected data size to confirm that the traversal is complete.
FAQs
Q: What should I do if an error occurs during scanning: invalid cursor (master node idx out of range)?
This error is typically caused by cursor invalidation resulting from cluster node failover or scale-in. Catch the exception, reset the cursor to 0, and restart the scan. Do not ignore the error and continue iterating with the original cursor, as this will cause an infinite loop.
Q: How can I prevent a global SCAN from entering an infinite loop?
Exception handling must be implemented for the scan process, and a maximum number of retries must be set: when a cursor exception is captured, restart the scan from 0; if the retry limit is exceeded, terminate the scan and throw an exception.
Q: How do I scan data in only a specific shard?
To perform a targeted scan, append the NODEID parameter to the end of the SCAN command. For the specific command syntax, refer to the actual support of the instance.