tencent cloud

Batch Compute

Product Introduction
What's Batch?
Scenarios
Strengths
Glossary
Quota & Limits
Data Security
Purchase Guide
Getting Started
Preparation
Using CLI - Submit a Job
Using CLI - Compute Environment
Configuring a Job
Using Console
Console User Guide
Task Template Management
Job Management
User Guide
How to create images usable by BatchCompute
Windows Custom Images
Environment Variables
Entering COS & CFS Paths
Event Notice & Callback
Cloud Access Management
Command Line Interface
Preparation
Quick Start
Running Remote Package
Mapping Remote Storage
Use Cases
Building Cluster Using Compute Environment
Example: 3ds Max 2018 Rendering
Example: Deep Learning
API Documentation
History
Introduction
API Category
Making API Requests
Compute Environment APIs
Configuration Viewing-related APIs
Task Template-related APIs
Job-related APIs
Data Types
Error Codes
SDK Documentation
Instructions
FAQs
Contact Us
Glossary

Mapping Remote Storage

PDF
포커스 모드
폰트 크기
마지막 업데이트 시간: 2025-08-19 15:09:44

Operation Scenarios

Remote mapping is used as an auxiliary feature by BatchCompute for storage services to map remote storage, such as COS and CFS, to a local folder.

Prerequisites

Complete the preparations as instructed in Preparations and learn how to configure the general part of the custom information.

Directions

Uploading input data file

1. Create number.txt with the following content:
1
2
3
4
5
6
7
8
9
2. Log in to the COS Console and click Bucket List on the left sidebar.
3. Select the ID of the created bucket, click Objects, select input, and upload number.txt as shown below:



Viewing and modifying demo

Note:
Modify the general part of the custom information in 3_StoreMapping.py as instructed in Preparations.
Open 3_StoreMapping.py in an editor.
# custom (Change to your info)
imageId = "img-m4q71qnf"
Application = {
"DeliveryForm": "PACKAGE",
"Command": "python ./codepkg/sumnum.py",
"PackagePath": "http://batchdemo-xxxxxxxxx.cos.ap-guangzhou.myqcloud.com/codepkg/codepkg.tgz"
}
StdoutRedirectPath = "your cos path"
StderrRedirectPath = "your cos path"
InputMapping = {
"SourcePath": "cos://batchdemo-xxxxxxxxx.cos.ap-guangzhou.myqcloud.com/input/",
"DestinationPath": "/data/input/"
}
OutputMapping = {
"SourcePath": "/data/output/",
"DestinationPath": "your output remote path"
}
Compared to 2_RemoteCodePkg.py, some of the parameters in the custom part are modified as follows:
Configuration Item
Description
Application
Modify `Command` to `sumnum.py`.
InputMapping
Input mapping.
`SourcePath` remote storage address: modify this address to the path of the **input** folder in "Preparations". For more information, please see Getting COS Endpoints.
`DestinationPath` local directory: retain the original setting for the time being.
OutputMapping
Output mapping.
`SourcePath` local directory: retain the original setting for the moment.
`DestinationPath` remote storage address: modify this address to the path of the **output** folder in "Preparations". For more information, please see Getting COS Endpoints.
sumnum.py is composed as below: Open input/number.txt, add up all numbers in each row, and write the result into output/result.txt.
import os

inputfile = "/data/input/number.txt"
outputfile = "/data/output/result.txt"

def readFile(filename):
total = 0
fopen = open(filename, 'r')
for eachLine in fopen:
total += int(eachLine)
fopen.close()
print "total = ",total
fwrite = open(outputfile, 'w')
fwrite.write(str(total))
fwrite.close()

print("Local input file is ",inputfile)
readFile(inputfile)

Submitting job

Run the following command to run the Python script. The demo encapsulates the job submission process by using Python scripts and the BatchCompute command line tool.
python 3_StoreMapping.py
The returned result is as follows, indicating that the job is successfully submitted:
{
"RequestId": "8eaeb01e-94a6-41a1-b40f-95f15417c0b4",
"JobId": "job-97smiptb"
}
If the submission operation fails, check the returned value or contact us for assistance.

Viewing status

Please see Viewing Status in Quick Start.

Viewing result

1. Log in to the COS Console and click Bucket List on the left sidebar.
2. Select the ID of the created bucket, click Files, and select output as shown below: BatchCompute copies the output data from the local directory to the remote directory. The execution results of 3_StoreMapping.py are stored in result.txt, which is automatically synced to COS.


result.txt is composed as below:
45


도움말 및 지원

문제 해결에 도움이 되었나요?

피드백