FTP Server Tool

Overview

The COS FTP Server can be used to directly operate objects and directories, uploading/downloading/deleting files and creating folders in COS via the FTP protocol. This tool is developed with Python, which makes the installation easier.

Description

Upload mechanism: adopts the stream upload and is not saving the uploaded file locally. It works only if the working directory is configured according to the standard FTP protocol, and no actual disk storage space is occupied.
Download mechanism: The downloaded file is directly returned to the client in the stream download mode.
Directory mechanism: Bucket serves as the root directory of the entire FTP Server, and multiple subdirectories can be created under Bucket.
Binding to multi-buckets: Multiple buckets can be bound at the same time.

Binding to multi-buckets: Multiple buckets can be bound via different FTP Server working paths (home_dir). Therefore, ensure a unique home_dir is assigned to each bucket and user information.

Restriction on deletion: The delete_enable option can be configured for each FTP user in the new FTP Server to identify whether the FTP user is allowed to delete files.
Supported FTP commands: put, mput, get, rename, delete, mkdir, ls, cd, bye, quite, and size.
Unsupported FTP commands: append and mget (The native mget command is not supported, but batch download is allowed on certain Windows clients, such as the FileZilla client.)

The FTP Server tool does not support resuming upload from breakpoint.

Getting Started

System Environment

• OS: Linux. It is recommended to use the CVM of Tencent CentOS series. Windows systems are not supported for now.
• Python interpreter version: Python 2.7. For more information on the installation and configuration, see Installing and Configuring Python.
• Dependency package:
  • cos-python-sdk-v5 (≥1.6.5)
  • pyftpdlib (≥1.5.2)

Use Limits

Applicable to COS XML version.

Installation and Operation

Download the FTP Server tool at cos-ftp-server. The installation steps are as follows:

1. Run setup.py to install FTP Server and its relevant dependent libraries (network required).

   python setup.py install   # Your account sudo or root permission may be required.

2. Copy the configuration sample file conf/vsftpd.conf.example and name it conf/vsftpd.conf. See Configuration File of this document to correctly configure bucket and user information.
3. Run ftp_server.py to start FTP Server.

   python ftp_server.py

   FTP Server can also be started in the two following ways:
   • Execute the nohup command to start it in the backend process:

     nohup python ftp_server.py >> /dev/null 2>&1 &

   • Execute the screen command to run it at the backend (you need to install the screen tool):

     screen -dmS ftp
     screen -r ftp
     python ftp_server.py
     #Switch back to the main screen with shortcut key:
     Ctrl+A+D 

Stop Operation

• If FTP Server is running directly or running at the backend with the screen command, you can stop it with the shortcut key combination Ctrl+C.
• If FTP Server is started with the nohup command, you can stop it by the following way:

  ps -ef | grep python | grep ftp_server.py | grep -v grep | awk '{print $2}' | xargs -I{} kill {}

Configuration File

The configuration sample file of the FTP Server tool is conf/vsftpd.conf.example. Copy and name it vsftpd.conf, and then configure it as follows:

[COS_ACCOUNT_0]
cos_secretid = COS_SECRETID    # Replaced with your SECRETID
cos_secretkey = COS_SECRETKEY  # Replaced with your SECRETKEY
cos_bucket = examplebucket-1250000000
cos_region = region   # Replaced with your bucket region
cos_protocol = https
#cos_endpoint = region.myqcloud.com
home_dir = /home/user0
ftp_login_user_name=user0
ftp_login_user_password=pass0
authority=RW
delete_enable=true# true allows the FTP user to delete files by default; false prohibits the user to delete files.

[COS_ACCOUNT_1]
cos_secretid = COS_SECRETID    # Replaced with your SECRETID
cos_secretkey = COS_SECRETKEY  # Replaced with your SECRETKEY
cos_bucket = examplebucket-1250000000
cos_region = region   # Replaced with your bucket region
cos_protocol = https
#cos_endpoint = region.myqcloud.com
home_dir = /home/user1
ftp_login_user_name=user1
ftp_login_user_password=pass1
authority=RW
delete_enable=false

[NETWORK]
masquerade_address = XXX.XXX.XXX.XXX        # If FTP SERVER is located behind a gateway or NAT, you can assign the gateway's IP address or domain name to the FTP through this configuration item.
listen_port = 2121                       # Ftp Server's listening port (default: 2121). Note that the firewall needs this port opened.

passive_port = 60000,65535             # passive_port can be used to set the port range for the passive mode. The default range is (60000, 65535).

[FILE_OPTION]
# By default, the maximum size of a single file is 200G. We do not recommend going beyond the limit. 
single_file_max_size = 21474836480

[OPTIONAL]
# For the following settings, take the default settings unless otherwise needed. Fill in an appropriate integer if necessary.
min_part_size       = default
upload_thread_num   = default
max_connection_num  = 512
max_list_file       = 10000                #  The maximum number of files to be listed by `ls` command. It is not recommended to set it too big. Otherwise, high latency of `ls` command will occur.
log_level           = INFO                 # Set the log output level.
log_dir             = log                  # Set the directory to store logs. Default is the **log** directory under the **ftp server** directory.

• To bind each user to a unique bucket, add the section of [COS_ACCOUNT_X].
  The section for each COS_ACCOUNT_X is described as follows:

  • The username (ftp_login_user_name) and the home directory (home_dir) under each account must be unique, and the home directory must be a directory that exists in the system.

  • The number of users logging in to each COS FTP Server simultaneously cannot exceed 100.

  • endpoint and region will not take effect at the same time. To use the public cloud COS service, enter the region field correctly. The endpoint is commonly used in the privatized deployment environment. When both 'regionandendpointare entered,endpoint` will take precedence.

• The OPTIONAL part in the configuration file is used to adjust the upload performance for advanced users. You can obtain an optimal uploading speed by reasonably adjusting the part size and the number of concurrent upload threads based on the server performance. General users can keep the default settings without adjustment.
  Meanwhile, the limit option for the maximum number of connections is provided. If you do not want to set a limit to it, enter 0, meaning no limit to the maximum number of connections (a reasonable evaluation is required based on your server performance).

FAQs

If any error occurs or you have any question on the upload limit while using FTP Server, see FTP Server Tool FAQs.