COSFS Tool

Last updated: 2019-11-04 18:00:53

PDF

FAQs About Features

How do I mount a bucket with a temporary key?

You can mount a bucket using a temporary key (STS) by following the steps below:

Step 1. Create a temporary key configuration file, such as /tmp/passwd-sts, for the COSFS command option "-opasswd-file=[path]" to specify the key configuration file. For more information on temporary keys, see Generating and Using Temporary Keys. Below is an example of temporary key configuration file:

COSAccessKeyId=AKIDXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX  # Below are the Id, Key, and Token fields of the temporary key
COSSecretKey=GYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
COSAccessToken=109dbb14ca0c30ef4b7e2fc9612f26788cadbfac3
COSAccessTokenExpire=2017-08-29T20:30:00  # Expiration time of the temporary token in GMT time, which should be in the same format as shown here

COSFS will determine whether it needs to reload the configuration from the key file based on the time configured in COSAccessTokenExpire.

To prevent key leakage, you should set the key file permission to 600 in COSFS by running the following command:

chmod 600 /tmp/passwd-sts

Step 2. Run the COSFS command. Specify the role as sts using the command option "-ocam_role=[role]" and specify the key file path using "-opasswd_file=[path]", as shown below:

cosfs examplebucket-1250000000 /mnt/cosfs -ourl=http://cos.ap-guangzhou.myqcloud.com -odbglevel=info -oallow_other -ocam_role=sts -opasswd_file=/tmp/passwd-sts

How do I check the mounting parameter options and version number provided by COSFS?

You can check the parameter options provided by COSFS using the cosfs --help command and check the COSFS version number using cosfs --version.

How do I view the logs generated by COSFS?

COSFS-generated logs are stored in /var/log/messages in CentOS and in /var/log/syslog in Ubuntu. If any error occurs when you use the service, please send us the logs for the corresponding time period.

How do I mount a directory in a bucket?

When running the mounting command, you can specify a directory in the bucket as shown below:

cosfs examplebucket-1250000000:/my-dir /mnt/cosfs -ourl=http://cos.ap-guangzhou.myqcloud.com -odbglevel=info

my-dir must begin with /.

In versions below v1.0.5, the mounting command is:

cosfs 1250000000:examplebucket:/my-dir /mnt/cosfs -ourl=http://cos.ap-guangzhou.myqcloud.com -odbglevel=info

How can a non-root user mount COSFS?

If you are not a root user, you are recommended to create a .passwd-cosfs file in the Home directory, set the permission to 600, and then mount COSFS using the mounting command. In addition, you can also specify the key file path using the "-opasswd_file=path" option and set the permission to 600.

Does COSFS support mounting over HTTPS?

Yes. Mounting methods over HTTP and HTTPS are as shown below:

-ourl=http://cos.ap-guangzhou.myqcloud.com
-ourl=https://cos.ap-guangzhou.myqcloud.com

If the version of the NSS library that libcurl depends on is v3.12.3 or higher (check the NSS version using curl -V), you can mount the bucket over HTTPS by run the following command:

echo "export NSS_STRICT_NOFORK=DISABLED" >> ~/.bashrc
source ~/.bashrc

How do I set auto-mounting at startup for COSFS?

Add the following to the /etc/fstab file. The "_netdev" option specifies that the command can be executed only after the network is ready:

cosfs#examplebucket-1250000000 /mnt/cosfs fuse _netdev,allow_other,url=http://cos.ap-guangzhou.myqcloud.com,dbglevel=info

How do I mount multiple buckets?

If you need to mount multiple buckets simultaneously, write a line for each bucket to be mounted in the /etc/passwd-cosfs configuration file, with the content of each line being in the same format of that for a single bucket, such as:

echo examplebucket-1250000000:AKIDXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:GYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY >> /etc/passwd-cosfs
echo log-1250000000:AKIDXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX:GYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY >> /etc/passwd-cosfs

How do I make a mounted directory accessible to other accounts on the server?

To make the mounted directory accessible to other accounts, specify -oallow_other during mounting.

Are there any restrictions on filenames in a COSFS-mounted directory?

In a COSFS-mounted directory, you can create a file whose name does not contain /. On a Unix-like system, the / character is a directory separator, so / is not allowed in filenames in a COSFS-mounted directory. In addition, when creating a file whose name contains special characters, make sure that such characters are not used by shell; otherwise, file creation may fail.

FAQs About Troubleshooting

What should I do if error message "unable to access MOUNTPOINT /path/to/mountpoint: Transport endpoint is not connected" is displayed and COSFS becomes inaccessible?

You can check whether the COSFS process exists using the ps ax|grep cosfs command. If the COSFS process is down due to faulty operations, run the following command to remount it:

umount -l /path/to/mnt_dir
cosfs examplebucket-1250000000:/my-dir /mnt/cosfs -ourl=http://cos.ap-guangzhou.myqcloud.com -odbglevel=info

If the COSFS process error is not caused by faulty operations, you can check whether the fuse version of the server is below v2.9.4. The libfuse on versions below v2.9.4 can cause exceptional exit of the COSFS process. In this case, it is recommended to update the fuse version or install the latest version of COSFS as described in Installation and Usage.

What should I do if the Content-Type of a file uploaded via COSFS is changed to "application/octet-stream"?

COSFS automatically sets the Content-Type of a file uploaded to COS by comparing the file's suffix against /etc/mime.types. If a Content-Type error occurs, check whether the configuration file exists on the system. For Ubuntu, you can add it using "sudo apt-get install mime-support". For CentOS, you can add it using "sudo yum install mailcap". You can also create this file manually, where one line is added for each file format, as shown below:

image/jpeg                                      jpg jpeg
image/jpm                                       jpm jpgm
image/jpx                                       jpx jpf

What should I do if "Bucket not exist" is displayed during mounting?

Check the parameter -ourl to ensure that the bucket part is not included in the URL. The correct format is:

-ourl=http://cos.ap-guangzhou.myqcloud.com

Why can't I write to files that I could write to previously?

Because of the adjustment to the COS authentication policy, using the COSFS tool below v1.0.0 will cause the policy check to fail. You can install the latest COSFS tool and mount it again.

What should I do if an error such as "Input/Output ERROR" occurs when using the COSFS tool?

Please follow the steps below to identify the cause of error.

  1. Check whether the server can access the COS domain name normally.
  2. Check whether the account is configured correctly.

After confirming that the above configurations are correct, open the /var/log/messages log file on the server and locate the log entry for s3fs, which can help you identify the cause of error. If the error persists, contact Tencent Cloud technical support team by submitting a ticket.

I've set auto-amounting at startup for COSFS using /etc/fstab, but the error "wrong fs type, bad option, bad superblock on cosfs" occurs when I run "mount -a". Why?

This error happens generally because the fuse library is missing on your server. It is recommended to install the fuse library by running the following command:

  • CentOS
    sudo yum install fuse
  • Ubuntu
    sudo apt-get install fuse

Is it normal to see a lot of 404 error codes in the system log /var/log/messages?

In the internal logic of COSFS, a HEAD request is used to determine whether the parent directory and the file exist, and the 404 error does not necessarily mean that the program is running erroneously.

Why does the file that I see in COS have a size of 0?

Generally, when you write data to COSFS, a file with a size of 0 will be first created in COS, and the data will be written to the local cache file.
During the write process, the result of the mount point ls command shows the change in file size. When the file is closed, COSFS will upload the data written to the local cache file to COS. If the upload fails, you may only see a file with a size of 0, in which case you can try copying the failed file again.

Are the files in the COSFS cache directory the same as those in COS? Can I use them directly?

No. The files in the cache directory are used to accelerate reads and writes in COSFS and may include only a part of the files in COS.

I have copied a file to COSFS using the rsync command and the progress has reached 100%, but I only see a temporary file on the server. Why?

The rsync command creates a temporary file in the mounted directory. The 100% progress only means that the temporary file has been 100% written locally. After that, it will be uploaded to COS and then renamed and copied.
Typically, it takes more time to copy data to a COSFS-mounted directory using the rsync command than using the cp command.

What should I do if COSFS uses up the disk capacity?

Uploads and downloads using COSFS involve disk file caching. When you upload or download a large file, if you don't specify the -oensure_diskfree=[size] parameter, the disk where the file is cached will be used up. You can use the -oensure_diskfree parameter to specify that when the remaining capacity of the disk is less than [size] MB, COSFS will minimize disk capacity usage (in MB). If you specify the -ouse_cache=[path] parameter, the cache file will be located in the path directory; otherwise, it will be in the /tmp directory.

For example, you can configure COSFS to reduce disk capacity usage when the remaining capacity is less than 10 GB by running the following command:

cosfs examplebucket-1250000000 /mnt/cosfs -ourl=http://cos.ap-guangzhou.myqcloud.com -odbglevel=info -oensure_diskfree=10240

What should I do if the error message "fuse: failed to open /dev/fuse: Operation not permitted" is displayed when I use docker to mount COSFS?

To start a docker image, you need to add the parameter --privileged.

Can I use a directory as a shared cache directory for multiple mount points?

It is not recommended that multiple mount points share a cache directory. The cache directory contains metadata used by COSFS, and sharing it may mess up the metadata.

What should I do if the error message "/bin/mount:unrecognized option --no-canonicalize" is displayed when I use COSFS for mounting?

Lower versions of the mounting tool do not support the --no-canonicalize option. Please update the tool. You are recommended to download v2.17 and then mount it again. The installation command is as follows:

tar -jxvf util-linux-ng-2.17.tar.bz2
cd util-linux-ng-2.17
./configure --prefix=/usr/local/util-linux-ng
make && make install
mv /bin/mount{,.off}
ln -s /usr/local/util-linux-ng/bin/mount /bin

What should I do if mounting fails?

Step 1. Check whether the mounting command and the key configuration file are correct and whether the COS service is accessible based on the log files and error messages.

Step 2. Check whether the server time is correct by running the date command. If it is incorrect, correct it using the date command and then mount it again, such as date -s '2014-12-25 12:34:56'.

Is it normal that the time of a mounted directory changes to 1970-01-01 08:00 when I use the ls -l --time-style=long-iso command?

Yes. After you unmount the mount point, the time of the mounted directory will return back to the time before mounting.

Can a mounted directory be non-empty?

You can mount a non-empty directory using the -ononempty parameter, but you are not recommended to do so because a problem may occur when the mount point and the original directory have files with the same path.

Why does it take the ls command so long to return when I run it in a COSFS directory?

If there are a lot of files in a mounted directory, executing the Is command requires a HEAD operation on each file in the directory, so it takes a lot of time to read the directory system before the command returns. Note: You are recommended not to enable IO hung which may result in unnecessary restarts.