Using UPX to Sync Local Files to Upyun Object Storage

Last week, while U-File was open for registration, I registered an account and got 2 TB of storage with unlimited capacity for free. After waiting for quite a while, the application was finally approved. I logged into the console to take a look and found that it should be using Upyun Object Storage. I tested several tools provided by Upyun, and they all worked normally.

The only slight regret is that Upyun Storage does not yet support image pull. What's strange is that the object storage services of the three major providers—Alibaba Cloud OSS, Tencent COS, and Baidu BOS—as well as smaller providers such as Qiniu Cloud all support image pull, yet Upyun has not followed suit.

Since image pull is not supported, and I need U-File to accelerate my image hosting through image mirroring, I had to use another solution. Therefore, I found the UPX command-line tool in Upyun's documentation. It includes a sync command that can be used to synchronize a local path to Upyun Object Storage. Although it cannot implement image pull, cloud storage data can be synchronized with local data, which can also be regarded as another form of mirroring.

Project Introduction

UPX is a command-line cloud storage management tool designed specifically for developers. It supports file uploads and downloads, incremental file synchronization, directory creation and deletion, file deletion (including asynchronous batch file deletion), and more.

Open-source project: https://github.com/upyun/upx

For more details, see the project's README

This article mainly explains how to use upx on CentOS to synchronize local files to Upyun Storage.

Installing UPX

# 64 位系统下载
wget -O /usr/local/bin/upx http://collection.b0.upaiyun.com/softwares/upx/upx-linux-amd64-v0.2.6
# 32 位系统下载
wget -O /usr/local/bin/upx http://collection.b0.upaiyun.com/softwares/upx/upx-linux-386-v0.2.6

# 给予权限
chmod +x /usr/local/bin/upx

After installation, you can enter upx directly to check whether it was installed successfully.

Command Introduction

All commands support -h for viewing usage instructions.

Logging In to Upyun

Enter the upx login command, then enter the service name, operator, and operator password in sequence and press Enter to log in. Seeing Welcome to indicates that the login was successful.

You can enter upx info to view information. CurrentDir is the current directory, and Usage is the amount of storage already used.

UPX also supports logging in to multiple accounts. You can use upx sessions to view all login sessions and upx switch to switch between sessions. Since I do not have any additional accounts, this will not be demonstrated here.

Syncing Files

First, use cd to open the directory to be synchronized, then use upx sync to perform the synchronization.

For example, the upx sync 2020/ /2020/ command synchronizes the 2020 folder in the current directory to the 2020 folder under the root directory in Upyun Storage.

After synchronization is complete, the above prompt will appear. Connect to U-File using an FTP tool, and you can see that the 2020 directory has been completely synchronized.

This method can also be used to synchronize the static resources of other website applications.

WordPress

upx sync wp-content/uploads/ /wp-content/uploads/

Typecho

upx sync usr/uploads/ /usr/uploads/

Flarum

upx sync public/assets/ /public/assets/

Never synchronize the website's root directory directly, because it may contain configuration files such as WordPress's wp-config.php, which contains important information such as database credentials. After being uploaded to Upyun Storage, it could be downloaded by anyone, seriously compromising security.

Scheduled Synchronization

Currently, upx sync uses incremental synchronization and does not yet support real-time synchronization, so scheduled tasks are the only way to implement periodic synchronization.

Here, I use the scheduled tasks in the Baota Panel, which are somewhat easier to use than Crontab.

Note that when executing upx sync in a scheduled task, you need to change the synchronization directory so that it uses the system root directory.

I set it to synchronize once every minute. After one minute, I checked the log and could see that the synchronization was working normally.

+=============+=========+===========+================+===========+=============+
| SYNC_EXISTS | SYNC_OK | SYNC_FAIL | SYNC_NOT_FOUND | DELETE_OK | DELETE_FAIL |
+-------------+---------+-----------+----------------+-----------+-------------+
|         571 |       0 |         0 |              0 |         0 |           0 |
+-------------+---------+-----------+----------------+-----------+-------------+
----------------------------------------------------------------------------
★[2020-05-30 13:50:04] Successful
----------------------------------------------------------------------------