CloudStor to/from HPC¶
NOTE: Cloudstor has now shutdown. This document is no longer useful.
Hi Kay
Yes you can use the command line using “rclone”. Usage looks simple but initial setup is not pretty: https://support.aarnet.edu.au/hc/en-us/articles/115007168507
I have not yet tried this myself. I have added it to my list of things to try as it might be useful for others. Particularly if I document it better than what Aarnet has.
You can get rclone from here: https://github.com/rclone/rclone Install using the instructions for the “Linux installation from precompiled binary”. But you will need to install it into your own home directory as you cannot use sudo.
If you get it working do let me know. It might be a while until I get a chance to try this myself.
Go to https://cloudstor.aarnet.edu.au/plus/settings/personal Settings / General Sync Password [ New password ] [ Change Password ]
Have installed rclone on the cluster from rpm rclone-v1.48.0-linux-amd64.rpm
$ rclone config
No remotes found - make a new one
n) New remote
n/s/q> n
name> CloudStor
Storage> 24
** See help for webdav backend at: https://rclone.org/webdav/ **
URL of http host to connect to
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
1 Connect to example.com "https://example.com"
url>
<https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/>
Name of the Webdav site/service/software you are using
Owncloud
user> user@institute.edu.au
950654
Remote config
--------------------
[CloudStor]
type = webdav
url = https://cloudstor.aarnet.edu.au/plus/remote.php/webdav/
vendor = owncloud
user = 950654
pass = *** ENCRYPTED ***
--------------------
y) Yes this is OK
Current remotes:
Name Type
==== ====
CloudStor webdav
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q
~/$
Test
$ rclone copy --progress --transfers 1 TEST.txt CloudStor:/
$ rclone ls CloudStor:/
ermdc13 mlake/$ rclone copy --progress --transfers 2 CloudStor:/Zebedee Zebedee
Transferred: 76.238M / 76.238 MBytes, 100%, 5.449 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 12 / 12, 100%
Elapsed time: 13.9s
ermdc13 mlake/$
ermdc13 mlake/$ rclone copy --progress --transfers 4 CloudStor:/Zebedee Zebedee
Transferred: 76.238M / 76.238 MBytes, 100%, 7.482 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 12 / 12, 100%
Elapsed time: 10.1s
ermdc13 mlake/$ rclone copy --progress --transfers 8 CloudStor:/Zebedee Zebedee
Transferred: 76.238M / 76.238 MBytes, 100%, 8.070 MBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 12 / 12, 100%
Elapsed time: 9.4s
.config/rclone/rclone.conf