How to Upload Data Into Nextcloud

To upload your data into Nextcloud you can rely on three different ways:

1. From the browser interface
2. From Coding
3. By using a file manager that supports WebDAV

Upload Data From The Browser Interface

You can easily upload your material from the Nextcloud browser interaface by clicking on "New" and "Upload files" or "Upload folders":

It is recommended to not upload files bigger than 10 GB within the same request. 

Upload Data From Coding

Users who are members of a Nextcloud group will see the corresponding folder displayed in the Jupyter file manager on the left-hand side of the screen. You can manage that folder just like any other folder you have: to upload files, simply drag them into it or use the “Upload” button. You can move or delete your files and folders directly from Coding.

Upload Data By Using a File Manager

To upload larger files/folders into Nextcloud, it is recommended to rely on file managers like WinSCP. These tools are also more user-friendly and easy to use.

The WebDAV connection option is required for the tool to connect to Nextcloud correctly.

The following example uses WinSCP.

In the 'Username' and 'Password' fields, enter your EO account username and the Nextcloud app password respectively. Finally, go to 'Advanced Site Settings' and 'Directory', as shown below, and enter the same path as shown in the image below, replacing YOUR_USERNAME with your actual username.

Upload Data with Rclone

The most reliable way to upload very large datasets into Nextcloud is via Rclone. In the following example you will see how to install and configure Rclone into your local machine.

It is recommended that you use Rclone in case you need to transfer very large files (hundreds of Gigabytes or Terabytes), as Rclone allows you to resume the upload even following any network interruptions.

1. Install Rclone

To install Rclone, just open the terminal into your local machine and type:

winget install Rclone.Rclone

Once it is installed, you can verify the installation with:

rclone version

You may need to restart your terminal after the installation.

2. Configure Rclone with your credentials

At this point, you need to configure Rclone with the Nextcloud endpoint and your credentials (username and App Password). Run the following command and substitute YOUR_USERNAME with your actual Nextcloud username and YOUR_APP_PASSWORD with your App Password:

rclone config create nextcloud webdav vendor=nextcloud url="https://storage.biomass.pal.maap.eo.esa.int/remote.php/dav/files/YOUR_USERNAME/" user="YOUR_USERNAME" pass="YOUR_APP_PASSWORD"

3. Upload your files

To upload your files, just run the following command, replacing path/to/your/local/file with your dataset path and TargetFolder with the name of the target Nextcloud folder in which to upload your files:

rclone copy "path/to/your/local/file" nextcloud:TargetFolder --transfers 4 --checkers 4 --webdav-nextcloud-chunk-size 100M --fast-list --checksum -P

4. Verify the correct upload

The post-transfer check can be performed by running the following command:

rclone check "path/to/your/local/file" nextcloud:TargetFolder --download