Overview of Depositing with the Command Line
Vault makes it easy to deposit data via four different methods. This article provides an overview of depositing data with the Command line.
In this article
- About the Command Line
- About Rclone with Vault Support
- How to Download Rclone with Vault Support
- Related content
About the Command Line
The command-line utility for programmatically depositing content into Vault has been developed using a fork of the popular cloud storage command line tool Rclone.
With this tool, you can perform the following tasks with Vault:
- Sync a directory
- Create a collection
- Upload files (from local disk or other cloud providers)
- Upload folders (from local disk or other cloud providers)
- Download files
- Download folders
- Download a collection
- List collection
- List folders
About Rclone with Vault Support
Prerequisites:
- An active Vault account
- A macOS (both classic Intel-based Macs and the newer Apple Silicon Macs), Windows, or Linux machine
- Basic familiarity with the command line
Known Limitations:
The command line utility currently has some known limitations:
- Not all features of Rclone are supported - only strict command-line features are currently supported.
- Example of an unsupported Rclone feature: mounting storage systems
- Free disk space is required when performing cloud-to-cloud transfers (e.x. copying files from AWS S3 into Vault) because files are temporarily stored on the machine where Rclone runs.
- Uploaded files are currently not mutable, meaning if a file has been uploaded, it cannot be reuploaded with the same name and different content or updated content.
How to Download Rclone with Vault Support
To deposit content with the command line tool, you first need to download and install Rclone with Vault support.
How to Download and Install Rclone on macOS
We currently support macOS (both classic Intel-based Macs and the newer Apple Silicon Macs). Follow each step below to get started, or check out Vault’s installation and usage guide for Rclone with Vault support on GitHub.
Step 1: Download and install Rclone with Vault Support
We suggest you use Terminal.app (or any other terminal emulator) and curl or wget to download the binary. After you download curl or wget the file needs to be made executable with chmod.
Each command is separated by a $ symbol in the instructions. Make sure you run each command separately when installing Rclone with Vault Support, and delete the $ symbol before running each command.
Intel-based Macs
|
$ curl --output rclone -L https://github.com/internetarchive/rclone/releases/download/v1.62.2-vault-20231002151045-374e2c594/rclone_1.62.2-vault-20231002151045-374e2c594_Darwin_x86_64 $ chmod +x rclone |
Apple Silicon Macs
|
$ curl --output rclone -L https://github.com/internetarchive/rclone/releases/download/v1.62.2-vault-20231002151045-374e2c594/rclone_1.62.2-vault-20231002151045-374e2c594_Darwin_arm64 $ chmod +x rclone |
Step 2: Check if the Install was Successful
To check if the binary works, you can follow these instructions to print out version information about the program. To run the command you can stay in the directory where the binary is located and run it from there:
| $ ./rclone version |
Step 3: Configure Rclone Vault Backend
Prerequisite: Vault credentials are required to complete this step.
To access Vault, Rclone will need to know your Vault credentials and the Vault API endpoint.
Replace alice and secret to match your credentials:
| $ ./rclone config create vault vault username=alice password=secret endpoint=https://vault.archive-it.org/api |
This will create a configuration file (or extend it, if if already exists) and will add a section for Vault. Rclone uses a single configuration file, located by default under your HOME directory.
You can always ask Rclone to show you where your configuration file is located with:
| $ ./rclone config file |
Step 4: Run a test
To check if everything works, you can run the following to display information about the configured Vault user you set up in Step 3.
| $ ./rclone config userinfo vault: |
If you see a similar output to the one below, congratulations - your Rclone with Vault support is now ready to use!
| $ ./rclone config userinfo vault: DefaultFixityFrequency: TWICE_YEARLY FirstName: LastLogin: 2022-07-02T00:29:20.364793Z LastName: Organization: SuperOrg Plan: Basic QuotaBytes: 1099511627776 Username: Roosevelt |
Step 5: Use Rclone with Vault Support to Deposit Content
- Once you have successfully completed step 4, you now have access to Vault’s API and can perform any task you like to upload, download, or sync content to Vault.
- Available Tasks:
If you are interested in further resources, check out:
How to Download and Install Rclone on Windows
Follow each step below to get started, or check out Vault’s installation and usage guide for Rclone with Vault support on GitHub.
Each command is separated by a $ symbol in the instructions. Make sure you run each command separately when installing Rclone with Vault Support, and delete the $ symbol before running each command.
Step 1: Download and install Rclone with Vault Support
Download the latest binary: Rclone with Vault for Windows x64 64bit: https://github.com/internetarchive/rclone/releases/download/v1.62.2-vault-20231002151045-374e2c594/rclone_1.62.2-vault-20231002151045-374e2c594_Windows_x86_64.exe
In (the rare) case you have an ARM-based computer running Windows, please download: https://github.com/internetarchive/rclone/releases/download/v1.62.2-vault-20231002151045-374e2c594/rclone_1.62.2-vault-20231002151045-374e2c594_Windows_arm64.exe.
Important: We do not sign the executables, which is why Windows will issue warnings about an untrusted source and will suggest that you delete the file.
To ensure the downloaded file is the same as the one we published, you can compare the checksum of the file you downloaded against a list of checksums we publish alongside each release. The filename ends with _checksums.txt - on Windows you can generate various hash sums of a file with certutil, a pre-installed command line utility; examples.
Once downloaded, it may be convenient to rename the file. You can do this in your Explorer or with the Command Prompt with the ren command. Please make sure the file has an .exe extension, otherwise, Windows may not recognize it. An error you may see would be rclone is not a recognized internal or external command).
Step 2: Check if the Install was Successful
To check if the binary works, follow these instructions to print out version information about the program. To run the command you can stay in the directory where the binary is located and run it from there:
| $ rclone.exe version |
Step 3: Configure Rclone Vault Backend
Prerequisite: Vault credentials are required to complete this step.
To access Vault, Rclone will need to know your Vault credentials and the Vault API endpoint.
Replace alice and secret to match your credentials:
| $ rclone.exe config create vault vault username=alice password=secret endpoint=https://vault.archive-it.org/api |
This will create a configuration file (or extend it if it already exists) and add a section for Vault. Rclone uses a single configuration file located by default under your HOME directory.
You can always ask Rclone to show you where your configuration file is located with the following:
| $ rclone.exe config file |
Step 4: Run a test
To check if everything works, you can run the following to display information about the configured Vault user you set up in Step 3.
| $ rclone.exe config userinfo vault: |
If you see a similar output to the one below, congratulations - your Rclone with Vault support is now ready to use!
| $ rclone.exe config userinfo vault: DefaultFixityFrequency: TWICE_YEARLY FirstName: LastLogin: 2022-07-02T00:29:20.364793Z LastName: Organization: SuperOrg Plan: Basic QuotaBytes: 1099511627776 Username: Roosevelt |
Step 5: Use Rclone with Vault Support to Deposit Content
- Once you have completed step 4, you can access Vault’s API and perform any task you like to upload, download, or sync content to Vault.
- Available Tasks:
If you are interested in further resources, check out:
How to Download and Install on Linux
The instructions separate each command by a $ symbol. When installing Rclone with Vault Support, make sure you run each command separately and delete the $ symbol before running each command.
Step 1: Download and install Rclone with Vault Support
Download the latest release depending on your architecture:
For convenience, you can rename the downloaded file to e.g. rclone with your File Explorer or the mv command.
Finally, set executable permissions:
| $ chmod +x rclone |
Step 2: Check if the Install was Successful
To check if the binary works, follow these instructions to print out version information about the program. To run the command you can stay in the directory where the binary is located and run it from there:
| $ ./rclone version |
Step 3: Configure Rclone Vault Backend
Prerequisite: Vault credentials are required to complete this step.
To access Vault, Rclone will need to know your Vault credentials and the Vault API endpoint.
Replace alice and secret to match your credentials:
| $ ./rclone config create vault vault username=alice password=secret endpoint=https://vault.archive-it.org/api |
This will create a configuration file (or extend it, if if already exists) and will add a section for Vault. Rclone uses a single configuration file, located by default under your HOME directory.
You can always ask Rclone to show you where your configuration file is located with:
| $ ./rclone config file |
Step 4: Run a test
To check if everything works, you can run the following to display information about the configured Vault user you set up in Step 3.
| $ ./rclone config userinfo vault: |
If you see a similar output to the one below, congratulations - your Rclone with Vault support is now ready to use!
| $ ./rclone config userinfo vault: DefaultFixityFrequency: TWICE_YEARLY FirstName: LastLogin: 2022-07-02T00:29:20.364793Z LastName: Organization: SuperOrg Plan: Basic QuotaBytes: 1099511627776 Username: Roosevelt |
Step 5: Use Rclone with Vault Support to Deposit Content
- Once you have successfully completed step 4, you now have access to Vault’s API and can perform any task you like to upload, download, or sync content to Vault.
- Available Tasks:
If you are interested in further resources, check out:
Related content:
Last updated October 29, 2023.
Comments
Please sign in to leave a comment.