Use S3-compatible tools with Object Storage
Create an S3 credential for Object Storage, configure s3cmd or an SDK with it, and manage the objects of your buckets from the tools you already run.
Object Storage answers the S3 protocol, so you manage objects with the tools you already run, such as s3cmd and the boto3 library. Each tool authenticates with an S3 credential, and then lists, uploads, downloads, and deletes the objects of the buckets that credential reaches.
Prerequisites
- A bucket for the credential to reach. Refer to Create a bucket.
- Access to Azion Console, for the Console procedure. Refer to Access Azion Console.
- A personal token, for the API procedure.
- An S3 client: s3cmd for the command-line procedures, or Python with the boto3 library for the SDK example.
Create a credential in Azion Console
A credential carries a set of capabilities, and each capability is one checkbox on the create form.
| Console label | API value | What the credential may do |
|---|---|---|
| List Files | listFiles | View the files stored in the bucket: ListObjects and ListObjectsV2. |
| Read Files | readFiles | Access and download files: GetObject and HeadObject. |
| Write Files | writeFiles | Upload or modify files: PutObject, CopyObject, CreateMultipartUpload, UploadPart, and CompleteMultipartUpload. |
| Delete Files | deleteFiles | Remove files: DeleteObject, DeleteObjects, and AbortMultipartUpload. It requires Write Files. |
| List All Bucket Names | listAllBucketNames | Retrieve the names of the available buckets: ListBuckets. |
| List Buckets | listBuckets | View bucket details, including metadata and configurations: HeadBucket, ListMultipartUploads, and ListParts. |
To create the credential:
Access Azion Console > Object Storage.
Select Create Credential, the control that configures a new credential to reach your Object Storage bucket.
Under General, in Name, enter a name that says what uses the credential, such as ci-upload.
Enter the date on which the credential stops working.
Select one checkbox per operation the credential allows, from the six in the table above.
Azion Console reports “Credential has been created” and shows the secret key once, in a copy dialog. Copy the secret key before you close the dialog.
The credential exists in your account, and you hold the only copy of its secret key.
Create a credential using the API
POST /v4/workspace/storage/credentials creates the credential and returns the key pair. The body accepts four fields:
| Field | Required | What it sets |
|---|---|---|
name | Yes | Identifies the credential in your account. |
capabilities | Yes | The operations the credential allows: listFiles, readFiles, writeFiles, deleteFiles, listAllBucketNames, and listBuckets. |
buckets | No | An array of bucket names the credential reaches. Omit the field and the credential reaches every bucket in the account. |
expiration_date | No | The moment the credential stops working, in UTC ISO 8601. The field is null when you omit it. |
The field is the plural buckets, and it takes an array. A singular bucket field is accepted and ignored, and the credential it creates reaches the whole account. For the S3 operations each capability covers, refer to S3 compatibility.
To create the credential:
Replace [TOKEN VALUE] with your personal token, and my-static-bucket with the bucket the credential reaches:
A 201 carries the credential, with the access key and the secret key your S3 client needs:
Copy data.access_key and data.secret_key into the tool you configure next.
The credential reaches the buckets in data.buckets, and DELETE /v4/workspace/storage/credentials/{id} removes it.
Configure s3cmd
s3cmd is a command-line tool for S3 and other cloud storage services, and s3cmd --configure asks for each connection setting in turn. To configure it:
Install the s3cmd package on the machine that manages the objects.
Confirm that your system PATH holds s3cmd, so the command runs from any directory.
Enter the access_key and the secret_key of the credential you created.
Enter us-east-005, the region the Object Storage endpoint is named for.
Enter s3.us-east-005.azionstorage.net.
Enter %(bucket).s3.us-east-005.azionstorage.net, which addresses each bucket as a subdomain of the endpoint.
Enter an encryption password and the path to your GPG program. Gnu Privacy Guard is free and open-source encryption software, and it adds a layer of cryptographic privacy and authentication to the S3 communication.
Enter true.
Leave the answer blank when your machine connects directly.
s3cmd signs a request with the keys you entered and reports the result:
s3cmd now reaches your buckets with that credential. You can save the answers into an .s3cfg file, and run s3cmd --configure again to change them.
Manage objects with s3cmd
Every command addresses a bucket or an object by its s3:// URL.
| Command | What it does |
|---|---|
s3cmd ls | Lists every bucket in the account. |
s3cmd ls s3://my-bucket | Lists the contents of the bucket. |
s3cmd put file.xml s3://my-bucket/file.xml | Uploads a file into the bucket. |
s3cmd get s3://my-bucket/file.xml file-2.xml | Downloads the object into a local file, which you compare with the original. |
s3cmd del s3://my-bucket/addrbook.xml | Deletes an object. |
s3cmd info s3://my-bucket/file.xml | Returns the metadata of an object without downloading it. |
Run s3cmd --help to list every command the tool accepts.
Connect with an SDK
Any S3 SDK reaches the same endpoint with the same credential. boto3 is the library most Python projects use for S3 storage, and its client takes the endpoint, the region, and the key pair:
The script prints one line per object stored in the bucket:
The endpoint signs requests with SigV4, and it also answers a path-style address, s3.us-east-005.azionstorage.net/<bucket>/<key>. It serves object management, not delivery to end users: an application backed by a connector is what answers a request from the internet.