S3 API operations

IBM Spectrum® Protect supports basic S3 application programming interface (API) operations for data backup and retrieval. However, some S3 API client function calls are not supported by IBM Spectrum Protect.

The IBM Spectrum Protect S3 API is packaged with the IBM Spectrum Protect server. The IBM Spectrum Protect S3 API enables an S3 client application to use containers for storage management. The API includes function calls that you can use in an application to run several operations.

IBM Spectrum Protect implements parts of the S3 protocol as summarized in Table 1. IBM Spectrum Protect follows the Amazon Web Services (AWS) implementation as closely as possible. However, the S3 protocol is not fully standardized and limitations exist. One example is the HTTP headers that are used in requests and responses. The HTTP protocol specifies that the header keys are not case-sensitive. One S3 client might send ETag as a header key, while another S3 client sends Etag. Case inconsistency in the headers is a known S3 client issue.

Tip: Use an existing S3 client software development kit (SDK). You can use an S3 SDK to resolve heading and other issues and to manage the protocol. If you use an application and do not know how it is implementing the object client protocol, try the operation with the AWS S3 Command-Line Interface (CLI) commands or another well-tested application.

Table 1. S3 REST API operations, parameters, and guidelines
S3 REST API operation	Function	Supported parameters	Requirements
Put Bucket	Creates a file space for the object client node on the IBM Spectrum Protect server. This file space acts as a bucket for all client objects.	Bucket	The bucket name must comply with the Amazon S3 bucket name guidelines, except for requirement that the name must be globally unique. The bucket name must be unique only within the client node. The Amazon S3 bucket rules are more restrictive than the IBM Spectrum Protect server's filespace name restrictions. For more information about Amazon S3 bucket rules, see Bucket restrictions and limitations. Bucket names must follow these guidelines: The name must be 3 - 63 characters long. Only lowercase letters (a-z), numbers, periods, and hyphens are allowed in the name. The name must start and end with a letter or number. The name cannot be formatted as an IP address. Restriction: The character set is restricted for bucket names to support maximum client compatibility. However, UTF-8 characters are permitted in object keys.
Delete Bucket	Deletes a bucket from IBM Spectrum Protect. In IBM Spectrum Protect, buckets are represented by file spaces.	Bucket	Before the bucket can be deleted, ensure that the bucket is empty. Deleting a bucket starts a DELETE FILESPACE operation on the IBM Spectrum Protect server. The DELETE FILESPACE command that is started on the server is an asynchronous operation and might take some time to complete. After the DELETE FILESPACE operation is started, the DeleteBucket operation requests the bucket name. The requirement for the bucket to be empty is only restricted if the DELETE bucket function is called through the S3 interface. If you issue the DELETE FILESPACE command directly on the server, no such restriction applies.
List Buckets	Provides a list of buckets for this object client node. In IBM Spectrum Protect, buckets are represented by file spaces.	No parameters are available.	IBM Spectrum Protect does not save the filespace creation date, so the creation date of a bucket is not an accurate timestamp.
Put Object	Creates an object in IBM Spectrum Protect.	Bucket Key Content-MD5 Content-Length x-amz-storage-class	Object names: An object key must follow the object naming guidelines in Object keys. Object keys must be 1 - 1024 characters long and consist of valid UTF-8 characters. As noted in the guidelines, several characters must be avoided. These characters might be acceptable as part of an object key. However, to avoid possible issues with applications, do not use the characters in object keys or elsewhere. Object sizes: The S3 standard limits single object PUT requests to 5 GB. Larger requests are rejected. Each part of a multipart object is limited to 5 GB. Although a size of 5 GB is allowed per single object in PUT requests, use the multipart object operations for objects that exceed 100 MB. Only the STANDARD storage class is supported.
Get Object	Retrieves an object or a range of data within the object from IBM Spectrum Protect.	Bucket Key Range If-Match If-Modified-Since If-None-Match If-Unmodified-Since	Server performance can be affected when multiple small GET requests are issued. The optimal GET request size is 5 MB or larger. For GET requests that exceed 5 GB, data is sent over a single IBM Spectrum Protect session. In this situation, clients might not get data quickly enough. To avoid this issue, consider requesting multiple ranges of the same object in parallel.
Head Object	Returns information about a specified object.	Bucket Key If-Match If-None-Match If-Modified-Since If-Unmodified-Since	See the list of supported response elements in List objects.
Delete Object(s)	Deletes a single object or multiple objects.	Bucket Key
List Objects	Retrieves a list of objects or directories for a specified bucket.	Bucket Continuation-Token Delimiter Prefix Start-After Supported response elements: Name CommonPrefixes Contents ContinuationToken Delimiter IsTruncated KeyCount MaxKeys NextContinuation Token Prefix StartAfter	A bucket corresponds to a file space on the server. A delimiter can be used to specify a character to group keys as defined in the S3 API. The prefix is used to search for object agent files under the current node definition with the specified prefix. When used with the delimiter, this operation can generate a hierarchical representation of a directory structure.
Create Multipart Upload	Initiates the construction of a multipart object.	Bucket Key x-am-storage-class	Multipart object operations are used to upload larger objects. Objects that are larger than 100 MB must be uploaded by using multipart object operations. In this way, the object can be uploaded in parallel streams. Objects as large as 5 TB can be created within a multipart object. A single part can be up to 5 GB. Parts must be at least 5 MB except for the last part. The S3 protocol allows for up to 10,000 parts. In IBM Spectrum Protect, each part is stored as an object, and a head object is added. These objects are then grouped with the head object. For example, a multipart object that was backed up with 10 parts appears as 11 objects in IBM Spectrum Protect queries such as the QUERY OCCUPANCY command. After the object is committed by using the S3 protocol, it appears as a single object in the S3 APIs, but you might see multiple objects in IBM Spectrum Protect. Important: The S3 client must either commit or abort a multipart object or the operation might fail to complete processing: If the multipart object is not committed, the object is not viewed as an object by the S3 APIs and is not processed. If the multipart object is not aborted, the spaces that are used by the parts continue to be used and the parts are not deleted. Ensure that no multipart objects are left either uncommitted or unaborted. All rules in Object names apply. Only the STANDARD storage class is supported.
Upload Part	Uploads one part of a multipart object.	Bucket Key partNumber UploadID Content-Length Content-MD5
Complete Multipart Upload	Completes a multipart upload by assembling previously uploaded parts.	Bucket Key UploadID
Abort Multipart Upload	Halts a multipart upload after it is started. All parts of the specified non-committed object are deleted.	Bucket Key UploadID
List Parts	Lists the parts that were uploaded for a specific multipart upload.	Bucket Key UploadID Supported response elements: Bucket Key Part StorageClass UploadId	A bucket corresponds to a file space on the server. Use the key and upload ID that are issued in the List Parts operation to start a multipart upload.
List Multipart Uploads	Lists multipart uploads for a specified bucket.	Bucket Delimiter Key-Marker Max-Uploads Prefix Supported response elements: Bucket Delimiter IsTruncated KeyMarker MaxUploads NextKeyMarker Prefix Upload	The operation returns all objects that match a specified prefix with an optional delimiter. A bucket corresponds to a file space on the server. The listing includes uploads that are created but not committed or aborted.