Naming constraints

Edit online

Object storage types vary in their support for characters and length of file and path names. Avoid transfer problems by correctly naming storage elements.

Bucket and container names

Many object storage platforms, including Amazon S3, Google Cloud Storage, and Azure, require DNS-compliant bucket names, with extra constraints specific to certain platforms. The following are general naming rules. For more information about the bucket and container names, go to the storage platform documentation site.

Rules for DNS compliance:

  • Names must be 3 - 63 characters long.
  • Names must be a series of one or more labels, with next to labels separated by a period a (.).
  • Labels can contain lowercase letters, numbers, for Amazon S3 hyphens (-), and for Google Cloud Storage and Azure dashes (-).
  • Names must start and end with a lowercase letter or a number, and labels cannot start or end with a period. Periods might not be next to another period or a hyphen and nor can a hyphen be next to another hyphen. For example, (..), (--), (-.), and (.-) are not valid.
  • A container name must be a valid DNS name.
  • Labels cannot be formatted as IP addresses. For example, 192.00.00.20.

Object names, key names, and blob names

In general, object names, key names, and blob names must be a sequence of Unicode characters whose UTF-8 encoding is one to 1024 bytes long. This format applies to Amazon S3, Google Cloud Storage, and Azure. For more information about object, key, and blob names, go to the storage platform documentation site.

The following character sets are generally safe:

  • Alphanumeric characters: 0-9, a-z, A-Z
  • ! - _ . * ' ( )

The following characters might require special handling, such as URL encoding or referencing as HEX:

  • & $ @ = ; : + , ?
  • Spaces
  • ASCII character ranges 00-1F hex (0-31 decimal) and 7F (127 decimal).

Avoid the following characters:

  • \ { } ^ % ` [ ] " < > ~ # |
  • Nonprintable ASCII characters (128-255 decimal characters)
Convert specific characters in filepaths before transfer:
  • Add the <replace_illegal_chars> option that is in the <file_system/> section of the aspera.conf file to replace all instances of the characters that don't meet the naming requirement for the object storage.
  • For example, this replaces all instances of the characters: <replace_illegal_chars>\ ` { } ^ < > ~ # % " </replace_illegal_chars>.

Object storage platform Additional information on object name, key name, or blob name requirements
Amazon S3
  • The object key uniquely identifies the object in an Amazon S3 bucket.
Google Cloud Storage
  • Names cannot contain Carriage Return or Line Feed characters.
  • Avoid control characters that are not valid in XML 1.0 (#x7F -#x84 and #x86-#x9F).
Azure
  • Blob names are case-sensitive.
  • Avoid blob names that end with a period, a forward slash (/), or a sequence of the two.
  • Blob names cannot contain more than 254 path segments, where a path segment is the string between delimiter characters (such as the forward slash) that correspond to the name of a virtual directory.

Object metadata names (keys) and values

Object metadata is a set of name-value pairs. Users can often add customized metadata names, within the constraints of the object storage platform.

Object storage platform Object metadata name requirements
Amazon S3
  • Name-value pairs must conform to US-ASCII when using REST, and to UTF-8 when using SOAP or browser-based uploads (POST requests).
  • When using the REST API, user-defined metadata names must begin with x-amz-meta-.
  • PUT request headers are limited to 8 Kb, of which 2 Kb might be user-defined metadata. User-defined metadata is calculated as the total bytes of the UTF-8 encoded name and value.
Google Cloud Storage
  • Custom metadata names must begin with x-goog-meta-.
  • Each individual metadata entry is limited to 32768 bytes, and 512 Kb for the total metadata server.
Azure
  • Metadata names must follow the naming rules for C# identifiers.
  • The combined size of the name-value pair might not exceed 8 Kb.