Generic S3 connection

To access your data from a storage service that is compatible with the Amazon S3 API, create a connection asset for it.

Create a Generic S3 connection

To create the connection asset, you need these connection details:

  • Endpoint URL: The endpoint URL to access to S3
  • Bucket(optional): The name of the bucket that contains the files
  • Region (optional): S3 region. Specify a region that matches the regional endpoint.
  • Access key: The access key (username) that authorizes access to S3
  • Secret key: The password associated with the Access key ID that authorizes access to S3
  • The SSL certificate of the trusted host. The certificate is required when the host certificate is not signed by a known certificate authority.
  • Disable chunked encoding: Select if the storage does not support chunked encoding.
  • Enable global bucket access: Consult the documentation for your S3 data source for whether to select this property.
  • Enable path style access: Consult the documentation for your S3 data source for whether to select this property.

For Credentials and Certificates, you can use secrets if a vault is configured for the platform and the service supports vaults. For information, see Using secrets from vaults in connections.

Federal Information Processing Standards (FIPS) compliance

This connection can be used on a FIPS-enabled cluster (FIPS tolerant); however, it is not FIPS-compliant.

Generic S3 connection setup

For setup information, consult the documentation of the S3-compatible data source that you are connecting to.

Supported file types

The Generic S3 connection supports these file types:  Avro, CSV, delimited text, Excel, JSON, ORC, Parquet, SAS, SAV, SHP, and XML.

Table formats

In addition to Flat file, the Generic S3 connection supports these Data Lake table formats: Delta Lake and Iceberg.

Learn more

Related connection: Amazon S3 connection