Using S3 proxy to access S3 and S3 compatible buckets

External applications and query engines can access the S3 and S3 compatible buckets that are managed by watsonx.data through S3 proxy.

watsonx.data on Red Hat® OpenShift®

watsonx.data SaaS on AWS

To access the S3 and S3 compatible buckets:

Get the S3 proxy end point from the watsonx.data information window. Click the i icon on the home page to open the information window.
Replace the S3 endpoint with the S3 proxy endpoint.
```
<cas endpoint>/cas/v1/proxy
```
Replace the access key with the encoded value as follows:
```
base64{<instanceid>|ZenAPIkey base64{username:<apikey>}}
```
Note: To get the Base64 encoded string, use one of the following commands:
- ```
printf "username:<apikey>" | base64
```
- ```
echo -n "username:<apikey>" | base64
```

Java code example to use S3 proxy:

        String bucketName = "bucket1";
        String keyName = "folder1/file1";
        # replace the target object store endpoint with the CAS proxy endpoint
        String endpoint = "<cas endpoint get from About page>/cas/v1/proxy";
        /** Replace the Access Key with watsonx.data user name and API key following the below base64 encoded method.
        * CPD base64{<instanceid>|ZenAPIkey base64{username:<apikey>}}
        * SaaS base64{<crn>|Basic base64{ibmlhapikey_<user_id>:<IAM_APIKEY>}}
        */
        String accessKey = "encoded value";
        String secretKey = "any string";

        BasicAWSCredentials cos_cred = new BasicAWSCredentials(accessKey, secretKey);
        EndpointConfiguration cosEndPoint = new EndpointConfiguration(endpoint, "us-east");
        AmazonS3 s3Client = AmazonS3ClientBuilder.standard().withPathStyleAccessEnabled(true)
                .withCredentials(new AWSStaticCredentialsProvider(cos_cred))
                .withEndpointConfiguration(cosEndPoint).build();
        GetObject.GetObjectTest(s3Client, bucketName, keyName);

Note: For information about S3 REST API permissions, see S3 REST API permissions.

Note: To use S3 proxy externally, you have to import the CA certificates and add them in your cluster. To get the certificates, run:

echo QUIT | openssl s_client -showcerts -connect <cas host>:443 | awk '/-----BEGIN CERTIFICATE-----/ {p=1}; p; /-----END CERTIFICATE-----/ {p=0}' > cas.cert

To import the certificates to your local Java truststore, run:

sudo keytool -import -trustcacerts -cacerts -storepass changeit -noprompt -alias cas-cert -file ./cas.cert