Configuring an Amazon AWS CloudTrail log source that uses an S3 bucket with an SQS queue

If you want to collect AWS CloudTrail logs from multiple accounts or regions in an Amazon S3 bucket, configure a log source on the QRadar® Console so that Amazon AWS CloudTrail can communicate with QRadar by using the Amazon AWS S3 REST API protocol and a Simple Queue Service (SQS) queue.

About this task

Using the Amazon AWS S3 REST API protocol and a Simple Queue Service (SQS) queue instead of with a directory prefix has the following advantages:
  • You can use one log source for an S3 bucket, rather than one log source for each region and account.
  • There is a reduced chance of missing files because this method uses ObjectCreate notifications to determine when new files are ready.
  • It's easy to balance the load across multiple Event Collectors because the SQS queue supports connections from multiple clients
  • Unlike the directory prefix method, the SQS queue method does not require that the file names in the folders be in a string sorted in ascending order based on the full path. File names from custom applications don't always conform to this.
  • You can monitor the SQS queue and set up alerts if it gets over a certain number of records. These alerts provide information about whether QRadar is either falling behind or not collecting events.
  • You can use IAM Role authentication with SQS, which is Amazon's best practice for security.
  • Certificate handling is improved with the SQS method and does not require the downloading of certificates to the Event Collector.

Procedure

  1. Create the SQS queue that is used to receive ObjectCreated notifications.
  2. Create an Amazon AWS Identity and Access Management (IAM) user and then apply the AmazonS3ReadOnlyAccess policy.
  3. Configure the security credentials for your AWS user account.
  4. Add an Amazon AWS CloudTrail log source on the QRadar Console using an SQS queue