Amazon Web Services protocol configuration options

Access Key ID/Secret Key

Standard authentication that can be used from anywhere.

EC2 Instance IAM Role

If your QRadar managed host is running in an AWS EC2 instance, choosing this option uses the IAM role from the metadata that is assigned to the instance for authentication. No keys are required. This method works only for managed hosts that are running within an AWS EC2 container.

The Access Key ID that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Access Key parameter is displayed.

The Secret Key that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Secret Key parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role ARN parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role Session Name parameter is displayed.

Assume Role External ID is an optional identifier that is required to assume a role in a different account.

If the account administrator, to which the role belongs, provides you with an external ID, then insert that value in the Assume Role External ID parameter.

This value can either be a string, a passphrase, a GUID, or an account number. For more information, see AWS documentation Using an external ID for third-party access.

The Kinesis Data Stream from which to consume data.

Initial Position in Stream

This option controls which data to pull on a newly configured log source. Select Latest to pull the latest data that is available. Select Trim Horizon to pull the oldest data that is available.

Kinesis Worker Thread Count

The number of worker threads to use for Kinesis Data Stream processing. Each worker thread can process approximately 10000 - 20000 events per second depending on record size and system load. If your log source is not able to process the new data in the stream, you can increase the number of threads here to a maximum of 16. The allowed range is 1 - 16. The default value is 2.

Checkpoint Interval

The interval (in seconds) at which to checkpoint data sequence numbers. Each record from a shard in a Kinesis Data Stream has a sequence number. Checkpointing your position allows this shard to resume processing at the same point if processing fails or a service restarts. A more frequent interval reduces data duplication but increases Amazon Dynamo DB usage. The allowed range is 1 - 3600 seconds. The default is 10 seconds.

Kinesis Application

Leave this option blank to have this log source consume data from all available shards in the Kinesis Data Stream. To have multiple log sources on multiple event processors scale log consumption without loss or duplication, use a common Kinesis Application across those log sources (Example: ProdKinesisConsumers).

Partition

Select this option to collect data from a specific partition in the Kinesis Data Stream by specifying a partition name.

Forwards only the original event that was added to the Kinesis Data Stream.

Kinesis logs wrap the events that they receive with extra metadata. Select this option if you want only the original event that was sent to AWS without the additional stream metadata through Kinesis.

The original event is the value for the message key that is extracted from the Kinesis log. The following Kinesis logs event example shows the original event that is extracted from the Kinesis log in highlighted text:

{"owner":"123456789012","subscriptionFilters":["allEvents"],
"logEvents":[{"id":"35093963143971327215510178578576502306458824699048362100",
"message":"{\"eventVersion\":\"1.05\",\"userIdentity\":{\"type\":\"AssumedRole\",
\"principalId\":\"ARO1GH58EM3ESYDW3XHP6:test_session\",\"arn\":\"arn:aws:sts::123456789012:assumed-role\/CVDevABRoleToBeAssumed\/test_visibility_session\",
\"accountId\":\"123456789012\",\"accessKeyId\":\"ASIAXXXXXXXXXXXXXXXX\",\"sessionContext\":{\"sessionIssuer\":{\"type\":\"Role\",\"principalId\":\"AROAXXXXXXXXXXXXXXXXX\",
\"arn\":\"arn:aws:iam::123456789012:role\/CVDevABRoleToBeAssumed\",\"accountId\":\"123456789012\",
\"userName\":\"CVDevABRoleToBeAssumed\"},\"webIdFederationData\":{},\"attributes\":{\"mfaAuthenticated\":\"false\",
\"creationDate\":\"2019-11-13T17:01:54Z\"}}},\"eventTime\":\"2019-11-13T17:43:18Z\",\"eventSource\":\"cloudtrail.amazonaws.com\",
\"eventName\":\"DescribeTrails\",\"awsRegion\":\"ap-northeast-1\",\"sourceIPAddress\":\"192.0.2.1\",\"requestParameters\":null,\"responseElements\":null,
\"requestID\":\"41e62e80-b15d-4e3f-9b7e-b309084dc092\",\"eventID\":\"904b3fda-8e48-46c0-a923-f1bb2b7a2f2a\",\"readOnly\":true,\"eventType\":\"AwsApiCall\",
\"recipientAccountId\":\"123456789012\"}","timestamp":1573667733143}],"messageType":"DATA_MESSAGE","logGroup":"CloudTrail\/DefaultLogGroup","logStream":"123456789012_CloudTrail_us-east-2_2"}

Select this option for the collected events to flow through the QRadar Traffic Analysis engine and for QRadar to automatically detect one or more log sources.

When you select this option, the Log Source Identifier Pattern can optionally be used to define a custom Log Source Identifier for events that are being processed.

If you selected Use As A Gateway Log Source, you can define a custom log source identifier for events that are being processed and for log sources to be automatically discovered when applicable. If you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources.

Use key-value pairs to define the custom Log Source Identifier. The key is the Identifier Format String, which is the resulting source or origin value. The value is the associated regex pattern that is used to evaluate the current payload. This value also supports capture groups that can be used to further customize the key.

Define multiple key-value pairs by typing each pattern on a new line. Multiple patterns are evaluated in the order that they are listed. When a match is found, a custom Log Source Identifier is displayed.

If QRadar accesses the Amazon Web Service by using a proxy, select this option.

If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password fields.

If the proxy does not require authentication, configure the Proxy IP or Hostname field.

The maximum number of events per second that QRadar ingests.

If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle.

The default is 5000.

Access Key ID/Secret Key

Standard authentication that can be used from anywhere.

EC2 Instance IAM Role

If your QRadar managed host is running in an AWS EC2 instance, choosing this option uses the IAM role from the metadata that is assigned to the instance for authentication. No keys are required. This method works only for managed hosts that are running within an AWS EC2 container.

The Access Key ID that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Access Key parameter is displayed.

The Secret Key that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Secret Key parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role ARN parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role Session Name parameter is displayed.

Assume Role External ID is an optional identifier that is required to assume a role in a different account.

If the account administrator, to which the role belongs, provides you with an external ID, then insert that value in the Assume Role External ID parameter.

This value can either be a string, a passphrase, a GUID, or an account number. For more information, see AWS documentation Using an external ID for third-party access.

Log Stream

The name of the log stream within a log group. If you want to collect logs from all log streams within a log group, leave this field blank.

Filter Pattern

Type a pattern for filtering the collected events. This pattern is not a regex filter. Only the events that contain the exact value that you specified are collected from CloudWatch Logs. If you type ACCEPT as the Filter Pattern value, only the events that contain the word ACCEPT are collected, as shown in the following example.

{LogStreamName: LogStreamTest,Timestamp: 0,Message: ACCEPT OK,IngestionTime: 0,EventId: 0}

Event Delay

Delay in seconds for collecting data.

Other Region(s)

Deprecated. Use Regions instead.

Forwards only the original event that was added to the CloudWatch Logs.

CloudWatch logs wrap the events that they receive with extra metadata. Select this option if you want to collect only the original event that was sent to AWS without the additional stream metadata through CloudWatch Logs.

The original event is the value for the message key that is extracted from the CloudWatch log. The following CloudWatch Logs event example shows the original event that is extracted from CloudWatch Logs in highlighted text:

{LogStreamName: 123456786_CloudTrail_us-east-2,
Timestamp: 1505744407363, Message: 
{"eventVersion":"1.05","userIdentity":{"type":
"IAMUser","principalId":"AAAABBBCCCDDDBBBCCC",
"arn":"arn:aws:iam::1234567890:user/<username>",
"accountId":"1234567890","accessKeyId":
"AAAABBBBCCCCDDDD","userName":"User-Name",
"sessionContext":{"attributes":
{"mfaAuthenticated":"false","creationDate":
"2017-09-18T13:22:10Z"}},"invokedBy":
"signin.amazonaws.com"},"eventTime":
"2017-09-18T14:10:15Z","eventSource":
"cloudtrail.amazonaws.com","eventName":
"DescribeTrails","awsRegion":"us-east-1",
"sourceIPAddress":"192.0.2.1","userAgent":
"signin.amazonaws.com","requestParameters":
{"includeShadowTrails":false,"trailNameList":
[]},"responseElements":null,"requestID":
"11b1a00-7a7a-11a1-1a11-44a4aaa1a","eventID":
"a4914e00-1111-491d-bbbb-a0dd3845b302",
"eventType":"AwsApiCall","recipientAccountId":
"1234567890"},IngestionTime: 1505744407506,
EventId: 335792223611111122479126672222222513333}

Select this option for the collected events to flow through the QRadar Traffic Analysis engine and for QRadar to automatically detect one or more log sources.

When you select this option, the Log Source Identifier Pattern can optionally be used to define a custom Log Source Identifier for events that are being processed.

If you selected Use As A Gateway Log Source, you can define a custom log source identifier for events that are being processed and for log sources to be automatically discovered when applicable. If you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources.

Use key-value pairs to define the custom Log Source Identifier. The key is the Identifier Format String, which is the resulting source or origin value. The value is the associated regex pattern that is used to evaluate the current payload. This value also supports capture groups that can be used to further customize the key.

Define multiple key-value pairs by typing each pattern on a new line. Multiple patterns are evaluated in the order that they are listed. When a match is found, a custom Log Source Identifier is displayed.

If QRadar accesses the Amazon Web Service by using a proxy, select this option.

If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password fields.

If the proxy does not require authentication, configure the Proxy IP or Hostname field.

The maximum number of events per second that QRadar ingests.

If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle.

The default is 5000.

Access Key ID/Secret Key

Standard authentication that can be used from anywhere.

EC2 Instance IAM Role

If your QRadar managed host is running in an AWS EC2 instance, choosing this option uses the IAM role from the metadata that is assigned to the instance for authentication. No keys are required. This method works only for managed hosts that are running within an AWS EC2 container.

The Access Key ID that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Access Key parameter is displayed.

The Secret Key that was generated when you configured the security credentials for your AWS user account.

If you selected Access Key ID / Secret Key or Assume IAM Role, the Secret Key parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role ARN parameter is displayed.

If you enabled Assume an IAM Role, the Assume Role Session Name parameter is displayed.

Assume Role External ID is an optional identifier that is required to assume a role in a different account.

If the account administrator, to which the role belongs, provides you with an external ID, then insert that value in the Assume Role External ID parameter.

This value can either be a string, a passphrase, a GUID, or an account number. For more information, see AWS documentation Using an external ID for third-party access.

For more information, see Amazon S3 Event Notifications (https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html).

When you use this option to extract original event with SQS, the original event might be in a specific JSON element. If so, you must specify the name of the top-level JSON element that contains the original event. This option also unescapes any data that is contained within that element.

For example, when the Message element is used, it takes that root element and unescapes the nested JSON if necessary:

{ "Type" : "Notification", "MessageId" : "6d11936e-2361-5dc1-a689-c590f69c73da", 
"Subject" : "Test Notification", "Message" : "{\"eventVersion\":\"2.1\", \"eventSource\":\"aws:s3\", 
\"awsRegion\":\"us-east-1\", \"eventTime\":\"2020-04-01T17:47:39.107Z\"}" }

The unescaped data then appears as this extracted original event:

{"eventVersion":"2.1", "eventSource":"aws:s3", "awsRegion":"us-east-1", "eventTime":"2020-04-01T17:47:39.107Z"}

If you don't select Use As A Gateway Log Source and you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources.

If you selected Use As A Gateway Log Source, you can define a custom log source identifier. This option can be defined for events that are being processed and for log sources to be automatically discovered when applicable. If you don't configure the Log Source Identifier Pattern, QRadar receives events as unknown generic log sources.

Use key-value pairs to define the custom Log Source Identifier. The key is the Identifier Format String, which is the resulting source or origin value. The value is the associated regex pattern that is used to evaluate the current payload. This value also supports capture groups that can be used to further customize the key.

Define multiple key-value pairs by typing each pattern on a new line. Multiple patterns are evaluated in the order that they are listed. When a match is found, a custom Log Source Identifier is displayed.

If QRadar accesses the Amazon Web Service by using a proxy, select this option.

If the proxy requires authentication, configure the Proxy Server, Proxy Port, Proxy Username, and Proxy Password fields.

If the proxy does not require authentication, configure the Proxy IP or Hostname field.

The maximum number of events per second that QRadar ingests.

If your data source exceeds the EPS throttle, data collection is delayed. Data is still collected and then it is ingested when the data source stops exceeding the EPS throttle.

The default is 5000.