Audit logging data statistics

Platform services audit logging data is generated on your clusters:

Learn more about the amount of audit data that is generated. Use the information to help you tune audit policies, allocate disk space, and prepare SIEM to handle audit records.

Platform services audit logging

Multiple platform services generate audit records. Some of the services such as authentications (auth-idp), authorization (auth-pdp) and mutation advisor generate a large volume of audit data.

The following table lists information about each platform service:

Sample audit logs for each service

• platform-identity-management-audit.log
• platform-identity-provider-audit.log
• pdp-audit.log
• mutation-advisor-audit.log (file annotation)
• mutation-advisor-audit.log (process annotation)

platform-identity-management-audit.log

{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:168ac3b0-4cc2-11e9-9451-57b95a7e8968","action":"create","requestPath":"/identity/api/v1/teams","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<Cluster-IP>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"Test Team","actions":{"teamId":"test-team","name":"Test Team","directoryList":[]},"typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:49.707Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:16b2be10-4cc2-11e9-9451-57b95a7e8968","action":"update","requestPath":"/identity/api/v1/teams/test-team","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<Cluster-IP>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"test-team","actions":{"teamId":"test-team","name":"Test Team","users":[{"userId":"testuser","userBaseDN":"uid=testuser,ou=people,dc=ibm,dc=com","roles":[{"id":"crn:v1:icp:private:iam::::role:Operator"}]}],"usergroups":[{"name":"security","userGroupDN":"cn=security,cn=platform,ou=cloud,ou=isl,ou=groups,dc=ibm,dc=com","roles":[{"id":"crn:v1:icp:private:iam::::role:Operator"}]}],"directoryList":[null]},"typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:49.969Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:16d5af60-4cc2-11e9-9451-57b95a7e8968","action":"update","requestPath":"/identity/api/v1/teams/test-team/resources","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<Cluster-IP>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"test-team","actions":"crn:v1:icp:private:k8:mycluster.icp:n/default:::","typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:50.198Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:16f74120-4cc2-11e9-9451-57b95a7e8968","action":"read","requestPath":"/identity/api/v1/teams/test-team","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<Cluster-IP>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"test-team","typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:50.418Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:1a1e71c0-4cc2-11e9-9451-57b95a7e8968","action":"delete","requestPath":"/identity/api/v1/teams/test-team","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"curl/7.47.0","address":"<Cluster-IP>:8443"}},"target":{"id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","name":"test-team","typeURI":"service/security/group"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:46:55.708Z","kubernetes.container_id":"4b1871f2f163856e3e3f56723fa16c543af3b1386588d311c2b4a07436122671","kubernetes.container_name":"platform-identity-management","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}

platform-identity-provider-audit.log

{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:d3f579f0-4cc1-11e9-8103-77a98aa80e0c","action":"authenticate","requestPath":"/v1/auth/identitytoken","initiator":{"typeURI":"service/security/account/user","name":"admin","credential":{"type":"token"},"host":{"user-agent":"Go-http-client/1.1","address":"<Cluster-IP>:8443"}},"target":{"id":"5342e7942a91434bebc79a2683e6ad4a426348f079c1d87480d1ada5e39a4706","typeURI":"service/security/credential"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:44:57.999Z","kubernetes.container_id":"5342e7942a91434bebc79a2683e6ad4a426348f079c1d87480d1ada5e39a4706","kubernetes.container_name":"platform-identity-provider","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:d423b5e0-4cc1-11e9-8103-77a98aa80e0c","action":"authenticate","requestPath":"/v1/auth/identitytoken","initiator":{"typeURI":"","name":"","credential":{"type":"token"},"host":{"user-agent":"Go-http-client/1.1","address":"<Cluster-IP>:8443"}},"target":{"id":"5342e7942a91434bebc79a2683e6ad4a426348f079c1d87480d1ada5e39a4706","typeURI":"service/security/credential"},"observer":{"id":"target"},"severity":"normal","outcome":"success","reason":{"reasonType":"HTTP","reasonCode":200},"eventTime":"2019-03-22T16:44:58.302Z","kubernetes.container_id":"5342e7942a91434bebc79a2683e6ad4a426348f079c1d87480d1ada5e39a4706","kubernetes.container_name":"platform-identity-provider","kubernetes.pod":"auth-idp-8jdlx","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}

pdp-audit.log

{"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", "initiator": {"typeURI": "service/security/account/user", "host": {"address": "iam-pdp.kube-system.svc.cluster.local:7998", "user-agent": "lua-resty-http/0.11 (Lua) ngx_lua/10013"}, "name": "admin", "credential": {"type": "token"}}, "kubernetes.namespace": "kube-system", "kubernetes.pod": "auth-pdp-vvtbt", "requestPath": "/v1/authz", "observer": {"id": "initiator"}, "eventType": "activity", "origination": "cli", "eventTime": "2019-03-22T16:46:50.851788", "kubernetes.container_id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "severity": "normal", "reason": {"reasonCode": "200", "reasonType": "HTTP"}, "version": "v1.0", "action": "authorize", "outcome": "success", "id": "icp:1739793c-4cc2-11e9-8903-e6096785c0b3", "kubernetes.container_name": "iam-policy-decision", "target": {"typeURI": "security/policy", "id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "name": "iam-policy-decision"}}
{"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", "initiator": {"typeURI": "service/security/account/user", "host": {"address": "iam-pdp.kube-system.svc.cluster.local:7998", "user-agent": "lua-resty-http/0.11 (Lua) ngx_lua/10013"}, "name": "admin", "credential": {"type": "token"}}, "kubernetes.namespace": "kube-system", "kubernetes.pod": "auth-pdp-vvtbt", "requestPath": "/v1/authz", "observer": {"id": "initiator"}, "eventType": "activity", "origination": "cli", "eventTime": "2019-03-22T16:46:51.085948", "kubernetes.container_id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "severity": "normal", "reason": {"reasonCode": "200", "reasonType": "HTTP"}, "version": "v1.0", "action": "authorize", "outcome": "success", "id": "icp:175d307a-4cc2-11e9-8903-e6096785c0b3", "kubernetes.container_name": "iam-policy-decision", "target": {"typeURI": "security/policy", "id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "name": "iam-policy-decision"}}
{"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", "initiator": {"typeURI": "service/security/account/user", "host": {"address": "iam-pdp.kube-system.svc.cluster.local:7998", "user-agent": "lua-resty-http/0.11 (Lua) ngx_lua/10013"}, "name": "admin", "credential": {"type": "token"}}, "kubernetes.namespace": "kube-system", "kubernetes.pod": "auth-pdp-vvtbt", "requestPath": "/v1/authz", "observer": {"id": "initiator"}, "eventType": "activity", "origination": "cli", "eventTime": "2019-03-22T16:46:55.608122", "kubernetes.container_id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "severity": "normal", "reason": {"reasonCode": "200", "reasonType": "HTTP"}, "version": "v1.0", "action": "authorize", "outcome": "success", "id": "icp:1a0f3c8c-4cc2-11e9-9566-e6096785c0b3", "kubernetes.container_name": "iam-policy-decision", "target": {"typeURI": "security/policy", "id": "f6ffd930a71856089866433b9f943e3af2b0f638a99d082fd15155f840517b3d", "name": "iam-policy-decision"}}

mutation-advisor-audit.log (file annotation)

{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:2bd64a33-db41-4c96-9910-7e9ecb7a1b76","action":"update","requestPath":"/opt/ibm/identity-provider/logs/identity_provider.log.1.gz","observer":{"id":"target"},"initiator":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/auth-idp-gk76f/platform-identity-provider/421a6c6850d2e259a91e25b222b06efa4e3e22320820d7f3fbc2a1924675984e","credential":{"type":"container"}},"target":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/auth-idp-gk76f/platform-identity-provider/421a6c6850d2e259a91e25b222b06efa4e3e22320820d7f3fbc2a1924675984e","name":"/opt/ibm/identity-provider/logs/identity_provider.log.1.gz","typeURI":"service/data/file"},"severity":"critical","outcome":"success","eventTime":"2019-05-19T22:26:30.625Z","kubernetes.container_id":"3297527d587232e539315eca371f95c62a8b1a1c2f7ea1f5896444582b7093ad","kubernetes.container_name":"mutation-advisor","kubernetes.pod":"vulnerability-advisor-ma-file-annotator-58c5bdcbd5-5r65p","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:fd44c075-6a2a-46ff-9d11-1c9a8929c8d0","action":"create","requestPath":"/opt/ibm/identity-provider/logs/identity_provider.log.2.gz","observer":{"id":"target"},"initiator":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/auth-idp-gk76f/platform-identity-provider/421a6c6850d2e259a91e25b222b06efa4e3e22320820d7f3fbc2a1924675984e","credential":{"type":"container"}},"target":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/auth-idp-gk76f/platform-identity-provider/421a6c6850d2e259a91e25b222b06efa4e3e22320820d7f3fbc2a1924675984e","name":"/opt/ibm/identity-provider/logs/identity_provider.log.2.gz","typeURI":"service/data/file"},"severity":"critical","outcome":"success","eventTime":"2019-05-19T22:26:30.625Z","kubernetes.container_id":"3297527d587232e539315eca371f95c62a8b1a1c2f7ea1f5896444582b7093ad","kubernetes.container_name":"mutation-advisor","kubernetes.pod":"vulnerability-advisor-ma-file-annotator-58c5bdcbd5-5r65p","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}
{"typeURI":"http://schemas.dmtf.org/cloud/audit/1.0/event","eventType":"activity","id":"icp:caf11dcf-8996-481a-b064-478b7ac47c60","action":"delete","requestPath":"/var/lib/prometheus/data/01DB3SHSTH94KTG4RF2KF5M6E1","observer":{"id":"target"},"initiator":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/mcm-prometheus-5894b6655f-nrlbk/prometheus/90daf5abeb06786912840473654291d799893a53c7137074e0be11216b848574","credential":{"type":"container"}},"target":{"id":"crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/mcm-prometheus-5894b6655f-nrlbk/prometheus/90daf5abeb06786912840473654291d799893a53c7137074e0be11216b848574","name":"/var/lib/prometheus/data/01DB3SHSTH94KTG4RF2KF5M6E1","typeURI":"service/data/file"},"severity":"critical","outcome":"success","eventTime":"2019-05-19T22:28:28.749Z","kubernetes.container_id":"3297527d587232e539315eca371f95c62a8b1a1c2f7ea1f5896444582b7093ad","kubernetes.container_name":"mutation-advisor","kubernetes.pod":"vulnerability-advisor-ma-file-annotator-58c5bdcbd5-5r65p","kubernetes.namespace":"kube-system","origination":"cli","version":"v1.0"}

mutation-advisor-audit.log (process annotation)

{"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", "initiator": {"credential": {"type": "container"}, "id": "crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/key-management-lifecycle-56b76dc775-dqd4f/icp-audit-service/17128f445f8a7402f22ed4ff571eea637ac3a6292024e2309a9c83d53fb0cef8"}, "kubernetes.namespace": "kube-system", "kubernetes.pod": "vulnerability-advisor-process-ma-annotator-6966664857-m9xgs", "requestPath": "cron", "observer": {"id": "target"}, "eventType": "activity", "origination": "cli", "eventTime": "2019-05-20T06:26:14+0000", "kubernetes.container_id": "3298edd78b411e2bab1311329da58a85e4a30cf2d844e2b8eceb45e34ab58b0f", "severity": "critical", "version": "v1.0", "action": "create", "outcome": "success", "id": "icp:c231a79c-10a1-4225-a226-51d0c9f070c4", "kubernetes.container_name": "mutation-advisor", "target": {"typeURI": "service/compute/process", "id": "crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/key-management-lifecycle-56b76dc775-dqd4f/icp-audit-service/17128f445f8a7402f22ed4ff571eea637ac3a6292024e2309a9c83d53fb0cef8", "name": "cron"}}
{"typeURI": "http://schemas.dmtf.org/cloud/audit/1.0/event", "initiator": {"credential": {"type": "container"}, "id": "crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/think-blue-demo-app-68b77bdc57-dj7lq/icp-audit-service/f6854681151bda2d2335dd6206e632d535ae789fe9f3a0a071f92fe193ed918f"}, "kubernetes.namespace": "kube-system", "kubernetes.pod": "vulnerability-advisor-process-ma-annotator-6966664857-m9xgs", "requestPath": "cron", "observer": {"id": "target"}, "eventType": "activity", "origination": "cli", "eventTime": "2019-05-20T06:35:44+0000", "kubernetes.container_id": "3298edd78b411e2bab1311329da58a85e4a30cf2d844e2b8eceb45e34ab58b0f", "severity": "critical", "version": "v1.0", "action": "delete", "outcome": "success", "id": "icp:81762981-f340-4bed-a193-4bac86085cf6", "kubernetes.container_name": "mutation-advisor", "target": {"typeURI": "service/compute/process", "id": "crn:v1:icp:private:k8:172.16.26.19:n/kube-system::container:kube-system/think-blue-demo-app-68b77bdc57-dj7lq/icp-audit-service/f6854681151bda2d2335dd6206e632d535ae789fe9f3a0a071f92fe193ed918f", "name": "cron"}}

Experiment and statistics

An experiment ran on a cluster with 1 master node, 1 management node and 2 worker nodes. A background script performed tasks to trigger target services to generate audit logs. The default configuration was used for auth-idp and auth-pdp. In the mutation advisor configuration, file crawler time was updated from 24 hours to 5 minutes. Process crawler did not change. The experiment ran for 5 hours for each service. Audit data size was collected as the experiment was running.

The following table illustrates a data size growth pattern in 15 minute time intervals. The data sizes are shown in kilobytes (KB).