IBM Db2 Event Store considerations for GDPR readiness

For PID(s): 5737-E53

Notice

This document is intended to help you in your preparations for GDPR readiness. It provides information about features of IBM® Db2® Event Store that you can configure, and aspects of the product’s use, that you should consider to help your organization with GDPR readiness. This information is not an exhaustive list, due to the many ways that clients can choose and configure features, and the large variety of ways that the product can be used in itself and with third-party applications and systems.

Clients are responsible for ensuring their own compliance with various laws and regulations, including the European Union General Data Protection Regulation. Clients are solely responsible for obtaining advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulations that may affect the clients’ business and any actions the clients may need to take to comply with such laws and regulations.

The products, services, and other capabilities described herein are not suitable for all client situations and may have restricted availability. IBM does not provide legal, accounting, or auditing advice or represent or warrant that its services or products will ensure that clients are in compliance with any law or regulation.

Table of Contents

  1. GDPR
  2. Product Configuration - considerations for GDPR Readiness
  3. Data Life Cycle
  4. Data Collection
  5. Data Storage
  6. Data Access
  7. Data Processing
  8. Data Deletion
  9. Data Monitoring
  10. Responding to Data Subject Rights

GDPR

General Data Protection Regulation (GDPR) has been adopted by the European Union ("EU") and applies from May 25, 2018.

Why is GDPR important?

GDPR establishes a stronger data protection regulatory framework for processing of personal data of individuals. GDPR brings:

  • New and enhanced rights for individuals
  • Widened definition of personal data
  • New obligations for processors
  • Potential for significant financial penalties for non-compliance
  • Compulsory data breach notification

Read more about GDPR

Product Configuration - considerations for GDPR Readiness

Offering configuration

The following sections provide considerations for configuring IBM Db2 Event Store to help your organization with GDPR readiness.

Configuration to support data handling requirements

An application administrator can add users to IBM Db2 Event Store through the User Management page of the administration console. Only users who are explicitly added by an IBM Db2 Event Store administrator can access the web client or connect to the database using the APIs.

Data in the database is stored in shared file systems, such as IBM Cloud Object Storage (IBM COS), or NFS. The credentials for accessing the shared file systems are stored as Kubernetes secrets on the IBM Db2 Event Store cluster and can only be accessed by the root user.

IBM Db2 Event Store provides only an authentication mechanism. It does not provided fine-grained access control or auditing capabilities.

Configuration to support data privacy

Because IBM Db2 Event Store does not have fine-grained access control, the application has a limited ability to ensure data privacy. While only authenticated users can access the data, any authenticated user can access all of the data in the database.

If you need to ensure data privacy, you should use IBM Db2 Event Store as a single-user application.

Additionally, IBM Db2 Event Store does not have any capabilities to differentiate between personal information and sensitive personal information.

Configuration to support data security

When users access the IBM Db2 Event Store web client, they must enter their username and password. Similarly, when users access IBM Db2 Event Store through APIs, they must authenticate using the following methods:
  • For the JDBC and client APIs, users must supply their username and password
  • For the REST API, users must provide a bearer token

However, as previously stated, once a user logs in, they can access all of the data in the database. IBM Db2 Event Store does not have fine-grained access controls to limit who can access specific data, such as personal information or sensitive personal information.

Data Life Cycle

IBM Db2 Event Store is an event processing engine that ingests and stores large amounts of event data in standard parquet format in a shared storage file system. After the data is ingested, it can be queried at any time.

Depending on your requirements and the table schema that are used in your database, the data that is ingested and stored might contain personal information or sensitive personal information.

In order to authenticate users, IBM Db2 Event Store stores usernames and passwords. Alternatively, you can configure a connection to your LDAP server to manage passwords.

Customer business logic and processes

IBM Db2 Event Store extends and optimizes Spark SQL processing. Client applications written in Scala or Python use Spark SQL and SQL/OLTP extensions provided by IBM Db2 Event Store to create tables, ingest data into the database, and query the data in the database.

If personal information or sensitive personal information is part of your business logic that is used to interact with the database, the data might be submitted to IBM Db2 Event Store using Spark SQL.

Authentication
Users and client applications must provide valid credentials to authenticate to IBM Db2 Event Store:
  • Users must specify their username and password
  • Client applications that use the JDBC API or the client APIs must specify a username and password
  • Client applications that use REST APIs must provide a bearer token, which you can obtain by authenticating to IBM Db2 Event Store with your username and password.

After successfully authenticating to IBM Db2 Event Store, the connection can be used to add rows to the database tables or to query the database.

The credentials are stored in the authentication service that is deployed as part of the IBM Db2 Event Store cluster.

Transaction logs

IBM Db2 Event Store does not support transactions.

Audit logs

IBM Db2 Event Store does not support auditing.

Diagnostic information
IBM Db2 Event Store includes two types of error logs:
  • Application error logs, which can be accessed only by users with valid application administrator credentials.

    The application error logs include entries from different services. The log files are periodically purged. An application administrator can configure the log retention period in the Settings.

  • Database error logs, which can be downloaded from the web client by any authenticated IBM Db2 Event Store user.

The diagnostic information in IBM Db2 Event Store might contain some personal information depending on the level of tracing is turned on.

The diagnostic information include error logs, crash reports and performance reports.

Monitoring
IBM Db2 Event Store includes several types of monitoring:
  • A database monitoring interface, which can be accessed by any authenticated IBM Db2 Event Store user. Users can access this interface through the web client or through the REST APIs.
  • An application monitoring interface, which can be accessed only by users with valid application administrator credentials. Admins can access this interface through IBM Data Platform Manager in the web client or through the REST APIs.

    If a non-admin user tries to access the application monitoring interface through the REST APIs, some of the values in the response might be blank or set at 0.

You can also monitor the database by reviewing the logs. For more information, see Diagnostic information.

Database catalog tables

IBM Db2 Event Store does not have a database catalog. However, the metadata for the database, tables, and cluster nodes are partially stored in the shared storage and partially stored in ZooKeeper, which is a coordinating service for components in the cluster. The metadata might include names of columns, tables, and elements of the cluster. It is possible that some of the names are derived from personal information. For example, if you dynamically created a table that contains all of the employees who report to a specific director, the name of the table could be derived from the director's name and employee ID.

Configuration files

IBM Db2 Event Store uses configuration files on the cluster, in the web client, and in client applications. The configuration files can contain connection information for various IBM Db2 Event Store services. Some of the parameters contain URIs that are required to make a successful connection to the services, such as ZooKeeper and your shared storage volumes. Other parameters deal with SSL. It is unlikely that the configuration files include any personal data.

Cluster environment
IBM Db2 Event Store Enterprise Edition runs on a cluster with one or more nodes in a public or private cloud environment. The IBM Db2 Event Store cluster environment is orchestrated using Kubernetes.

After you install and deploy IBM Db2 Event Store on a cluster, several environment variables that are used to connect to your shared storage file system are set. None of the environment variables include personal data. Environment variables that include access IDs and secret keys are stored as Kubernetes secrets in the appropriate containers during deployment.

Personal data used for online contact with IBM

IBM Db2 Event Store clients can submit online comments/feedback/requests to contact IBM about IBM Db2 Event Store subjects in a variety of ways, primarily:
  • Public comments area on pages in the IBM Db2 Event Store community on IBM developerWorks
  • Public comments area on pages of IBM Db2 Event Store documentation in IBM Knowledge Center
  • Public comments in the IBM DB2 space of dWAnswers
  • Feedback forms in the IBM Db2 Event Store community

Data Collection

IBM Db2 Event Store does not provide an explicit mechanism to collect personal data. The only personal information that is collected occurs when you set up a username and password for a user.

However, personal data can exist as part of the data that is added to the IBM Db2 Event Store database.

Data Storage

Protection

Data in IBM Db2 Event Store is protected by encryption. IBM Db2 Event Store uses SSL/TLS to encrypt data in motion, but does not provide a mechanism to encrypt data at rest. However, you can protect the data in IBM Db2 Event Store by using the built-in encryption options for your shared storage volumes:
  • On IBM COS, data is automatically encrypted without administrator intervention
  • On NFS, it might not be possible to encrypt data.

Additionally, the solid-state drive where the logs are stored should be encrypted with LUKS.

Remember: Any authenticated user has access to all of the data in the IBM Db2 Event Store database.

Additional considerations

The following data storage mechanisms are used by IBM Db2 Event Store. You should consider these factors as you assess your GDPR readiness.

Storage of account data

In IBM Db2 Event Store, account data for IBM Db2 Event Store users is stored in the authentication service. The service resides in the cluster, but you can optionally connect the authentication service to an LDAP server.

Storage of client data

In IBM Db2 Event Store, data is first stored in logs in a local solid-state drive on the cluster. Then, it is shared and moved to shared storage.

When you install and deploy IBM Db2 Event Store, you can specify the physical location and the credentials of the shared storage if you are using IBM COS. The credentials are stored as Kubernetes secrets in the IBM Db2 Event Store cluster. If you are using NFS, the shared data is made accessible on the host by the system administrator.

When you create a table, you can specify an index for the table. As with any data in the table, the data in the index resides in both the local solid-state drive on the cluster and in the shared storage file system.

Additionally, when you create a table, you can optionally specify a value for the time to live parameter (ibm.eventstore.ttl.timeunit) for the data in the table. After the time-to-live passes, the data is automatically purged from the table and the indexes.

Data Access

Who can access data in your offering?

As previously stated, there is no support for fine-grained access control to objects in the IBM Db2 Event Store database. However, users can access the data only if they are authenticated to IBM Db2 Event Store. Once a user is authenticated, the user can query any of the data in the database. (The user is not limited to querying data that they inserted into the database.)

It is your responsibility to ensure that there is no personally identifiable data stored in IBM Db2 Event Store that cannot be shared across multiple users in IBM Db2 Event Store.

IBM Db2 Event Store provides Spark extensions for OLTP access. Any Spark user with the valid IBM Db2 Event Store credentials will have access to data in the IBM Db2 Event Store database and any data in the logs that has not been shared and added to the database.

Only users with valid application administrators credentials can access the admin console (IBM Data Platform Manager) and the application monitoring REST API calls.

If users access IBM Watson™ Studio, only project collaborators can access local data sets and remote data sources. By default, each member of the project needs to have access to the remote data source. For relational databases, the project administrator can optionally provide a shared encrypted credential that can be used by all members of the project to connect to that remote data source.

Additional considerations

There are two different administrator roles in IBM Db2 Event Store. However, both of these roles can be performed by the same user:
  • The first type of administrator is a privileged administrator, a user that has root access to the physical cluster where IBM Db2 Event Store is deployed. The privileged administrator installs the product on the cluster and can execute kubectl commands to perform ongoing maintenance tasks.

    No other IBM Db2 Event Store users can access the physical machines in the cluster or execute kubectl commands.

  • The second type of administrator is an application administrator, a user who has admin user access to the IBM Db2 Event Store web client. By default, the IBM Db2 Event Store web client is configured with the admin user and a default password.

    The application administrator can add other authenticated users to IBM Db2 Event Store and can monitor activity on the IBM Db2 Event Store cluster through the web client user interface.

    An application administrator can assign users one of the following user roles:
    • Admin: an application administrator.
    • User: a non-administrative user. All users have the same privileges to access the web client and the IBM Db2 Event Store database, such as:
      • Creating tables
      • Inserting data into the database
      • Querying the database
      • Monitoring the database

Data Processing

Encryption of data being sent over your network
  • Traffic between clients applictaions and the server is encrypted using SSL/TLS
  • Communications between IBM Db2 Event Store and shared storage file systems, such as IBM COS, are encrypted using SSL/TLS
  • For NFS, the system administrator is responsible for securing the access to the remote file system.

Encryption of stored data

See the recommendations in Data Storage for encrypting the data that is stored in IBM Db2 Event Store.

What happens to the data?

As data is ingested into IBM Db2 Event Store, it is stored in logs on solid-state drives on the cluster file system. The data is then shared and sent to shared storage file systems, such as IBM COS and NFS.

The data in the shared storage can only be accessed by a user with valid IBM Db2 Event Store credentials.

You are responsible for managing the credentials for the shared storage.

Data Deletion

Article 17 of the GDPR states that data subjects have the right to have their personal data removed from the systems of controllers and processors without undue delay.

IBM Db2 Event Store is not a forward-facing application for customers and thus does not provide any mechanisms for data subjects to request or control data deletion. All data deletion related to the product can only be accomplished by authorized users.

You cannot explicitly delete data in an IBM Db2 Event Store database. However, when you create a table, you can specify a value for the time to live parameter (ibm.eventstore.ttl.timeunit) for the data in the table. The TTL period specifies how long data persists in the table before it automatically expires and is purged from the table.

Alternatively, you can drop the table that contains the data or uninstall IBM Db2 Event Store.

Application administrators have the ability to delete users from the integrated LDAP directory server.

If you use the integrated data science features from IBM Watson Studio, application administrators also have the ability to delete content in IBM Watson Studio. And any users who are granted sufficient privileges in a project can delete content that they created. Data inside a project, such as a CSV file, can be deleted individually or by deleting the entire project.

If you uninstall IBM Db2 Event Store, you should clean the disks after uninstalling to ensure that no personal information remains on the disks.

Data Monitoring

In IBM Db2 Event Store, data is ingested into a log and then into a table in shared storage. Users can query the data in the log or in the table by running a query using any of the provided access methods.

Authenticated IBM Db2 Event Store users can monitor the database by using the following mechanisms:
  • The database monitoring interface in the web client
  • The database monitoring calls in the REST APIs
  • The database error log, which can be downloaded from the web client
Additionally, authenticated application administrators can monitor the IBM Db2 Event Store cluster by using the following mechanisms:
  • The IBM Data Platform Manager interface in the web client
  • The application monitoring calls in the REST APIs

Because IBM Db2 Event Store must be installed in a secure environment to achieve the data protection requirements of GDPR, use these security mechanisms to regularly monitor the security state of the product and environment. Consult the product information related to those solutions for details on how to monitor the regular state of system security.

An effective security monitoring and management protocol needs to cover many areas including:

  • Overall system security and access
  • Product configuration
  • Product monitoring
  • Monitoring of log and trace data produced by the product

Individual customer needs will vary. Use the tools and functions that are mentioned above as part of developing this overall security management solution for your specific needs.

Responding to Data Subject Rights

The customer is responsible for meeting data subject rights through their database application logic and business processes. As enumerated in above sections IBM Db2 Event Store has some restrictions on how the data can be accessed, modified or deleted.