Several factors stand out in the 2020 Forrester Wave™ on data management for analytics that demonstrate a marked change from traditional thinking about enterprise architectures. We are now at an inflection point where organizations should no longer think of databases, warehouses, data lakes and streaming data repositories as separate components of an architecture, but rather a cohesive whole. Big data, real-time data and more traditional data assets are all important considerations in an enterprise data architecture.

Organizations are seeking high levels of automation through differentiators like AI and machine learning, as well as solutions that minimize data complexity across heterogenous types and environments allowing them to make more effective digital transformations. Much of that data integration, of course, also relies on the strength of data governance available as part of the solution and its ability to deliver data quality and data privacy simultaneously for multiple business units. For that reason, customers should seek a comprehensive data management framework for an edge in making business decisions and delivering competitive advantage.

Automate and optimize enterprise data management with machine learning and AI

AI and machine learning have driven considerable value by improving insights. However, they can also be used to improve the automation within data management systems as well. One example is adaptive workload management and resource optimization. Machine learning is used for a feedback mechanism that compares expected runtimes against actual runtimes for workloads, then adjusts available resources to overcome shortfalls. Additionally, this can be set up with specific workload classes with unique performance targets for each one.

Machine learning can also be used to optimize queries. While automated query optimization has existed previously, adding machine learning introduces the ability to learn from experience and base future decision making on that experience. Instead of suggesting a “best” option that has routinely proven sub-optimal in practice, the machine learning query optimizer will receive feedback and refine query paths based on the real results of each execution. This can significantly reduce query times and allow database administrators (DBAs) to do less monitoring and more high-value work.

Better platform and data integration through containerization and data virtualization

With organizations embracing all data sources, whether unstructured, semi-structured or structured across a multitude of deployments, it’s vital that their enterprise architecture easily accommodate the collection and analysis of all data types wherever it’s located. A good starting point is for enterprise data management solutions to be integrated to a platform that is open source, cloud-native and built to accommodate containerized solutions. An example is Red Hat OpenShift. The common, open source platform allows the data management architecture to be spread across on-premises, multiple-clouds, and even multiple cloud vendors with ease. Moreover, containerization gives organizations the chance to easily add more data management, analytics or governance capabilities as needed.

The second crucial component of data integration, the ability for all data to be easily accessible in an efficient way, is possible through data virtualization. Data virtualization allows data to be accessed at one point without the need to copy data or move it. In essence, all data may be used in analytics and AI projects whether it’s big data, real-time data, new data or cold data. No time is wasted on manually moving or copying the data, and organizations can forego additional expenses that would have been incurred as a result of moving the data or storing copies of the data. As an added bonus there’s no fear of lowering the data quality with partial or redundant copies.

Using data governance to improve data quality and boost regulatory compliance through good stewardship

Data governance has a considerable impact on self-service capabilities, which is highlighted by Forrester as something organizations should look for in data management vendors. Data quality must be high, meaning the data must be clean and well organized as part of the holistic data management architecture being built in organizations. This is true whether you’re building a business intelligence dashboard or an AI data model. Otherwise, undue time is spent by data scientists and others performing data analytics to find usable data before drawing insights. For instance, using an intelligent metadata catalog allows organizations to track data lineage better across its life cycle and visually explore it while solving for “lost in translation” issues across multiple organizational stakeholders. To learn more about one cataloging solution available on a data and AI platform, read the Watson Knowledge Catalog eBook.

Since organizations are also data stewards, access restriction is another important component of effective data governance and self-service. Because some data is sensitive and should only be handled by those with a legitimate business need, categorizing this data and restricting its access is not only a good practice for data privacy, but may also be required for regulatory compliance. Having an established access system as part of the data governance program means that data users can find data without fear of breaking compliance or lowering data security, while those who need accessibility to more sensitive data for business processes can do so without undue hurdles.

The automation, data integration and data governance framework necessary to build a holistic data management architecture for analytics can be found in IBM Cloud Pak for Data and the family of Db2 enterprise data management capabilities that runs on it.

To learn more about Forrester’s assessment of IBM’s enterprise data management for analytics offering download the 2020 Forrester Wave™. Or, if you have questions, schedule time with one of our data management experts who would be happy to discuss topics like data governance, data integration and automation more with you.