What huge volume of data are required for smart AI

Share this post:

Smart AI algorithms in all their glory, but without a steady supply of large amounts of reliable data, they are useless. I will describe how data on-site opens great opportunities and cloud solutions so that even smaller companies can keep up.

In this blog post I will talk about AI, hybrid architectures, and multi-clouds. I will conclude with the transformation that is happening right now, meaning that all companies and organizations get a chance to use AI in smart ways.

Data is the beginning of everything

It all starts with large amounts of data. Data that is collected, managed and will create AI applications such as machine learning that can add value to the companies that use them. This is what I discussed in more detail during the session “The Journey to AI with data, IoT and sustainability”, from the online conference Nordic Cloud and AI Forum by IBM. My main point is data.

I believe that smart ways to use machine learning and AI require large amounts of data that you can trust. Very few has it today, in a secure way, with a stable supply, and that manages compliance with the rules and regulations.

How much data are we talking about?

It is all about the terabyte level. I know that many experiments are made in isolated environments and they often work well. However, in production environments, a completely different level of solutions is required to make it work. To illustrate this, we need to understand that it is a matter of running ten thousands of algorithms at the same time to get the insights which drives value and that all these algorithms require access to enormous amounts of data. It could be up to petabyte levels.

What value are we striving for?

An example of value that can be created, is addressed at the conference, concerns consumers increased focus on sustainability. Some surveys show that many consumers are willing to change their buying habits to reduce climate impact and that they believe that traceability is very important for the goods they buy.

Traceability in this case means that a consumer wants to be sure that a product has not been transported unnecessarily far or produced in an environmentally hazardous manner.

For a manufacturer of a product, and the store that sells it, to be able to show this, data is required that is often difficult to collect. AI solutions can significantly simplify the work of collecting this data, by examining, for example, invoices and waybills by machine. This type of work is resource-intensive and difficult to do in a good way for a person who must do it manually by examining data in business systems and other applications, or paper documents.

In essence, the core value from AI is its ability to drive extreme automation. Insights generated from an AI algorithm can remove manual steps in a process, and make decision making more pointed by having the relevant information presented when it is needed and where it is needed. I.e. having an AI algorithm reading e-mails, and generating responses or tasks with only exceptional human intervention, can drive down cost of manual work significantly.

Why is multi-cloud needed?

There are many such examples. What they have in common is that they are very data-intensive. How does this lead to the need for multi-cloud hybrid architectures? I think that modern cloud solutions make it possible to have big visions, start on a small scale and grow quickly. You do not need to buy hardware and software licenses, invest in premises or hire to get AI investments started. Multi-Cloud technology makes AI accessible for everyone.

With the technology platform in place, it is important to build an architecture for efficiently handling data -secure and compliant management and provisioning (APIs) of data which can be trusted. Without it, it is not possible to get the optimal value from AI investments.

Cloud gives everyone the same chance to invest in AI solutions. The differences between small and large companies disappear, as the same amount of data is required regardless of a company’s size.

Find out more where I discuss this further in the session “The Journey to AI with data, IoT and sustainability”, from the online conference Nordic Cloud and AI Forum by IBM.