With critical information changing each millisecond and data volumes ballooning, how could ZE supercharge its data and analytics platform and enable its clients to outmaneuver their competitors?

By refreshing its storage architecture, ZE accelerated the delivery of complex analytics on big data to its clients, enabling it to diversify its offerings and gain market share.

When making mission-critical decisions, every second counts—especially when you are a major player in global energy, commodities and financial markets. However, with volumes of time-series data ballooning, mobilizing the right information to drive business decisions has become increasingly complex.

For ZE, a data management company that specializes in delivering hot data and cutting-edge analytics platforms, the dawn of big data presented an opportunity to expand, as well as growing pains to overcome.

Aiman El-Ramly, Chief Strategy Officer, explains: “At ZE, we help our clients manage their market exposure. To do this, we provide them with vast arrays of data that we extract, transform and load with clear and rigorous visualization and automated integration tools.

“With this up-to-the-second information, our clients can make better decisions and refine their market strategies almost in real time. To keep providing accurate information at this speed in the age of big data, brings fresh challenges to our doorstep.

“The amount of data available for businesses to analyze and inform their strategies has significantly increased, and as the data landscape has changed, so too have our clients’ demands for sourcing and interpreting information.

“To optimize their agility, our clients require tools that help them easily visualize and understand market trends. At the same time, they want to reduce their costs by moving to a hands-off approach to IT infrastructure.

“To meet our clients’ evolving needs, we decided to diversify our offering and move to a Platform-as-a-Service [PaaS] model—providing our clients with rich data streams as well as cloud-hosted infrastructure and software to more effectively deliver data and analytics.”

For ZE to successfully transition to a PaaS model, it had to confront several technical challenges which, if left unchecked, could have negatively impacted the versatility of its services.

Nader El-Ramly, Chief Product Officer, elaborates: “During the first iteration of our cloud it became clear that we were not sufficiently equipped to meet the demands of an evolving data landscape and changing client demands. The limitations of the performance of our storage infrastructure came to the fore over this period; as a result, our service was sometimes running slower than our clients needed it to perform—limiting their dexterity.

“In our industry, low performance levels can have a significant impact on our services. For instance, once we have pooled and collected vast arrays of time-sensitive data, such as price indexes, our software then builds forward price curves, which our clients use as tools to inform their strategic decisions and investments.

“It is important that these curves are generated and updated in real time so that our clients stay ahead of the curve—so to speak—and gain first-mover advantage. Any delay in generating this information can have a big impact on our clients. In technical terms, our many racks of disk storage couldn’t smoothly handle the load produced by our clients’ requests.

“We tried to solve this by adding more spinning disk storage into our architecture, but this approach could not keep pace with the increasing volumes of data, and simply pushed up the cost and complexity. The performance issues caused a bottleneck, leaving our data-processing systems starved of data and sometimes taking a significant period to return analytics.

“In this way, our storage architecture was limiting our ability to make the most of our processing power. It would also regularly break—increasing maintenance costs and diminishing the availability of our services. To make the cloud a true success and bring our services up to speed with the exponential growth of data, we needed to re-think our storage architecture.”

Since deploying IBM flash storage, ZE has dramatically accelerated performance, reduced latency, cut maintenance costs, and unleashed the ability to develop new capabilities and services.

Nader El-Ramly explains: “The most immediate improvement that we saw from moving to flash storage was that processing times were much faster. This was because the bottleneck we were experiencing in storage was gone—allowing us to get data to the CPUs as fast as they can process it.

“Indeed, we had to adjust from measuring in minutes, to seconds and milliseconds. Being able to process analytics at this speed means that we can often exceed the extremely challenging Service Level Agreements (SLAs) we have in place with clients, even as the volumes of data and frequency of reporting increase.”

Aiman El-Ramly adds: “With such quick processing times, the FlashSystem solution has unleashed the potential for ZE to grow its customer base and enhance its offerings.

“As our ability to process our clients’ requests has accelerated, our clients’ behavior and expectation has also changed. In particular, we are now able to complete more complex requests and generate a much larger volume of forward pricing curves in near real time.

“Where previously our clients approached us for modeling thousands of different curves, they now expect hundreds of thousands of curves. Introducing IBM flash storage was also a non-disruptive way to buy more breathing space for a separate re-architecting of our software to better address big data demands.

“Being able to diversify our services and process a much larger amount of data more quickly keeps our customers happy and has enabled us to expand. After the implementation of flash storage, we were able to grow our PaaS customer base by 30 percent in one year alone!”

In addition to developing new capabilities and services, ZE has made significant energy-efficiency savings by moving to enterprise flash storage and acquired a common set of management tools for both flash and disk storage.

“IBM FlashSystem 900 and IBM FlashSystem 840 have enabled us to make significant efficiency savings,” continues Nader El-Ramly. “Flash storage is, by design, more durable and more compact, however we weren’t expecting to have consolidated by so much. With IBM flash storage, we moved from two racks of disk storage to just 2U of flash storage—in turn, reducing energy and cooling costs.

“On top of this, we maximized the capacity of our storage with the virtualization capabilities included with IBM Storwize V7000, which also makes managing our storage much easier. When we first deployed the Storwize technology, we achieved a 60 percent compression ratio. We then invested in a second generation V7000 to leverage the enhanced compression engine, leaving the first to act as our second tier of storage beyond the FlashSystem 900 and FlashSystem 840.

“Also, adopting flash storage has eliminated the frequent and disruptive work of replacing failing disks in our old infrastructure. Today, if there is an issue with a disk on the Storwize V7000, the device automatically notifies IBM and they instantly send out a replacement—usually before we even notice there is an issue.”

With IBM FlashSystem 900, IBM FlashSystem 840 and Storwize V7000 technology powering faster analytics of big data, ZE PowerGroup can help its clients expand their trading operations by enabling them to make more decisions, faster. And by helping to increase their confidence in the quality of data and reporting, the upgraded ZE platform can help clients improve tolerance of risk, opening up new trading opportunities.

Aiman El-Ramly concludes: “We have been able to give our clients more confidence in their data delivery infrastructure, which in turn makes them more tolerant to market risks and helps them to grow their operations. In this way, flash storage has become a true asset to our company, enabling us to provide new pathways to growth for our clients.

“Throughout this project, we have enjoyed working with IBM, and were pleased to see their dedication to helping ZE grow. With IBM systems spurring us to grow and enhancing our competitive advantage, we have been able to provide better services that help our clients boost their versatility.”

Established in 1995, ZE (link resides outside of ibm.com) is the developer of ZEMA, a comprehensive software solution for data management and business process automation challenges. By providing unrivaled data collection, analytics, curve management, and integration capabilities, ZEMA offers end-to-end automated business process solutions for clients in all markets and industries. ZE also provides a full range of support and implementation services ensuring full success across all elements of the project.