Large language models may dominate headlines, but a different class of AI could change how businesses predict the future. Compact and efficient time series models are transforming forecasting across industries.

IBM’s TinyTimeMixer (TTM) exemplifies this trend. With fewer than one million parameters, TTM delivers robust predictions without the computational demands of its larger counterparts.

“Forecasting can be a powerful tool when applied correctly,” IBM Technical Strategist Joshua Noble explains. “The ability to predict demand, revenue, costs, device failure or market changes are all powerful assets for a business at any size.”

The AI industry has recently seen a surge of interest in smaller, more efficient language models. These compact models aim to deliver performance comparable to larger counterparts while requiring less computational power and memory. For instance, Mistral AI garnered attention with its Mixtral 8x7B model, which uses a mixture of experts approach to achieve high performance with a relatively small parameter count.

This trend towards “AI lite” reflects a growing focus on practical deployment and accessibility, potentially democratizing AI technology for a wider range of applications and devices.

Gated attention: The key to efficiency

TTM swaps out traditional machine learning self-attention—where every element in a sequence weighs its relationship to all others—for gated attention, a mechanism that selectively controls simple perceptron blocks to link time series variables. This streamlined approach sharpens focus and slashes computational costs in training and fine-tuning, resulting in a lean, efficient model that excels at time series tasks.

The Beijing Air Quality dataset is a real-world test case showcasing TTM’s ability to forecast PM2.5 air pollution levels using historical data and meteorological variables. This application demonstrates the model’s potential in environmental monitoring and urban planning.

While time series models show promise, challenges remain. Noble cautions, “Forecasting, like most AI, depends on good data and predictable patterns. There are some phenomena that simply aren’t very predictable, and no model will be able to work around that.”

Fine-tuning addresses model limitations through a streamlined process: prepare data, load model, evaluate, fine-tune and reassess. Its impact is clear: For Beijing air quality forecasts, fine-tuning cut evaluation loss from 0.426 to 0.253, significantly improving prediction accuracy. This real-world example demonstrates fine-tuning’s power in enhancing model performance for specific tasks.

Fine-tuning for precision

The fine-tuning process involves splitting the dataset, loading the pre-trained model, establishing baseline performance, fine-tuning training data with early stopping, and final evaluation. This approach enhances the model’s ability to capture complex data patterns, making more accurate predictions.

TTM’s forecasting pipeline handles complex time series data, incorporating both target variables and external factors. Going back to the PM2.5 forecasting example, this approach allows TTM to capture intricate relationships between various elements affecting air quality. By considering multiple variables simultaneously, the model provides more accurate and nuanced predictions, accounting for the complex interplay of factors influencing air quality over time.

The IBM watsonx platform brings these capabilities to a broader audience. The platform enables users to train, validate, tune and deploy models efficiently, democratizing AI-driven forecasting for businesses of all sizes.

As time series models like TTM evolve, their impact on business forecasting grows. These models offer powerful tools for navigating uncertainty, from supply chain optimization to market trend prediction.

Noble summarizes the potential of these models: “Foundation models trained on time series data can help to reduce the barrier to entry for this kind of forecasting because they have much of the training data already built in.”