Delta Lake and Talend

Modern data architecture with Delta Lake and Talend

Read Article

About Wavicle Data Solutions

Delta Lake and Talend

Modern data architecture with Delta Lake and Talend

Read Article
Delta Lake and Talend

Modern data architecture with Delta Lake and Talend

Read Article
Delta Lake and Talend

Modern data architecture with Delta Lake and Talend

Read Article

Wavicle Insights, Opinions, Commentary, and More.

Using Big Data to Better Predict Your Recovery: COVID-19’s Impact on Demand Forecasting

Date: Monday May 11, 2020

Perhaps like never before, the Coronavirus pandemic has thrown the global supply chain into a tailspin. From toilet paper to crude oil, the world has seen dramatic and surprising shifts in supply and demand that will prompt all industries to look at demand forecasting differently in the months and years to come.

While it’s been a tough and seemingly endless road, we know this will end. Business will begin to rebound one way or another. Eager for that day, many businesses are starting to think through their road to recovery once the pandemic eases – specifically, when it will happen and what it will look like.

With some states starting to relax restaurant and retail restrictions, how will demand for goods and services change, and how quickly will it happen? How can we anticipate and react to second and third waves of health infections, with more closures and re-openings? These are among the difficult questions that must be factored into forecasting moving forward.   

Post-pandemic forecasting demands new methods and models

Whether you’re a hospitality company, food or beverage supplier, CPG, or other organization, you might rely on historical sales, revenue, and seasonality to predict future demand of products and services. But with COVID-19, the unthinkable happened – a near total halt in consumer activity for an unknown period of time. Year-over-year, month-over-month, or week-over-week data may no longer be valid or useful. What happens if the data to inform your demand curves becomes irrelevant? 

Like it or not, Coronavirus is ushering in a new era of business continuity planning. New methods are needed to forecast future demand. 

“The pandemic is forcing a lot of organizations to shift their forecasting strategies from top-down or bottom-up to a more fine-grained approach,” says Duane Lyons, National Restaurant Practice Lead, Wavicle Data Solutions. “External data sources will become much more important, and will need to include real-time data about what consumers are doing right now –  not just regionally, but at the retail level.”   

This means companies need to be creative about their forecasting models and methods and the data that fuels them. Here’s how forward-thinking companies are using big data to generate insight on traffic, sales, and revenue so they are poised to take advantage.

Hotels anticipate return to travel

Recently, we worked with a large hotel chain to help determine when travelers will start visiting their properties in various markets. The hotel brand wanted to be ready at the right time to ramp up their advertising, staffing, food inventories, and other operational functions to optimize revenue, while meeting customer expectations. To their credit, the hotel is working to avoid the worst-case scenario: excited travelers arriving at a luxury resort only to find sub-par service and limited food and drink options.

Core to our strategy was analyzing different data sets than what the hotel previously relied upon. Traditionally, many hotels relied on prior bookings and seasonality to predict future reservations. In the travel business, however, another key indicator for future hotel bookings is airfare – a vast majority of consumers book air travel before finalizing accommodations.  

By collecting the airline industry’s leading third-party “looker and booker” data, and combining it with other data sets into a data warehouse (i.e., past hotel bookings, Google search, COVID-19 trends, airport, weather), correlations can be tracked, with values assigned to demand fluctuations. By processing hundreds of terabytes of data, self-learning algorithms can generate new demand scenarios based on real-time consumer activity. We recommend testing a single market as a proof of concept, then applying the same method to other markets given success and refinement.

Supply chain complexities grow more tangled with unexpected forces 

Food and consumer goods companies that serve commercial, institutional, and consumer markets have faced an interesting dynamic during the pandemic. As commercial demand has declined due to “shelter-in-place” mandates, consumer demand has risen.

Let’s take a meat processing and distribution company as an example. Since many restaurant dining rooms are closed or limited due to local social distancing and shelter-in-place mandates, large meat producers have experienced a sharp decrease in demand from restaurants. However, they are experiencing significantly greater demand among grocers, as consumers are cooking more at home nowadays.  

The meat company must package their meat differently for two distinct buyers. When and where should the meat company plan to start packaging for commercial bulk sales again? When will the spike in retail demand ebb?

In the past, the meat producer might rely on product SKUs shipped to distribution centers in prior daily or weekly time periods. But this “steady-state” demand forecasting no longer reliably informs projections in times of unprecedented disruption. Complicating matters signifcantly are major meat plant closures due to COVID-related employee illness, wreaking havoc across the supply chain while creating scarcity. 

Is foot traffic data key to predicting the consumer goods rebound?

As Coronavirus hot spots shift from region to region and stay-at-home mandates are enforced and lifted state by state, consumer goods manufacturers will need more localized and real-time information to inform demand forecasts. Monitoring movement of people in specific areas via cell phone data can provide cues when store and restaurant traffic will begin to pick up. Consumer intelligence providers such as Placer.ia and Cuebiq sell foot traffic data that reflect real-time locations of people based on cell phone data. Likewise, health statistics about new and declining infections by region or state may tell us to expect shifts in traffic patterns in certain locales. 

Other data sources are still emerging or are yet unknown. Take, for example, streaming media: digital entertainment behemoths are providing valuable glimpses into consumer behavior via streamed content. Home security systems like RING are proving helpful to law enforcement. What secondary data sources would be useful for your industry?

There is no definitive consumer signal when business will return to normal. It varies by industry. Auto manufacturers need to order parts to make cars, and dealerships can’t open without inventory. The complexities of supply chain create an urgent need to predict demand increases, and traditional approaches aren’t enough.

Build your big data skillsets before competitors do it first

Eighty percent of executives say they will lose competitive advantage if they don’t use data effectively.1 Moreover, consulting firm McKinsey & Company published a new guide (“The Restart,” May 2020) outlining specific actions CEOs should take to effectively reignite their business economies. Core to each step? Data. That said, reality bites: only 31% of companies claim to be data-driven. 2

For companies that attempt this new way of demand forecasting with internal teams, be prepared: there is no magic bullet. Teams should possess a combination of data science, data engineering, and advanced analytics. Companies across industries – but particularly mainstream consumer brands – are intent on learning how consumers’ “fear factor” might impact future purchasing behavior. As such, they are elevating the need for AI and machine learning as acquiring, integrating, and analyzing secondary data become ritualized to forecasting processes.

Rob Saker, a retail and CPG leader at Databricks, a technology partner to Wavicle, asserts that demand forecasting at finer levels of granularity will almost certainly require new modeling techniques. “To perform good data science, data is properly aligned with the algorithms employed. But there isn’t always just one solution to a problem – some may work better at one time than at others,” he says. “With each training and forecasting cycle, and as new data arrives, the evaluation of multiple techniques will be necessary.”

For a more technical view of fine-grained demand forecasting, read Databricks’ recent article “New Methods for Improving Supply Chain Forecasting.” 

It’s time to optimize the value of your predictions

The Coronavirus pandemic is a watershed event to rethink how demand forecasting is done, because the massive disruption caused by COVID-19 may not be our last. As such, data-savvy companies are experimenting with their models to more effectively learn how to expect the unexpected.

Here’s the silver lining: many businesses will start taking incremental steps to think about risk management differently, and will do it better over time. Using advanced analytics to predict when and how your business rebounds after extreme events will become a new standard for business continuity planning. And for leaders in competitive industries, big data will differentiate those who merely fight for survival, and those who thrive in the face of adversity.


1 CIO Review
2 Harvard Business Review