When data surprises you, have the courage to act

To marketers, social media and other forms of consumer data represent new struggles. To turn these struggles into viable strategies, marketers need fresh and innovative approaches — but they should also follow the example set over 70 years ago by a World War II-era mathematician.

In 1943, near the time World War II momentum shifted against the Axis powers, the Allies faced a particular problem: They were sending a lot of planes on bombing runs over Germany, but not many of them were coming back.

The dilemma eventually fell to Abraham Wald, an Austrian-born Jew who’d earned his PhD at the University of Vienna before emigrating to the United States as the war broke out. Once in the U.S., he joined the Statistical Research Group, a collection of mathematics luminaries tasked with helping the Allies to solve complex questions that could win the war — such as how to mitigate aircraft losses.

Before coming to Wald, researchers from the Center for Naval Analyses had performed a study of returning planes. At first brush, the researchers’ conclusion seemed reasonable and intuitive: The Allies needed to add armor to the parts of the planes that drew the most gunfire.

Wald turned this idea on its head. He argued the Naval researchers were mistaken to analyze only the planes that returned from battle. In all likelihood, these planes survived because they absorbed gunfire in places that were already adequately protected. By not considering the planes that were shot down, the researchers lost sight of the central question they needed to solve: Which parts of their aircraft were most vulnerable to enemy fire?

Wald reasoned that surviving planes didn’t show damage in certain places because the damage would have been fatal. Instead of improving armor where Naval researchers saw lots of bullet holes, in other words, the Allies actually needed to fortify areas where the surviving planes hadn’t been hit.

As marketers who face mounting pressure to make data-driven decisions, here are three ways we can learn from Wald’s example.

1. The only thing worse than the wrong answer is the right answer to the wrong question.

The Naval researchers erred by asking the wrong question. The correct question wasn’t where surviving planes had been hit; it was where the planes that didn’t survive had been hit, and why the damage had been so devastating.

Suppose a piece of marketing content receives a large volume of likes or retweets, for example. Many marketers would assume the content must be effective. But a marketing piece’s goal generally isn’t to accrue likes; it’s to increase revenue. Revenue doesn’t correlate with retweets any more than the number of bullet holes on a 1940s bomber correlated with vulnerabilities in the plane’s design. When a marketer celebrates retweets and likes without understanding how they affect revenue, he or she is just like the Naval researchers who saw a cluster of bullet holes and assumed, “We should put more armor there.”

2. The best insights aren’t prescriptive. They combine data and human curiosity.

Wald didn’t rely solely on common sense to refute the Naval report. His reasoning can be summarized into something that sounds like simple logic, but to get there he completed a range of complicated calculations. That is, he used a combination of analytical thinking and human intuition.

The moral for marketers: Instead of expecting analytics to be prescriptive, think of them as muses for your own creative process. The goal isn’t to find an algorithm that spits out complete, neatly-packaged strategies. Rather, the goal is to find the meaningful signals in the data and to use this information as a starting point for creative strategies. If you have a hunch, validate it through data. If you’re unsure how to proceed, stories in the data can put you back on the right path.

3. It takes courage and conviction to be data-driven.

When data defies what everyone expects, marketers have to boldly stand their ground, just as Wald did.

Suppose a movie isn’t tracking well. The studio can use a variety of data science and natural language processing techniques to course-correct before opening day. By correlating financial histories with language patterns in consumers’ online statements, for example, the studio could identify what’s working and not working in its current ads and make adjustments. When the studios do this, they sometimes have to discard ideas in which they’ve already invested millions in P&A expenses. This is neither an easy nor a comfortable thing to do, but it can save millions more in the long run.

To use Wald’s example for today’s data problems, ask the following questions to evaluate data strategies:

Joshua Reynolds heads marketing for Quantifind. He has been spent the last 18 years advising senior leaders at disruptive companies across multiple sectors how to accelerate market adoption by focusing on an authentic purpose. Prior to joining Quantifind, he served as CEO at Blanc & Otus. Previously, he served for five years as the Global Technology Practice Director for H+K Strategies. He began his technology career as a Gartner analyst.