You Don't Have a Data Problem. You Have a Question Problem.

The pattern is familiar. The board wants better answers, so the organisation commissions a data programme: a new platform, a clean-up of every source, a set of dashboards. Two years and a large budget later, the platform works and the dashboards are live. The question the board actually asked is still unanswered.

The problem was never a shortage of data. It was that nobody wrote down the decision the data was for. This article sets out why starting with the data so often fails, and how starting with the decision gets you better answers with far less of it.

The symptoms are everywhere

• In a Gartner survey of 566 data and analytics leaders, only 44% said their team was effective at providing value to the organisation Gartner.
• In a later survey, the most common top challenge for chief data and analytics officers was being unable to measure the impact of their work on business outcomes. Only 22% of organisations had defined, tracked and shared business-impact measures for most of their data and analytics work Gartner.
• Research by BARC and Eckerson Group found that only about a quarter of employees actively use business intelligence tools, with little growth in seven years BARC.

You cannot measure the impact of analytics if you never said which decision it was meant to improve. And people do not use dashboards that answer questions they are not asking.

Why starting with the data fails

“Good data” depends on the job

The UK government’s Data Quality Framework defines quality as fitness for purpose: whether data is “good enough for the job it’s intended for” GOV.UK. That is the right definition, and it has an awkward consequence. Until you know what the data is for, you cannot say whether it is good enough, so a programme to “fix the data” has no finish line.

Poor data quality is expensive. Gartner estimates it costs organisations at least $12.9 million a year on average Gartner, though that figure is based on organisations’ own estimates. When researchers asked 75 executives to check the critical fields in the last 100 records their teams had created, only 3% of the results met even a lenient quality standard HBR. Notice the method: they checked a few fields that mattered, on recent real work. That is a decision-first approach to quality, and it took an afternoon, not a programme.

Technology on top of poor data gives wrong answers faster

The National Audit Office found that across government there was “a culture of tolerating and working around poor-quality data”, and that good data was wrongly treated as a free good NAO. Its advice to leaders started with “a clear sense of purpose”, and it warned that new technology on top of poor data may simply give wrong answers faster NAO. None of this is unique to the public sector. Any organisation that layers dashboards over data it has not judged fit for purpose has the same problem.

Start with the decision

Organisations that make better decisions perform better. Bain’s research across around 760 companies found that decision effectiveness and financial results were closely correlated in every country, industry and company size it studied HBR. Yet in a McKinsey survey, only 20% of respondents said their organisation excelled at making decisions McKinsey.

Douglas Hubbard’s work on measurement explains why data programmes go wrong. Organisations tend to measure what is easy and already well understood, and neglect the few uncertain things that would actually change the decision. His remedy is to model the decision first, work out which uncertainties matter, and measure only those Hubbard. Often the amount of data needed is startlingly small. His “rule of five” shows that a random sample of just five gives a 94% chance that the true median lies between the smallest and largest values.

Less can genuinely be more. Research on decision-making has found that, in uncertain conditions, simple rules that ignore much of the available information can match or beat complex models that use all of it Annual Review of Psychology.

A worked example

An organisation with 120 sites needs to decide which 20 to close or refurbish.

Starting with the data, it commissions an estate data platform: every attribute of every building, cleaned and joined. It takes two years, and the decision waits.

Starting with the decision, it asks what would actually change the ranking. Three things: running cost per square metre, the cost of the maintenance backlog, and how much each site is used. The first two are already known well enough from finance and surveys. The real uncertainty is use, so it samples badge and room-booking data on a handful of random days per site over two weeks. The shortlist is ready in a month, and every other attribute can wait until someone needs it.

The NAO made the same point about cost in government: departments know their total running costs, but often lack insight into what drives cost within a service. Its recommendation was a named senior owner for each service and simpler ways of measuring staff time, not a bigger cost model NAO.

What to do on Monday

1. Write the decision down. “Which 20 sites do we close?” is a decision. “Better estate insight” is not.
2. Name who makes it, and when. Analytics without a decision-maker is a report nobody reads.
3. Ask what would change their mind. Those few factors are the only data that matters for now.
4. Measure the biggest uncertainty first, with the smallest sample that will settle it.
5. Audit your dashboards. For each one, ask which decision it informs, who makes it and how often. Retire the ones nobody can answer for.
6. Judge the work by the decision. Did it get made, sooner and better? That is your measure of impact.

If you have plenty of data but not the answers you need, talk to us. We will start with your question, not your data.

Sources

1. Gartner (2023). Gartner survey reveals less than half of data and analytics teams effectively provide value to the organization.
2. Gartner (2025). Gartner survey finds one-third of CDAOs cite measuring data, analytics and AI impact as top challenge.
3. BARC and Eckerson Group (2022). Strategies for driving adoption and usage with BI and analytics.
4. Government Data Quality Hub (2020). The Government Data Quality Framework.
5. Gartner. Data quality: why it matters and how to achieve it.
6. Nagle, T., Redman, T. and Sammon, D. (2017). Only 3% of companies’ data meets basic quality standards. Harvard Business Review.
7. National Audit Office (2019). Challenges in using data across government, and Right data, right place, right time.
8. Blenko, M., Mankins, M. and Rogers, P. (2010). The decision-driven organization. Harvard Business Review.
9. McKinsey & Company (2019). Decision making in the age of urgency.
10. Hubbard, D. (2014). How to Measure Anything, 3rd edition. Wiley.
11. Gigerenzer, G. and Gaissmaier, W. (2011). Heuristic decision making. Annual Review of Psychology, 62.
12. National Audit Office (2025). Improving government’s productivity through better cost information.