Five things you need to know about building an evidence base

1 – Why your organisation needs evidence

‘Evidence’ is a buzzword that’s in danger of being over-used, but if an organisation needs to make a decision, it should ideally do so based on some reliable information. This helps it to make decisions which bear some relation to the real world so you can make more money, save more time, help more people, or whatever you’re aiming to do. It also helps it justify those decisions to the employees, shareholders, competitors or trade press who might question a new turn of strategy.

2 – You may already have one (in bits on a shelf somewhere)

Whether it’s about customers, markets, demographics, buying influences, relationships or the economy, it’s likely your organisation has access to some potentially useful data. Does this qualify you to claim you make ‘evidence-based decisions’? Do a few spreadsheets amount to ‘an evidence base’? It depends what you do with that information: if it’s kept up to date, how it’s analysed, if it’s duplicated in numerous different formats across the business and if senior decision makers actually have access to it. You’d be amazed how few do. Even the most savvy organisations have room to improve the value they get from their own data and other information which is freely available.

3 – How to go get some evidence

Having said all of the above, don’t start with the data. It’s very tempting to think about what information might be available and how you’d like to get your hands on it, but that’s a bit like going to the doctors and telling them which drug you’d like without discussing your symptoms. There may be all sorts of alternative options available once your requirements are fully understood. So think about the questions you really want to answer. Don’t even worry at this stage whether you believe they are answerable. A good supplier will want to get to the nub of the evidence you need, and will be able to tell you how close to it you can get with the data that’s out there and within the budget and time allowed.

4 – Be honest about how your organisation will use it

If you really need a nutcracker, don’t get persuaded to buy a sledgehammer. And the reverse is also true. Be realistic about the teams and individuals who will need to interact with the data, their technical and analytical capability, and the outputs you need from them. Sometimes just selecting and visualising some key datasets in a report is really all you need, and if you over-specify and make this too difficult, nobody will buy in to the evidence. Be watchful also for the ‘job retention scheme’: the colleague who’s telling you it’s all a bit complicated and you’d better let him or her keep hold of the data and dole out your ration if you ask nicely. Sharing information is powerful and it allows every stakeholder to add interpretation and ‘soft knowledge’ to the mix. An evidence base which exists on one person’s hard drive doesn’t deserve the name.

5 – Accept that data is never perfect

Anyone who works with government and commercial data regularly will sooner or later give you the data expert’s eye-roll. “How good is this dataset?” you’ll ask, and their eyeballs will shoot up to the ceiling as they launch into a tirade about its shortcomings. Do not despair. Every dataset has its limitations: its coverage, its scale or its currency, something will be less than ideal for your needs. Often data is aggregated into zones which are too large to give you all the insight you’d like at neighbourhood scale, or your Holy Grail information isn’t collected at all. This is where you need to employ proxy data which can provide a useful view of a related phenomenon. Or you can undertake some analysis, perhaps combining several datasets to derive a useful integrated measure. This is potentially dynamite, but you need to understand the modelling techniques being used, at least well enough to check if they make logical sense and that you’ll be able to justify them in a client meeting. Or you may decide that it’s worth collecting some data afresh. Meanwhile it’s worth bearing in mind that if the very best available data is frankly a bit below par, it’s still the very best available. Provided that you interpret results accordingly, it might give you a better edge than nothing at all.

No Comments Yet.

Leave a comment