1. It's tempting to think there's a hierarchy for data: That evidence from high-quality experiments is on top at Level 1, and other research findings follow thereafter. But even in healthcare – the gold standard for the "gold standard" – it's not that simple, says NICE in The NICE Way: Lessons for Social Policy and Practice from the National Institute for Health and Care Excellence. The report, produced with the Alliance for Useful Evidence (@A4UEvidence), cautions against "slavish adherence" to hierarchies of research. "The appropriateness of the evidence to the question is more important, not its place in any such hierarchy." NICE articulates how its decision model works, and argues why it's also relevant outside healthcare.
2. Don't let "little e" evidence kill off good Big Ideas. Take note, lean startups + anyone new to the ways of validating ideas with evidence. In their should-be-considered-a-classic, Is Decision-Based Evidence Making Necessarily Bad?, Tingling and Broydon examine the different uses of evidence in decision-making (MIT Sloan Management Review). As a predictive tool, sometimes it's flat wrong: Those iconic Aeron chairs in tech offices everywhere? Utterly rejected by Herman Miller's market-research focus groups. It's good to have a culture where "small" evidence isn't just an excuse to avoid risk-taking. But it's also good to look at "Big E" Evidence, assessing what research is predictive over time, and replace older methods (focus groups, perhaps).
3. 10+ years ago, Billy Beane famously discovered powerful analytic insights for managing the Oakland A's baseball* team, and as a reward was portrayed by Brad Pitt in Moneyball. Now a bipartisan group of U.S. federal leaders and advisors has published Moneyball for Government, intending to encourage use of data, evidence, and evaluation in policy and funding decisions. On Twitter at @Moneyball4gov. *The A's play not far from Ugly Research HQ, and much to our dismay, the Moneyball competitive advantage has long since played out. But Billy is still a great analyst; for the moment, we're just holding our breath.
4. We've barely scratched the surface on figuring out how to present data to decision makers. All Analytics did a web series this week, The Results Are In: Think Presentation From the Start (recordings and slide decks available). One of the highlights was a comparison to Joseph Campbell's hero's journey, by James Haight of Blue Hill Research (on Twitter @James_Haight).
5. We're wired to seek certainty. But Ted Cadsby argues the world is too complex for our simplified conclusions. He suggests probabilistic thinking to arrive at a "provisional truth" that we can test over time in Closing the Mind Gap: Making Smarter Decisions in a Hypercomplex World.
Five for Friday: 5-Dec-2014.