International Research and Developments
Pfeiffer J. and Sutton R, Evidence Based Management, Harvard Business Review, January 2006 pp. 63-74
“Executives routinely dose their organisations with strategic snake oil: discredited nostrums, partial remedies or untested management miracle cures. In many cases, the facts about what works are out there – so why don’t managers use them?” Pfeiffer and Sutton offer seven reasons why.
- There’s too much evidence. There are tens of thousands of books, journals, statistical reports, specialty television channels, web and business consultant sites advising management, and thousands more appear every year.
- There’s not enough good evidence. Much of the avalanche of management information is actually anecdotes, analogies, assumptions and assertions.
- The evidence doesn’t quite apply. Most management techniques are context specific – they work to some extent, with some people, in some circumstances, some of the time. Techniques developed for one purpose cannot be simply applied to another purpose, they have to be adapted and so there is always uncertainty.
- People are trying to mislead you. Because it’s hard to distinguish good advice from bad, and the pressures to produce quick success are so great, management authors and consultants sell short-term half-truths and quick fixes and managers are constantly enticed to believe and implement flawed practices.
- You are trying to mislead you. Managers routinely ignore or minimise negative evidence that clashes with their expectations while giving too much weight to evidence that is consistent with their beliefs and preferences.
- The side effects outweigh the cure. Even when the evidence is available and applied with reasonable competence and objectivity, the effects are too narrowly considered. For example, the new work reporting system is more comprehensive and accurate so planning can be more detailed. But it requires every employee to spend an average 15 minutes a day filling out the required forms and obtaining the necessary approvals – that’s nearly two weeks of lost productivity per employee per year.
- Stories are more persuasive anyway. Narratives – stories with characters who move along a connected beginning, middle and end – are more memorable, and more compelling, for people than any graph, chart or table.
Pfeiffer and Sutton identify the leading sector in evidence based management as the health sector and advocate that managers in every other arena should at least adopt the principles, and wherever possible adapt the practices, of evidence based management developed by and for the health sector. They urge managers to apply two simple rules to any proposed policy, practice or intervention:
- Demand a focus on efficacy; proof that the proposed action does make a difference
- Examine the logic of the proof presented – think critically about the explanations, unpack the assumptions and test that the cause-effect connections are real.
This publication is not available online as a free downlaod. It should be accessed through your library.
The Solid Facts: Social Determinants of Health, World Health Organisation, Copenhagen, Denmark, Second Edition 2003
Even in the most affluent countries, people who are less well off have substantially shorter life expectancies and more illnesses than the rich. Not only are these differences in health an important social injustice, they have also drawn scientific attention to some of the most powerful determinants of health standards in modern societies. They have led in particular to a growing understanding of the remarkable sensitivity of health to the social environment and to what have become known as the social determinants of health.
This publication outlines the most important parts of this new knowledge as it relates to evidence for public policy. The ten topics covered include the lifelong importance of health determinants in early childhood, and the effects of poverty, drugs, working conditions, unemployment, social support, good food and transport policy. To provide the background, it starts with a discussion of the social gradient in health, followed by an explanation of how psychological and social influences affect physical health and longevity.
In each case, the focus is on the role that public policy can play in shaping the social environment in ways conducive to better health: that focus is maintained whether we are looking at behavioural factors, such as the quality of parenting, nutrition, exercise and substance abuse, or at more structural issues such as unemployment, poverty and the experience of work. Each of the chapters contains a brief summary of what has been most reliably established by research, followed by a list of implications for public policy.
This publication is available in full at: http://www.who.dk/document/e81384.pdf
Nutley S, Davies H and Walter I, Evidence-Based Policy and Practice: Cross-Sector Lessons from the United Kingdom, Social policy journal of New Zealand, Issue 20, June 2003, pp. 29-48
This paper identifies key lessons learnt in the United Kingdom Public Sector quest for policy and practice to become more evidence based. The Annex to this paper provides outlines of and web links to specific initiatives across the public sector in the United Kingdom.
The overview of cross-sector experience has identified some progress, but also many outstanding challenges facing Evidence-Based Policy and Practice in public services in the United Kingdom. The authors identify a number of lessons related to four core requirements for improving evidence use in developing policy and practice.
The Nature of Evidence
- Research is only one source of evidence for public policy and practice.
- Agreement as to what counts as evidence should emphasise a “horses for courses” approach. “Ways and means” matrices – ways of understanding related to the most appropriate means for achieving each kind of understanding – are likely to be more beneficial in the long run than simple hierarchies of evidence.
A Strategic Approach to Knowledge Creation
- Stakeholder involvement in the creation of wide-ranging R&D strategies is crucial.
- Such strategies need to address capacity building as well as priority areas for future research.
Effective Dissemination and Wide Access
- Systematic reviews have the potential to increase access to robust bodies of knowledge but the cost of such reviews and the need for further methodological development in this area are barriers to progress.
- We know much about the features of effective dissemination but even good dissemination has its limits – “pushing” evidence out is not enough, there is also a need to develop the “pull” for evidence from potential end users.
Increasing the Uptake of Evidence
- Uptake needs to be defined broadly – there are many ways in which evidence might be utilised appropriately.
- There are a myriad of initiatives aimed at increasing the use of evidence in policy and practice but there is little systematic evidence on their effectiveness.
- Tentative evidence suggests that multi-faceted strategies that explicitly target barriers to change work best.
- Partnership models, which encourage ongoing interaction between evidence providers and evidence users, may be the way forward.
This publication is available in full at:http://www.msd.govt.nz/documents/publications/msd/journal/issue20/20-pages29-48.pdf
Little M, Axford N and Morpeth L, Risk and Protection in the context of services for children in need Journal of Child and Family Social Work, Issue 9 2004, pp105 - 117
This paper was included as a brief report in the June 2005 edition of CE Connect, the predecessor to INSPIRE. It is repeated here in more depth because the authors are concerned to contrast two different approaches to the use of ‘risk’ which are highly relevant to evidence based management.
The word risk in a scientific context has its roots in epidemiology. It generally refers to an individual’s chance of developing a disease. So we might say that children with poor diets have an increased risk of poor health in adulthood. This risk, generally expressed in numerical terms as a probability or odds ratio, is calculated by comparing the health of adults who had good and bad diets as children.
In social services, including children’s services it is common to talk about a child being ‘at risk’. This generally refers to a child who has been, or is more likely than others to be, maltreated. Professionals also talk about the risk of a child coming into care and, later in the process, children at risk of placement breakdown. This use of the word ‘risk’ is generally unhelpful for a number of reasons.
- First, the focus is more on service responses than child development.
- Second, there is not a great deal of specificity about what the child is at risk of.
- Third, the term ‘at risk’ is used to indicate greater chances of something happening, but it is rare for a numeric figure to be given to the size of the risk or for conclusions to be rooted in any sound evidence of contrasting outcomes for those exposed and not exposed to the risk.
- Fourth, even where a figure is given, it is rare for the important distinction between risk and probability to be made. For example, someone can be at high risk of developing a disorder – say five times more likely than other people – but still only have a tiny probability of actually succumbing – perhaps one in 100,000.
- Fifth, in the children’s services context ‘at risk’ is seldom linked to factors that might explain the increased likelihood of the event or circumstance occurring.
The authors propose a set of principles for incorporating a more rigorous and quantitative approach to developing risk assessments in social services settings:
This report can be found in full at:http://www.dartington.org.uk/documents/Risk%20and%20Protective%20Factors%20June%2003.pdf
Fit for purpose? Assessing Research Quality for Evidence Based Policy and Practice by Annette Boaz and Deborah Ashby
Efforts to promote quality products, services and practices currently abound in both the public and private sector. The evidence based policy and practice (EBPP) movement cannot exempt itself from this debate. Indeed one of the principal aims of EBPP has been to promote the generation and use of good quality research evidence in policy and practice. Unfortunately, traditional mechanisms for assuring research quality through peer review and quality standards (in the academic sector) and research professionalism (in the government and commercial sector) have failed thus far to deliver consistently high quality research. In particular, peer review has proved unreliable and in some fields there is a lack of consensus as to what ‘counts’ as good quality research.
Although improving the effectiveness of existing quality assurance procedures is an immediate challenge for the research community, this paper goes on to argue that future conceptualisations of research quality need to move beyond a fixation with methodological quality, to address the ‘fitness for purpose’ of research. After all, one of the strengths of research, compared to other sources of knowledge available to decision makers, should be that it is a quality assured product carried out to pre-agreed standards. A broader notion of research quality should help researchers and research users to feel confident about the use of evidence in policy and practice.
Paper can be found at: http://evidencenetwork.org/Documents/wp11.pdf
Is Evidence-Based Government Possible? Jerry Lee Lecture 2004
by Philip Davies PhD, Cabinet Office London
This lecture addresses whether evidence-based policy and evidence-based government is possible, and whether it is more than a rhetorical device. It attempts to define evidence-based policy and considers factors other than evidence that influence policy making and policy implementation. It also considers the types of evidence used by governments and the types of research that can contribute to that evidence. Some of the mechanisms that need to be in place for evidence-based government to occur are also discussed. The lecture concludes that evidence-based government is possible and is well established in the U.K. It argues that a broader conception of evidence is used by most government than by some academics, and that a wide range of methods for gathering and appraising evidence for government is required. Some implications for the Campbell Collaboration and the academic community are suggested.
The Paper can be found at: http://www.policyhub.gov.uk/downloads/JerryLeeLecture1202041.pdf#page=1
Other languages