Why governments do not always listen to ‘evidence,’ research and sound advice

by Dr Scott Prasser
June 2008

We have had more than a decade in Australia since the 1996 Howard Government gun buy-back. In that time, there have been many reports and statistics published, some even by Government bodies, that have shown that the buy-back failed miserably to do what former Prime Minister John Howard promised, that is, to make Australia a safer community. With this plethora of information discrediting the buy-back available, why do some Government Departments and politicians still believe the $1 billion in public money spent on buying back longarms and handguns from the lawful was a good policy? Why do some federal politicians still credit the buy-back as one of their greatest achievements? Dr Prasser of the University of Queensland offers some insights into why evidence-based research does not always equate to good government policy.

Our society has long been living in an ‘age of reason.’ Rationalism based on the principles of reason and logic, collation and analysis of information, and the assumption that ‘best choices’ can be made is the accepted way to conduct public discussions and to make public policy.
We see this everyday in almost every policy issue and every policy debate. It is shown by governments’ constant collection of information through their own public services, hiring of consultants, reports form special advisory commissions and numerous reports from investigatory bodies like parliamentary committees, royal commissions and other public inquiries. Interest groups also have increasingly resorted to using information, not just the perceived justness of their cause, to push their case.
All this reflects the view that solving policy problems was just a matter of collecting information and the research would point our policy makers in the inevitable right direction. This is very much the ‘engineering’ model of policy development - identify the problem, seek and research for information and apply the solution. Or as summed up by an American academic some years ago it was thought that all that was needed for effective policy development was research and “once society really knew the facts and figures...corrective action would inevitably follow.”
If only this were true. If only government would collect facts and figures and actually act on them. If only policy was just such a straightforward process of collecting information and application.
Yet it isn’t. And we have known it isn’t despite all the outward manifestations of information collection, consultation, research and formal reporting. Over fifty years ago British politician A. P. Herbert wrote that royal commissions for instance, were as much as “ritual” and were involved more in “digging in the truth” than for “digging it up.”
And so it seems now despite the latest fad to hit government bureaucracies of so called ‘evidence based’ policy advice - of providing advice and recommendations based on clear ‘evidence.’ Funny, some of us thought that had always been what policy advice was about. You know, providing independent policy advice reflecting sound analysis, an understanding of what works, and appreciation of administrative processes to underpin the policy.
But we have all known better for some time.
Many of us who have been commissioned to do research or consultancies for government often wonder why we bother. There must be a big room somewhere inside government where all those reports and reviews just get nicely pigeon-holed until the issue comes up again and the whole process is repeated and yet another report is produced. Over time these research reports mount up until their sheer weight gradually moves the policy ‘see-saw’ enough to gain some actual policy action to occur.
Well, there are several reasons why research does not always translate into policy action as easily as we would expect.
First, there is the very nature of policy problems themselves. Unlike business issues that can ultimately be gauged by profit margins, all aspects of policy are affected by values and ideology about the very nature of the problem. For instance, is child obesity or unwanted pregnancies in teenagers about individual behaviour and responsibility and morals or is it about social trends and pressures? Your response will depend on your values. And the way you define the issue based on these values will greatly influence what the so called policy ‘solutions’ and how these can be achieved. For instance, are they matters concerning individual responsibilities or an increased role for government?
Also, cause and effect in between policy problems and policy solutions is rarely direct or mono-causal. Look at the debates about housing affordability and increasing deaths from car accidents. The causes of these different policy problems are multiple and so must be the solutions. Also, policy problems are rarely sudden events. They are usually the culmination of long term trends so that fixing them, if at all, takes time, repeated efforts, trial and error about what works and considerable resources. Look how long it has taken to reduce cigarette smoking in our society.
In addition, the links between some of the proposed solutions and possible real impacts is sometimes tenuous. Too often we place too great a stress on ‘doing something’ or activities and outputs, and too little about policy outcomes and impacts. Sometimes, decision makers do not like what they receive in the research they have commissioned or in research presented from a variety of sources. This could be research highlighting that spending more money on education does not necessarily lead to measurable improvements in student performance. Or it could be research that indicates that the deployment of more police does not curb road deaths or reduce crime.
The reasons why such evidence does not always lead to appropriate policy action is that every policy is entangled with certain institutional self interest (eg education departments and teacher unions want to employ more teachers, police want more resources) so that evidence that contradicts this self-interest has another opponent to tackle. Such institutional self-interest is called ‘provider capture’ whereby those providing the services capture the decision makers into maintaining the status quo. That is why deregulation and transparency is so important in policy development. The public needs to know the evidence so that ‘provider capture’ and the flaws in their stances can be exposed.
Also, it is human nature to maintain a particular view, even if the evidence is suggesting otherwise. People like to maintain a consistency in their approach and hate to admit they were wrong. Governments are the same. Hence, governments, as you can observe every day, go to considerable effort not to admit that their policy needs revision and often pretend that there is no change to a policy even though the public knows a ‘back-down’ when it occurs.
Of course we all know that there have long been other drivers of policy other than just scientific based research, procurement of information, and application of knowledge to policy problems. We appreciate that values and ideology on how we view the world are important about what priorities are set, how issues are defined, and what solutions are proposed. Nevertheless, inside bureaucracy these days public servants are being exhorted to have a strong ‘evidence-based’ approach to policy. It sounds like something quite neutral, but it is more. It cannot be anymore divorced from the political environment that has always underpinned policy work now than in the past.
Second, the other impediment limiting the impact of research in policy solving is the nature of research itself. It comes in all sorts of forms. Some are digestible, some are not. Some are connected to policy problems. Some are more general in kind. Moreover, there are competing claims and competing ‘expertness.’ The debate over whether coffee is good for us or not, is just one example of this. What should policy makers do when confronted with such contradictory research findings - proceed cautiously is the answer. We sometimes need to wait for ‘evidence’ to accumulate to quite high levels before acting. This is frustrating for some, but inevitably necessary. Research begets research in a never ending spiralling of claims and counter claims.
The other issue is not the lack of data or research. Rather, we have lots of data, but it needs considerable processing to make it useful information in decision making. Census information tells us only on things that happened in the past. It needs interpretation to inform us of its importance, and even more analysis, sometimes of the speculative kind, to suggest what this might mean for the future and hence what corrective policy action may or may not be taken. At the same time, effort is required to collect the right data so that over time it is comparable, reliable and robust.
Third, policy players including elected officials, political parties, and interest groups are often selective in what research or ‘facts’ they want to use as the basis of their decisions. Certainly, they are keen to highlight that their decisions were based on research and investigations - hence all those inquiries, reports, and consultancies mentioned above. But often, the policy option has been long chosen and the reports our decision makers go on as ‘fishing expeditions’ to find the information that will best fit what they really want to do regardless. This is why some reports are discarded, others accepted. Nevertheless, the important point is that governments like to be seen to be acting on ‘research’ of some kind. Acting ‘rationally’ is as much about being seen to do so and being seen to do so means collecting information, commissioning research and appearing to heed its advice. These days, governments will rarely launch a policy and openly state it is based on a particular set of values or fundamental ideological beliefs. Elected officials never mention votes or elections as a factor affecting their policy choice - except when accusing their opponents that they are ‘playing politics.’ Instead, considerable efforts go into finding the information that best supports the action.
Fourth, research is sometimes not used by government because of a range of internal organisational dynamics. Hierarchy and top down decision making rules, and alternative viewpoints within government are not tolerated. This means research that does not fit the ruling clique will not rate. Also, government and departments reflect political and institutional interests, and ideologies. New research that challenges the status quo will be deflected, manipulated, smothered, and undermined in all sorts of ways. Its relevance will be questioned, its accuracy challenged, its conclusions exaggerated and its policy proposals sidelined. What is not said in all this is that the research is not accepted because it does not fit with the current orthodoxy, the existing world view, or the present political interests.
Exacerbating these organisational traits has been some other trends in government. Foremost of these has been the gradual ‘politicisation’ of the public service. Increasingly more and more of senior public service managers (and not so senior) are on limited term contracts, constantly jumping to meet the demands of their political masters. Government now gets the advice (backed by the appropriate ‘research’ and ‘evidence’ of course) they want rather than what they need.
Furthermore, a politicised and contracted public service is one of constant change. As ministers and governments come and go, so do the public servants. Consequently, there is a lack of continuity in the public service these days. The resultant high turnover of senior public servants means there is almost a sense of organisational amnesia - the public service is not even a good bureaucracy anymore as it has lost its capacity to ‘remember’ and file and maintain and use information and research. Again, this is another reason for all the research reports, consultations and the policy merry-go round.
We may have moved a long way from the days when policy choices were based on astrology, the entrails of goats, and visions from oracles or tricks from ‘wizards.’ At the same time ‘research’ should not be seen as some clear, pure and specified product. It comes in various forms, of varying quality and is commissioned by various groups with ulterior motives.
This does not mean we should not try to play the ‘evidence’ policy game. Indeed, in the long run it is developing more evidence and pushing it by repeated ‘pushing’ and propagation that will finally get research accepted. However, such efforts need to be directed not just at the key decision makers, but across the board so that the climate for change can be influenced to allow certain research to get seriously considered.

Scott Prasser is presently Visiting Fellow in the Law School at the University of Queensland and has worked in senior policy and advisory positions in federal and state governments. Scott writes extensively on a range of policy issues in both academic and the broader media and has just completed a major policy report for the Independent Schools Association of Queensland. Scott recently published Royal Commissions and Public Inquiries in Australia. Scott gained his Arts degree and Master of Public Administration from the University of Queensland and in 2004 was awarded his doctorate from Griffith University (School of Politics and Public Policy).