Successful academic health policy researchers have specialized expertise and work hard – traits that are prized in both academia and government. But while these high level characteristics translate easily, many aspects of the institutional structure of policy research in government and academia are quite different (and these, in turn, differ across countries). Both the kinds of research that are demanded and the ways that work is conducted vary across these two contexts.
The overarching goal of research in academia is to better understand the world. The most prized work provides a new perspective or insight into existing phenomena. Novel, creative work will identify a previously unrecognized problem, propose a new set of methodological tools, or offer a distinctive theoretical structure that ties together ideas or events that had been seen as unrelated [21,22,23]. This kind of work spawns a flood of follow-on research, normal science, filling in the details of a model. The corresponding metric of success in academia is articles in high quality academic journals and citation counts – measures of how many other researchers found that this work gave them a new and useful way of engaging with problems.
The merit of scholarly work depends critically on its creativity, novelty, and insight. Rigor and accuracy do also matter, but are not sufficient. Most of the onus for ensuring the accuracy of papers lies with the researcher. The external peer review system, in its design, can provide only a top-level review of the researcher’s approach, such as the design of the study, and pose a few questions. Reviewers rarely examine underlying data and often have inadequate time or training to assess the validity of findings . While researchers regularly call for replications and re-analyses, publications and promotions go to those who break new ground. The conventions of the academy also define what a high quality paper should cover. The data, methods, and results sections of a paper must adhere to certain discipline-specific standards, but authors have nearly free reign in choosing their questions and in discussing the implications of their results.
The growth of research funding for the social sciences, and of the field of public policy as an academic pursuit, have narrowed the gap between the goals of researchers in the academy (at least in this field) and those in government somewhat. Government funders of social science research expect investigators to address questions of policy significance and to make an effort to translate their findings into actionable suggestions (this is particularly true in England). Schools of public policy likewise encourage their faculty to study issues that policymakers will find relevant. Nonetheless, this gap remains an enduring concern, as evidenced by a steady flow of books, articles, and conferences and renewed attention to “knowledge transfer” between academia and policymakers .
While academic policy researchers do conduct research that may be useful to policymakers, the principal goal of research in government remains quite different from that of the academy-- to give policymakers information that will help them to solve a specific, pre-defined policy problem in real time. In contrast to the broad range of questions an academic might choose, most of the time a researcher in government is given a specific assignment to complete within a specific, often challenging, deadline. While a research team in academia generally conducts research itself, sometimes in collaboration with other research teams, in government most research (except rapid turnaround requests) is not conducted in-house but commissioned under contracts with external researchers . For example, at the Department of Health and Human Services, ASPE was asked to assist the Secretary in developing the benefit package that would be the standard for coverage under the Affordable Care Act. The agency conducted analyses in-house (comparing benefits offered in various markets, for example ) and used commissioned research (assessing actuarial values of various benefit designs) that would help answer that very concrete, policy relevant question .
For most questions, the permanent civil service staff is more likely to understand how much effort a question deserves, what the realistic bounds of the policy debate are likely to be, and how formally it needs to be answered. On more than a few occasions, the “old hands” in government make clear to time-limited academic appointees that internal bureaucratic politics, data limitations, or the likely uncertainty of any answer argue against devoting a lot of time or budget to what seemed to be an interesting and important policy question. But the academic’s training and socialization to seek creative solutions to new and difficult questions can be valuable when a novel policy problem is far from the questions that “old hands” have dealt with in the past.
The questions that flow to academics in government are usually very tough, out-of-the-box challenges that can best be answered by bringing together the worlds of institutional knowledge within government and of creative research in academia. Collaborations between governments and academics, often with academic researchers in government serving in a bridging role, can take the form of extended programs of policy-oriented research.
Such programs have generated useful results that altered the shape of policy. For example, the analysis prepared for Ministers in successive UK governments and expert enquiries on reforming the system for financing long-term care involved close collaboration between researchers at the Personal Social Services Research Unit at the London School of Economics and analysts in the social care analytical unit at the DH [27, 28]. (Wittenberg was for many years a member of both). The development and implementation of the Prospective Payment System (DRGs) in the US involved a decades-long collaboration between the Office of Research and Development in the Health Care Financing Administration and academics at several Universities [29,30,31]. In Israel, a group of Israeli academics began to study the issue of evaluating, measuring and reporting on the quality of community based health care. With time, these researchers established connections with health plans, in which they discussed the literature on quality measurement, compared different measures used in the different plans, and committed themselves to deploy agreed measures to improve quality of care, while committing not to reveal measures of quality in the different health plans. In 2004, Israeli, as The Director General of the Ministry of Health, an academic serving in government, and aware of this research, offered to place the program under the aegis of the MOH, along with full funding through the National Institute for Health Policy, leading to this academic-initiated activity transforming into the National Program for Quality Measurement in Community Care. It combines research and comparisons with other countries, a framework for health plans and providers to continually improve quality of care, and reporting of quality measures to the public.
As these examples suggest, while the primary intention of government-commissioned research is to address specific policy issues, not to generate academic publications of generalizable value, sometimes the two are quite compatible. In the field of health policy research, the most notable example of such a happy symbiosis is the RAND Health Insurance Experiment. In 1971, the US Department of Health, Education, and Welfare, which was then considering options for universal health insurance, funded the RAND Corporation (a US contract research firm) to was commissioned to produce estimates of the effects of alternative health insurance packages on service utilization. The results of the study remain of direct policy significance; the study also generated hundreds of peer-reviewed journal articles which have spurred further academic research [32, 33].
While the academic perspective may be useful in identifying a path to an answer for a challenging question, the academic’s usual approach is unlikely to be helpful in government. In contrast to the academy’s preference for novelty, research in government is most useful when it is straightforward; uses well-accepted, conventional methods; and can be replicated easily. Meta-analyses and reviews of existing literature are preferred over original research. In government, rigor and accuracy trump imagination and cleverness every time. In contrast to the academic model under which a small number (2–4) of voluntary peer reviewers assess the validity and interest of a completed work and pose a few questions, a major piece of governmental policy research, one that will be released publicly, may go through extensive internal review. Junior researchers within government may check the math; outside peer reviewers may be enlisted – and paid – to ensure that methods have been used appropriately; and legal staff, political staff, even public relations staff will review the product from their various perspectives. The framing of the question and the discussion section will garner at least as much scrutiny as the methods. The difference in processes means that at least according to formal requirements, a policy research report released by a government is often more likely to contain results that are accurate, verifiable, nuanced and replicable than a paper published in the most esteemed peer-reviewed journal (see https://aspe.hhs.gov/report/hhs-guidelines-ensuring-and-maximizing-quality-objectivity-utility-and-integrity-information-disseminated-public). Research on the quality of government-commissioned research in England finds that it is likely to be more accurate than other commissioned research, though it may not be published in a timely fashion [34, 35]. It is also likely to be less broadly interesting, provocative, and creative than academic research. It can be tremendously valuable if it is used by policymakers in shaping policy – or in rejecting a contemplated policy direction -- even if it is never ever cited again.
These differences in the nature of publication help explain why excellent policy analysts in the permanent civil service rarely migrate back into academia. Their commitment to doing the kind of research needed by government, and to publishing through the government process, often means giving up the chance to pursue their own research interests and to publish extensively in academic journals. Even the most well-established and effective policy researcher in government is very unlikely to have a resume of novel academic publications that would satisfy a university hiring committee. Moreover, if the researcher has become identified with specific policy directions (whether or not based on his or her research), and those policy directions are contested, return to academia may be more unlikely. While efforts are underway to facilitate movement back into academia, for example by encouraging tenure review committees to consider a broader range of publications (see, for example, http://wtgrantfoundation.org/grants/institutional-challenge-grant; http://www.cahs-acss.ca/wp-content/uploads/2011/09/ROI_FullReport.pdf), this difference in the nature and number of work products and tasks explains why it is so often challenging to move from full-time government service into academia.