?The jury is still out on whether artificial intelligence may potentially play a role in identifying important and impactful health-related research questions. This is the view of Profs John Garry, Maria Lohan (Queen's 肆客足球 Belfast) and Mark Tomlinson (Institute for Life Course Health Research) in an opinion piece for 肆客足球 World News.
- Read the original article below or click here for the piece as published.
John Garry, Mark Tomlinson and Maria Lohan*
Artificial Intelligence (AI) may be good at answering questions. But can AI help us to come up with good questions in the first place?
With this contribution published in the Journal of Global Health, we have considered whether Large Language Models (LLMs) may potentially play a role in identifying important and impactful health-related research questions.
Prioritising research questions
In general, specifying a good research question is difficult, but crucial. Imagine an important research question for your own field of interest. Further imagine that this question is possible to answer and that knowing the answer would lead to a large positive impact on the world. Almost certainly, academic researchers, policy practitioners, implementers, and funding organisations would all agree that trying to answer this question is a top priority.
Prioritising crucial health-related research questions is especially important. For each area of health, from heart disease to schizophrenia, it is vital to systematically identify the most pressing and potentially impactful set of priority research questions to improve health outcomes. Identifying the most crucial questions can't just be the purview of one constituency. Research translation to improve health usually relies on the confluence of policy makers, funders/doners as well as researchers. And since context matters, a geographical spread of this expertise is also important. Hence, setting research priorities is usually done by conducting a research project collating the views of a comprehensive range of researchers and practitioners in the particular health field and distilling from the data a rank-ordered list of priority questions.
A funding organisation may valuably peruse a priority list when specifying a research call. The list may form part of the objective evidence base enabling the funder to confidently defend its decisions to fund certain research themes (and sub-themes) rather than others.
Using AI?
Given the rapid rise of the use of AI across many aspects of health service provision, from diagnostics to organisational efficiency, it is perhaps no surprise that the potential contribution of AI tools—such as ChatGPT, DeepSeek, and others—to identifying priority research questions is becoming a live issue.
AI tools offer the appealing vista of saving time and money. Instead of organising expensive and complex surveys of stakeholders, an AI tool would systematically examine the vast array of stored human knowledge and identify the crucial research questions to address.
In fact, one study on the topic of pandemic preparedness has shown that an AI-based exercise produced a similar list of priority questions to a human-conducted study.
Enticing but opaque
One problem with this AI-based nirvana is explicability. In contrast to human conducted exercises where we can transparently detail the exact process of conducting the empirical project (reporting, for example, a description of the sample of researcher and practitioner survey respondents and how the collected data was statistically analysed), we are at something of a loss to explain exactly how AI-generated research questions are identified.
This is often referred to as the 'black box' problem. What exactly are tools such as ChatGPT doing when they are asked to generate research priority questions? The lack of clarity is off-putting, and such opacity constrains trust in the results. Even if the results appear plausible, if we don't know how they are produced we are likely to echo the concern of the late Irish prime minister and economist, Garret Fitzerald: “I can see that it works in practice, but does it work in theory?''
To mitigate this problem researchers utilising AI tools would have to provide a detailed exposition of exactly what resources the AI tool relied upon to identify research priority questions and precisely what criteria the tool used to rank-order them.
Dangers of excluding humans
Even if explainability was achieved, a further problem relates to the risk that an AI-generated set of priority questions could alienate the human practitioners who are so crucial to actually implementing any new health interventions on the ground. Phrased more positively, one of the advantages of human-conducted research priority setting exercises is that relevant researchers and health professionals are engaged in the process. Their views are listened to, recorded, and used to derive the findings. The process helps engender a democratic sense of inclusion in the development of the field in question.
If, in contrast, the priority questions are produced at the click of an AI button and metaphorically fall out a clear blue sky the academic and practitioner stakeholders may look askance at the findings.
This worry may be most pertinent where there is little existing stakeholder engagement in a particular field and an associated paucity of human-conducted research priority setting exercises. In such a context, using an AI tool to identify how to progress may exacerbate alienation.
Conversely, in a context of strong stakeholder networks and engagement, it may be perceived as more reasonable to augment existing knowledge of important research questions with an AI-based component.
Validity and Reliability
The more that it can be shown that AI-tools can produce priority research questions that are similar to those produced by human-exercises the more persuasive the pro-AI case is likely to be. Also, researchers advocating the use of AI tools should demonstrate that the results are not simply an arbitrary function of the particular AI tool being used, or that the results are skewed by minor tweaks to question prompts.
Potential
We should be appropriately sceptical about the use of AI tools to generate crucially important research questions that could have a profound impact on the funding of health research and the direction of certain fields. But acknowledging limitations and seeking to mitigate them may lead to the possibility of powerful AI research assistants effectively leveraging the vastness of existing knowledge to help direct further research that would unlock barriers to positive health outcomes.
An interesting avenue to explore is the precise type of contribution the AI tools may potentially make. Can AI tools potentially identify priority research questions that are essentially as good as the ones identified by humans—but are produced more quickly and affordably? Or is it the case that AI tools could perhaps identify priority questions that are qualitatively better than human-generated ones: perhaps entirely novel and ground-breaking? And could using AI tools be regarded as more democratic than a conventional human study given the vastly greater range of (electronically available) perspectives that can be considered?
The jury is still out, but intrigued.
*John Garry is Professor of Political Behaviour at Queen's 肆客足球 Belfast (QUB) in Northern Ireland. Mark Tomlinson is Co-Director of the Institute for Life Course Health Research at Stellenbosch 肆客足球, South Africa and Professor at the School of Nursing and Midwifery at QUB. Maria Lohan is UNESCO Chair based at the School of Nursing and Midwifery at QUB as well as at the Hitotsubashi Institute for Advanced Studies at Hitotsubashi 肆客足球 in Japan.
?
