ChatGPT-generated abstracts so convincing they sometimes fool reviewers

Scientific abstracts generated by artificial intelligence software were able to fool human reviewers, but only 32% of the time, a pre-print of a new study shows. The researchers suggest that authors who use AI when writing papers should disclose it to stop people from abusing the technology. The researchers selected 50 papers that were recently published in the high-impact journals JAMA, NEJM, BMJ, Lancet and Nature Medicine. They asked ChatGPT to generate abstracts based on the titles and name of the journal. Four blinded reviewers then examined 25 abstracts each, some original and some generated by ChatGPT. The reviewers correctly identified 68% of the generated abstract as being written by ChatGPT, but also thought that 14% of the original abstracts were AI-generated. The ChatGPT-generated abstracts also passed through the plagiarism detectors. The finding was worrying to lead author Dr. Catherine Gao, an instructor in pulmonary and critical care medicine at Northwestern University Feinberg School of Medicine. “This is concerning because ChatGPT could be used by ‘paper mills’ to fabricate convincing scientific abstracts,” she said in a Northwestern University press release. “And if other people try to build their science off these incorrect studies, that can be really dangerous.” Paper mills produce fake scientific work for profit. Convincing AI-generated abstracts could lead to fake submissions to journals or conferences, Dr. Gao said. “The ChatGPT-generated abstracts were very convincing because it even knows how large the patient cohort should be when it invents numbers.” she noted. For example, ChatGPT included high sample numbers for hypertension trials and fewer participants in monkeypox studies “Our reviewers commented that it was surprisingly difficult to differentiate between the real and fake abstracts,” Dr. Gao said. Read: Montreal hospitals developing artificial intelligence system to reduce ER wait times What did ChatGPT get wrong? Only 8% of the generated abstracts followed the journal’s formatting requirements. The generated abstracts were also easily detected by an AI output detector, with mean scores of 99%, meaning they were likely to be AI generated. The real abstracts scored a mean of 0.02% using the same detector. “We found that an AI output detector was pretty good at detecting output from ChatGPT and suggest that it be included in the scientific editorial process as a screening process to protect from targeting by organizations such as paper mills that may try to submit purely generated data,” Dr. Gao said. The researchers suggest that scientific publications and conferences should have authors disclose if they used AI to assist them writing the papers. Read: A new way to diagnose endometriosis quickly They also note how ChatGPT could help the scientific community by decreasing the burden of writing, especially for scientists writing in a language that is not their first language. “Generative text technology has a great potential for democratizing science, for example making it easier for non-English-speaking scientists to share their work with the broader community,” said senior author Dr. Alexander Pearson, a medical oncologist and associate professor of medicine at the University of Chicago. “At the same time, it’s imperative that we think carefully on best practices for use.” ChatGPT is a free tool available on the OpenAI website. It laucnhed on Nov. 30, 2022 and is built on the Generative Pre-trained Transformer-3 (GPT-3). The paper was published as a pre-print and has not been peer-reviewed. It is available on bioRxiv. Read: AI: risks and challenges

ChatGPT-generated abstracts so convincing they sometimes fool reviewers

Register for free access or login.

Related Topics