DEV Community

Cover image for How to find the actual study behind a "studies show" claim
Hanako H.
Hanako H.

Posted on

How to find the actual study behind a "studies show" claim

"Studies show that just having your phone on the desk makes you worse at thinking." It's the kind of sentence that goes into an essay or a report without a second look, because it sounds settled. It isn't made up, either: there is a well-known paper behind it. But there is also a newer review that pooled 33 studies and found no significant effect. Which one you cite changes what your sentence is allowed to say, and finding both takes about ten minutes once you know where to search.

The walkthrough below uses the Paper workspace in the osFoundry desktop app, which searches OpenAlex (a free, open index of scholarly papers) from your own computer and keeps each source attached to the sentence it supports. The same method works with any academic search; the app just keeps the pieces in one place. The paper writing and research workspace page describes the rest of it.

Mark the sentence that needs a source

Open Paper, click New paper and paste the draft. A paper in the Local workspace is saved on the computer, not on a server, which matters if the draft is coursework or anything unpublished.

Drag over the claim and click Create. That turns the passage into a thinking space: a side panel for notes and searches that belong to that one sentence.

The claim highlighted in the draft, with its thinking space open on the right

The app stores the 48 characters on either side of the quote along with it, so after the paragraph has been rewritten it can still find which sentence the notes were about.

Search with the researchers' words, not your sentence

Pasting the whole sentence into a search box mostly finds opinion pieces. Researchers have a name for most effects, and searching for that name finds the literature. Here it's the "mere presence" of a smartphone, so the search is mere presence of smartphone.

In the thinking space, click +, choose Academic search, type the key words, tick Open access and search.

Academic search for "mere presence of smartphone" with Open access ticked: 33,621 papers via OpenAlex

OpenAlex returned 33,621 open-access papers for that phrase. You don't need to read them. You need two kinds of result: the original study people are quoting, and anything that summarises many studies at once.

Check the famous paper against a review

The sentence in the draft traces back to "Brain Drain: The Mere Presence of One's Own Smartphone Reduces Available Cognitive Capacity" (Ward et al., 2017, Journal of the Association for Consumer Research). It's the paper that made the idea popular, with 787 citations in OpenAlex at the time of the search, and it does report that participants did worse on capacity tests when their phone was on the desk.

Citation counts measure how famous a paper is, not whether later work agreed with it. For that, look for a meta-analysis or a systematic review on the same question. The second result is one: Hartanto et al., 2024, "The effect of mere presence of smartphone on cognitive functions: A four-level meta-analysis". Click View to open it.

The 2024 meta-analysis opened with View: 166 effect sizes from 33 studies and 4,368 participants, no significant effect

The abstract has the numbers that matter. It combined 166 effect sizes from 53 samples in 33 studies, 4,368 participants in all, and found that the mere presence of a smartphone had no significant effect on cognitive outcomes (d = −0.02, 95% CI [−0.06, 0.01], p = .246). A 2022 attempt to replicate the original Brain Drain experiment, published in Acta Psychologica, is also in OpenAlex and is worth reading alongside it.

View looks for a free copy of the full text by itself: the open-access link from the search result first, then arXiv, then open copies of the DOI, then CORE and PubMed Central. If there isn't one, it says so rather than opening a paywalled page.

Save the source next to the sentence

Click Save source. The paper goes into References with its DOI and stays linked to the passage, so the next time you open the paper you can see which sentence each source is for.

The meta-analysis saved in References with its DOI, linked to the claim

Then rewrite the claim so it matches the evidence. Something like: "A widely cited 2017 study found that a nearby phone reduced performance, but a 2024 meta-analysis of 33 studies found no significant effect (Hartanto et al., 2024)." That is a more interesting sentence than the original, and it's one a reader can check. Collecting quotes and highlights for several claims this way is what the research notes side of the app is for.

"No significant effect" is not the same as proof of no effect. The confidence interval runs from −0.06 to 0.01, which means any real effect is likely to be very small. "Studies show it makes you worse at thinking" overstates that by a long way.

Top comments (0)