Are Scientific Papers Worth Reading?

~ J.S. Pailly

Hello, friends!

So over the course of the last few months, I’ve been learning about metascience.  I’ve been reading lots of metascientific articles and papers, and I’ve been watching a few metascientific lectures on YouTube.  For those of you who are unfamiliar with the concept, metascience is the scientific study of science itself, for the specific purpose of identifying fraud, correcting errors in the scientific process, and making science overall a more accurate and trustworthy thing.

Before I go any further with this topic, I think it’s extra important for you to understand who I am and what my perspective on science (and metascience) is.  I am not a scientist.  I have no professional or educational background in science.  What I am is a science fiction writer who wants to do his research so that science (as I portray it in my fiction) is accurate.  Well, somewhat accurate, or at least somewhat plausible.  At the very least, I want to make sure the science in my stories is not laughably implausible.

In order to do my research (as a science fiction writer), I have challenged myself to read peer-reviewed scientific papers.  I try to read at least one peer-reviewed paper each week.  As you can imagine, this is not easy.  These papers are packed full of jargon (some papers define their own jargon; most do not) and a whole lot of math (the kind of math where you see more of the Greek alphabet than Arabic numbers).

And now I learn, thanks to metascience, that the peer-review process is deeply flawed, and that science has way more problems than I ever realized.  There’s a lot of fraud going on, and also a lot of laziness and complacency, and scientists are not double checking each other’s work the way that they should.  That last problem—scientists not double checking each other’s work—is commonly known as the replication crisis.  It’s a problem which this article from Vox.com calls “an ongoing rot in the scientific process.”

No branch of science is immune to these problems, but I can take some solace in the fact that some branches of science seem to be more afflicted with problems than others.  Fields like medical science, computer science, and engineering (i.e.: the big money-maker sciences) are far more prone to fraud than fields like cosmology, astrophysics, or planetary science (i.e.: fields that I, as a science fiction writer, take the most interest in).  But still, as I said, no branch of science is immune.  Lazy and/or biased and/or unscrupulous researchers are everywhere.

And yet, despite some very valid concerns, I intend to keep reading these peer reviewed papers.  Why?  Because my alternative would be to get most of my science news and information from the popular press.  When it comes to science, the popular press has an annoying tendency to dumb things down, to gloss over boring (but important) details, and to hype up hypotheses that are the most likely to attract clicks and views but are the least likely to actually be true.  If I wrote my Sci-Fi based solely on what I read in the popular press, the science in my fiction would be laughably implausible.

I’d rather struggle through reading a peer-reviewed paper once a week.  Those papers may not be perfect, but reading them will get me much closer to the truth than relying on any other source of information currently available to me.

WANT TO LEARN MORE?

If you’d like to learn more about metascience and the replication crisis, I suggest checking out some of the links below.  These links are organized from “easiest and most accessible” at the top to “most technical” at the bottom.

21 thoughts on “Are Scientific Papers Worth Reading?”

  1. I have read (or listened) to a few books on the subject and as someone who is scientifically trained it hurts my soul to see just how far from the see scientific ideal so many have fallen. It irks me that when people try to replicate some research and show different results, journals, who should be willing to present the new data, refuse to publish. Especially if it’s a famous piece of research that gets a lot of citations. It often takes metascientists to point of the flaws when the reviewers are meant to do that. I recently read an article from a few years ago that was ostensibly about one thing but was mainly an attack on another archaeologist’s ideas about autism and used outdated information to say essentially “Autistic people can’t do xyz, so they’re just a burden”.

    Liked by 4 people

    Reply

    1. I always knew science in general and the peer review process in particular were not perfect. It’s been upsetting to learn more about metascience and discover that the problems are way worse than I imagined.

      With the stuff I research, the worst that could happen is I get some space facts wrong. But the cases of bias and fraud in other fields are downright terrifying. Your autism example is apt. When that sort of thing goes unchallenged and has the veneer of being legitimate science, it can do a lot of real harm to a lot of real people.

      Liked by 3 people

      Reply

  2. You are a brave and noble lad to enter the arena of jargon. I understand why – I use jargon in my own field. But it makes cross-discipline work difficult, and many papers are inaccessible outside of their own rarified world.

    Liked by 2 people

    Reply

    1. The more you read in any given field, the easier the jargon gets. And jargon does have it’s value. It saves a lot of time to talk about a moon’s libration rather than talking about the way a moon appears to wobble back and forth due to changes in its orbital velocity. But if you’re new to a field and you don’t know the lingo yet, it does make things a lot harder.

      Liked by 1 person

      Reply

  3. I think you said it best the other day when you mentioned that scientific papers should be viewed as part of an overall conversation. Sometimes it can make sense to drill down into a particular part of that conversation, but most of us don’t have the time or energy to do it consistently or comprehensively.

    There’s a lot to be said for reading books by experts in the field. They can give a broad overview, including an expert assessment of what parts of the scientific literature are worth paying attention to and which portions aren’t taken seriously by the field, something difficult to glean by looking at individual papers. (It does pay to read multiple authors with different viewpoints.) The trick is finding books that are detailed enough to be interesting while still being understandable. Ideally, you want ones with citations, so you can drill down into the details when you need them.

    Wikipedia can also be useful, as long as you don’t take it itself as authoritative, but use it as a way to discover key papers and books in the field.

    It’s also worth noting that there are relatively trustworthy news outlets that do make an effort to get the science right. Admittedly none are perfect. Some I currently try to monitor are Ars Technica, Live Science, Science News, Science Alert, and Universe Today.

    Liked by 3 people

    Reply

    1. The broader conversation is really important. That’s why I try to read a paper a week, because I often end up reading responses and refutations to the previous research I read. That gives me a better sense of what the conversation is really about and where it might be heading.

      Books are good too, if they cite sources. There was one book I read a while back that had some really interesting speculation about life on rogue planets. But there were no sources cited, nor any suggestions for further reading. I’ve avoided reading anything else by that author ever since.

      Live Science and Universe Today are pretty good. I turn to those a lot. I’ve read Ars Technica more sporadically. And then there’s Scientific American. They don’t cover nearly as much space news as I’d like, but I consider them a highly reliable source.

      Wikipedia has its place in my research process too. It’s a good place to start. And even reports in the popular press can be useful places to start.

      Liked by 2 people

      Reply

      1. Some authors don’t do actual citations, but they at least name the scientists who came up with a theory or did particular experiments, enabling you to find the papers. But yeah, a lot of the shallower pop-science stuff doesn’t do even that.

        The Ars Technica site has a lot of technology news mixed in, but they have a straight science feed, which is what I’m subscribed to.

        I’ve had mixed experiences with Scientific American. Their straight news is reliable enough. But I found too many of their opinion pieces verging on, if not outright, unscientific. And they limit free views. I used to have a subscription, but had technical problems with it, and didn’t find they had enough unique content to justify keeping it.

        Liked by 2 people

      2. I recently dropped my Scientific American subscription, mainly because I wasn’t getting as much space news as I wanted, but also because of some of the things you said. I didn’t really mind their opinion pieces, because they were very clearly opinion pieces; but again, that’s not really what I wanted to read.

        All that being said, if I come across a Scientific American article that’s relevant to my interests, I’ll read it. They tend to be a pretty reliable source as well.

        Liked by 2 people

  4. When I see astronomy articles in the popular media, I will sometimes track the discovery back to a reliable source, which frequently turns out to be a scientific paper, written for peer scientists not the general public. Given that, I usually won’t wade through it and try to understand every graph and every paragraph – but I will read the introduction at the front and the conclusion or summary at the end. My hope is always to increase my knowledge in some small way and to confirm whether the original story is reliable.

    I have always naively thought that it would be helpful if the scientists who author papers could include a plain language section at the end of each paper which describes in a few paragraphs the purpose of the paper and the conclusions it reaches so that you, me and the journalists who write about it can have a clear understanding of the concepts.

    I’ve noticed that sometimes even the media officer (at the university or other organisation) doesn’t always grasp the concepts either…..

    Liked by 3 people

    Reply

    1. It would be useful if the scientists’ home institution put out press releases that accomplished your suggestion. Sometimes I stumble onto such a release. I imagine a lot of the popular press I see is based on press releases.

      Liked by 3 people

      Reply

      1. The quality of those press releases varies widely. There are a bunch of sites that basically just collect and publish all them for a particular field. ScienceDaily.com is the most general one I’m aware of, although there are lots of more specialized ones. The nice thing is they usually do link to the paper the press release is about.

        Liked by 3 people

      2. Oh, it’s worse than that, my friend! A lot of science news gets filtered through the Associated Press or some other organization like them before it gets picked up by national or local news outlets. It’s like a game of telephone, with several steps between the researchers and the general public.

        Liked by 2 people

    2. Yeah, I’ve noticed that too. Press releases from universities and other institutions are often hit or miss.

      I have seen plain language summaries included with some papers. I think there are a few journals that do require that. But it’s not a common thing.

      And in most cases, just reading the abstract and the conclusion is enough. Although sometimes I do come across juicy bits of extra info when I read the paper in its entirety. I also sometimes catch red flags hidden in the body of the paper, like researchers outright admitting that they’re only sharing the data that supports their hypothesis.

      Liked by 2 people

      Reply

    1. I’ve heard a lot of snarky comments about researchers just skimming through these papers or not reading them at all. I don’t know if that’s true. But I do hear a lot of snarky comments about it. 😉

      Like

      Reply

  6. Oh, dear! It never occurred to me that peer reviewed stuff has problems, but it has to be better than someone going on YouTube spouting medical conspiracy theories for the idiots who say, “I do my own research and make up my own mind.” I do think you are brave! To be honest, I use New Scientist and such to get ideas, and books to do research. Even children’s books can be helpful to explain stuff I don’t understand. I also ask someone who knows a subject better than I do. For example, I wrote a short story set in 19th century Australia, about bushrangers, and asked a friend who knows ALL about that era. She made a few minor suggestions and I was able to submit it with a clear conscience. Another time I was editing a science fiction publication and passed on a hard SF story to a physicist I knew. He did make a correction. (The author admitted he had got it wrong, for creative reasons, but hoped we wouldn’t notice. He is now selling to Analog)

    Liked by 1 person

    Reply

    1. No source is ever going to be perfect, but relying on the popular press and YouTube videos is bound to get you into trouble.

      I’ve left errors in my stories on purpose too. The story comes first, as far as I’m concerned. Then again, I’m not writing hard Sci-Fi, so I think I’m allowed a bit more leeway.

      Like

      Reply

  7. Unless you are a scientist, then the science fiction novel can’t be completely correct. Because you won’t have a real connection with science. It takes years of hard work and research, study, and more. But we’re talking about science fiction. It’s all made-up.

    Liked by 2 people

    Reply

    1. I think making an effort to learn this stuff helps, though. Even if the science isn’t perfect, it’s easier for readers to suspend their disbelief if the science kind of matches up with reality.

      Liked by 1 person

      Reply

      1. I can see your point. Here’s a view. First, I like the idea of learning and sharing, and I’ve always enjoyed science fiction, provided there’s a real-world aspect, most notably, human behavior. I think that’s why I always like the 4th Star Trek movie the best, the one where Kirk, Spock, and McCoy retrieve a whale from the past, never mind all the propaganda, but that’s also a notable, for people are easily propagandized. Here’s the point: Enjoy writing and sharing. There are a ton of readers, however, if you make points based upon faulty understanding of data and scientific studies, that will live their lives in oblivion, thinking they know and feeling they’re smart because they read these books. You can’t help that, but writers do promote fantasy in error. Think of Clive Cussler. Fantastic writer. Full of BS when it comes to real science. Fun to read. But discernible readers will be able to see his own contradictions. Unfortunately, all too-many readers like to think they’re smart when they can’t even see contradictions, and they become pseudo-scientists in the blogging world, and elsewhere, claiming the sky is falling because pc politicians say that. **Hope you write some cool books.

        Liked by 1 person

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.