Category: Behind the Science

Sciency Words: Academic Paper Mills

~ J.S. Pailly ~ 6 Comments

Hello, friends!  Welcome back to Sciency Words, a special series about those weird and wacky words scientists use.  In this week’s episode of Sciency Words, we’re talking about:

ACADEMIC PAPER MILLS

A paper mill is a factory that produces paper.  It’s a perfectly legitimate business.  An academic paper mill is a business that, in an almost factory-like manner, cranks out fraudulent academic papers.

This term came up in my ongoing research about research.  Academic paper mills are a growing concern in the scientific community.  An extraordinary number of these paper mill papers have gone through the peer review process and been published in highly respected journals.

Distressingly, even when the origins of a paper mill paper are exposed, publishers do not always make that clear.  As this article from Nature explains:

Publishers almost never explicitly declare on retraction notices that a particular study is fraudulent or was created by a company to order, because it is difficult to prove.

Even so, that same article from Nature says that at least 370 published papers have been retracted since January of 2020 due to their suspected paper mill origins.  Another 45 have been flagged with “expressions of concern” by the journals that published them.  And since academic journals started cracking down on paper mill papers, it seems that some researchers have decided to voluntarily retract their own research “without stating the reason for retraction.”

Based on what I read in that Nature article, as well as in other articles like this one from Chemistry World, I get the sense that this is a bigger problem in some scientific fields than it is in others.  Fields like biomedical science, computer science, and engineering seem to be getting paper milled the hardest—in other words, fields where there’s the most money to be made and where researchers are under the most pressure to rack up publication credits.

For my own purposes as a science fiction writer who wants to do his research, I read a fair number of academic papers: mostly papers on astrobiology and planetary science.  I doubt I have to worry much about paper mill papers in those fields.  There are, however, other red flags I know to look out for.

How to Be Smart

~ J.S. Pailly ~ 14 Comments

Hello, friends!

So there’s this anecdote I heard once, way back when I was a kid, about a math teacher who didn’t know the value of pi.  This teacher had to stop in the middle of class and look the number up in a book.  Naturally, this drew some snarky comments from the students.  The teacher replied, sagely: “Why should I waste valuable brain space on information I can easily look up?”

Why indeed?

I haven’t been doing much research lately.  Right now, I’m trying to pick the habit up again, and I thought I’d start by doing a little research on how to do research.  Specifically, I thought I could use a refresher course on how to tell the difference between facts and fabrications on the Internet.  I wound up reading several papers (this one, this one, and this one), and I still have at least one more paper (this one) that I want to read.  So what have I learned so far?

Well, the main take away from my research on research is that a lot of people implicitly share the philosophy of that math teacher who didn’t know the value of pi.  I may not know the answer, but I know where to find the answer, and in the end that’s good enough.  And maybe it is good enough, so long as you recognize that you’re getting your information from an external source.

Unfortunately, according to this paper from the Journal of Experimental Psychology, the act of using a search engine can trick our brains into thinking we know more than we actually we do.  In a series of memory-related tests, people tended to overestimate their “unplugged knowledge” and underestimate their dependency on Internet search engines.  You don’t even have to have successful search engine results to get this inflated knowledge ego.  As the paper explains:

The illusion of knowledge from Internet use appears to be driven by the act of searching.  The effect does not depend on previous success on a specific search engine, but rather generalizes to less popular search engines as well (Experiment 4a).  It persists when the queries posed to the search engine are not answered (Experiment 4b) and remains even in cases where the search query fails to provide relevant answers or even any results at all (Experiment 4c).

I don’t think the lesson here is that we should stop using the Internet for research.  Rather, I think the lesson is that we need to stay humble.  It’s a little too easy to forget where our information comes from when information comes so easily through the Internet.  Unlike that math teacher who had to spend time flipping through a book to find the value of pi, I can just google it—or, faster yet, I can ask Siri.  But that does not mean I actually know the answer any better than that math teacher did.

P.S.: Yes, I did all my research for today’s post using Google.

Blue People Don’t Know That They’re Blue

~ J.S. Pailly ~ 6 Comments

Hello, friends!

So before anybody asks: no, the Eiffel 65 song “Blue (Da Ba Dee)” was not inspired by the work of Benjamin Lee Whorf.  I checked.  The two things are totally unrelated.

I told you about Benjamin Whorf in last week’s episode of Sciency Words.  Whorf, along with Edward Sapir, was one of the key researchers in the development of the linguistic relativity hypothesis (a.k.a. the Sapir-Whorf hypothesis).  While reading Sapir and Whorf’s original papers on the topic (from 1929 and 1940, respectively), I noticed something.  The two of them seem to spend more time defending linguistics in general as a legitimate science than they do explaining or defending linguistic relativity in particular.

I can imagine the kinds of skepticism early linguists must have faced.  I mean, language is just language, right?  You learn your grammar.  You learn your vocabulary.  And that’s that.  What more is there for this “science” of linguistics to study?

In his 1940 paper, titled “Science and Linguistics,” Whorf uses an analogy.  Imagine a race of people who, for whatever reason, are only capable of seeing the color blue.  They can see light blues and dark blues and medium blues.  The variety of shades of blue they can see must seem very impressive to them.  But in the end, all they see is blue.  If such a race of people existed, Whorf tells us, you can expect that their language would not have a word for blue.  These blue people would have absolutely no concept of blueness.  How could they?  How could anyone have a concept of blueness unless they could compare and contrast blue with other colors?

In a similar way, linguists compare and contrast languages, and by doing this they can learn far more about how language works, beyond the obvious grammar and vocabulary stuff.

Benjamin Whorf seems to have been pretty optimistic about the future of this relatively new science called linguistics.  As someone writing about 80 year later, I can say Whorf’s optimism was well founded.  Linguistics is, in my opinion, one of the coolest sciences we have!

P.S.: And seriously, the Eiffel 65 song was not inspired by Whorf, or intended as a tribute to Whorf, or anything like that.  It seems like it must have been, but apparently it was not.

Do Planets Have Genders?

~ J.S. Pailly ~ 17 Comments

Hello, friends!

So a while back, I got some unsolicited feedback from a person I know in real life.  This person had seen one of the illustrations I’d drawn for this blog, and she was incensed—absolutely incensed—that I would depict the planet Saturn as female.  You see, Saturn is a very masculine planet.  That’s a fact, apparently.

A lot of my thinking about planets—including my thinking on the gender identities of planets—was shaped by a book called Venus Revealed, by David Grinspoon.  That book was my first serious introduction to planetary science.  In a section titled “Men are from Venus, Women are from Mars,” Grinspoon has this to say:

At first I tried being completely gender-neutral in my writing, but this was unsatisfying because, to me, Venus is not just a “thing.”  Venus is not, in my mind, inanimate, and so “Cousin It” will never do to describe him… or her.

In that same section, Grinspoon does a little cross-cultural analysis and finds that Venus has been “a real gender bender” across human cultures and human history.  Sometimes she’s male; other times he’s female, depending on which mythological tradition you’re looking at.  And some cultures have apparently assigned different genders to the Morning Star and Evening Star, thus effectively making Venus genderfluid.

So do planet’s have genders?  No, of course not.  But much like David Grinspoon, I can’t see the planets as purely inanimate objects.  Planets have too much personality for that.  And since I think of the planets as having personalities, then, for better or worse, I also think of them as having genders.

For purely arbitrary reasons, I tend to think of Saturn as female.  But if you’d prefer to think of Saturn as male, or as something else entirely, that’s okay.  I’m not going to fight you over it.  I can love Saturn (and all the other planets, too) just the same, no matter what gender identities we pretend they have.

P.S.: While doing research for this post, I ended up reading a lot about how astrology assigns genders to planets (and also to numbers, elements, constellations, etc). I don’t want to dive too far down that particular rabbit hole, but I thought I should at least share this article on the subject. I used to think astrology was just silly. Now I think it’s problematic for reasons that go beyond mere pseudoscience.

Is There Life on Earth?

~ J.S. Pailly ~ 5 Comments

Hello, friends!

Let’s imagine some space aliens are cruising by our Solar System.  They turn their scanners on our planet and see… what?

Among other things, they’d notice that Earth’s landmasses are partially covered with a strange, green-colored substance.  Of course, you and I know what that green substance is.  It’s chlorophyll.  But would those extraterrestrial observers, who have no prior knowledge of our planet, be able to figure that out?  Even if they did, would they realize what chlorophyll is used for?  Maybe.  Probably not, though.

Which brings me to my all-time favorite scientific paper: “A search for life on Earth from the Galileo spacecraft,” by Carl Sagan et al.  I love this paper in part because it’s so clearly and concisely written, with jargon kept to a minimum.  Sagan was, after all, a talented science communicator.  But I also love this paper because its conclusions are so shocking, so eye-opening.

In 1990, NASA’s Galileo spacecraft turned all its high-tech instruments toward Earth and detected… not much, actually.  Galileo did pick up radio broadcasts emanating from the planet’s surface.  Aside from that, though, Galileo’s data offered highly suggestive (but also highly circumstantial) evidence on Earthly life.  The lesson: finding life on other planets is hard.  Even using our very best equipment, it was hard for NASA to detect signs of life right here on Earth!

At least that’s what I got out of reading Sagan’s Galileo experiment paper.  And based on various commentaries I’ve read or heard about this paper, that seems to be the lesson other people got out of it too.  So I was surprised to hear Sagan himself, approximately seven-and-a-half minutes into this interview, saying the exact opposite.

We’ve flown by some sixty worlds.  We claim that we haven’t found life anywhere, and that that is a significant result.  That is, that we would have found life had it been there.  But this has never been calibrated.  We’ve never flown by the Earth with a modern interplanetary spacecraft, all instruments on, and detected life here.  And so Galileo, because of this peculiar gravity assist VEEGA trajectory, permits us to do that.  And as I’ll describe tomorrow, we find life five or six different ways, including intelligent life.  And this then means that the negative results that we find elsewhere are, in fact, significant.

I’ve been puzzled by this for a while now, but I think I’ve finally figured out why Sagan would say this.  It’s politics.

On the one hand, scientists need to understand the challenges they’ll face (including the limitations of their own equipment) in searching for life on other worlds.  That really is, I think, the purpose of the Galileo experiment paper.  On the other hand, it would not do to say on public television, to cantankerous taxpayers and the listening ears of Congress, that NASA spends millions of dollars on space probes that are not even capable of detecting life right here on Earth.

Space exploration is expensive.  And like all expensive types of research, sooner or later the researchers involved have to learn how to play politics.

No Gospel Truth in Science

~ J.S. Pailly ~ 5 Comments

Hello, friends!

So there’s this notion in the popular press that when a new scientific paper comes out, that paper should be taken as the final definitive word on an issue.  Science has spoken.  This is a scientific fact now.  But that is not how science works.

When new research is published, you should expect there will be followup research, and then that followup research will be followed up by even more research.  A new scientific paper really shouldn’t be seen as a proclamation of fact but rather as the beginning of a dialogue among scientists, or perhaps as the continuation of a dialogue that’s already in progress.

The recent detection of phosphine in the atmosphere of Venus has turned out to be a fantastic example of this ongoing dialogue in action.  The initial research was published in two separate papers (click here or here).  Basically, astronomers found the spectral signature of phosphine (PH3) in the Venusian atmosphere, and they were at a loss to explain where all that phosphine could be coming from.

Based on everything we currently know about Venus, those two papers tried to rule out several possible explanations.  Such a large quantity of phosphine could not be created by Venus’s atmospheric chemistry.  It could not be spewing out of volcanoes on Venus’s surface.  It could not be delivered to Venus by asteroids or comets.  One very intriguing possibility that could not be ruled out: maybe there’s life on Venus.  On Earth, phosphine is produced almost exclusively by living things.

But those two papers were not the definitive final word on the matter.  A dialogue had begun.  Soon, followup research came out suggesting that phosphine could be spewing out of volcanoes after all.  It would still be pretty shocking to discover that Venus has enough active volcanoes to produce that much atmospheric phosphine—but it be nowhere near as shocking as discovering Venus has life.

And then even more followup research came out with this paper, which points out possible errors in the original research and suggests that we may be dealing with a false positive detection.  Venus might not have phosphine after all, or maybe it doesn’t have as much as originally believed.

And the dialogue continues.  More research will come.  More responses will be published, and then there will be responses to those responses, and so forth until the scientific community reaches some sort of consensus about this Venusian phosphine business.  And even then, that scientific consensus still might not be the 100% final word on the matter.

Based on the way the popular press reports science news, you could easily get the impression that scientific papers should be treated as gospel truth.  You would be understandably confused, then, when one scientific paper comes out refuting the findings of another.  Subsequently, you may come to the conclusion (as a great many people apparently have) that science must not know anything at all.  Science just keeps contradicting itself, it seems.

But scientific papers are not meant to be taken as gospel truth.  They’re part of an ongoing back-and-forth dialogue.  So the next time you hear about some new scientific discovery on the news, remember that scientific papers are not intended to be bold proclamations of fact.  And when you hear about some new paper refuting older research, you’ll understand what’s going on.

Daily Life with Dinosaurs

~ J.S. Pailly ~ 7 Comments

Hello, friends!

There’s an important science fact that I wish more people were aware of.  Birds are not merely the descendants of dinosaurs.  According to a taxonomic system called cladistics (also known as phylogenetic systematics), birds are dinosaurs.  To quote this article from DinoBuzz:

Using proper terminology, birds are avian dinosaurs; other dinosaurs are non-avian dinosaurs, and (strange as it may sound) birds are technically considered reptiles.  Overly technical?  Just semantics?  Perhaps, but still good science.

So with that in mind, the following statements are 100% true:

  • I often wake up to the sound of noisy dinosaurs outside my window.
  • I sometimes see dinosaurs swimming in the river near my house.
  • I hate it when dinosaurs poop on my car.
  • I enjoy eating dinosaur meat.  Sometimes I put dinosaur meat on sandwiches or in salads.

Anyway, what sort of experiences have you had with dinosaurs in your daily life?  Please share in the comments!

P.S.: Have you seen those dinosaur-shaped chicken nuggets in the grocery store? They’re cute.  I’m just not convinced that they’re made from 100% real dinosaur meat.

Science is Wrong About Everything

~ J.S. Pailly ~ 6 Comments

Hello, friends!  So one day when I was a little kid, I got into a huge argument with another kid in school.  I’d said something about how Earth is a sphere, like all the other planets.  The other kid told me (firstly) that Star Trek isn’t real and (secondly) that the earth is flat.

As evidence, the other kid told me to just look around.  It’s obvious that the world is flat.  If I needed more proof, I could look at a map.  More kids soon jumped into this argument.  They all agreed: the earth is flat, and also I’m a huge nerd for watching so much Star Trek.  I was outnumbered, and being outnumbered was further proof that I must be wrong.

I went home so mad that day.  How could those other kids be so stupid?  I was right.  Everybody else was wrong.  I’m tempted to turn this into a metaphor for Internet culture, but that’s not the point I want to make today.

Yes, when those other kids said the Earth is flat, they were wrong.  But when I said the Earth is a sphere, I was wrong too.  Less wrong, obviously.  But still, I was wrong.

Isaac Asimov’s essay “The Relativity of Wrong” is a brilliant summation of how science works.  It should be required reading for every human being (click here to read it).  As Asimov explains:

[…] when people thought the earth was flat, they were wrong.  When people thought the earth was spherical, they were wrong.  But if you think that thinking the earth is spherical is just as wrong as thinking the earth is flat, then your view is wronger than both of them put together.

As Asimov goes on to explain, there was a time, long ago, when educated people really did believe the world was flat, and they had good reasons for thinking it to be so.  But then discoveries were made.  New knowledge was learned, and people came to think of the world was a sphere.  Then more discoveries were made, and people started to think of the world as an oblate spheroid (round, but slightly bulgy at the equator).  And then still more discoveries were made, and even the oblate spheroid model turned out to be slightly inaccurate.

People (including people on the Internet) will gleefully point out that science has been wrong about stuff in the past; therefore, science could be wrong about stuff today—stuff like evolution, climate change, general relativity—also stuff like vaccinations and COVID-19.  When science is wrong so much, why pay attention to science at all?

Well, it’s true.  In absolutist (this-or-that-ist) terms, science is wrong.  Science is always wrong, about everything, all the time.  Science is full of educated guesses and close approximations of observed reality.  It’s not perfect.  It will never be perfect.  But with each new discovery, science is a little less wrong today than it was yesterday.  And you can trust science to keep being less and less wrong, even if it will never be 100% right.

And that process of constant refinement and improvement, that process of getting closer and closer to the truth—that’s something worth paying attention to, something worth taking seriously, don’t you think?

P.S.: I’ll concede that those kids in school were right about one thing.  I was, and still am, a huge Star Trek nerd.

Galactic Census Report: How Many Civilizations Are in Our Galaxy?

~ J.S. Pailly ~ 5 Comments

Hello, friends!  Have you heard the news?  Scientists have determined that there should be at least thirty-six alien civilizations in our galaxy right now.

Here’s the actual research paper from The Astrophysical Journal (warning: paywall).  As you might imagine, this research is based on some key assumptions.  And the authors do make it clear that they are making assumptions.  Reasonable assumptions, they argue, but still… ASSUMPTIONS!!!

The first major assumption is this: any Earth-like planet with an Earth-like chemical composition that happens to have an Earth-like orbit (i.e.: habitable zone) and has been around for an Earth-like period of time (approximately 4.5 billion years) has a reasonably good chance of developing Earth-like intelligent life.

The second major assumption is this: once a civilization advances to the point that it can start broadcasting its presence to the rest of the universe, that civilization will also have advanced to the point of being able to destroy itself.  Earth-like intelligent life has a tendency, the authors argue, for self-destruction.  Maybe it’ll be nuclear weapons, or maybe a climate catastrophe of some kind.  Or, I don’t know, maybe an increasingly globalized society will make itself more vulnerable to some sort of global pandemic.

Obviously the authors make other assumptions as well, but those two are the big ones.  When plugging numbers into a modified version of the Drake Equation, the most pessimistic assumptions yield an estimate of 36 civilizations in our galaxy (with a margin of error that could push that number all the way down to 4).  The most optimistic assumptions yield an estimate of 928 civilizations (with a margin of error that could push the number all the way up to 2908).

As I’ve said before, scientific papers should never be taken as proclamations of absolute fact.  That’s especially true for papers like this one.  Scientific papers are part of an ongoing back-and-forth conversation in the scientific community.  What do we currently know?  How much do we still have to learn, and what should our expectations realistically be?  That’s what this paper from The Astrophysical Journal is really about: setting expectations for SETI research.

So what should our expectations be, based on those two key assumptions the authors made?  Well, even under the most optimistic scenarios, our nearest neighbors are predicted to be hundreds (if not thousands) of lightyears away—far enough away that they’d be very, very, very difficult to find using our current technology, and establishing two way communications would be virtually impossible.

So maybe we’re not alone in the universe, but we may as well be.

Sciency Words: Null Hypothesis

~ J.S. Pailly ~ 9 Comments

Hello, friends!  Welcome to Sciency Words, a special series here on Planet Pailly where we look at the meaning and origins of scientific terms.  Today on Sciency Words, we’re talking about:

THE NULL HYPOTHESIS

Whenever there’s a big scientific discovery in the news, my first question is always: should I take this seriously?  The answer is usually no.  The popular press may say one thing, but when you dig into the actual science, you often find the facts do not support the hype.

So when I started reading about a second possible planet in the Proxima Centauri system, I wanted to know: should I take this seriously?  In this article from Scientific American, the astronomers who discovered this possible planet are quoted as saying:

Since the very first time we saw this [potential planetary] signal, we tried to be its worst enemy.

The astronomers are then quoted saying:

We tried different tools to prove ourselves wrong, but failed.  However, we have to keep the doors open to all possible doubt and skepticism.

For me, this is the most reassuring thing any scientist could say.  Too often in popular culture, scientists are portrayed a certain way.

For a multitude of reasons, this is not a real scientist.

But no, good scientists are not out to prove to the world that they’re right.  They’re trying as hard as possible to prove to themselves that they’re wrong.  Which brings me to the null hypothesis.

According to the Oxford English Dictionary and other sources (like this one), the term “null hypothesis” can be traced back to British statistician Ronald Fisher.  Fisher first wrote about the null hypothesis in 1935, in a book titled The Design of Experiments.

As a way of introducing the concept, Fisher tells us the story of a woman who claimed to have an oddly specific talent.

A lady declares that by tasting a cup of tea made with milk she can discriminate whether the milk or the tea infusion was first added to the cup.

The Design of Experiments, by Ronald Fisher

Fisher then describes an experiment to test this woman’s claim.  She’s given eight cups of tea, four with the milk added first, and four with the milk added afterward.

In the context of this experiment, the null hypothesis predicts that the woman will not be able to tell which tea is which—she’s only guessing.  Or to put that in sciencier language, the null hypothesis asserts that there will be no statistically significant relationship between the way this woman’s tea was prepared and the way she believes her tea was prepared. As Fisher explains:

[…] it should be noted that the null hypothesis is never proved or established, but is possibly disproved, in the course of experimentation.  Every experiment may be said to exist only in order to give the facts a chance of disproving the null hypothesis.

The Design of Experiments, by Ronald Fisher

A null hypothesis is usually paired with an “alternative hypothesis,” which asserts that a statistically significant relationship does exist.  In Fisher’s tea tasting example, the alternative hypothesis would be that the woman really can tell which tea is which.  You can never really prove that either the null hypothesis or the alternative hypothesis is true, but a well designed experiment should be able to prove that one hypothesis or the other is false.

Going back to that possible planet in the Proxima Centauri system, the article from Scientific American does not explicitly mention the null hypothesis; however, the spirit of the null hypothesis is clearly in play.  Astronomers are trying their best to prove that that planet does not exist, and so far they can’t do it.  And that’s enough to convince me that I should take this new planet seriously (at least for now).

Next time on Planet Pailly, we’ll find out what this not-yet-disproven planet might look like.