Let’s look at the definition of literacy. If you type “define literacy” into Google, you get this:

lit·er·a·cy
/ˈlidərəsē,ˈlitrəsē/
noun

• the ability to read and write.
  synonyms: ability to read and write, reading/writing proficiency;
• competence or knowledge in a specified area.
  “wine literacy can’t be taught in three hours”

So clearly a literate person can both read and write. Could we then assume that “science literacy” means the ability to “do” science? I think so. That’s the definition I am going with. Oh wait! That doesn’t mean you have to be a scientist. As Chad Orzel always like to say, “science is what makes us human” (read more in his excellent book Eureka: Discovering Your Inner Scientist).

But what does it mean to DO SCIENCE? As a human, I would expect a scientific literate person to be able to do the following:

• Build a model based on experimental evidence (physical model, conceptual model, mathematical model).
• Use and understand some existing models (you can’t build everything yourself).
• Understand the limitations of science.
• Think of an experiment to test a particular model.

Really, it’s all about models. It is not about “things.” Science literacy is not about knowing stuff about science. It’s not about knowing the difference between a planet and a dwarf planet (because really, who cares what you call Pluto—it’s still awesome). Knowing that light travels faster than sound doesn’t mean you are a scientist any more than knowing the first 100 digits of pie means you are mathematician.

Measuring Scientific Literacy

Now we get to the reason for this post. It started with this NPR post: Pop Quiz: How Science-Literate Are We, Really? I thought—oh hey, a science quiz. I might win at this little contest. I love science. So I took the 11 question science quiz. The quiz sucks (in my opinion).

What’s wrong with the quiz? The first problem is that it only tests the “facts” and not real science. For example, there is a question about whether lasers use light or sound. If I get that wrong, does that mean I don’t know anything about science? No. It means I don’t know anything about lasers. Ok, I will admit that I would expect reasonable people to know that lasers use light—but it’s not really a test of science literacy.

The second problem is that I didn’t get a perfect score. Why not? Am I a dummy? Well, yes—but still I should get these correct. One of the questions asked: True or False, The universe began with a huge explosion? I thought about this for a second and considered “true”—but was it an explosion? An explosion is the process of something expanding into something else. For the Big Bang (the event, not the TV show), the universe itself expanded. I don’t think it would classify as an explosion. Of course the quiz expected the answer of “true”—so I’m science illiterate.

Are There Better Science Questions?

Here is the real problem: testing for science “facts” is easy. Testing for understanding in science is pretty darn tough—especially if you want a quiz in multiple-choice format. It’s difficult, but I think it’s possible.

I have a couple of examples. I don’t remember where this comes from, but here is an awesome question that tests if students understand the scientific model of the phases the moon.

If the moon was a cube instead of a sphere, what would it look like (from Earth) if it was half way between full and new moon?

1. A curved line inside a square.
2. A fully illuminated square.
3. A square that is half illuminated and half dark.
4. A dark square.

Maybe this question would be better with pictures. Oh well. I think the best answer would be “a dark square”. Maybe I will make a VPython model to show this sometime. However, the point is that this is NOT a recalling “fact” question. It’s still not the best test of scientific literacy because it is entirely possible that a human understands science but has not considered the phases of the moon model.

How about another one? Suppose you have a frictionless cart with a fan that pushes with a constant force. You release this cart from rest and you find that it constantly increases speed over a distance of 1 meter. What will happen if you repeat the experiment with a distance of two meters?

1. The cart will keep speeding up.
2. The cart will reach a constant speed.

Still not the best question, but it has enough information that a human could build a simple model about the cart just from the question. The best answer to this question is that the cart will continue to increase in speed. There is no evidence to suggest that it would stop speeding up.

What other questions could you ask? I’m not sure. But I think a better test of scientific literacy could be created. If your goal is to test the general population’s knowledge of scientific “results” (or you could call them facts if it made you happy), the linked quiz might work. Still, some of the questions should be changed for clarity.