The CRAAP Detector—A Tool For Evaluating Information Resources

by Kari Swanson, illustration by Jennie Bernstein 

What is research

This post doesn’t have anything to do with what’s in your baby’s diapers… unless you’re looking for valid information about what’s in your baby’s diapers, in which case it might be a very useful tool for you. The CRAAP test doesn’t have anything to do with crap and a whole lot to do with determining if the information you’ve found, regardless of whether that information is in a print resource or online, is valid, partly valid, or if it’s… well, just plain crap.

What is the CRAAP test? The CRAPP test is a tool that can be used to facilitate evaluating information resources. It was developed by librarians at the California State University at Chico’s Meriam Library (www.csuchico.edu/lins/handouts/eval_websites.pdf). CRAAP is an acronym that stands for Currency, Relevance, Authority, Accuracy, and Purpose. The CRAAP test poses questions in each area that it assesses to help you to determine if a particular source is more or less valid—it’s really a fluid scale not a black or white answer.

While the CRAAP test was developed by librarians for use by college students in evaluating resources to support research papers, it can be used by anyone to evaluate the validity of any resource used to answer a particular question.

Currency

The first questions in the CRAAP test are about the currency of the source. They ask questions about the timeliness of the information:

  • When was the information published or posted?
  • Has the information been revised or updated?
  • Does your topic require current information, or will older sources work as well?
  • Are the links functional?

It is very important to think about whether the answer to your question requires current information and, if it does, to determine if the source you are evaluating is current, has been revised or updated and (if it is an online resource) has functional links. An online resource that contains a bunch of broken links is almost certainly not up to date. And if a print resource has a copyright date before your grandparents were born you might want to consider more recent material.

Relevance

The next set of questions in the CRAAP test are about how well the resource relates to your information need:

  • Does the information relate to your topic or answer your question?
  • Who is the intended audience?
  • Is the information at an appropriate level (i.e. not too elementary or advanced for your needs)?
  • Have you looked at a variety of sources before determining this is one you will use?
  • Would you be comfortable citing this source in your research paper [or to answer your question]?

If you are using the CRAAP test to evaluate a resource for answering a particular question, but not necessarily for a research paper, it is important for you to think about what kind of information resources you think will answer your question and if the resource you are evaluating is that kind of resource. For example, if you have a question that you think will best be answered by a scholarly research article and the resource you are evaluating is a newspaper article about the research the newspaper article will probably not thoroughly answer your question, because it will probably only provide a very brief summary of the research. And, some newspaper articles and blog posts about scholarly research are notoriously bad at summarizing scholarly research and occasionally present conclusions that the research does not actually support.

Authority

The next group of questions in the CRAAP test relate to the authority of the source of the information in the resource:

  • Who is the author/publisher/source/sponsor?
  • What are the author’s credentials or organizational affiliations?
  • Is the author qualified to write on the topic?
  • Is there contact information, such as a publisher or email address?
  • Does the URL reveal anything about the author or source? Examples: .com .edu .gov .org .net

Again, even if you are not writing a research paper it is still important to think about the authority of the source of information in a resource. Anyone can publish anything on the Internet, so it is important to understand who the source of information is when evaluating a resource in order to determine if the resource is valid. URL’s can reveal information about source, because some URL’s can only be used by certain kinds of organizations. For example, only academic institutions can have a .edu URL and only government agencies can have a .gov URL.

Accuracy

This set of questions pertains to the reliability, truthfulness and correctness of the information:

  • Where does the information come from?
  • Is the information supported by evidence?
  • Has the information been reviewed or refereed?
  • Can you verify any of the information in another source or from personal knowledge?
  • Does the language or tone seem unbiased and free of emotion?
  • Are there spelling, grammar or typographical errors?

The type of resource provides clues to whether information has been reviewed or refereed by other experts. Scholarly articles are usually peer-reviewed, which means experts in a field have reviewed the research and determined that it meets rigorous standards. Even if a resource isn’t peer-reviewed, the information it presents can be supported by evidence in the form of citations or links to other sources of information (which may or may not themselves be valid). Be wary: satire, spoofs and intentional falsehoods abound on the internet. There are whole web sites dedicated to non-existent species of animals that unfortunate people have been tricked into believing are real (e.g. look up “tree octopus”).

Purpose

The last set of questions in the CRAAP test pertain to the reason the resource exists.

  • What is the purpose of the information? Is it to inform, teach, sell, entertain or persuade?
  • Do the authors/sponsors make their intentions or purpose clear?
  • Is the information fact, opinion or propaganda?
  • Does the point of view appear objective and impartial?
  • Are there political, ideological, cultural, religious, institutional or personal biases?

Knowing the purpose of a resource can help you to determine whether it is valid or reliable. You may find good information on a commercial site that sells products related to your question, but it probably shouldn’t be the only resource you use to answer the question since it is quite likely that they will present information that is biased in favor of their products. Sometimes it is easy to determine bias and sometimes it is much more difficult. Sometimes to answer these questions you have to consider answers to previous questions. For example, you may need to consider an author’s affiliations and expertise to determine if there is bias of some kind or whether a resource he or she wrote is fact, opinion or propaganda.

Answering all of the questions in the CRAAP test will help you to determine if a resource is more or less valid for the purposes you need it. If you are not confident that a resource meets the level of reliability or validity that you need to answer your question you can move on to other resources. If you find a resource that meets the level of reliability or validity that you need, but you want or need more information you can use that resource as a means to find related resources by looking for other resources that it cites or that cite it. When you’re doing your personal research to assist you in your decision making, it is a good idea to ensure it all passes the CRAAP test even if you won’t be publishing it anywhere.

And, if you aren’t sure or you get stuck, ask the expert searchers: your local public or academic librarians. We excel at knowing the difference between CRAAP and crap!

_______________________

Kari Swanson

 Kari Swanson is a college librarian, editor, photographer who invests much of her free time supporting other women in their breastfeeding journeys. Kari lives with her two children and husband in beautiful Northeast, USA.

Hyperemesis Gravidarum and Ondansetron: a critical response to media coverage

by Kari Swanson

In the interest of full disclosure: I am the survivor of two pregnancies with Hyperemesis Gravidarum during which I was given Zofran and/or ondansetron. Both of my children, currently ages 9 and 4, are developmentally normal, with no health problems attributable to my use of ondansetron.

 

Birth defects from Zofran in pregnancy

A recent headline in the Toronto Star proclaims in bold face “Birth defects blamed on unapproved morning sickness treatment.” The lengthy piece about the drug ondanestron, which is sometimes prescribed off-label to women with Hyperemesis Gravidarum (HG), is written as an exposé of drug companies and physicians gone wrong, the result of which is “vulnerable” pregnant women being prescribed a harmful drug that causes their babies to be born with extraordinary birth defects as a result. This would be horrifying if it was true, of course, but, it’s not. This article is not a scientific article. It is not a scientific literature review. It is pseudo-scientific sensationalism. One might think the goal of such an article would be to protect women and children, but like most pseudo-science involving medicine it presents very real public health risks.

According to the Hyperemesis Education & Research Foundation web site, HG is

“…a severe form of nausea and vomiting in pregnancy. It is generally described as unrelenting, excessive pregnancy-related nausea and/or vomiting that prevents adequate intake of food and fluids. If severe and/or inadequately treated, it is typically associated with:

  • loss of greater than 5% of pre-pregnancy body weight (usually over 10%)
  • dehydration and production of ketones
  • nutritional deficiencies
  • metabolic imbalances
  • difficulty with daily activities”

HG is not morning sickness. HG is just plain sickness. In my first pregnancy there were many days when I vomited in excess of once an hour. I lived with constant nausea. I vomited before getting out of bed when I woke up in the morning. I vomited in the shower. I vomited on the side of the road while driving to work. I vomited in my flower garden while weeding. I believe my record was 38 times in one day. I vomited so violently and so frequently at one point that tiny blood vessels broke in my face and eyes. It certainly wasn’t the pregnancy glow I had envisioned.

I tried all of the remedies that everyone, including my obstetricians, suggested: crackers, sips of water, lemon, ginger ale, ginger tea, ginger snaps, candied ginger, extra vitamin B, extra sleep. Nothing worked. I lost weight. At 3 months pregnant I weighed about 11% less than I did before my pregnancy. I had several visits to the ER for IV hydration. More than half way through my pregnancy, after an all-night stay in the ER for IV fluids, my OB finally prescribed Zofran. For me it was a miracle drug, because it meant that I was able to keep at least one meal down every day. It didn’t completely eliminate the symptoms, but it did make them much more manageable. When I became pregnant again 5 ½ years later and started vomiting numerous times every day at 5 weeks pregnant I asked for Zofran. I still experienced nausea and vomiting throughout my second pregnancy, but nothing like I experienced the first time. I only required IV hydration in the ER once the second time around.

HG presents serious risks to a woman’s health. Complications of HG include: dehydration, malnutrition, damage to tooth enamel, renal failure, jaundice, ruptured esophagus, and deconditioning of the heart muscle, just to name some. Some of these complications can be and have been fatal. In addition, HG can cause long term health effects. Some women experience PTSD. Others, like me, develop complications of their complications: prolonged dehydration caused me to develop kidney stones.

HG also presents risks to the child. Fetal complications of HG include: premature birth, low birth weight, neural tube defects, and congenital heart defects, among others. Also, according to the Hyperemesis Education & Research Foundation, “…prolonged stress, malnutrition and dehydration in the mother can potentially put an unborn child at risk for chronic disease (e.g. diabetes, heart disease) in later life.” And, HG can also cause fetal or neonatal death.

Clearly Hyperemesis Gravidarum is a serious health condition. It is not something that can be or should be ignored or treated lightly. Women die. Babies die. When considering treatment options, women and their healthcare providers must weigh the benefits and risks of particular treatments. The decision is not about a minor inconvenience. It is very often a matter of mitigating potential harmful or life-threatening effects.

The Toronto Star article cites data recorded in the US Food & Drug Administration (FDA) Adverse Event Reporting System (FAERS). The reporters cite this data as if it presents irrefutable proof that ondanestron is a dangerous drug that caused harmful effects to babies. But, the truth of the matter is it does no such thing. The FDA states on the FAERS site:

FAERS data do have limitations. First, there is no certainty that the reported event (adverse event or medication error) was actually due to the product. FDA does not require that a causal relationship between a product and event be proven, and reports do not always contain enough detail to properly evaluate an event. Further, FDA does not receive reports for every adverse event or medication error that occurs with a product. Many factors can influence whether or not an event will be reported, such as the time a product has been marketed and publicity about an event. Therefore, FAERS data cannot be used to calculate the incidence of an adverse event or medication error in the U.S. population.

Let me reiterate: the FDA does not require that a causal relationship between a product and event be proven. This means that random, purely coincidental health conditions may be reported as side effects of a drug. FAERS data are useful for looking for trends or potential side effects that might have been caused by a drug, but they are not proof that a side effect is caused by a drug. FAERS data should be used for further study. They should not be construed as concrete evidence of a causal relationship.

The use of ondansetron in pregnancy has been studied. The Toronto Star article sites some of the research, but a news article is not a scientific literature review. The reporters do not present the totality of research on the subject, and what they do present is presented in a manner that shows bias in favor of their own assertion: that ondansetron causes birth defects. There are, however, numerous other scientific studies that indicate otherwise. Anyone can easily research the topic for herself (or himself) by utilizing the freely available medical research database PubMed.

Although it has been studied, the use ofondansetron in pregnancy is considered an off-label use. It is unfortunate that the Toronto Star article presents off-label use of medications so negatively. Off-label prescribing is common and sometimes is the best or only treatment option for certain conditions. While it is true that sometimes off-label use of a medication might later be proven (by research) to be of no therapeutic value, or worse: harmful, sometimes off-label use later becomes an FDA approved use after further research supports it. According to WebMD certain beta-blockers once only used for the treatment of high blood pressure and used off-label to treat heart failure later became approved prescription treatments for heart failure.

Despite the fact that use of ondansetron to treat HG is off-label, the prescribing information for Zofran (ondansetron from GlaxoSmithKline) states:

Pregnancy Category B. Reproduction studies have been performed in pregnant rats and rabbits at daily oral doses up to 15 and 30 mg/kg/day, respectively, and have revealed no evidence of impaired fertility or harm to the fetus due to ondansetron.

Pregnancy Category B is defined by the FDA as follows: “Animal reproduction studies have failed to demonstrate a risk to the fetus and there are no adequate and well-controlled studies in pregnant women.”  The safety and categorization of drugs varies from country to country. For example, acetaminophen is US FDA Pregnancy Category C (less safe to use in pregnant women than ondansetron!), but in Australia acetaminophen is Australian Pregnancy Category A, which means Australia considers acetaminophen to be safer for pregnant women than the US FDA does.

Ondansetron is not entirely without risks or side effects.   The FDA, like Health Canada, issued warnings about potential heart risks, specifically a heart rhythm problem called QT prolongation. However, those risks very clearly apply to people with Long QT syndrome, those with underlying cardiac defects, those with low potassium or magnesium, and people taking other medications that can cause QT prolongation. They did not withdraw the drug from the market. Many thousands of people have taken ondansetron with no apparent harm to their hearts.

Furthermore, many women, including those who have not taken ondansetron or any other drug, give birth to babies with birth defects every year. According to the US Centers for Disease Control and Prevention (CDC), “Birth defects occur in about 3% of all live births.” Recent CDC data on the prevalence of birth defects in the US between the years 2004 and 2006 show an estimated prevalence of 4.71 in 10,000 babies born with atrioventricular septal defect, for example. HG is associated with an increased risk for fetal cardiac defects, but women without HG and who have not taken ondansetron also give birth to babies with heart defects. According to the National Heart, Lung and Blood Institute, “doctors often don’t know why congenital heart defects occur.” Leaping to the conclusion that the heart defect of one infant reported in FAERS was caused by ondansetron is wildly inappropriate.

I do not speak for all women who have experienced HG, but I and more than one of my “HG sisters” found the Toronto Star article disturbing. It is sensationalist journalism that has the potential to cause women or their healthcare providers to delay or avoid effective treatment at the risk of their/their patients’ own or their babies’ immediate or long-term health or even at the risk of their lives.

For another look at the safety of odansetron during pregnancy, see this Huffington Post piece breaking down a Danish study of 1,970 births where the drug was used during pregnancy.

 

kariswansonTLBKari Swanson, MLS, is a daughter, sister, wife, mother of two, member of Generation X and an admin for The Leaky B@@b Facebook page. She has been an academic librarian for 15 years. She blogs occasionally over at Thoughts from BookishMama