Identify Bias and Assess Credibility of Resources

What to know about teaching evaluation skills

  • Evaluating sources is not a skill students will be taught in one week or in one course - you're laying the foundation for deeper skills that will be honed in a student's specific discipline. 
  • You're teaching students the skills they need to evaluate viewpoints no matter where they find them - i.e. from vetted sources and from Twitter.
  • You will need to revisit the teaching of these skills throughout the semester. 
  • Evaluating sources is intuitive to you - that is not the case for early researchers.

Suggested flipped classroom approach

Try assigning this video (~8 minutes) for homework and basing the next class discussion on the content. You may use the questions in the video (pasted below) for homework points. The class discussion text below is an extended version of the video script - it may be helpful in devising more homework questions or class discussion.

Questions from video:

(on credibility):

  • Do you think you are good at choosing trustworthy sources? Think about the last time you trusted a writer or host of a show. What made you trust them? Were you right?
  • Have you ever posted a news story on social media, or passed along something you learned? Did you think twice about its credibility? How do you investigate credibility?
  • Can you think of a time that someone passed along bad information to you or a group of people? What effect did that have?

(on bias):

  • Bias is not predictive of perspective. Let’s go back to the doctor example. How does her experience with terminally ill patients affect her ideas around assisted suicide? Can you imagine it influencing her to embrace it? What about reject it? How might her experience influence either of those perspectives?
  • Identity and life experience influence one’s perspective, but always dig deep. Can you think of a time that your identity or experience had an effect on your perspective? Would it bother you if someone said that “you think X because you’re Y”?
  • Can you think of an example of the word bias being used to describe an unfair situation? How can people be affected by bias unjustly?

Class Discussion: What does it mean to be credible?

Everything below is based upon what I have seen RHE 306 students struggling with in their work. I assess students' ability to evaluate information in RS 3&4. 

Authority / expertise: someone’s education or job title can give them authority.
  • The authority has to be relevant to the controversy.
    • Ex. Is a medical doctor an authority? On what? Is he an authority on fracking? Is he an authority on the health effects fracking has on water supplies? Maybe, what kind of doctor is he? Is he a podiatrist?

Authority and credibility of a venue

  • Where do you go for reliable information? What news sources do you trust? When you have a question about something, how do you choose whom to ask?
  • Ex. You want to know more about water treatment in Austin. How safe is our drinking water? How is water treated? What chemicals are used? Are the chemicals safe? Whom would you ask?
    • City of Austin Utilities can give you water safety reports that tell you the levels of toxins or metals in your water. Where is this info published? (on their website for their customers)
    • A chemist could tell you more about the chemicals used to treat the water. Where is this info published?
    • A medical doctor could tell you more about the health effects of those chemicals. Where is this info published?
    • An engineer could tell you more about how water is transported in a city and how the facilities to treat the water are constructed. Where is this info published?
      • If you answered an academic journal, yes that is correct. That is where a lot of this specialized research is first published.
      • If you answered a newspaper, magazine or website, that is also correct. Journalists and reporters write about this research from academic journals for the general public, usually in language non-specialists can understand.
  • A note about television news and comedy news programs:
    • Comedy news programs such as Daily Show and John Oliver:
      • Who writes for these shows? What is their background? Are they journalists? Politicians? No, they are comedy writers. They may be very smart and informed, but they are comedy writers.
      • What is the purpose of these shows? To satire, to parody, to entertain.
    • News programs and news channels (Fox, MSNBC, Bill O’Reilly, Rachel Maddow, PBS Newshour)
      • Who writes for these programs and who is the host? What is their expertise?
      • What is the purpose of these shows and channels? How do they make their money? How does advertising affect how news is presented?
      • Who is the audience? Can you tell if the writer(s) or host is speaking to a particular point of view? A particular audience? How can you tell?

Credibility of a source based upon evidence presented by the author

  • When do you need evidence or proof to trust a source?
  • When an author presents evidence, does he make it easy for you to trace from where he got that evidence?
  • What kinds of evidence are most convincing?
    • The type of evidence needs to be relevant to the controversy
      • Ex. How likely is it that Austinites will vote in favor of increased train lines?
        • Public opinion polls could help you gauge attitudes towards public transportation in Austin
        • Past voting records: How did people vote on this issue in the past?
        • Exit polls, surveys and focus groups could give you insight into why people voted the way they did or how they will vote in the future. Who conducts these polls and where do they publish the results?
        • Demographics: Who rides the bus? Where do people in Austin work? How far is their commute? This could be Census or American Community Survey information
      • Ex. The plastic bag ban in Austin. Is it effective?
        • Effective at what? How would you measure effectiveness? Less bags being manufactured? Less bags in landfills? Less bags littering the environment?
        • Someone who works in Austin parks could tell you if there are less bags littering parks, water, etc.
        • Someone who works for city sanitation could tell you if there are less bags in Austin landfills.
        • Someone who works at a plastic bag manufacturer or distributer could tell you if their business has been impacted by the ban.
          • Where is the above information published? For city services, you could try websites.
          • Companies often release financial reports to shareholders and employees.
          • Typically, a newspaper or magazine will have articles written by journalists who have investigated and synthesized this data or conducted interviews and report it all to the general public. 

Class Discussion: What does it mean to be biased?

When discussing bias with students, please avoid:

  • Oversimplifying bias by explaining in terms of race, ethnicity, religion, age, sexual orientation. Identity and life experience influence perspective and should be considered, but it undermines the speaker's credibility to disregard other factors. It also stymies deeper, richer analysis of what constitutes authority and perspective. In fact, it reminds me of Donald Trump dismissing Judge Curiel's ability to try a fair case based upon his Mexican heritage. 
  • Avoid avoiding the topic of bias. Bias, the word, has been stigmatized for students (think of 'bias incidents' in college). Your job is to get them thinking about how we use this term and how bias affects how we perceive and participate in arguments. 

In class discussion

  • When considering a controversy, it is important to think about who cares about this issue and why. What is their stake in the argument? What are their commitments and investments?
    • What are commitments and investments? How does what one is committed to or invested in affect how they perceive the world?
      • A job or career can be a commitment that affects how you view a topic.
        • Ex. A doctor has experience with dying people, people who have incurable conditions. How might this experience inform their viewpoint on assisted suicide?
        • Ex. Apple, as a company, has made some strong commitments to ensuring users’ privacy by encrypting data. How does this commitment affect an Apple employee’s thinking about the topic? What inside information might the Apple employee have that you or I don’t have?
        • Ex. Conversely, how does an FBI agent view Apple’s data encryption commitment? How does encrypting data affect an FBI agent’s ability to do their job?
      • A degree or education can be an investment that affects how you view a topic.
        • Ex. A marine science major who studied the Gulf Coast knows a lot about how pollution and oil spills affect wildlife there. How might this inform their viewpoint on offshore drilling?
      • Life experience can be an investment or a commitment that affects how you view a topic.
        • Ex. A first generation college student may feel isolated or overwhelmed about going to college because he or she doesn’t have family members who modeled success in college. How might this affect how they view the availability, or lack thereof, of scholarships and other support aimed at students like them?
        • Ex. A woman seeks asylum in the US after escaping from Boko Haram. How does her experience affect her views on women’s rights? On terrorism? On refugees? On human rights? On sexual servitude?
  • Remember: We are accustomed to thinking of bias in negative terms. One’s job, education, experiences and identity inform the positions we take on topics and controversies. A biased argument is not an invalid argument. Bias tells why someone cares about the topic and gives us insight into what is important to whom in a controversy.

Resources, Activities and Assignments:

Assignment / class discussion: Assign students to watch this video on bias and credibility. You may use the questions in the video (pasted above) for homework points. In a flipped classroom approach, students will come prepared to the next class meeting to discuss these concepts as a group.

In class activity 1: Name that Bias!
This exercise is designed to get students thinking about author, audience and publication. Because they aren't reading an entire piece, they won't be able to consider alternative views or language. Set up a scenario for your students and give them a series of examples of viewpoints. Ask them to identify the bias and discuss how they came to that conclusion. You may use the fracking scenario below (and in the Powerpoint) or create your own.

Scenario: The Texas legislature is considering legislation to regulate fracking.

  • President, Plains Exploration and Production Company (producers of natural gas), writes a letter to the editor of the Houston Chronicle about the number of employees in his company working in hydraulic fracturing.
  • Professor of Geology at the University of Austin, and member of the board of the Plains Exploration and Production Company, publishes an article in the newsletter of America's Gas Alliance about studies denying a link between fracking and groundwater contamination.
  • Head of the West Texas Cotton Farmers Association, is interviewed on NPR about how the drought and fracking has impacted farming.
  • A West Texas rancher, writes a piece for Texas Monthly about fracking and its effect on livestock
  • Head of the Texas Medical Association, blogs about the impact of the chemicals used in fracking on human health.

In class activity 2:  This approach essentially models what they will need to do for RS 3/4.

  • Modeling in class: Use your own scenario or the one provided below to model the process in class. If you choose to create your own scenario, you may want to find articles from conservative and liberal publications (see list here) on the same topic and compare them. 
    • Scenario: Use this October 2012 op-ed from the New York Times (via LexisNexis) about doctor assisted suicide ("Four Myths about Doctor Assisted Suicide" by Ezekiel J Emanuel. 10/28/12). This topic allows you to discuss language (doctor assisted suicide versus euthanasia) and stakeholder viewpoints (doctors, patients, families, the poor). 
    • Remember to tell your students that uncovering bias is not a checklist activity - don't worry if the publication does not suggest bias, for example. 
    • Walk through the article with them discussing bias as it relates to the author, publication and language.
      • In addition to looking up authors and publications with Google, you can also use Academic Search Complete or LexisNexis to further investigate an author and publication with your students. Search the author to see what else they’ve written (that is also held within the database); check for a brief biography of the author in the record itself or in the full text of the article; click on the source title to find out more about that source. 
      • Does the author recognize opposing viewpoints in the article? Does he deride or mock his opponents? Does he use dismissive language?
      • Think about value-laden terms: What do you associate with the word euthanasia? Doctor-assisted suicide? Right to Die? Death with Dignity?

Homework assignments or in class group activity: Find a few longer viewpoint articles from magazines or trade journals and ask your class to read them. Then discuss the articles in class, being sure to talk about all the considerations listed above. You may structure this as group work, too, and have students discuss in groups and then report out in order to lead to a larger classroom conversation. You may also use one of the following worksheets to structure an in-class activity or as a take home assignment for an assigned article or one they find and plan to use for their paper.