Summary

This assignment constitutes the fourth of eight creative challenges that undergraduate students complete for Writing with Artificial Intelligence, an undergraduate writing course. In this creative challenge, a writing assignment adapted from Professor Whitney Gregg-Harrison’s (University of Rochester) Critical Media Analysis Project, students will work to develop their critical AI literacy competencies by using the “18 Pitfalls in AI Journalism,” a critical framework for analyzing logical fallacies, unsubstantiated hype, methodological errors, and interpretive errors in media “hype” about AI.

For this creative challenge, students are also encouraged to leverage what they learned from the first creative challenge, which introduced them to critical AI literacies, including Postman’s “Five things we need to know about technological change” and the Civics of Technology’s “Technoskeptical Framework.” This assignment builds on those critical AI literacies by introducing the “Eighteen Pitfalls to Beware of in AI Journalism.”

Introduction

Your first creative challenge — Key Benefits of Writing Without AI for Students — asked you to reflect on what you and other writers gain by writing without AI. For that assignment you reviewed Postman’s “Five things we need to know about technological change” and the Civics of Technology’s “Technoskeptical Framework.” Then, for the third challenge, you continued developing critical AI literacies by collaborating to develop a bot and then reflecting on whether that new medium constrains or enhances human creativity and agency.

Now, for this challenge, you’ll continue developing your AI literacies — a competency recently conceptualized by the MLA CCCC Taskforce on Writing and AI as a foundational competencies for knowledge workers in the age of AI.

What is Critical AI Literacy and Why Does It Matter?

In our contemporary knowledge economy, critical AI literacy is a fundamental skill for students, professional writers, and educators alike. It encompasses not just a technical understanding of AI capabilities but also awareness of the risks, rewards, limitations, biases, and potential societal impacts of these technologies. As the MLA-CCCC Joint Task Force on Writing and AI argues, cultivating critical AI literacies is crucial across academic and professional contexts. Developing this literacy involves increasing knowledge of AI tools, experimenting with them, and resisting quick fixes in favor of curiosity and nuanced understanding. By strengthening our critical AI literacies, we can foster ethical AI use and responsible public engagement as these powerful technologies become more prevalent.

The Creative Challenge

For this challenge, you will select one article, video, or other media piece that exaggerates or perpetuates hype around AI capabilities. Using the “18 Pitfalls in AI Journalism” framework from the AI Snake Oil blog, you will create an annotated version identifying and analyzing specific instances where these pitfalls are present. You may choose between four to five of the pitfalls to analyze:

1. Attributing agency to AI
2. Suggestive imagery
3. Comparison with human intelligence
4. Comparison with human skills
5. Hyperbole
6. Uncritical comparison with historical transformations
7. Unjustified claims about future progress
8. False claims about progress
9. Incorrect claims about what a study reports
10. Deep-sounding terms for banal actions
11. Treating company spokespeople and researchers as neutral parties
12. Repeating or re-using PR terms and statements
13. No discussion of potential limitations
14. Limitations de-emphasized
15. Limitations addressed in a “skeptics” framing
16. Downplaying human labor
17. Performance numbers reported without uncertainty estimation or caveats
18. The fallacy of inscrutability 

Your target audience is the general public that reads sites such as the AI Snake Oil blog, The Atlantic, The New Yorker, or Writing Commons. Through this creative challenge, you will practice applying a critical lens to AI media narratives and develop skills for discerning hype from reality.

Requirements

1. Length: Aim for 500 to 750 words across your annotations.
2. Design Messages for Maximum Impact
3. Employ a Professional Writing Style. 
4. Citation style: APA 7 or hyperlinked URLS – just make sure that you are consistent!
5. Required Template
   • To aid your analysis, use the template Kapoor and Narayanan use in Eighteen pitfalls. For examples of what I’m asking for, see this example from CNN and this example from the NYT.

Readings for this Creative Challenge

The primary readings for this challenge are

1. Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process (For the practice exercise.)
2. Eighteen Pitfalls to Beware of in AI Journalism and/or the Eighteen pitfalls checklist
3. The text you are citing
4. A third choice. For this third choice, I encourage you to reference the critical technology frameworks we reviewed in the first creative challenge:
   • MLA-CCCC Joint Task Force on Writing and AI Members (Byrd, A., Flores, L., Green, D., Hassel, H., Johnson, S. Z., Kirschenbaum, M., Lockett, A., Losh, E. M., & Mills, A.) (2023). Statement on writing, artificial intelligence, and critical digital literacies. Modern Language Association. https://aiandwriting.hcommons.org/working-paper-1/
   • Postman, N. (1998, March). Five things we need to know about technological change [Conference presentation]. Denver, CO, United States. https://www.cs.ucdavis.edu/~rogaway/classes/188/materials/postman.pdf
   • Technoskeptical Framework – 5 Critical Questions about Technology (n.d.). Civics of Technology. https://www.civicsoftechnology.org/curriculum
   • Technoskeptical Framework Handout (n.d.). Civics of Technology. https://docs.google.com/document/d/1gCCh-BGrJQK6rtaNsCpi2BbFjmUy2JlFQBAdHIwHQKg/edit

Schedule

Step 1 – Practice at Media Analysis

Working collaboratively

1. use the ”Eighteen pitfalls checklist” to critique “Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process”
2. share your group’s analysis with the class. 
3. provide a link to your group’s analysis at the course sandbox. Each group will have 3 minutes to share their analysis. Make sure all group members are identified

Step 2 – Identify a Media Source to Analyze

Working individually,

1. use Google Scholar or a tool such as ChatGPT-4o, Perplexity, Research Rabbi, Semantic Scholar, Elicit, or Consensus to search for and identify a media item to focus on. You want to find a media example that “hypes” AI. Your source can come from anywhere, including edtech and business domains. If you’re using a non-text type of media (e.g., a video or an audio podcast), you can work with the transcript for the color-coded highlights.
2. reread the “18 Pitfalls” framework 
3. use “gdocs highlighting” to annotate the media you’ve chosen to analyze. Your audience consists of regular readers of “AI Snake Oil” and of people who are encountering the site for the very first time via your article. This creates a tricky balance: you don’t want to bore the regular readers by rehashing things they already know in excruciating detail, but you are going to need to provide some context for those readers who are encountering this site for the first time. You also can’t necessarily assume that your readers have read anything from our course readings.Follow the template Kapoor and Narayanan use in Eighteen pitfalls.  For example, see this example from CNN, and this example from the NYT. Identify errors in reasoning, bias, failure to address counterarguments. Your goal in the intro and conclusion is to contextualize your analysis (e.g. relate it to the framework that Kapoor and Narayanan provide, and to other sources we’ve already read) and to offer readers a “take home message” about what your analysis shows us.

Assessment Criteria

Argument & Analysis:

• The introduction clearly motivates the analysis that follows
• The implications for the analysis (why does it matter?) are discussed explicitly in the intro and outro
• The annotations are supported using well-chosen evidence and good reasoning
• There is a clear “take home message” in the outro

Engagement with Sources and Evidence:

• You engage with at least 3 sources
• Examples from your chosen media item are clearly highlighted with colors corresponding to your key and annotated with comments sharing your analysis 
• The examples are analyzed in relation to both the “18 pitfalls” framework and at least one other reading we’ve reviewed this semester
• Each source is paraphrased, quoted, and/or summarized without plagiarism. APA 7 is used

Organization & Coherence:

• Each annotation is focused, coherent, and clearly relevant to the analysis
• There is a clear connection between the annotations, such that taken together, they build toward the “take-home message” in the introduction.

Presentation & Style:

• The critical analysis has an interesting, accurate, and genre-appropriate title
• The writing in the intro, conclusion, and annotations is appropriate for the genre. If choices deviate from genre conventions, is it intentional, for a clear purpose?
• There is effective, thoughtful use of diction, grammar, and stylistic writing style.

Step 3 – Peer Evaluation

Working collaboratively in small groups, with the entire group’s focus on one peer’s text at a time, engage in Structured Revision – How to Revise Your Work. Use the following organization to focus your group critique

• Inspect the Document @ the Global Level
• Inspect the Document @ the Section Level
• Inspect the Document at the Paragraph Level
• Inspect the Document at the Sentence Level + Sentence-level Perspective

Step 4 – Submission Instructions – Deliverables

1. Upload to Canvas the gDoc link for your group’s analysis of “Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process
2. Upload to Canvas a .pdf version of your individually-authored Media Analysis. Be sure your link enables edit-view privileges. If you used a GAI tool to author your reflection, keep the chat log archived in case I need to review it.