Introduction to the Creative Challenge

In the first creative challenge — Key Benefits of Writing Without AI for Students — you reflected on what you and other writers gain by writing without AI. For that assignment you reviewed Postman’s “Five things we need to know about technological change” and the Civics of Technology’s “Technoskeptical Framework.” Then, for the third challenge, you continued developing critical AI literacies by collaborating to develop a bot and then reflecting on whether that new medium constrains or enhances human creativity and agency.

Now, for this challenge, you’ll continue developing your AI literacies — a competency recently conceptualized by the MLA CCCC Taskforce on Writing and AI as a foundational competency for knowledge workers in the age of AI.

What is Critical AI Literacy and Why Does It Matter?

In our contemporary knowledge economy, critical AI literacy is a fundamental skill for students, professional writers, and educators alike. It encompasses not just a technical understanding of AI capabilities but also awareness of the risks, rewards, limitations, biases, and potential societal impacts of these technologies. As the MLA-CCCC Joint Task Force on Writing and AI argues, cultivating critical AI literacies is crucial across academic and professional contexts. Developing this literacy involves increasing knowledge of AI tools, experimenting with them, and then reflecting on their constraints and affordance — what rhetoricians call “Rhetoric & Apparatus Theory“. By strengthening our critical AI literacies, we can foster ethical AI use and responsible public engagement as these powerful technologies become more prevalent.

In 2024, the MLA-CCCC Joint Task created an incredibly useful breakdown of the dispositions, knowledge base, and actions of AI-literature writers: “Student Guide to AI Literacy.” A must read for students, writers, and teachers alike, these guidelines are intended to proscribe ethical use of AI. According to these guidelines, AI literacies are associated with these understandings and behaviors:

“You are becoming a literate user of GenAI when you can do the following: 

1. You have a basic understanding of how GenAI technologies work.
   • You can distinguish between AI and GenAI.
     • AI refers to systems that predict outcomes based on statistical models derived from large datasets. 
     • GenAI produces text, images, and videos in response to prompts. Large language models (LLMs), a type of GenAI, use a diverse range of text, often drawing on digitized texts and text from the internet, to produce text that is near or even indistinguishable from text composed by a human. 
   • You can explain that LLMs essentially predict the likelihood that parts of words will appear successively, producing text based on their training on digitized forms of human-written text. 
   • You can identify that LLMs involve various forms of human intervention, including feedback and content moderation, that influence their performance. 
2. You understand the policies and frameworks for the ethical use of GenAI outlined by your instructors and institutions.
   • You can identify and follow relevant guidelines on using GenAI for academic purposes. 
   • You credit GenAI contributions in your work through appropriate citation or attribution. 
   • You can discuss your process transparently with your instructors and peers.
3. You know how to prompt GenAI to produce useful outputs. You can provide a prompt that suits the purpose of a learning task. 
   • You use effective techniques, including experimenting, practicing, and refining prompts, to achieve desired outputs. 
   • You use flexible strategies to adapt to new and evolving GenAI tools. 
4. You evaluate the relevance, usefulness, and accuracy of GenAI outputs.
   • You select strategies appropriate to the context, purpose, and audience for a task. 
   • You can analyze GenAI outputs to determine whether the results align with the purpose of a task. 
   • You recognize when using GenAI is not appropriate for a writing or research task. 
   • You check the accuracy, correctness, and relevance of GenAI outputs against credible sources. 
5. You monitor your own learning as you use GenAI tools.
   • You understand and can articulate why you used GenAI in a writing, reading, or research task. 
   • You can explain how using a GenAI tool for writing contributed to your work. 
   • You can reflect on how your use of GenAI affects your creativity and development as a writer. 
   • You can reflect on your learning needs and make effective decisions about when to avoid relying on GenAI as a learning tool. 
6. You recognize that GenAI is fundamentally different from human communication.
   • You can explain differences between text produced by a GenAI tool and authentic human communication. 
   • You recognize that while GenAI is a useful aid in writing, written communication happens only between human writers and readers. 
   • You can evaluate whether the output from a GenAI prompt used as part of your writing process or product might result in miscommunication. 
7. You understand the potential harms of GenAI, both those inherent to the technology and those that arise from misuse. You can identify how the growth of AI technologies impacts the environment. 
   • You can identify how the growth of AI technologies impacts labor. 
   • You can evaluate GenAI outputs for bias in language, culture, gender, ethnicity, and other social biases. 
   • You can examine and explain privacy, intellectual property, and data security risks of GenAI. 
   • You can assess the implications of bypassing human feedback and communication in writing processes” (MLA-CCCC Task Force on Writing and AI, 2024).

The Creative Challenge

For this challenge, you will select one article, video, or other media piece that exaggerates or perpetuates hype around AI capabilities. To find a good topic for analysis, you may want to peruse Facebook for ads on new AI tools. Using the “18 Pitfalls in AI Journalism” framework from the AI Snake Oil blog, you will create an annotated version identifying and analyzing specific instances where these pitfalls are present. You may choose between four to five of the pitfalls to analyze:

1. Attributing agency to AI
2. Suggestive imagery
3. Comparison with human intelligence
4. Comparison with human skills
5. Hyperbole
6. Uncritical comparison with historical transformations
7. Unjustified claims about future progress
8. False claims about progress
9. Incorrect claims about what a study reports
10. Deep-sounding terms for banal actions
11. Treating company spokespeople and researchers as neutral parties
12. Repeating or re-using PR terms and statements
13. No discussion of potential limitations
14. Limitations de-emphasized
15. Limitations addressed in a “skeptics” framing
16. Downplaying human labor
17. Performance numbers reported without uncertainty estimation or caveats
18. The fallacy of inscrutability 

Your target audience is the general public that reads sites such as the AI Snake Oil blog, The Atlantic, The New Yorker, or Writing Commons. Through this creative challenge, you will practice applying a critical lens to AI media narratives and develop skills for discerning hype from reality.

Requirements

1. Length: Aim for 500 to 750 words across your annotations.
2. Design Messages for Maximum Impact
3. Employ a Professional Writing Style. 
4. Citation style: APA 7 or hyperlinked URLS – just make sure that you are consistent!
5. Required Template
   • To aid your analysis, use the template Kapoor and Narayanan use in Eighteen pitfalls. For examples of what I’m asking for, see this example from CNN and this example from the NYT.

Readings for this Creative Challenge

The primary readings for this challenge are

1. Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process (For the practice exercise.)
2. Eighteen Pitfalls to Beware of in AI Journalism and/or the Eighteen pitfalls checklist
3. The text you are citing
4. A third choice. For this third choice, I encourage you to reference the critical technology frameworks we reviewed in the first creative challenge:
   • MLA-CCCC Joint Task Force on Writing and AI Members (Byrd, A., Flores, L., Green, D., Hassel, H., Johnson, S. Z., Kirschenbaum, M., Lockett, A., Losh, E. M., & Mills, A.) (2023). Statement on writing, artificial intelligence, and critical digital literacies. Modern Language Association. https://aiandwriting.hcommons.org/working-paper-1/
   • Postman, N. (1998, March). Five things we need to know about technological change [Conference presentation]. Denver, CO, United States. https://www.cs.ucdavis.edu/~rogaway/classes/188/materials/postman.pdf
   • Technoskeptical Framework – 5 Critical Questions about Technology (n.d.). Civics of Technology. https://www.civicsoftechnology.org/curriculum
   • Technoskeptical Framework Handout (n.d.). Civics of Technology. https://docs.google.com/document/d/1gCCh-BGrJQK6rtaNsCpi2BbFjmUy2JlFQBAdHIwHQKg/edit

Schedule

Step 1 – Practice at Media Analysis

Working collaboratively

1. use the ”Eighteen pitfalls checklist” to critique “Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process”
2. share your group’s analysis with the class. 
3. provide a link to your group’s analysis at the course sandbox. Each group will have 3 minutes to share their analysis. Make sure all group members are identified

Step 2 – Identify a Media Source to Analyze

Working individually,

1. use Google Scholar or a tool such as ChatGPT-4o, Perplexity, Research Rabbi, Semantic Scholar, Elicit, or Consensus to search for and identify a media item to focus on. You want to find a media example that “hypes” AI. Your source can come from anywhere, including edtech and business domains. If you’re using a non-text type of media (e.g., a video or an audio podcast), you can work with the transcript for the color-coded highlights.
2. reread the “18 Pitfalls” framework 
3. use “gdocs highlighting” to annotate the media you’ve chosen to analyze. Your audience consists of regular readers of “AI Snake Oil” and of people who are encountering the site for the very first time via your article. This creates a tricky balance: you don’t want to bore the regular readers by rehashing things they already know in excruciating detail, but you are going to need to provide some context for those readers who are encountering this site for the first time. You also can’t necessarily assume that your readers have read anything from our course readings.Follow the template Kapoor and Narayanan use in Eighteen pitfalls.  For example, see this example from CNN, and this example from the NYT. Identify errors in reasoning, bias, failure to address counterarguments. Your goal in the intro and conclusion is to contextualize your analysis (e.g. relate it to the framework that Kapoor and Narayanan provide, and to other sources we’ve already read) and to offer readers a “take home message” about what your analysis shows us.

Assessment Criteria

Argument & Analysis:

• The introduction clearly motivates the analysis that follows
• The implications for the analysis (why does it matter?) are discussed explicitly in the intro and outro
• The annotations are supported using well-chosen evidence and good reasoning
• There is a clear “take home message” in the outro

Engagement with Sources and Evidence:

• You engage with at least 3 sources
• Examples from your chosen media item are clearly highlighted with colors corresponding to your key and annotated with comments sharing your analysis 
• The examples are analyzed in relation to both the “18 pitfalls” framework and at least one other reading we’ve reviewed this semester
• Each source is paraphrased, quoted, and/or summarized without plagiarism. APA 7 is used

Organization & Coherence:

• Each annotation is focused, coherent, and clearly relevant to the analysis
• There is a clear connection between the annotations, such that taken together, they build toward the “take-home message” in the introduction.

Presentation & Style:

• The critical analysis has an interesting, accurate, and genre-appropriate title
• The writing in the intro, conclusion, and annotations is appropriate for the genre. If choices deviate from genre conventions, is it intentional, for a clear purpose?
• There is effective, thoughtful use of diction, grammar, and stylistic writing style.

Step 3 – Peer Evaluation

Working collaboratively in small groups, with the entire group’s focus on one peer’s text at a time, engage in Structured Revision – How to Revise Your Work. Use the following organization to focus your group critique

• Inspect the Document @ the Global Level
• Inspect the Document @ the Section Level
• Inspect the Document at the Paragraph Level
• Inspect the Document at the Sentence Level + Sentence-level Perspective

Step 4 – Submission Instructions – Deliverables

1. Upload to Canvas the gDoc link for your group’s analysis of “Not quite eye to A.I.: student and teacher perspectives on the use of generative artificial intelligence in the writing process
2. Upload to Canvas a .pdf version of your individually-authored Media Analysis. Be sure your link enables edit-view privileges. If you used a GAI tool to author your reflection, keep the chat log archived in case I need to review it.