PROCESS: Groups of 2 - 4 students were each given a poetry packet containing excerpts from four (4) poems written by humans, and four (4) poems written by Artificial Intelligence (AI). Each group then attempted to reach a consensus about the authorship of each poem. Finally, groups posted their decisions on the board so the comparative results could be discussed as a class.

Note: In the photos below, papers that are the same color are the same poem...

POEMS: (Without giving away which poems were used in this modified Turing Test......) We analyzed four (4) poems from ChatGPT and other AIs, as well as four (4) poems from these distinguished human authors...

RESULTS: Students found it very difficult to distinguish between poetry written by humans versus machines:

• Chelsea Martin's poem was identified as human-written only 50% of the time

• Marcus Wicker's poem was identified as human-written only 40% of the time

• Rupi Kaur's poem was identified as human-written only 50% of the time

• e.e. cummings' poem was identified as human-written only 40% of the time

• Human-written poems were correctly identified only 18 out of 40 times (45% accuracy)

• AI-written poems were correctly identified only 18 out of 40 times (45% accuracy)

• AI-written poetry fooled students 55% of the time

This activity was repeated during a subsequent AP Computer Science A class; our advanced placement Java student correctly identified poem authorship only 26 out of 56 times (46% accuracy); they correctly identified human-written poems 14 out of 28 times, and AI-written poems 12 out of 28 times. Of the four classes, none were able to reach a correct consensus on any of the eight poems; in fact, the only consensus reached was a unanimously wrong claim by the AP CSA students that Marcus Wicker's poem was AI-written.

It's worth noting that Alan Turing set a 30% threshold for AI fooling humans, but the AI in this activity fooled students 55% of the time!