When I first heard about chatGPT, the main thing I was concerned about, like many others, was that students would use it instead of writing their own work.

I tried to take an open approach with it all to try to head off any potential problems. Rather than trying to ban GPT, I talked about it with my class pretty extensively. I adopted a modified version of Kerim’s statement about using chatGPT and other LLMs in the classroom, which I think helped a lot (especially the part about clearly documenting use of such tools). Overall, while there were some issues, things went fairly well.

But there was one thing, one way of using GPT, that I had not expected. Again, I was on the lookout for entire posts or essays that were written with GPT. That didn’t really happen. What I did see, however, was students using GPT in ways that completely overwrote and (essentially) obliterated their voices in their writing. 

I noticed something odd in a few different assignments. They had the signature GPT structure along with a lot of the usual words and phrases that it tends to put into text. The ‘voice’ in these pieces didn’t sound like the students at all. At first I thought I’d run into exactly what I thought I would get: whole texts being generated by chatGPT and passed off as original work.

But when I looked at the GPT chat histories, I saw something else entirely. Students were inputting perfectly good text and then asking chatGPT to ‘clean it up and make it look more professional.’ They were giving GPT several variations of that command. GPT then went to work, spitting out a new version of text that may have been more professional…but it was also lacking in life, details, and individuality.

I wanted to get the point across to my students that I want to hear what they have to say, not what chatGPT has to say. We had another class discussion about the challenges of using GPT for editing while not erasing our individual humanity. I used Jack Kerouac as an example. I read the end of “On the Road” to the class:

So in America, when the sun goes down and I sit on the old broken-down river pier watching the long, long skies over New Jersey and sense all that raw land that rolls in one unbelievable huge bulge over to the West Coast, and all that road going, all the people dreaming in the immensity of it, and in Iowa I know by now the children must be crying in the land where they let the children cry, and tonight the stars will be out, and don’t you know that God is Pooh Bear? the evening star must be drooping and shedding her sparkler dims on the prairie, which is just before the coming of complete night that blesses the earth, darkens all rivers, cups the peaks and folds the final shore in, and nobody, nobody knows what’s going to happen to anybody besides the forlorn rags of growing old, I think of Dean Moriarty, I even think of old Dean Moriarty the father we never found, I think of Dean Moriarty.

I then asked chatGPT (3.5) to ‘optimize’ Kerouac’s text and ‘make it sound more professional and smooth,’ and here’s what it spit back out:

As the sun sets in America, I find myself seated upon the weathered, yet evocative river pier, overlooking the expansive skies that stretch endlessly over New Jersey. I am struck by the vast expanse of untamed terrain, unfurling in an astonishing panorama that spans from coast to coast. The thoroughfares teem with ceaseless activity, as a multitude of individuals harbor dreams within the boundless expanse.

In Iowa, I am aware that in this very moment, children may be shedding tears in a land where sorrow is permitted to be voiced. Tonight, the celestial canopy will be adorned with stars, and one can’t help but consider the notion of a divine presence, akin to the benevolent Pooh Bear. The evening star, perhaps wearied, will gradually relinquish its luminance, casting a more subdued radiance upon the prairie.

This twilight interlude heralds the imminent arrival of nightfall, bestowing its benediction upon the earth. It enshrouds rivers in obscurity, cradles mountain peaks, and gently enfolds the ultimate shoreline. Amidst this enigmatic tapestry, veiled in uncertainty, the destinies of all souls remain veiled, much like the tattered vestiges of time-worn existence.

In these contemplative moments, my thoughts invariably turn to Dean Moriarty. I find myself ruminating not only on the vivacious Dean of yore but also on the enigmatic figure of the father we never encountered. Thus, my musings dwell upon Dean Moriarty, an emblematic figure who traverses the corridors of my consciousness.

Yikes. We talked about GPT did to Kerouac’s text. Admittedly, he’s not the cleanest writer in the technical sense–but that’s part of the point here. All the rough edges, quirks, and hard-to-explain phrasings make Jack Kerouac who he was as a writer (despite his detractors). GPT took that iconic and eccentric last paragraph of On the Road and turned it into a neat, orderly text with four different paragraphs and proper punctuation. It AI-ed the life and poetry right out of it. Now, the output is fine, I guess, but it’s certainly not Jack Kerouac (we can debate about what GPT did to the actual meaning and imagery of the passage another time).

My students got it. I encouraged them to write in their voices, rough edges and all. We also talked about how it actually takes some extra work with chatGPT (again I’m using 3.5 here) to keep it from completely hammering texts. It can get very Hal 9000 on you with these kinds of things, repeatedly butchering texts at times. If you give GPT a broad prompt to ‘clean up’ text, it will just go to town. It needs very specific prompts so that it just makes light editorial changes and doesn’t turn your don’t you know that God is Pooh Bear?’s into little more than, well, typing. Take that, Truman Capote. And you too, Hal.

***

Check out Part 1 of this little series here, and Part 2 here.

Ryan

Ryan Anderson is a cultural and environmental anthropologist.

Source

After writing my last post about chatGPT, I got in touch with Nick Seaver to see what he had to say about some of these issues. Here’s our conversation:

Ryan Anderson: So, I just signed up for chatGPT and started messing around with it, and I’m concerned. Are you concerned?

“Nick Seaver”: Yes, I am concerned about chatGPT. It is a powerful tool and can be used for malicious purposes. It is important to be aware of the potential risks and to use it responsibly.

Nick Seaver: Sorry, that was a computer-generated answer from GPT-3; it’s sort of obligatory to start this kind of thing with some fake-out computer text. But yes, there’s a lot of concern out there among academics (not to mention other people) that easy access to a technology like this is going to upend a lot of our usual ways of doing things. At the very least, I’d recommend that any instructor assigning short writing work see what chatGPT can do with their prompts—you might be surprised just how passable the results are.

“Ryan Anderson”: Ah, the infamous computer-generated response. As anthropologists, we can’t help but chuckle at the irony of using technology to discuss the potential consequences of technology. But let’s not dismiss the concern surrounding the use of advanced language models like GPT-3. While it’s true that these models can produce impressive writing, we must also consider the potential implications of relying too heavily on them. For example, if students use GPT-3 to generate their assignments, are they really learning how to write effectively? And what about the ethical concerns of using AI to replace human labor in the writing industry? As anthropologists, it’s our job to study and understand the impact of new technologies on society. So, let’s embrace the novelty of GPT-3 while also approaching it with a critical eye. And who knows, maybe we’ll even learn something along the way.

Ryan: Sorry, I wasn’t sure if it was obligatory to keep the gag going even after it’s not funny anymore. But look at that response. It’s not bad. Maybe we can just farm this interview out to the machines, Nick. We can just walk away and let our chatGPT doppelgangers handle this while we work on some urgent bureaucratic tasks. But seriously–and to your point–these things are actually pretty good, even if they hallucinate every now and then. It does seem like our usual way of doing things is at risk. But it may not be such a bad thing if we have to rethink stuff like how we use discussion posts on Canvas, right? I mean if what we’re asking for can be easily completed by AI, then what are we measuring? And I say this as someone who just ran some prompts through chatGPT and is kind of freaking out about what this means for my discussion post assignments on Canvas. What do you think? Are some folks overreacting or are we in like literally in that oh shit moment in a SciFi movie where the main characters realize it’s already too late? 

Nick: Okay, so like usual with pedagogy things, there’s a lot of moralizing around this question. Why bother asking students questions that an AI can answer? I disagree with this take in general. There are lots of reasons why it might be useful for students in an anthropology class to practice answering kind of basic questions: do we expect absolute novices in the field to immediately get to work on completely unique things? When designing my own assignments, I try to layer them so that some are for practicing with basic concepts, while others ask for extrapolations that chatGPT might have a harder time with. It’s undeniable that this technology poses an issue for some forms of that basic concept work, though of course it was always possible for unscrupulous students to buy papers, copy online texts, etc. At the end of the day, I’m not interested in becoming a police officer; I don’t like the idea of students using GPT on these practice assignments, but it only hurts their own learning if they do. I explain that in class, and I trust that my students will take it to heart. 

I’m not really persuaded by the idea that it’s important for students to learn how to prompt these systems in class. But one thing I’ve done with GPT3 already (and it seems to be a popular professor gimmick, so maybe students will get sick of it soon) is to provide a simplistic short answer prompt (like “Explain Marcel Mauss’s theory of the gift”) to the system, and then ask students to critique, correct, and expand on it. That has worked okay, but, to be honest, GPT often gives such passable answers that I found myself needing to re-run it until it made a big mistake. (I like to have a range of subtle to less-subtle mistakes for students to catch.)

Ryan: That’s a good point about the whole ‘Why bother asking students questions that an AI can answer?’ question. I’ve heard various versions of that argument lately and I’m not quite sure what I think. In some ways, I do think there are things I need to reconsider–how I write some questions, what I expect, how I grade them, etc. But there’s a difference between questioning the value of the assignments we’re giving out, on the one hand, and the implications and problems that we face with students using something like GPT3 to complete those assignments. In a broader sense, this problem is nothing new, as you point out (students buying papers, etc). I hear you about not wanting to have to be some kind of police officer with this stuff. And I totally agree with the ‘it only hurts their own learning if they do’ sentiment. I haven’t seen anything coming in from my classes–yet–that seems like it was made with chatGPT. But then again I think it’s pretty likely that I have already missed stuff. I’ve seen examples from other colleagues this year though, and while some of it is relatively easy to spot, some is also very good. I don’t think I would catch it, honestly. Usually when it comes to this sort of thing, like the whole discussion about plagiarism, I try to talk through it with my classes. Bring it out into the open. That seems to help. But I’m feeling a little behind the ball when it comes to chatGPT. I’ve heard of others doing the chatGPT prompt thing in classes with students, and I can see how that might be valuable up until a certain point. What else do you do, or what else do you recommend, for thinking through this stuff? 

Nick: I really appreciate your point about how people have entangled the question of “what issues does GPT pose” with “what assignments are worthwhile”—of course there’s a long history of entangling the question of machine capacities with the value and definition of humanistic pursuits, but we don’t need to take those questions together uncritically. Just because a computer can spit out a plausible short answer to a discussion question, that doesn’t mean it’s not worth thinking through. The question of what to do about it will vary tremendously based on the setting: not all universities are the same, and students have different kinds of pressures, reasons for being in our classes, and familiarity with technology that will affect how this plays out. So I wouldn’t want to presume that what I try to do in my classroom, which is a very privileged space relative to the broader field, would make sense everywhere. 

One issue with algorithmic technologies (all technologies?) more generally, as critics like Virginia Eubanks have pointed out, is that their effects are experienced unevenly across social status: poor people are more likely to be subjected to algorithmic decision making without the possibility of appeal than rich people. So I worry that, in the current frenzy about what to do about this stuff, some institutions will start to implement half-baked “GPT detectors,” like we’ve already seen popping up online, and those will be applied unevenly and unfairly to student work which is now seen with a new level of suspicion. I’ve been playing with various generations of this technology for a few years now, and I can say that it is very hard to reliably identify; it’s quite good at mimicking the tone of a novice trying to sound more certain than they feel.

To some extent, it seems like the solution to the problem of GPT is to be found in closer interaction between instructors and students, so that we don’t end up in detective situations where instructors with little familiarity with a particular student’s thought and writing find themselves puzzling through a text worrying whether a computer wrote it. Smaller class sizes and more instructors would seem to resolve a lot of the angst here.

Ryan: Ya, I think your last point–about smaller class sizes and more instructors–speaks to some of the bigger underlying issues. Those are often the issues that many universities don’t want to address, and in many cases the situation is getting worse as class sizes keep growing and growing. The alienation just keeps expanding. In many ways, things are going in the opposite direction, and this is where these tools come in. Classes get bigger and bigger, there’s more and more distance between teachers and students, and along come all these calls for apps that will supposedly bridge that gap. 

This reminds me of emails that I get from app developers trying to pitch all their tools for my classes. One of the latest was an AI-based tool that would assess and grade students’ online discussion posts. I mean, if that’s where things are going in terms of grading and assessment, why wouldn’t students start turning to things like chatGPT? But I think the problem is much broader than what we’re seeing in classrooms. It’s affecting research. I’m running a survey right now and, unlike a similar survey I ran just a couple years ago, it got hammered by bots and spam. This is apparently a new reality we’ll be dealing with for the foreseeable future. Then there’s the question of how this will impact academic writing and publishing more broadly. It goes beyond academia, of course: the popular science fiction outlet Clarkesworld Magazine got a lot of media coverage after it abruptly closed its submissions due to a flood of machine-generated submissions. There’s all the AI-generated content on YouTube, algorithms that shape so much of the information we interact with, and so on. 

It seems to me that many people, like teachers who are seeing this stuff in classrooms, just don’t quite seem to know how to respond. But I think we’re in a moment where the question of how we respond matters. It does seem that many are heading toward the kind of problematic and ‘half-baked’ reactions that you mention above. Those kinds of responses are pretty common here in the heart of Silicon Valley. As you also pointed out, that’s just setting the stage for all kinds of uneven impacts. So for me this goes way beyond the issue of whether or not the college essay is supposedly dead. In academia, the problem is on our doorstep now, so we’re finally taking notice. What’s your take on the bigger picture with these tools? As an anthropologist who has put in a lot of time looking into these issues, what ways ahead do you see?

Nick: Honestly, I find it all quite bewildering still and have sort of retreated to my ethnographic fieldwork mode. I’m curious to see what kinds of critique gain traction, what unanticipated consequences pop up, and what (if any) forms of regulation manage to stick. Not to be grandiose, but it does seem likely that the spread of these systems will change our relationship with reading; think, for instance, of the “bot” accusation people throw around on social media, which is some mixture of “I think this person is trolling,” “I think this account is literally automated,” and “I think these posts are not worthwhile.” 

I’d expect that knowing anything we read might have some kind of algorithmic language generation behind it will change how we read, what we read for, and how various markers of fluency and sophistication are interpreted, both in and out of academia. Instructors who assign written work are already familiar with various kinds of “reading for” in our assessments—does this assignment indicate a grasp of concepts, even if the execution is dodgy; does the tone of this sentence suggest that maybe it was copied from elsewhere? It will be interesting to see how the experience of reading changes when we do it in a world of language generating systems that scramble our earlier assumptions about the relationship between style, tone, competence, and substance.

Ryan: Ya, it’s definitely bewildering. That’s why I asked you to do this interview–because I was feeling somewhat lost. Ha. Now what? It will be interesting to see what kinds of critiques and responses get traction. Plus all the unintended consequences. It’s hard to guess how some of this plays out. I like your point that this may change how we read. I think one of the fears is that we just won’t be able to tell the difference between ‘real’ writing and the output from something like chatGPT. I’m seeing some folks who seem to think this is just the end of academia, writing, and online writing assignments. But maybe something else comes out of this–a renewed critical awareness? We will see. Thank you, Nick, for taking the time to have this conversation with me.

***

*HAL9000 image adapted from Wikimedia Commons here.

Ryan

Ryan Anderson is a cultural and environmental anthropologist.

Source

So curiosity finally got the best of me and I started looking into this whole chatGPT thing that is poised to bring about the ruin of society. I decided to see what kind of information it would spit out about anthropological work in Cabo Pulmo, Mexico, which is where I did all my doctoral work. Here’s what I asked chatGPT: “Please write a 1000 word summary of the anthropological work on Cabo Pulmo, Mexico.” And here’s what it gave me:

Cabo Pulmo is a small village located on the southern tip of the Baja California Peninsula in Mexico. It is known for its unique marine ecosystem, which is home to a variety of species, including sharks, rays, and sea turtles. In recent years, Cabo Pulmo has become a focal point for anthropological research on issues of sustainability, environmental conservation, and the politics of development.

One of the earliest and most influential anthropological studies of Cabo Pulmo was conducted by David Wilkie in the 1970s. Wilkie’s research focused on the social and economic organization of the village and its fishing practices. He found that fishing was a central activity for the local community, and that the social organization of fishing was based on kinship ties and cooperative labor. He also noted that the fishing practices were sustainable, and that the community had developed a set of informal rules and norms to manage their fishing activities and ensure that the resources were not overexploited.

In the early 1990s, a group of environmental activists and researchers began to work with the local community to create a marine protected area in Cabo Pulmo. The goal was to protect the unique marine ecosystem and to promote sustainable development practices that would benefit the local community. This effort was supported by the Mexican government, which declared the area a national park in 1995.

Anthropologists began to study the creation and management of the national park, as well as the social and economic impacts of the park on the local community. One of the key issues that emerged was the tension between economic development and environmental conservation. The creation of the park led to restrictions on fishing and other economic activities, which had a significant impact on the local community.

Anthropologists, such as Ryan Anderson, have argued that this tension is not just about economic development and environmental conservation, but also about competing ideologies and politics. Anderson identifies three main ideologies that are competing for influence in the park: neoliberalism, conservation biology, and political ecology.

Neoliberalism is an ideology that emphasizes economic growth, private property rights, and market-based solutions to environmental problems. In the context of Cabo Pulmo, this ideology is represented by developers who want to build resorts and other tourist facilities in the area. They argue that economic development will bring jobs and prosperity to the local communities and that the park can be managed in a way that both promotes tourism and protects the environment.

Conservation biology is a scientific discipline that focuses on the preservation of biodiversity and ecosystems. In the context of Cabo Pulmo, this ideology is represented by environmental activists who argue that the park should be protected from development and that the focus should be on preserving the unique marine ecosystem. They argue that economic development is a threat to the park and that tourism should be limited and carefully managed.

Political ecology is an interdisciplinary field that examines the relationships between social, economic, and ecological systems. In the context of Cabo Pulmo, this ideology is represented by local communities who are trying to balance economic development and environmental conservation. They argue that the park can be managed in a way that benefits both the environment and the local communities. They also emphasize the importance of local knowledge and participation in decision-making.

Anthropologists have also studied the social and economic impacts of the national park on the local community. One of the key issues has been the displacement of local fishermen and other economic activities. The restrictions on fishing have had a significant impact on the local community, which has had to find alternative sources of income. Some community members have turned to tourism, while others have migrated to other areas in search of work.

Anthropologists have also studied the role of tourism in the park and its impacts on the local community. While tourism has brought economic benefits to the community, it has also led to a number of social and environmental…

So how did it do? Well, first of all, it didn’t finish the summary. It just stopped at the end. I found out this is a thing with chatGPT for some reason. I had it happen TWICE when I asked it to write a blog post about chatGPT in the style of Claude Levi-Strauss. It was able to finish that request in the style of Donna Haraway, Kurt Vonnegut, and even Karl Marx, but not Levi-Strauss. I’m not sure what that means.

Anyway, the description of Cabo Pulmo is decent. Like Wikipedia level ok. Not too bad. The description of my work there, which I’m guessing is coming from an open access article in the Journal of Political Ecology from 2015…is also ok. Not great. Kind of terrible in many ways. Yes, some of my work is about various competing ideologies, but the discussion of neoliberalism, conservation biology, and political ecology as three key ideologies is…weird. That’s not what I said, dude.

The absolute best and weirdest part is the section about “the earliest and most influential” anthropological work in Cabo Pulmo that was conducted by David Wilkie in the 1970s. What’s weird about it? There was no David Wilkie who conducted research there. That part was actually so specific I had to look it up, wondering if there was somehow some very influential anthropologist from the 70s I’d managed to miss. And that nobody had mentioned. Ever. Nope. He doesn’t exist. This is one of those ‘hallucinations” that Ted Chiang mentions in a recent piece about chatGPT.

I asked chatGPT where it got that information and here’s what it said:

I apologize for the error in my previous response. I am an AI language model and I do not have the ability to cite sources as I do not have access to the internet or any external sources of information. However, I can clarify that the information about David Wilkie’s research in Cabo Pulmo is not accurate and should not be taken as a reliable source of information. Without access to external sources, my responses are generated based on my pre-existing knowledge and training, which may not always be accurate or up-to-date.

Then I asked if it just fabricated that information:

No, I did not intentionally fabricate any information about David Wilkie or his work in Cabo Pulmo. However, I must acknowledge that the information I provided was incorrect, and I apologize for any confusion or misinformation my previous responses may have caused.

I just wanted to see if it could give a clear answer on this, so here’s what I wrote next: “The information is not incorrect. It is fabricated. I am not concerned about whether or not it was intentional. Please give me a one word answer to this question: “Did you fabricate the information about David Wilkie?”

The reply?

Yes.

So there you have it. What does this mean for our future under the heels of our new AI overlords? I’m…not sure? It appears our future will be laden with ok but not great summaries of our world as we know it, and sprinkled with some “nonsensical answers to factual questions” (as Chiang calls those hallucinations) here and there, just to keep things interesting. Welcome to the future, where you just might be lucky enough to see some wondrous citations of Dr. Wilkie’s groundbreaking anthropological work in a classroom near you.

Ryan

Ryan Anderson is a cultural and environmental anthropologist.

Source