ChatGPT and Learning

Over on the ACOUP blog, University of North Carolina Historian Bret Devereaux has produced a useful critique of ChatGPT as a tool for writing college essays. In it, he first presents his definition of an essay, the one that guides his classroom assignments:

Writing an essay thus involves a number of steps, of which communication is merely the last. Ideally, the essay writer has first observed their subject, then drawn some sort of analytical conclusion about that subject, then organized their evidence in a way that expresses the logical connections between various pieces of evidence, before finally communicating that to a reader in a way that is clear and persuasive.

…

ChatGPT is entirely incapable of the first two steps (though it may appear to do either of them) and incompetent at the third; it’s capabilities are entirely on the last step (and even there generally inferior to a well-trained human writer at present).

The reason that some people are saying that ChatGPT is a threat to the college essay is that they misunderstand the purpose.

[There is] a consistent problem in how we teach students, which is that we rarely explain our pedagogy (our ‘teaching strategy’) to the students. That tends to leave many assignments feeling arbitrary even when teachers have in fact put a great deal of thought into why they are assigning what they are and what skills they are supposed to train.

…

[The] most important thing I am trying to train is not the form of the essay nor its content, but the basic skills of having a thought and putting it in a box that we outlined earlier…. What we are practicing then is how to have good thoughts, put them in good boxes and then effectively hand that box to someone else. … at no point in this process do I actually want the essays. 

The reason I’m writing this blog…well…essay, is that Devereaux’s approach is similar to how our Senior Project worked back when I was teaching Management Information Systems. For their final class in the MIS major, students were broken into teams and given a Request For Proposal for some sort of computer system. They had to be small problems, because they had to be completed and documented within the eight week quarter. So, it might be an appointment scheduling system, or an inventory management system or the like. Students would first design the system (and turn in a response to the RFP), then build (and document) it, and then, finally, deliver the completed project, complete with user manual and budget documentation. It was great fun, for me, at least, but I had to keep pushing back against feature creep on the part of the students.

They were always going out and finding plug-in code that would produce snazzy calendars or pretty inventory graphics, even if there was no requirement for them in the RFP. I had to keep telling them that I didn’t want an inventory system, and I certainly didn’t want three different inventory systems by three different teams. What I wanted was for them to have the experience of team development of a project, to a schedule and with certain deliverables. From all reports, ChatGPT is not capable of generating workable code, and it certainly isn’t capable of producing a response to an RFP, a user manual, or a detailed budget.

 

My Debt to Van Halen

Eddie Van Halen has died of cancer, age 65 (ten years younger than me). For my two decades teaching Management Information Systems in the business school at a small regional university, he was the inspiration for a key lesson I taught my students.

You see, the last class the Business MIS majors took was essentially a capstone, Systems Project. In that class I would break them into teams and give each team a ~20 page Request For Proposal for the new software system I wanted them to build. Depending on what was trending that year, they were tasked to build a ride-hailing system called Ober, a lost airliner search coordination system, or a hospital staffing and appointment tool, and so forth, different each year.

Of course, I didn’t need a fishing fleet scheduling tool, or a  drug control inventory system. What I wanted was for them to have the experience of going from a set of written requirements, through the design and documentation process, followed by the joys of team software builds. How well it worked, while important, was secondary.

One of the lessons I wanted them to take away was the absolute necessity of carefully reading the RFP, line by line, with their finger on the screen and their tongue between their lips. Enter Van Halen.  The band’s contract with the venue they were performing at included a rider that said there would be a bowl of M&Ms in the backstage area, but the bowl could contain no brown M&M’s.  As band member David Roth later explained:

Van Halen was the first to take 850 par lamp lights — huge lights — around the country. At the time, it was the biggest production ever. In many cases, the venues were too outdated or inadequately prepared to set up the band’s sophisticated stage.

If I came backstage, having been one of the architects of this lighting and staging design, and I saw brown M&Ms on the catering table, then I guarantee the promoter had not read the contract rider, and we would have to do a serious line check of the entire stage setup.

In order to emulate the Van Halen Rider, I would bury critical formatting information in obscure parts of the document — the requirement that all margins be 0.75″, set in between paragraphs specifying that no drugs were allowed on the worksite, and that no metals from pre-apartheid South Africa would be used (this was a reduced copy of a real government RFP from the 70’s).

When the teams turned in the initial proposal, the first thing I would do is get out a ruler and check the margins and other formatting requirements. Invariably, they had missed them, and had gone with the default MS Word or Libre Office settings.

The object wasn’t to be mean, but to ensure that they understood they had to read the entire RFP in detail, instead of skimming over it the way students are wont to do. It wasn’t until the students were graduated and out in industry that some of them told their friends “I finally understand what he was trying to tell us.”

Thanks, Van Halen.