Manny's Reviews > ChatGPT-Based Learning And Reading Assistant (C-LARA): Second Report

If you were wondering why I've been posting so little the last few weeks, then wonder no more. Available for free download here.
___________

But seriously...

ChatGPT-based Learning And Reading Assistant (C-LARA) is a project that's been taking up most of my time for the last year. The basic idea was to build a web platform that lets people create easy-to-read multimodal texts in foreign languages, and have ChatGPT-4 do as much of the work as possible. Chat appears in two roles. With its software component hat on, it writes the texts, cuts them up into roughly sentence-length pieces, adds glosses to the words, marks them with root forms, and adds TTS audio. With its software engineer hat on, it's written most of the codebase. This report gives you a detailed picture of what we've done.

You may think there are too many details - does it really need to be 144 pages long? But it hasn't primarily been written for you, it's been written for ChatGPT-4. Although the AI is responsible for the greater part of the work performed in the project, it periodically has to be reinitialised, and then I need to tell it who it is again. Having this report available makes the job easy: I can give it the text in half a dozen instalments, which takes a quarter of an hour, and then it's up to speed again.

For humans, here are some of the bits you might find interesting. First of all, we've made C-LARA easier to use. There's a new top level called "Simple C-LARA", which lets you create a multimodal text with an initial request and a couple of button presses. You choose the text language and the glossing language, provide a sentence or two telling the AI what to write, and it creates a short illustrated text for you. (The illustrations comes from DALL-E-3). There are straightforward options to edit and correct when it gets things wrong. You can also paste in an existing piece of text if you prefer, and tell the AI to annotate it instead. For example, that's how I created the multimodal Norwegian passage in my review of Jon Fosse's Melancholia I-II.

Second, we've done some work to evaluate C-LARA. We describe an experiment we presented at the ALTA 2023 conference late last yeat, where we created six texts of widely different kinds in English, Faroese, Farsi, Mandarin and Russian and carefully checked how often the AI was making mistakes as it wrote and annotated them. Not surprisingly, it's much better at some languages than others. It turns out to be nearly as good at Mandarin as it is at English, but it's clearly worse at Russian, worse again at Farsi, and having serious problems with Faroese. Of course, it's miraculous that it can do anything in Faroese, an obscure Scandinavian language spoken by about 50,000 people. We repeated the experiments a few months later, and found it had improved a good deal in English. We also analyse the codebase to quantify the AI's contribution to writing it. We found that it had written nearly all of the simple modules and done the greater part of the work on the middling difficult ones. There were a couple of top-level pieces of functionality, in particular "Simple C-LARA", where it couldn't deliver: they required an overview of the whole project, and its context window doesn't seem to be up to it yet. I'm curious to find out what happens in GPT-5.

Third, we present some case studies where people have started to use C-LARA. We have a primary school teacher in Holland who teaches a weekly class for Romanian kids whose parents don't want them to forget their heritage language. We'd never used C-LARA with Romanian, a language I know nothing about, but Lucretia just tried giving it requests in Dutch and said it produces cute, funny little Romanian stories that the kids like. We've also been collaborating with linguists at the University of New Caledonia, who are using it for a couple of the Indigenous languages there. Here, the AI can't help with the writing; they need to create the texts by hand, but that's possible too. They're pleased with the results, which are freely available on the web.

In the appendices we give more details, including examples with step-by-step screenshots showing how to create a C-LARA text. Try it out, the platform is already a whole lot of fun to play with! It's just amazing what you can do with an AI to help you.
___________
[Update, Apr 9 2024]

One of the most useful things about writing this kind of document is that it forces you to think carefully about what tasks you should be planning to do next. We put together a long list in the "Further work" section (§9.1), and we've already started. More about that in a recent post on the C-LARA blog.