For the Progress Forum: While only a narrow domain of human progress, I've long been hoping for technological accelerations to humanities consumption and scholarship. I think there is a serious case that the future of books, especially of works which require commentary, will involve some sort of LLM interface. I experimented a little with this below for a very specific application - reading poetry in a foreign language. I would welcome any comments on this, I see this as being applicable to domains beyond this as well.

It is possible to translate poetry, but not fully. The compression of unique sounds, nuanced double meanings, and hidden references is not an experience that can be fully captured in another language. In previous ages this process was rarely attempted: anyone interested in reading and appreciating poetry was an educated individual who would have already learned the languages necessary, whether Sanskrit, Greek, Chinese, or Persian.

But poetry in translation is still powerful, still captures at least some of the magic that keeps the poems alive. The austere quiet world of the translated poems of Basho can move one incredibly even if the translations lack the extreme economy of phrases, the cutting words which play with the meter, and the seasonal associations which reference hundreds of years of waka that Basho’s original intended readers would have appreciated.

Translated poetry is a practical necessity for the present age. People do not have the time to learn languages, and there are more poems accessible to them from more languages than ever before. While an educated late Roman might speak Punic out in the country, he could read Latin and Greek comfortably, and if he were Christian he might even stretch himself to read Hebrew, but there were no languages outside of these. Now we are aware that this Roman coexisted with Indian and Chinese scholars with their own sets of languages, not to mention the hundreds of poetries that flourished before and after. A modern well-educated person might have learned a language or seven, but human knowledge is only truly accessible once it is translated into the home language, or at least a common one acquired in schooling (English, Mandarin, Standard Arabic, Hindi, etc).

This is a limitation, not a necessity. If we accept the thesis that literature, especially poetry, is better understood in its original language, that opens opportunities to pursue this better understanding. And while the sure method is to laboriously learn the language to fluency, there are alternatives.

Reading in the original confers several distinct advantages, some of which can be more easily captured than others. Poetry often, though not always, gains from sound, and simply reading a poem out loud and knowing its meaning gets you some benefit. Bilingual editions of many works approximate this, allowing the reader to at least hear the sound play even if it is separated from the meanings. For example, an Italian reader will get the full beauty of Dante in a single unadulterated stream, but an English reader can read the English, read the Italian out loud to appreciate the terza rima, and with greater time and effort mentally combine them into an estimate of the original.

thus did a living light shine all around me,

leaving me so swathed in the veil of its effulgence

that I saw nothing else.

così mi circunfulse luce viva,

e lasciommi fasciato di tal velo

del suo fulgor, che nulla m’appariva.

Paradiso XXX.49 Hollander translation

As for the double meanings, tricks of grammar and syntax, these are more difficult to convey, but can still be communicated as long as one has a basic knowledge of the original language. Annotated editions of poems attempt to do this through extensive footnotes, taxing the reader’s time moderately in return for greater appreciation and enjoyment. The amount of notes and their linguistic depth vary significantly between editions, dealing with this tradeoff based on the calculations of the editor and what the publisher is willing to support. Some editions explain a few puns, but leave the rest of poetic meaning-making to the scholars, others come close to approximating teaching the original language in miniature.

One might argue that any external supports, any reinforcing of a poem’s meaning through explanation rather than feeling, is inauthentic and not worth doing. Poetry is a natural expression from one’s mother tongue, and fluent speech in a language is required to read it as such.

I concede that this is possible, but many of the poets I read certainly do not agree. While modern vernacular poetry has tended to emphasize its rootedness in spoken language, poetic language has more often been understood as something that is precisely abstract and scholarly, the languages themselves inappropriate for casual use. Classical Chinese, poetic Latin, and Ancient Greek are languages that were rarely if ever spoken casually, and many of the writers of the poems did not speak them at all. There are centuries of poems written in Classical Chinese by Japanese poets who had no spoken command of the language beyond chanting formulas in Japanese pronunciation, and these poems are still beautiful.

To restrict poems to their original spoken language would also be to disallow them to move through time. Milton and Shakespeare, to say nothing of Spenser and Chaucer, are best read by native English speakers in editions that explain and contextualize how meaning and pronunciation has changed. Even across regions explanation can help, one does not need to grow up speaking Irish English to fully appreciate Joyce nor Southern English for Faulker; missed slang or puns can be elaborated by helpful editors. Abstract processes of language comprehension are and always have been an acceptable part of the enjoyment of poetry.

Certain modern editions of old poems get close to conveying this enjoy of foreign language texts through careful teaching. One of my favorite books is Archie Barnes’ Chinese Through Poetry. It took me an incredible amount of time and energy to work through (I ended up making flashcards for the characters one by one), but being able to go straight to reading Wang Wei and Li Bai in the original was extremely rewarding, and the poems I’ve read character by character are now some of my favorites.

Other editions experiment with new techniques but do not always succeed in helping the neophyte, but are extremely heartening for those such as me who long for a better way. A tour director and guide-lecturer Toshiharu Oseko wanted to be able to convey Basho in the original to interested audiences and has published the poet’s complete haiku with extensive grammatical explanations. While these may be difficult for someone without the stomach for classical Japanese grammar, and the English explanations are sometimes quite lacking, the resulting experience is wonderful.

Such efforts are rare though and rarely available for most poets and most languages.

At present, it is best to read poetry in the original. It is better to read a poem in translation than not at all, and it is even better to read it with explanations and commentary which brings one closer to the original. The problem is that such learning takes time, and that time can only be lessened by the specialized and hard work of editors and scholars, who are much too few.

But what if there were a better way?

The practicalities around languages have changed so that an educated American only knows one language where an educated ancient Roman (or modern Indian) would have known three. These practicalities might change again, either if there were more reason to learn languages, or it became easier to acquire them.

There is likely less reason to learn languages, AI translation technology is improving so quickly that practical barriers to basic comprehension are falling quickly. But the other side of this technology might make the kind of deep appreciation and comprehension I discuss above much easier.

Large Language Models (LLMs) are extremely effective at taking extremely involved linguistic tasks and automating them. Language learning as an individual is so frustrating because the tasks involved (looking up words, parsing grammar, making flashcards, etc) are tedious and time consuming. Fortunately these are the sorts of tasks that LLMs are very good at.

OpenAI’s new GPT Chat has displayed some incredible advanced capabilities in this regard and I decided to try to use it to read some poetry I have long desired to, but have been barred from due to lack of time and resources.

Al-Mutanabbi is a Classical Arabic poet from the 10th century who is often cited as the greatest in the language. Nassim Nicholas Taleb writes “Ask a polyglot who knows Arabic who he considers the best poet — in any language — and odds are that he would answer Almutanabbi.” However, Al-Mutanabbi is tragically undertranslated, and in an age of increasing interest in world literature and the Islamic world, the only versions we have in English are from the earlier Orientalist scholars whose translations most agree have failed to fully capture the original.

I would love to read a good, nuanced translation of Al-Mutanabbi with plenty of notes, but even more I’d like to be able to appreciate the complexities of the original without having to spend months on the rudiments of Arabic’s notoriously hard grammar. Perhaps AI could assist with this.

After trying GPT Chat on a few languages I knew, I decided to put it to the test with a randomly chosen short poem from Al-Mutanabbi’s Divan. I do not know Arabic and have never studied it, however I have learned the alphabet (shakily) from tackling Persian and have a general knowledge of grammatical terms. I chose the following poem:

لا تَحسُنُ الوَفرَةُ حَتّى تُرى

مَنشورَةَ الضِفرَينِ يَومَ القِتال

عَلى فَتىً مُعتَقِلٍ صَعدَةً

يَعُلُّها مِن كُلِّ وافي السِبال

The first thing I ask of GPT is to give me a literal translation and to make it a little easier for me by rewriting it in Roman letters.

This is promising. The language is a bit obscure and there seems to be some failure to put all the words together correctly, but already I can start to grasp the images of this battle poem.

Next I ask it to give me a version showing me each word in the translation alongside the original poem so I could start to grasp the syntax, a method I know from editions of Chinese poetry.

Already I’m able to start picking out features. The one bit of Arabic I know is that “al” means “the,” and sure enough each “al” is translated as such. But I wanted to get deeper and understand the grammar of each word a little more deeply. I ask first about “la” and then “tahsunu.”

Things are starting to make sense. The poem actually starts with “no,” in a way that an English poem reliant on word-order meaning could not. I now understand what the verb is doing and even wha the subject is. I start producing my own more literary translation of the poem as I work through GPT’s explanations.

I seek a deeper explanation of a word to see if GPT is capable of expanding on nuances. I’m happy to find that it can tell me about “al-wafrata” (“abundance”) and also reference the particular Arabic root, a system I’ve heard mentioned in quiet respect by grammarians, and which I now trust GPT to slowly teach me.

I progress slowly through the poem asking GPT to go through each word so I can understand the mechanics of the sentence and the line, attempting to find areas where I can enhance my own parallel translation.

The second half of the poem gives me a bit of trouble.

“On a captive youth, a ladder

He rises on it from the faintheartedness of the lions”

“On a youth” means what exactly? And why is the latter just there, but the next line has the youth rising on “it” from the faintheartedness of the lions.

Analyzing each word in turn, I begin to grasp which grammatically can be the subjects or direct objects or indirect objects. I realize that GPT has difficulty connecting meaning between lines and this is where I, the human, can come in to help it.

Combining the words in the two lines I can grasp that the youth is climbing on the ladder away from the lions, and the image starts to become clear. Al-Mutanabbi, a warrior poet, does not enjoy the abundance of plunder but rather enjoys this one image from it. A captive youth escaping, but also ascending, a metaphor made real. And while lions are normally intimidating and courageous, he is rising away from the lions’ fear, these brutes whose power cannot match the steely resolve of an escaping youth mirroring the courage of the Biblical prophet Daniel. The youth may even be on the opposing side of the battle, but the image of freedom is what truly lasts when the dust settles. I begin to understand the beauty of this short poem.

Abundance does not become beautiful until,

the page of two fates on the day of battle, you see,

a captive youth on a ladder

rising above the fear of the lions

All I need to complete the poem as to better understand the last few words. But GPT freezes, which is understandable considering the demand on OpenAI’s servers. This should prove no problem though, I can refresh my browser and start the investigation from the beginning and just ask about the fourth line. I feed GPT the same initial prompt and get this:

The translation is completely different. I tried it a few more times and it produced several more completely different translations. The “ladder” turned into a “hill,” then to a “mountain”, the “fates” turned into “rivers,” the “lions” into “valleys” and then to “birds of prey.” What’s more, when I asked GPT whether some of the words were ambiguous, whether its suggestions were all possible meanings, it would typically deny its earlier version as a real possibility.

The grammar of the poem completely changed too. GPT would carefully explain why a noun could only possibly the object of a verb, and then switch in the next iteration to making the same claim about its possessive function. Even the reading of the Arabic letters themselves changed and its explanations stopped making sense.

I asked a friend who has actually studied Arabic, and he pointed out the grammatical explanations it was feeding me were not entirely correct either. GPT’s explanation of case seems to have been influenced by explanations of Arabic grammar, but also the grammar of other languages. A simple webpage did a much better job of explaining in brief how all these words functioned and showed that GPT had quickly entered the dangerous territory of “not even wrong.”

GPT appears to have been “hallucinating,” making up correct-sounding responses as if they were facts with no real way to differentiate between its own fictions and ground truth. When I tried to get to reach out to something it could trust, it informed me that this was not currently an enabled capability.

The poem remains untranslated. Even going back to the old state of the art, Google Translate, turned up no paths forward.

While I had a wonderful experience appreciating my own poem, sadly I do not feel as if I have succeeded in experiencing the genius of Al-Mutanabbi.

My experiment with GPT Chat failed, but this was because of the limitations of current technology, not limitations on what is possible. GPT Chat already seems to work better with languages that are better represented in its database, and while it cannot reliably do the sort of question-answer format I prefer for my learning, it and other LLMs can already assist in other language learning tasks such as making interlinear translations or sorting difficult words.

The key issue in my experiment was that this interface currently cannot explore ground truth texts. Hallucination would not be a fatal quality of such a model if it were possible to seamlessly look into sources. My ideal language helper would be able to pull from all known dictionaries (including machine translation of foreign ones), compare them for differences, look up commentaries of similar poems from the same era to understand strange grammatical uses, and give me passages from the relevant references to check its work. Ideally hallucination will become a solved problem, but even after that I’d love to have access to the real grounded resources that a scholar or AI ought to draw on.

Even if what I experienced was a hallucination, it was still a vision of a better possible future. This way of working allowed me to get to an appreciation of a poem in a foreign language I had almost no knowledge of in about 30 minutes without any teacher or supporting materials. As it stands, the technology does not work, and likely I will need many months of learning and access to a teacher to get to the place I thought I had arrived at. At the current pace of technology, it may be faster to wait for OpenAI to release their next model.

There is a huge potential for the small ways LLMs help people read to become big ways that reading is transformed. We might appreciate poetry, from all ages and all places, like no one else before if we can get the right tools for the job.