Human vs AI: Who’s Better at Turning Marcel Proust’s Words into Photos?

In July of 2022, my partner Annabelle Matter and I traveled to Normandy, France, to shoot a project for the weekly Le Figaro Magazine that we’d been researching for months: translating into photographs a text by French writer Marcel Proust, renowned for his highly detailed and soulful descriptions.

We’d selected an article entitled “Impressions de Route en Automobile”, widely considered to have been a prototype for the author’s massive 3,200-page masterpiece “Remembrance of Things Past”. First published in November of 1907, it narrates a part-factual, part-imagined road trip in a chauffeur/mechanic-driven taxi between the small towns of Cabourg and Lisieux.

The article “Impressions de Route en Automobile” by Marcel Proust in a November 1907 issue of Le Figaro.

Illustrating the Essay’s Locations

Illustrating a 114-year-old text that is part fact and part fiction posed a series of new challenges compared to our previous project, in which we’d narrated Napoleon’s life through 300 photographs that fused historical context with current esthetics.

Napoléon – L’esprit des lieux.

While all the Napoleonic locations actually existed (though many took extensive research and/or negotiations to locate and access), those described in the “Impressions” fall into three categories: Some are actual places that existed at the time of Proust’s writing and still do today, some existed then but have since been lost to war or urban development, and others are purely fictional creations cobbled by the writer from a variety of impressions. This required a mix of strategies and artistic approaches.

We’ve consistently found that networking through associations is a very effective tactic, although it is time-consuming and therefore requires an early start. There are numerous organizations of Proust enthusiasts in France (and worldwide), and through them we were able to connect with the author and historian Jean-Paul Henriet, a renowned specialist on the topic of Proust and Normandy, who acted as our guide on location.

Thanks to his extensive knowledge of Proust’s life and works we were able to pinpoint the locations of specific landscapes and buildings, often more precisely than the descriptions in the “Impressions” allowed on their own. He also guided us to places that were similar or equivalent to those that had vanished.

For imaginary places, we took a more radical approach that relied on luck and serendipity: Following the historical road maps from 1907 which we were able to source from antique document dealers, we let our instincts guide our wanderings until we encountered scenes that resonated with the mental images and emotions we had personally experienced through reading Proust’s text. That non-linear process yielded equal parts of frustration and delight – and we got the shots.

AI Image Generation Explodes on the Scene

Le Figaro Magazine published our photo essay in November of 2022, right at the time when the text-to-image AI phenomenon exploded on the scene. My Visual 1st conference partner and principal chair, Hans Hartman, suggested that Annabelle and I extend the project by “feeding” the same textual descriptions that had inspired our photographs to an AI image generator, to see how its results would differ from our own artistic interpretations.

We chose DALL-E 2 for its ability to work directly with the original French text. Since the objective was to obtain a comparison rather than to achieve a predetermined result, we input the relevant paragraphs to the engine as-is, with only the addition of the terms “realistic color photograph” to guide the rendering style.

Roughly speaking, about 80% of results could be described as somewhat close, 10% well off base, and 10% quite good. Strangely, while DALL-E 2 always complied with “realistic” and “photograph”, it had more trouble with “color” and consistently offered several renditions in black and white. The versions we selected for the five comparisons below are the ones that came closest to our own images. No enhancements were made to the content generated by the program.

Paragraph #1

In this paragraph, Proust describes fleeting views of “old and wobbly” traditional Normandy houses that he glimpses from his moving car. He goes into some detail about the rose bushes and trees that grow alongside them.

Alexis and Annabelle
DALL-E 2

Surprisingly DALL-E 2’s flowers are not roses, but gladiolas. The trees which feature prominently in Proust’s description are a very minor part of this image. The architectural style of the house however is fairly convincing.

Paragraph #2

In this paragraph, Proust describes the moment when the plain surrounding the city of Caen becomes visible to him from the road.

Alexis and Annabelle
DALL-E 2

In this case, no matter how many variations were asked for, the AI insisted on generating an aerial view rather than a ground-level view.

Paragraph #3

In this paragraph, Proust describes houses at a location in the town of Lisieux that feature a particular architectural style and are decorated with carvings of saints or demons.

Alexis and Annabelle
DALL-E 2

The AI gets the architectural style absolutely spot on, but it misses the figurative carvings altogether, possibly because its training set didn’t contain these types of images.

Paragraph #4

Here Proust describes a section of the porch of a 13th-century cathedral in Lisieux which features a series of closely spaced stone columns. He visits at night, and his driver uses the headlights of the car to sweep across the porch so that Proust can see the carvings.

Alexis and Annabelle
DALL-E 2

The AI did a remarkable job of understanding the subject matter and lighting. However since the text doesn’t specify the number of columns, DALL-E 2 is at a disadvantage versus our ability to go on-site.

Paragraph #5

Here Proust engages in a cultural digression referencing the author La Fontaine as well as various painters. Rather than an actual scene, he describes an archetype of a “voyager”, or traveler, who can be imagined riding a horse at sunset on a beach.

Alexis and Annabelle
DALL-E 2

Here the previous situation is flipped: While DALL-E 2 doesn’t have the advantage of being on location in Normandy, it does have access to plenty of images of beaches, sunsets, and riders – whereas, at the time we were on location, no equestrians appeared at sunset.

AI Image Generation is Impressive

While this exercise was too narrowly defined for us to draw sweeping conclusions on the value of generative AI in the creative process, we were impressed with its ability to produce a usable starting point for an image. Whether or not that starting point is preferable to a photograph is likely to vary depending on the final result desired, but it was clear to us that in most cases producing something that can be considered a legitimate work of art should require a significant amount of “post-AI” work by the artist/author.

On reviewing the results, Hans Hartman offered the following thoughts from his perspective as a leading industry analyst:

If DALL-E were human, according to Maslow, she’d master the creation of visuals from clear-cut descriptions first and foremost before tackling the higher echelons of multi-faceted literary passages and turning these into esthetic pictures. But DALL-E is not human and is apparently firing on all cylinders tackling multi-faceted and multi-interpretable literary texts as well as descriptions of well-defined objects.

Despite also producing a fair amount of unsuccessful visuals, DALL-E proved at times to be remarkably capable of creating images that represented challenging descriptions, such as a particular architectural style, landscape composition, a scene with car lights sweeping across a porch, and a beach scene featuring a galloping horse.

Surprisingly, DALL-E stumbled in some instances of well-defined descriptions – mistaking gladiolas for roses, and showing an aerial rather than a ground-level view. Going forward, it’s safe to assume we’ll be seeing fewer and fewer of these types of mistakes (plus a variety of other misinterpretations and non-sensical imagery) and the battlefield will center around the artistic eye of the serious photographers versus that of advanced generative AI solutions. That battle has already started, as Alexis and Annabelle demonstrated. Place your bets!


About the author: About the photographers/authors: Annabelle Matter and Alexis Gerard are a photographer couple/unit whose work merges the past and present into “time-fused” images by integrating historical insight with contemporary artistic sensibility and perceptions. Their projects have included retelling the life of Napoleon through on-site photographs in the book “Napoleon l’Esprit des Lieux“, translating Marcel Proust’s prose into photos for the Figaro Magazine, and contributing to numerous publications including GEO and Le Point. They are represented by AKG Images and Farmboy Fine Arts.

To contact Annabelle and Alexis please use the form at https://www.annabellematteralexisgerard.photography.

Discussion