Can AI Win a Photography Competition? Take the Turing Test
The Royal Meteorological Society, which runs the Weather Photographer of the Year competition, has posed an intriguing question: Can artificial intelligence (AI) win a photography competition?
To answer this, the Society drew up a Turing test in which the viewer is invited to guess which is an AI image and which is an actual award-winning photo.
The Turing test, created by Alan Turing in 1950, is a test of a machine’s ability to exhibit intelligent behavior indistinguishable from a human.
Five of the below images are real photos that were shortlisted finalists in the 2021 Weather Photographer of the Year competition. The other 11 are generated from either DALL-E, Midjourney, or Stable Diffusion. The answers are at the bottom of the article.
Recreating Photos With AI
The Royal Meteorological Society attempted to recreate winning photos from its weather photography competition with AI. It took three winners from the 2021 competition and tried to artificially copy it using the three leading AI image generators.
The above images show the AI attempting to recreate 2021’s overall winner, a photo of a foggy autumn morning in northern Italy taken from a hilltop church.
“The prompt mentions only that a church is involved rather than explicitly defining it as the viewpoint, and that ambiguity leads Midjourney and Stable Diffusion to produce photographs of an Italian-style rural church surrounded by morning mist. DALLE-2 produces an image much closer to the winner, capturing a foggy valley (albeit containing a church tower) lit by a low elevation morning sun,” writes the Society.
Prompts refer to text written by human operators that inform the AI what picture to generate.
“These results highlight the importance of using the right prompt and right theme with the right AI artist, and show that this type of direct comparison is probably not a fair competition,” adds the Society.
To get the AI to produce competition-worthy images that could fool people into believing they are real photographs, the Society tweaked the prompts to include phrases like “award-winning,” “4K,” and describing real locations and events.
Separated into four themes, viewers are able to get an idea of how the different AI image generators perform. The Society describes DALL-E as “excellent at capturing physics and structural features,” Midjourney as “the most creative with the prompts and has perhaps the best lighting,” and Stable Diffusion as excelling at “composition.”
The Society concludes: “There is still a reasonable degree of human creativity required – in dreaming up the text prompts, in refining them to make the generated images more suitable, and in selecting the best images from those produced to take forward.
“Many generated images still contain artifacts that immediately give them away as fakes – and they are still far better at creating artistic stylizations than photorealistic images. But, these gaps are reducing with every new hardware and software development.”
Turing Test Results
As for the Turing test, the images are small and low-resolution making them hard to discern, but the five real photos are (b), (d), (i), (l), and (n). An anagram of the world blind.
Image credits: All images courtesy of the Royal Meteorological Society.
