news

Research shows that computers can match humans in art analysis

Release Date: March 5, 2013

Tarakhovsky and Shamir

Jane Tarakhovsky is the daughter of two Russian artists, and it looked like she was leaving the art world behind when she emigrated to the US in 1998 and decided to become a computer scientist after receiving her bachelor’s degree in history at 15 years old at an art school in Russia. But her recent research project at Lawrence Technological University has demonstrated that computers can compete with art historians in critiquing painting styles.

While completing her master’s degree in computer science earlier this year, Tarakhovsky used a computer program developed by Assistant Professor Lior Shamir to demonstrate that a computer can find similarities in the styles of artists just as art critics and historian do.

In the experiment, published in the ACM Journal on Computing and Cultural Heritage and widely reported elsewhere, Tarakhovsky and Shamir used a complex computer algorithm to analyze approximately 1,000 paintings of 34 well-known artists, and found similarities between them based solely on the visual content of the paintings. Surprisingly, the computer provided a network of similarities between painters that is largely in agreement with the perception of art historians.

For instance, the computer placed the High Renaissance artists Raphael, Da Vinci, and Michelangelo very close to each other. The Baroque painters Vermeer, Rubens and Rembrandt were placed in another cluster.

The experiment was performed by extracting 4,027 numerical image context descriptors – numbers that reflect the content of the image such as texture, color, and shapes in a quantitative fashion. The analysis reflected many aspects of the visual content and used pattern recognition and statistical methods to detect complex patterns of similarities and dissimilarities between the artistic styles. The computer then quantified these similarities.

According to Shamir, non-experts can normally make the broad differentiation between modern art and classical realism, but they have difficulty telling the difference between closely related schools of art such as Early and High Renaissance.

Tarakhovsky utilized her knowledge of art to demonstrate the versatility of an algorithm that Shamir originally developed for biological image analysis while working on the staff of the National Institutes of Health in 2009. She designed a new system based on the code and then designed the experiment to compare artists.

She also has used the computer program as a consultant to help a client identify bacteria in clinical samples. “The program has other applications, but you have to know what you are looking for,” she said.

Tarakhovsky believes that there are many other applications for the program in the world of art. “This is just the tip of the iceberg,” she said.

“My professors at LTU have provided me with a broad perspective and have encouraged me to go to new levels,” she said. Her experience demonstrates that women can succeed in scientific fields and that people can make the transition from subjects that are more in demand now that the economy is driven by technology.