Machine learning goes for baroque and paints 'brand new' Rembrandt
Facial recognition software has analysed 346 Rembrandt paintings to create an all-new work in the artist's style.
Who's your favourite artist of centuries past? How badly would you like to see a new work by them? Technology may be the key, if a newly completed project is any indication.
A team of Microsoft data technicians and art historians have used digital analysis, facial recognition and 3D printing to create an entirely new painting in the style of Dutch Baroque painter Rembrandt van Rijn (1606-1669).
The project, called Next Rembrandt, took 18 months to complete. It was a collaboration between ING, Microsoft, Delft University of Technology, The Mauritshuis and Museum Het Rembrandthuis.
The first step in generating a new Rembrandt painting was data collection. This involved creating a digital database of Rembrandt's entire body of work, some 346 paintings. These included high-resolution 3D scans of the paintings, upscaled by deep learning algorithms.
This produced 150GB of digitally rendered graphics. These were examined, pixel by pixel, to determine the idisoyncratic elements that make up a Rembrandt: The harsh lighting that is so characteristic of his paintings, how his portraits are commonly posed, the average age and race of his chosen subjects, their attire, how much facial hair they have, even the composition of their faces.
Comparison of Rembrandt portraits showing similar poses and proportions.
These demographics pointed to a Caucasian male, between the ages of 30 and 40, with a beard, black clothes, white collar, and a hat, seated angled to the right, gazing out of the canvas at the painter.
"To master his style, we designed a software system that could understand Rembrandt based on his use of geometry, composition, and painting materials. A facial recognition algorithm identified and classified the most typical geometric patterns used by Rembrandt to paint human features. It then used the learned principles to replicate the style and generate new facial features for our painting," the website reads.
"Once we generated the individual features, we had to assemble them into a fully formed face and bust according to Rembrandt's use of proportions. An algorithm measured the distances between the facial features in Rembrandt's paintings and calculated them based on percentages. Next, the features were transformed, rotated, and scaled, then accurately placed within the frame of the face. Finally, we rendered the light based on gathered data in order to cast authentic shadows on each feature."
Rendering the image took over 500 hours, but the work was not yet done. Making the digital file into a tactile painting was the final step. For this, the algorithms generated a height map, a sort of topography for the brushstrokes, based on analysis of 3D scans of Rembrandt's paintings. The final painting was produced in 13 layers of paint-based UV ink to create a facsimile of Rembrandt's painting style in oils.
The project was originally conceived as a promotional one, but it does raise an interesting dialogue about the relationship between technology and art. While some might question the artistic merits of a digitally generated Rembrandt, it's important to remember that without the artist himself, the algorithm would have had no paintings to copy.
It's also interesting to think about in terms of art forgery. The practise of passing off artworks under famous names has been thriving for millennia. In its current state, Next Rembrandt is unlikely to be of use; for starters, the forgers would need to get hold of some pretty advanced software. The final product would also be a problem, since it would take an expert very little time at all to establish that it was printed in ink, not paint.
If someone invented an oil-painting robot that could output the images devised by the algorithms, on the other hand...