This Wild Robot Taps AI to Paint Whatever You Tell It To

Remember the olden days? When AI tools like Dall-E and Stable Diffusion turned your short text prompts into digital art? That's so 2022… 

Meet Frida, an AI-driven robot out of Carnegie Mellon University that transforms your prompts into physical paintings, complete with bold brushstrokes in a variety of techniques. Perhaps most strikingly, the bot can change course as it paints to mimic the iterative nature of making art.     

"It will work with its failures and it will alter its goals," Peter Schaldenbrand, a Ph.D. student at CMU's School of Computer Science and one of the robot's creators, said in a video describing the project. 

Frida aims to explore the intersection of robots and creativity, says the team, which presents its research paper this May at the IEEE International Conference on Robotics and Automation in London. Robots have produced art before and even exhibited it. But Frida is designed expressly to collaborate with humans using the same sort of generative intelligence that drives experimental tools like the AI chatbot ChatGPT. 

Frida stands for Framework and Robotics Initiative for Developing Arts, but it also shares a name with famed painter Frida Kahlo. The off-the-shelf machine, found in a basement, looks nothing like Kahlo or any other human, though. For now, it's simply a robotic arm with a paintbrush affixed, a configuration that underscores the team's insistence that Frida is a "robotic painting system, not an artist." 

"Frida is not generating the ideas to communicate," Schaldenbrand said. That's where humans come in, communicating goals for Frida with text inputs. They can also show the bot images in a style they like or even flash photographs they want to see represented as a painting. The researchers are even playing around with audio prompts (they played ABBA's song Dancing Queen, an excellent choice, and asked Frida to paint it). Frida suggests appropriate paint colors on screen, then humans mix it in the robot's palette. 

"Frida can help someone experiencing physical barriers to creating visual art," Schaldenbrand told me. "Frida can also help people who don't have time to engage in art as it can automate some of the tedious elements. We are now moving forward and working with people to see the range of Frida's capabilities."

Enlarge Image

Frida probably won't ever reach the renown of its namesake, but some impressive skills set it apart from other artsy robots, whose input images generally match their final goal. Creating art is a dynamic, constantly evolving process, and after planning its trajectory in a simulated environment, Frida uses machine learning to evaluate and progress in real time. The robot does its planning, as CMU robotics professor James McCann notes in the video, "in a space of meaning instead of a space of outputs." 

Frida doesn't prize precision like most robots and might, for example, incorporate a "mistake" like an errant spot of paint into its final product. Each painting takes hours to complete, and the results are often whimsical and vividly colorful. 

"There's this one painting of a frog ballerina that I think turned out really nicely," Schaldenbrand said. "It is really silly and fun, and I think the surprise of what Frida generated based on my input was really fun to see."

Enlarge Image

To train their multiple AI components, the team input current news headlines and further trained them on images and text representative of diverse cultures to avoid an American or Western bias. 

AI's role in generating visual art, composing songs and even writing poetry and movie scripts is generating excitement, but also raising ethical and copyright concerns among artists and lawyers. AI art isn't created in a vacuum. It works by absorbing and reconstructing existing art created by humans. As machine-made art improves, will those humans -- actual graphic designers, illustrators, composers and photographers -- find themselves edged out of work? 

Some artists I've spoken with describe feeling jittery about the complex questions raised by AI art. Others -- like Steve Coulson, an avid comics fan who wrote a comics series drawn entirely by Midjourney -- are embracing what they view as an inevitable shift. The comics artists Coulson has long loved "have an eye for dramatic composition and dynamic narrative that I strongly doubt machine learning will be able to match," Coulson says. "But as a visualization tool for nonartists like myself, it's a hell of a lot of fun." 

Frida's inventors share a similar perspective. 

The armed bot, the research paper says, "is a robotics initiative to promote human creativity, rather than replacing it, by providing intuitive ways for humans to express their ideas using natural language or sample images." 

Editors' note: CNET is using an AI engine to create some personal finance explainers that are edited and fact-checked by our editors. For more, see this post.