Adobe Retrained Its Generative AI, and I'm Impressed With the Upgrade
I put the Adobe's Firefly image creator to the test. It's not perfect, but it's a bubbling fountain of creative output. Also new: generative AI in Illustrator and Adobe Express.
- Shankland covered the tech industry for more than 25 years and was a science writer for five years before that. He has deep expertise in microprocessors, digital photography, computer hardware and software, internet standards, web technology, and more.
It can be fun to use Adobe's Firefly to generate scenes never before seen by human eyes. The prompt for this image was "A mountain scene with grass and wildflowers in the foreground and conifers in the background fading into the mist. a groundhog looks up in the foreground. A storm looms in the clouds above the mountains."
If you're hoping artificial intelligence will get your creative juices flowing faster, Adobe on Tuesday revealed three big changes to its Firefly family of generative AI tools. The engine that powers Photoshop's image generation is getting a big upgrade, and generative AI is coming to Adobe Illustrator designs and Adobe Express layouts.
Firefly already let you turn text prompts into pictures on the Firefly website and in Adobe's Photoshop image-editing software, but a second-generation AI model offers more detail and better image quality, said Alexandru Costin, Adobe's generative AI leader. The model isn't yet available in Photoshop, but you can expect it to arrive after some testing online.
In my testing, I did indeed find the results and the user interface much better. Adobe trained the new AI model on twice as many images, and it offers higher resolution, better detail like skin pores, and the ability to steer generation with photography parameters like lens focal length and depth of field.
Generative AI starts with the same basic AI methods that have been used for years: train a system to recognize patterns in real-world data. But generative AI goes a step further with the ability to create new material like text, images, speech or video based on its own understanding of those patterns. That's revolutionized computers, lifting them out of their plodding, literal ways and giving us a taste of what a truly smart machine might be like.
Trying Adobe's upgraded Firefly generative AI
Although Firefly results sometimes are unconvincing, it's just plain fun to create fanciful and entertaining images, especially when using more forgiving art styles like paintings, cartoons and watercolors. I enjoyed typing in prompts, seeing what the AI would produce, then tweaking the prompts for more useful or outrageous results.
The interface is improved, too. A high-level option to choose between photorealism and a more artistic illustration style is helpful, as are options for square, landscape and portrait aspect ratios. Sliders for visual intensity and style strength give you a choice of images that are dramatic, understated or something in between. You can upload a reference photo to steer the style of the output.
Adobe's Firefly generated this image of a ghoul wearing a heavy metal outfit mountain biking through a post-apocalyptic wasteland. Just don't look too closely at the pedals.
One of my standard prompts to test AI, a parachuting elephant, produced better results than the first-gen Firefly, though the technology still struggles with the parachute cords. In one image, it managed to construct a wooden frame to hold the elephant, which surprised me.
Creating "a kindly doctor in a hospital room for a pharmaceutical ad" produced a variety of generally acceptable portraits. Firefly produced a variety of races and genders, but every one of them had the obligatory scrubs and stethoscope.
The images Firefly created of a spikey electric guitar were satisfyingly dangerous looking if not always endowed with the correct number of frets, pickups and strings.
My testing also showed there's still a lot of work to be done. In images of the stereotypical hacker hunched over a keyboard, they sometimes wore their hoodies backwards. My quest for a photorealistic red crab waving its claws in the air went unfulfilled, with distorted claws, extra eyes and other problems. Firefly did a passable job generating a groundhog, but Adobe's training data evidently doesn't include enough picas — a more unusual rodent found in the high mountains.
Adobe's Firefly presents a quartet of options, in this case prompted with "a kindly doctor in a hospital room for a pharmaceutical ad." It offers a range of genders and and races, but perhaps because of the stock imagery it's trained on, I never once saw an older doctor among the several dozen images I created. The Firefly website lets you rate the images and submit feedback.
And another test I like to run, an image of an angry crocodile leaping out of a stormy ocean with lightning striking all around, still struggles with a plausible arrangement of teeth. My "ghoul in a heavy metal outfit rides a mountain bike through a post-apocalyptic wasteland" prompt showed that it's really hard to fit the geometry of a humanoid character to the mechanics of a bike. "A high-tech communications network spans the globe with data surging through the wires" produced a tangled mess.
But you can refine prompts to get better results. and the art style often produces more convincing subjects than photo style. Some art styles, like doodle drawing, are more forgiving than others. I particularly liked the watercolor option. But perfect Firefly is not.
Firefly comes to Adobe Illustrator
Adobe Illustrator, which designers use to create vector graphics like logos and diagrams, now gets Firefly text prompt abilities for the first time. As with Photoshop, the software will turn a text prompt into a quartet of illustration candidates you can choose from and refine with further editing. The illustrations are fully editable.
Adobe Illustrator gets Firefly's generative AI abilities, letting designers create vector illustrations like these cans with desert scenes.
And the Adobe Express app for building creations like flyers and social media videos gets a text prompt field of its own that you can use to build templates. For example, if you type in "pirate themed birthday party announcement for children," it'll spit out some layout options with custom art and fonts. It's in beta testing for now.
Read more: Here's What I Learned Testing Photoshop's New Generative AI Tool
Generative AI's creativity can be a problem if you're looking for factually accurate information for your lawsuit, travel itinerary or high school essay. But it can be a boon for creative uses, and Adobe is counting on Firefly to overhaul what's possible with its tools and help those who might lack expertise to spread their wings.
Sneak previews of AI-powered projects
AI showed up on Wednesday, too, the second day of Adobe's Max conference, when the company revealed several of its traditional "sneaks" — previews of technology under development it expects to ship eventually.
Ahead of Max, Adobe showed off Project Stardust, a new photo editing system that analyzes a scene, separating it into elements so that people can tap on subjects to select, move, delete, resize or modify them. It even notices shadows and alters them accordingly. That kind of AI-powered ability could ease the notoriously difficult process of changing only the part of a photo you want to change.
"Stardust is a new editing engine that we hope will revolutionize photo editing for potentially millions of people," said Stardust leader Mark Nichoson.
Using Adobe Firefly's more whimsical doodle drawing setting can help ease problems with realism, such as the lack of AI training data that shows elephants in the middle of the air.
Here are some of Adobe's other AI-powered sneaks:
Project Fast Fill brings generative AI to video, letting editors delete people from the background of one video frame and then extend that change to the entire video clip. It can also track a designated area so a change can follow subjects. Adobe showed one example adding a necktie to a walking person's outfit and another with a new foam pattern on the sloshing surface of a cup of latte.
Project Poseable lets people convert a 2D photo of a person into a 3D model in that same posture — sitting, lying down or in a fighting stance, for example. You can then reposition the limbs and joints of that skeletal model into a preferred pose, and with a text prompt build a new character style, like ogre or nurse. Adobe thinks it'll be good for storyboards, though for now it can't maintain a consistent character from one creation to the next.
And Project Draw & Delight turns a crude sketch accompanied by a text label into a cartoon. You can add extra material with new sketches and prompts, and it'll modify the existing design, preserving the character you've already chosen.
Firefly generates 3 billion images so far
In a few months of testing, Adobe customers have embraced AI rapidly, generating more than 3 billion creations so far, Costin said.
"Normally, features in Photoshop get a single percent utilization rate. Generative Fill [a Firefly Photoshop ability] got 10x that percentage in the first month," Costin said. "Customers love this tech."
Adobe's Firefly generative AI tool created this uncomfortably spiky guitar.
Adobe's bean counters also likely love it. Adobe subscription plans let you use Firefly many times per month, but in November, Adobe is raising Creative Cloud prices by about 9% to 10%.
The generation takes place on Adobe's cloud computing infrastructure and costs real money, especially given the high price tag of Nvidia processors that handle most generative AI work these days. Those big AI models typically don't fit in the memory of an ordinary laptop, but the industry is working on that problem at the same time companies like AMD, Intel and Apple are adding new acceleration abilities to their processors.
Other AI abilities emerging at Adobe Max:
- Adobe also is using AI to help you use AI. In Photoshop, it'll suggest completions to your text prompts to try to produce better results. This could help people who are short on experience in the latest field of computer science, prompt engineering. "If you don't know what to write, we're autogenerating prompts not only to statistically write a sentence but to try ... to generate images that look beautiful," Costin said.
- In Lightroom, a new AI-based lens blur effect will artificially blur backgrounds, simulating the bokeh that higher-end lenses can produce naturally to isolate subjects from the rest of a scene.
- Businesses can customize how Firefly works by uploading their own assets to steer the Firefly generation process in the right direction.
Editors' note: CNET is using an AI engine to help create some stories. For more, see this post.