Apple pioneer Bill Fernandez on AR, VR and the design of the future
Apple's first employee has a keen perspective on what's next.
- Jason is an award-winning journalist, editor, and video host with a passion for covering cutting edge innovation.
Bill Fernandez holds an Apple I.
EDITOR'S NOTE: This article was originally published on December 8, 2014 and is being republished today to commemorate Apple's 43rd anniversary (April 1, 1976).
At Apple, Bill Fernandez rode shotgun on the creation of the personal computer industry. He was on the original Mac team that made decisions that defined the modern user experience. He worked on QuickTime, which helped define Web video. He worked on HyperCard, which helped inspire HTML and the Web. So clearly, the man has some ideas about what makes a user interface work.
He also gets props for this: He's the guy who introduced Steve Jobs and Steve Wozniak.
Fernandez became the first official full-time employee at what was then known as Apple Computer Inc. Except for a brief absence early on, he remained with Apple until 1993 and became an expert in user interfaces.
I wrote an in-depth feature on TechRepublic about Fernandez's remarkable career. During the interviews with Fernandez, I got to pick his brain about the current state of user interfaces, his take on cutting-edge technologies, and what's next in computing. Not all of that material made it into the TechRepublic long form article, so I've pulled it together here -- there's a lot of wisdom about today's challenges in UI and a gut check on the sci-fi visions of the future.
I broke down Fernandez's comments into the most forward-looking topics.
The current state of UI
"The true user interface is the gestalt of many things: industrial design, ergonomics, internal hardware, performance, error handling, visual design, interaction, latency, reliability, predictability, and more. For this reason I'm glad to see my profession moving from the concept of user "interface" (where the user interacts with the screen) to user 'experience' (where more the above factors are considered)," said Fernandez.
"I can design the best user interface in the world on paper, but if it's not implemented well by the hardware and software engineers (e.g. the product is slow or buggy) then the product will fail. If you do a really good job of designing a product then everyone who sees it thinks it's so obvious that they could have done it themselves."
The challenge of flat design
"We are in a time of transition, and like the water becomes brackish where river water meets the ocean, the state of UI design is messy," said Fernandez. "There's some great stuff out there, much more than there used to be, but there's still a lot of trash, and there's a lot of well-meaning but misguided efforts. One example of this is in the migration from three-dimensional, photorealistic UI elements (window frames, push buttons, sliders, etc.) to 'flat' UI design. Years ago a friend asked what I thought Web pages of the future would be like and I said 'like magazines.' I thought we'd see flatter designs, expert typography, beautiful, magazine-advertisement-like page layouts, etc. That prediction is coming true with the trend toward 'flat' UI design...
"But in moving towards flat design we are losing much of the wisdom that was embedded in the old 3D style of UI, for example: a user must be able to glance at a screen and know what is an interactive element (e.g., a button or link) and what is not (e.g., a label or motto); a user must be able to tell at a glance what an interactive element does (does it initiate a process, link to another page, download a document, etc.?); the UI should be explorable, discoverable and self-explanatory. But many apps and websites, in the interest of a clean, spartan visual appearance, leave important UI controls hidden until the mouse hovers over just the right area or the app is in just the right state. This leaves the user in the dark, often frustrated and disempowered."
Tech's biggest obstacle
Bill Fernandez talks to Homebrew Computer Club members at the reunion in 2013.
"There have always been dreamers and visionaries of the future: flying cars, personal jetpacks, video phones, cashless payments, etc. The limitation has always been the hardware technology, which is always the limiting factor both in what is possible and in what is affordable by the masses," said Fernandez. "There are technological advances of the last decade in batteries, semiconductor sensors, low-power systems-on-a-chip, displays, spread-spectrum radio protocols, etc. that are just now finding their way into products. It seems unlikely to me that we're just around the corner from having teleportation, boundless free energy, interdimensional travel or magic that effortlessly transmutes lead into gold. But I do expect the relentless progress in science and technology to continue, and progressively for the science fiction of the past to become the hot new consumer product of today.
"Look for lots of new categories of personal devices -- like personal health trackers -- to emerge over time. One of the big things holding back the development of many kinds of products is our batteries. At present there's a new wave of wrist-mounted devices -- from Pebble, Google, Motorola, Samsung -- but batteries are really holding back what is possible. When a watch battery can hold as much energy as a car battery, and a large-screen computer-display-projector can be built into a watch, look for all your smartphone apps and capabilities to migrate into a wrist-mounted device rather than a belt-mounted device."
'Minority Report'-like touch panels
"The idea of having the transparent panels like you see on 'Minority Report' and 'Iron Man' and 'Avatar' are for the most part, impractical," said Fernandez. "By waving your arms, you don't get any precision. Tony Stark in 'Iron Man,' he's designing things. In real life, he'd have to spend hours drawing schematics, making precise mechanical diagrams, and there's no way you're going to do that in the air waving your hands. Your arms are going to become tired. So that's not really practical. It looks really good on the screen when you can take the camera through one of these displays to a person... But in real life people don't actually have that stuff...
"On the other hand, when we have things like 'Blade Runner,' which showed advertisements on every surface and when you saw 'Minority Report' where you'd walk down a hallway and an ad would be shown to you that's targeted, that's very definitely part of our media future. We already have it on websites, where you get targeted ads on each page that you go to. And [with] Apple's iBeacons where you can put your beacons in stores and your phone can show you ads for products that you're passing by. So, we're definitely stepping into that world."
Google Glass
"Years before Google announced their Glass project I used to paint this picture of the future: We will see a time when folks will be sitting in Starbucks gazing off into the distance, twitching their heads, waving their hands in mystical-looking incantations, and muttering to themselves," Fernandez said.
"Why? Because they will have projectors projecting virtual data screens into their eyes. As they move their heads their virtual data screens will update to show things (books, bookcases, apps, etc.) that they have arranged in space around them. Rings on their fingers will sense the positions, orientations and gestures of their hands. Voice interaction will let them speak voice commands and hear synthetic speech. So, for example, a user will turn his/her head to see a book, reach up to grab it and drag it down in front of him/her. Use a voice command to open the book to a section of contacts, use a combination of voice commands and gestures to find a desired contact, point to the one he/she wants and say 'dial that one.' So there will be a seamless, multimodal UI experience using sight, sound, voice, head motion, arm movement, gestures, typing on virtual keyboards, etc. There have been pieces of this in research for decades. Only now are we beginning to see some of these things emerge into the consumer market: with Google Glass, Siri, etc. We have further to go, but we're on the road."
Oculus Rift
"Oculus Rift is an interesting thing in that finally it looks like someone is putting high-quality virtual reality scenes into a product that will be affordable. It is, of course, going to be of very limited use because you have to be sitting somewhere. You can't be [using it] going about your daily life or driving a car," said Fernandez.
"[But] I like seeing things like Oculus Rift and Google Glass as kind of steps in the direction of actually bringing this vision I've had for years to life. So that's very interesting to see that happening in little baby steps."
Read TechRepublic's in-depth story: "Apple's first employee: The remarkable odyssey of Bill Fernandez"
