Antenna implanted in cyborg's skull gets Wi-Fi, color as sound

Neil Harbisson, a 31-year-old European artist, was born with a rare visual condition known as achromatopsia, which means that he can only see the world in black and white. Because he wanted to experience color, in 2004 he had an antennalike device, which he calls an "eyeborg," attached tightly to his head. The eyeborg could sense color around him and convert it to sound that he could hear as it was conducted along the bones of his skull. He then learned to associate the sounds of colors with their names.

A few months ago in December, Harbisson underwent a surgical procedure that more deeply integrated a new model of the eyeborg into his body via three holes he had drilled into the back of his skull.

Like the previous version, his new model can sense 360 different colors and pick up ultraviolet and infrared frequencies. This version of the eyeborg, however, has Wi-Fi and Bluetooth connectivity, so friends with smartphones can use an app to beam images directly to Harbisson's gear so he can "see" them without using his eyes.

I spent a fascinating half-hour chatting with Harbisson and I'm truly excited to meet him later in April at Moogfest, a festival celebrating the future of technology and sound, which I'll be covering for Crave. Here are the highlights of our interview.

Can you tell me a little about how your new eyeborg differs from the original?

I call it a cyborg antenna because it can do many more things than just convert color to sound. So it's an antenna that goes directly inside my bone. It's drilled inside the skull and then it has Bluetooth connections and allows me to connect to other antennas or other mobile devices or to the Internet directly so I can perceive things that are not in front of me.

So if somebody with a mobile phone wants to send me images from anywhere else, then I can receive these images directly inside my head and I can perceive them. The main difference is that it's now inside the skull and I can connect to other people's vision.

Before it was an explant, so it was pressuring the bone...now there's no pressure anymore because the sound goes directly inside the skull. There are three holes. One is for the audio input, the other is for the antenna, and the third one is actually a spare one.

Is it more comfortable than the other model?

It's different, yes. I don't feel any pressure now. The big difference as well is that if someone touches the antenna basically I feel it, which is something I didn't before. It's as if someone was touching your tooth or something. So this is very different. And not feeling the pressure is also very different. At the beginning I felt that I didn't have the antenna, so I felt weird because I didn't feel it. Otherwise it's just really good. It's a big, big difference physically.

You mentioned that you started dreaming in color: Did you actually see color?

To me color is never a visual perception. Color is a very specific perception of sound. So for people who see color it's the perception of vision -- color is a visual interference. But to me it's an audio interference. So my perception of color is through sound. So I started to hear specific electronic sounds in my dreams that give me color perception.

During a TED talk you said the way you were dressed represented a specific chord.

Yes, that's how I dress. I now think of dressing as music. It wasn't a joke really, I just decide how to dress depending on how I want to sound. We've also designed some clothes that sound as a specific music piece. So I have a tie that sounds like a piece by Sega Bodega -- I heard this piece of music and I liked it and then I transposed it into a tie so I can wear this specific piece of music. We also designed a wedding dress that sounds like the Wedding March. This friend of mine, she got married wearing the Wedding March.

Did you grow up musically inclined, or have you gotten more musically inclined as a result of wearing the eyeborg?

I grew up musically inclined, but color is sound-inclined. Music usually just has 12 notes, where color has 360 notes in an octave. So I'm more conscious of sound than music. So it's changed the way I perceive music. Color is so so microtonal that music sounds too simple; the translation of color into music would be to simplify it if we only have the 12 notes in an octave.

Can you ever turn the eyeborg off?

No, there's no on-off switch. But if the room is dark, if there's no color then there's no sound. So I could also cover the sensor with my hand or with something.

Does it ever get annoying or is it more like part of your background processing?

You know at the beginning, it was like sound interference constantly and then it just became normal and then it became beautiful. I'm used to hearing color as much as people are used to seeing it.

Have you ever had any brain scans done to examine if your brain's undergone any neuroplastic changes?

Yes, there was a television program and they wanted to test my brain so we went to this brain center in Valencia. They put me in this machine and tested the brain and they concluded that, yes, my brain was working differently. During the scan they showed me some images, and my brain was creating sound when I was looking at the images. It's what happens when I sleep as well. If I dream of the sky or something, I hear the sound of the sky and my brain creates this extra layer of color when I imagine things or see things.

So it's not directly related to the eyeborg anymore. It's the way your brain works from your imagination, for example.

Yes. And this is what happens when I sleep as well. It's not the software that creates the electronic sounds, it's my brain. And that's when I felt that I couldn't feel the difference between the software and my brain, because it really feels exactly the same. People can close their eyes and visualize a color, I can also -- I don't know what visualizing sound would be, soundalize? -- a color.

Tell me a little bit more about your cyborg advocacy and the Cyborg Foundation.

We develop different projects where we try to extend other people's senses. One of them has been with Moon Ribas [Harbisson's partner in the foundation] and she's had seismic sense for almost a year now. She doesn't feel disconnected from the cybernetic extension that she's been wearing, so she does feel it's a part of her and this extension allows her to sense whenever there's an earthquake in the world.

So now if she's in Barcelona but there's an earthquake in Japan, she feels a vibration in her elbow. And depending on the Richter scale of the earthquake, the vibration is higher or lower, so she feels a strong connection with the natural movement of the Earth. It has now become normal for her to feel earthquakes every 8 or 12 minutes because it measures from No. 1 on the Richter scale.

We also developed with another university in Barcelona something like a small crown that vibrates whenever someone gets close to you. So if someone gets close to you in the back, it vibrates and it gives you 360-degree perceptions of presence. It's now being developed even smaller so that it can be implanted and it doesn't need to be worn.

How does this earthquake technology distinguish between regular vibrations and earthquakes?

It connects to online seismographs. So her elbow is connected to the Internet and she has an account that collects all the online seismographs in the world.

She's kind of plugged into the Earth, isn't she?

Yes and I feel it's really, really beautiful because it's not an artificial reality. She can sense the natural movement of the Earth. She uses this as well in dance because she's a choreographer, so her main choreography piece is called "Waiting For Earthquakes" where she just stands on stage and doesn't dance until she feels an earthquake, and her dance intensity depends on the intensity of the earthquake. If there are no earthquakes there's no dance.

The Cyborg Foundation wants to defend cyborg rights. Are there infringements on cyborg rights at the moment or do you envision that there will be?

It's mainly defending that we should have the same rights as everyone else. It's not trying to create new rights, it's mainly just defending the basic rights. So in cases of not being allowed into a place because you use technology as a part of your body, that's something that we should defend.

Has that happened to you before? Have you been banned from going into certain places?

Yes, many times. For example, in shopping malls they don't like you if you have something that they don't know what it is, or if you explain to them what it is, they may not believe you and then ask you to leave. Also cinemas might not like you. They think I might be filming or doing some kind of espionage or something strange, so they don't want me to go into places.

Not being allowed to renew my passport had nothing to do with filming, you were just not allowed to appear on passport photographs with electronic equipment. I felt this was something against the right to have technology as part of your body.

It seems that the advent of wearable technology will help in this area.

Yes, I think we're going to enter now the age of wearable technology and this will help a lot with people who don't wear technology but who are technology. I think the next stage is that we will see that wearing technology is not comfortable. It's actually much better to become technology rather than wear it.

Tell me a little bit about what you're going to speak about at Moogfest.

I'll try to focus the talk on the idea that we can extend our creativity if we extend our body senses and perception by applying cybernetics to our body. Artists should have a particular interest in extending their senses because when you create new senses or extend existing senses it means you can express yourself through new senses that have never been explored before.

I never felt that I had a disability, because seeing in black and white was never a disability, it was a visual condition. I didn't actually want to change my vision, I wanted to perceive color, because to me, seeing in black and white is extremely positive. I don't find it negative in any way. I wanted to have a sense of color and that's why I chose sound, not sight, because I think sound can be much more precise than vision.

So when I started this project, the aim was not to perceive color like other people, the aim was to create a sense of color and a sense that I would be able to upgrade. When I was able to perceive 360 colors, then I included infrared and other colors. Now that I have Bluetooth, I can perceive colors that are not only in front of me, but colors that other people in other places of the world are perceiving. The aim was just to extend a sense.