The Right Glue
February 2011

Fictional interfaces

By Dean

Have you ever read a fiction book where the main character gets frustrated by an interface? Have you ever played a non-puzzle video game where your character has had trouble determining which buttons activate what doors, traps and/or lasers?

Unless you've been reading a lot more boring stuff than I have (or playing a lot of Myst) you should have noticed that this kind of thing just doesn't happen in fiction. Reality is a dark and desolate place where we have to deal with modal dialogues that interfere with us trying to get work done; where confusing, disorganized software with poorly designed and outright wrong interfaces are considered normal. In video games and books that happen to have computers in them, those computers are easy for everyone to use. You just have to suspend your disbelief that such a user experience is possible.

The utopian nature of such fictional usability aside, we can learn a lot from how fictional interfaces are presumed to work and try to bring them into reality to see if they really are that good.

Remember the computer system in Star Trek: The Next Generation? That thing was capable of understanding requests made in plain language (most of the time) and was able to form responses based on those requests. Unless the plot dictated otherwise, the Enterprise computer simplified the lives of everyone on board. The crew just talked to the computer and it responded. They didn't have to deal with keyboards or mice or mystifying error messages.

A system like that is incredibly hard to build. But we're doing it: IBM's built a system called Watson that is able to process and create spoken phrases in a somewhat intelligent manner. It's no ship's computer, but it's a big step in the right direction if we're ever going to do things the Picard way. Watson may not be doing anything other than trivia for now, but the language-processing software it employs will be applicable everywhere.

Another great fictional interface is the hologram. I don't mean an advanced system like a holodeck but rather using smaller holograms projected into real space to create something called augmented reality. We're already making a few AR systems (as can be seen in the Wikipedia article just linked) but those are not nearly at the level they need to be.

In the game Dead Space 2, holograms are an everyday part of life. They replace all physical interfaces that we have today, not requiring one to be in a specific location in order to access information networks. Today, we need to be in front of a computer monitor to see the output, and we need to be front of input devices to interact. We have a locality dependency on using computers.

Cell phones and other small devices mitigate this dependency by allowing one to carry a computer around whenever one is required. The locality is still there, though, since you need to pull out your cellphone. There's a waiting period, a transition between visualizing the world normally and visualizing it with a tool. Also, the small computer is not as capable as its larger counterparts that are decidedly less mobile.

Augmented reality could bring us information no matter where we are regardless of our proximity to computers. It would allow us to view content on virtual screens (if merely two-dimensional information is accessed), to blur the line between what exists physically and what exists solely as information. A lot of this information relates directly to the physical world anyway, such as map directions. In a sense a GPS device is an augmented reality device, in that it augments what you can see and hear.

But a GPS device is much too clunky for the world of fiction — it requires you to look at it to use it. Another system, this time from Neuromancer, is more appropriate in that some people have mechanical implants that allow images to be superimposed on top of what they're perceiving biologically. A system like this can display overlays onto reality so its user never has to look away from what he's doing in order to gain more information. It's already right there.

I'm moving steadily away from what is currently possible and moving into a futuristic world where people are cyborgs. That's where I think we should be going to make life simpler. We'll start with interfaces that are able to more correctly respond to what is meant rather than what is done, such as with Watson. Then we'll move beyond that to integrate computers into everything around us so that we don't have to be somewhere specific to access information. Further into the future still we'll be able to use tools like glasses with internal projector screens or indeed biological augmentation to allow for a personalized user experience.

And that's just one set of ideas from various sources. It's possible that the future of interaction design will be entirely different, something no author could have ever imagined. Or maybe we'll be stuck with what we have now forever. Whatever the future holds, we should always strive to make things easier for our users by taking small steps or large leaps in the right direction. Fictional interfaces are a great source for inspiration, since any software application and indeed any invention exists first as a fictional idea in its creator's head before it is brought into reality.

Latest comment:

Computer psychics

By Dean

I've always loved the term "psychic debugging" as coined by non-vampire blogger Raymond Chen. It refers to user-programmer interactions in which a user complains about a situation that shouldn't be possible given the information he's given to the programmer. The result of this is the programmer has to pull a solution out of invented scenarios that appear to be reading the user's mind.

A good demonstration of this kind of psychic power can be seen on the television show House, in which patients often lie to to their doctors for various reasons, leaving the doctors to use their psychic powers to determine the possible causes of their illnesses. While House isn't about software debugging, the psychic powers used in both situations are the same. I suspect that psychic powers manifest in every type of knowledge work.

Users who need help by definition don't know enough about what they're doing to do it. If this weren't the case, they wouldn't need help. But because of the Dunning-Kruger effect users in this situation can think something is important when it really isn't, and they can think things are unimportant when they are in fact vital to determining the cause of a problem.

This is normal human psychology (which I've covered before), and I don't fault anyone for it; however, the fact that it is normal human behaviour means that it's widespread. The enduring presence of misinformation fosters the latent mind-reading powers of computer scientists, turning them into computer psychics.

(Note that these clairvoyant powers apply only to debugging. There is no mechanism that I know of that allows software developers to be able to distinguish between what people say and what they want when defining a new software application. This distinction relies more on communication skills which are much squishier than psychic debugging and isn't something that all software developers are eventually capable of doing.)

As developers are asked more questions about their software (and as developers become more familiar with the software in the case of developers brought onto applications after they've already been written) they become more familiar with problems users face. At the same time they gain deeper knowledge of the system they're working on and the external applications with which it interacts, so they have more technical knowledge to draw upon.

So when a user faces a problem ("My can of walnuts isn't opening!"), he'll try his best to come up with the scenario that's causing it. He'll tell the developer what the problem is and maybe even the steps to reproduce the problem ("When I run my popcorn popper, then get my can of walnuts, I can't open it!"). Then the developer will see this and realize that the situation the user is describing is impossible ("Popcorn has nothing to do with walnuts."), but given what he knows about the user in question and the systems he's probably using, he's able to pull a likely scenario out of thin air ("You've unplugged your can opener to plug in your popcorn popper."). If he's right, it's an astonishing feat of herculean mental strength.

When he's wrong, though, it's like those first five times in an episode of House where they misdiagnose a patient: the patient's condition declines and his family become irate. But just like on TV, screwing up gives you information you didn't have. Even though it wasn't the can opener not being plugged in, at least the developer is thinking outside of the scope that the user defined. The user might be displeased that the programmer accused him of not knowing what things need to be plugged in, but he knows more than he did before the accusation was made. At that point they both realize that it's because the popcorn popper and can opener draw too many electrons from the circuit, diminishing the usefulness of both. Stupendous!

This kind of debugging appears impressive to any party involved other than the one who holds the knowledge himself. To him, it's just the application of the details he works with on an everyday basis. As Raymond often says in his psychic debugging posts, any of his readers should be able to pull off the same tricks that he manages as long as they pay attention. Knowing how it works doesn't make his powers any less captivating.

Psychic debugging is a sign of a developer who knows the particulars of his application and all of the systems around it. Once this level of parapsychology is achieved the developer can join the ranks of Charles Xavier, Gregory House and Raymond Chen, fighting bugs and Magneto wherever they may rear their ugly, helmeted heads.

Latest comments:
This is some kind of footnote. This webpage is awesome and can be viewed in any browser. Even ones that suck ass like Safari and Firefox. Isn't that awesome? This site is best viewed with browsers that aren't maximized on large-resolution displays (> 1024 pixels in width). But then again, if you are running a large resolution and browsing maximized, then you're a terrible person so you don't really deserve to see this site at its finest. Jerk. I mean, seriously. I spend all this time making a nice site and your silly browsing habits ruin its look. That's really cold, man. If you're using IE6, then in order to see the cool avatar effects you need to enable JavaScript. This site conforms 100% with the laws (both known and unknown) governing physical reality. No rights reserved by Dean Whelton (who is awesome) of any of the content, images, design, backend or electrons used in this site. Steal at your convenience. None of it is worth stealing anyway, so there. I have even made an RSS feed for more efficient theft of my intellectual property. Now, don't say I'm not generous. I guess if you want to know more about me, you can visit the about page. I actually made a real about page! It's more like a FAQ, though. You can contact me, too, if you feel like it. Are you really wasting time reading this? Go outside or something.