Interfaces, metaphors & multi-touch

Bill and Bob

If the desktop was useless, though, we’d have all run out and bought Microsoft Bob instead of Windows 98. Bob was a bolt on for Windows 3.1/95, much like MCE bolts onto WindowsXP. It replaced the Windows desktop with a different metaphor, which put the desktop in the context of a room. So instead of having the clock set on the taskbar, it was on the wall. You could arrange your room as you liked, and adorn it with objects (Windows shortcuts). It was all a bit whacky, though, and it introduced Bob – otherwise known as that f@$kink paperclip, before it drifted into obscurity.


Bob may be dead, but that’s not to say there’s no one working on alternatives to the Windows desktop. BumpTop combines the current desktop metaphor with a 3D room. This is similar to Bob, but without the embellishments or talking mutt. With a small room and a lot of data it could get messy fast, but it’s an interesting attempt.

Even thought the interface hasn’t advanced significantly the underlying operating system has evolved greatly: the desktop metaphor, which we are so accustomed to, is just a way of interacting with it.

The mouse made a huge difference to how we interact with our computer- it allowed us to make full use of the 2D display. Direct manipulation – the act of moving the mouse to interact with objects in the 2D world - is something that will long outlive the mouse. There is a reason we all use mice, which is that they are perfect for manipulating a 2D environment. When you try to manipulate a 3D environment, though, they suck.

3D environments

To interact with a 3D space we need additional inputs. That’s not a problem as we’ve had keyboards for ages, and they have inputs aplenty. If you play First Person Shooters, then you’re already well acquainted with how precise the keyboard and mouse combination can be. You can whisk your view around the world with impressive speed; HL2 even allows you to manipulate the world using the gravity gun – which has given birth to mods like Source Forts, where part of the game is building the map you are to play on by manipulating objects in the 3D world.

Source Forts, and to some extent HL2, show the problems associated with using the keyboard and mouse to directly manipulate objects in more than two dimensions. Source Forts would be a really fun game: if putting the blocks into place wasn’t so frustrating. Securing each block in the desired position and orientation is almost impossible. You can grab something and walk it around, but you can’t rotate the block into the correct orientation so you can lay it in place.


This may be a relatively new problem in games, but it’s an old one for 3D modellers. The tools used to model 3D objects for virtual worlds are an annoyance to use with a keyboard and mouse. You’re required to know an astounding amount of mouse gestures, hotkeys and operations before you can create even the most rudimentary structures. This is a massive problem, and one of the most compelling research topics bouncing around the hallowed halls of academia.