origins of the cinematic user experience

An interview with David Temkin,
founder and CTO of Laszlo Systems
by Sarah Allen, February 15, 2008

What is the “cinematic user experience?”
When this term originated we were in a world where most Web interfaces were static screen design. The user would be presented with a screen, then would click, and get another screen. Each screen was discrete. This was retrogressive over what was previously in place. What was previously in place was the graphical user interface, which is what attracted me to computing in the first place. This was an interface which included not just pictures of things, but objects that you could directly manipulate. You could drag an icon; you could drag or resize a window. There was an organic connection between your actions and what was happening on the screen. Because the graphical user interface had these visual objects that you could directly manipulate, there were an enormous amount of assumptions that a user could make about the objects on the screen. The metaphor of interacting with objects in the real world allowed the user to learn things that you couldn’t possible teach through another metaphor like command line or for that matter the web.

So first and foremost, getting back to what I just saying, it was about getting past this retrogressive phase of a user interface as a series of static pages. The second thing was getting past even the things that we were trying to get back to. If you think of the web as a discrete user interface, meaning I have discrete pages. We’re getting past that into something that is continuous, moving from discrete to continuous. However, both “discrete” and “continuous” aren’t that interesting -- those are geeky words. I don’t know how I originally arrived at the word “cinematic” but it seemed to encapsulate the motion and the interactivity of a kind of forward-looking user interface that took the best of the desktop, namely direct manipulation and the motion and interactivity and welded that to the branding and creative visual design that came out of the web.

The other hint of cinematic is you have drama. A lot of people said with our very early demos that this is what Hollywood has always said user interfaces are going to look like. Every day they look a little more like SciFi movies. We now live in Sci Fi from the perspective of the 1980s.

Can you describe the elements that make something a cinematic user experience?
An interface where two things are going on: one is that you have objects on the screen that are manipulable and respond to commands with motion; and the second piece is that motion actually tells a story about the role of those objects within the overall application and the role of those objects and those transitions within the task that you are trying to complete. Just as in a movie, a director seeks to cue the viewer from what was going on to what is happening next. In a movie, the camera might move, with a pan, or a zoom, you might have all sorts of other effects whose names I don’t know, but you probably do. The cinematic user experience is about using those effects and motion that are tied almost directly to our perceptual system to convey certain ideas and instantly infuse lots of assumptions that are bound to be correct about the application, the environment and the task. That’s what it is about, besides the cool, Hollywood connotation. And by the way, contrary to the most obvious interpretation of the term it is not about video.

When did you first start thinking about using animation in UI?
I’ve been involved in user interface design and advancing beyond desktop UI since the late 80s. I worked IRIS, a research lab at Brown University, where we created a system called Intermedia where there were documents with links within them that linked to another document that could contain different types of media objects. At the time it sounded like purely theoretical academic gobbledy-gook, but it was very similar to the web, which came out only a few years later. We used animation as part of that, just a little bit, but it came a lot more into play I my next job where I spent five years working on the Apple Newton PDA, or Personal Digital Assistant. There was a lot of animation within that interface, again mostly to convey the idea that you were dealing with a physical object. We had this notion of a drawer. You would click it and a drawer would slide out, it would make a drawer sliding noise, along with an animation. These little things would give the user all sorts of ideas and notions, way beyond that individual effect, about the remainder of the system. We also had a central metaphor of a scrolling notepad. To scroll the notepad, you would click the button down and it would go “fsst, fsst” like the noise of a piece of paper sliding along with the animation. Again it was a physical metaphor for what the machine would do. There were all sorts of effects like that in the Newton, but I was always attracted to taking it to the next level and creating a whole system, a visual, physical reactive system where the more you could leverage people’s instincts and intuitions about how the real world works and real-world objects work within the world of the digital, the better you would be doing in terms of actually conveying what that system might actually do.

Do you think that the ultimate end point of all this is a virtual reality system where we operate with real actual objects using some device of the future?
In 1989, I wrote a paper making the argument that the desktop made a huge advancement in usability for most people, mostly because things became visual and directly manipulable, but it stopped at some point. The desktop user interface never represented, for example, the network. There is no visual representation of the network, the people in the network, shared files or anything else. I argued that in order to get beyond the desktop you need something that is more completely visualized, more directly manipulable, but with a sense of magic, because if it directly mirrors the physical word than why bother having a computer at all.

But yes, that’s what I thought at the time. You know, we’ll all put on our gloves and our funny 3D glasses. I did actually get involved with graphics for 3D glasses and I was very excited by that. Since then I’ve kind of become disillusioned with that kind of visual presentation. I believe in 2D, two-and-a-half-D, animation, direct manipulation and in zooming interfaces. I’m not sure that I believe in immersive environments for anything other than video games. Now that may change. We have interesting alternatives to VR gloves. Now we have little webcams that can actually sense your physical position and do object recognition on the position of your hands and you don’t have to wear any funny stuff. Who knows? It could evolve there. We are probably five, ten steps in evolution from that, if it even goes there. But I think, in the here and now, we have barely begun to explore what can be done on a flat screen so I think there is a huge amount of head room within the input and output devices that we now have. I am very excited by touch screens, high resolution touch screens like what you see on the iPhone. The potential for more expressiveness visually, more direct manipulation, more natural interaction than graphic user interface, menus, mouse, pointer.

So do you remember when you actually coined the term?
It was probably somewhere around 2001 or 2002. It’s a marketing term. We were trying to come up with a way to summarize what we had that got people intrigued. We started doing what we were doing at Laszlo, and you remember this, Sarah, because at the time you were working at Macromedia and we came by to demo to you and to Jonathan Gay, who is the guy who invented Flash. It’s the kind of stuff that would get people incredibly excited and yet they couldn’t put a term on it. I’ll tell you two stories about this.

One time I had lunch with Norm Meyrowitz, who at one point was President of Macromedia, and at a later point was on a temporary leave. At that time, I had lunch with him and I showed him the Calendar and Dashboard demo applications. And he says, “this isn’t rich media… these are rich Internet applications.” To my knowledge that was the first time that phrase had ever crossed anybody’s lips -- that was his thing and you can imagine that got taken back to Macromedia and that became the RIA term you still hear today. But that is kind of a dull term – it does not get people excited.

We were struggling to find a good way to describe what is this thing? We didn’t want to say rich media. Rich media is tainted. It makes you think you are going to get some movies. The word “rich” is also tainted. You say rich and you think you are going to get some chocolately desserts that are heavy and expensive – 5MB downloads of chocolately desserts. Who wants that? I want my application.

Cinematic user experience conveys that first of all that you aren’t just watching you are interacting – the “user experience” part of it. We thought cinematic was an intriguing term, that had a non-techie spin to it. It was something that when non-techies saw it, they instantly understood this is a completely different type of product category and yet industry insiders would look at it and say things are moving on the screen, maybe you have a different technical architecture. They couldn’t see from the perspective of the common man or woman what this thing would really look like. We needed a term for that and that is what we ended up with.

What do you think is the best example of the cinematic user experience?
I’ll talk about the best example which has absolutely nothing to do with Laszlo’s technology, but I believe does have to with Laszlo aesthetic and its influence over time – the iPhone interface. They have all sorts of panels in the mobile industry where people talk about the iPhone and you listen to these executives and they say, “I don’t get it, my phone is faster, has more memory and its got 3G and it has MMS and it has all these features that the iPhone doesn’t and why doesn’t it sell more than the iPhone?” “It’s the user experience, stupid.” That’s what it is about.

The iPhone is a tactile experience where you have an almost organic connection between your hand and the objects behind the screen. You have a feeling of reality – those objects behind the screen exist. Of course, they don’t exist. They are pixels that are being drawn by software. But that’s the amazing thing about it. It’s almost a theatrical thing. The software is presenting something that is so real as to feel to the end user that they are dealing with actual objects. The cinematic user experience is the animation, the motion, the way that you actually move things around on the screen, the way that everything is presented as a visual. Everything has a little visual equivalent and is not ten menus deep. It’s a from-scratch design of a user interface that is appropriate to its form factor. The iPhone is a testament to the fact that at least in some quadrants of the industry a lot of hard work is being done to advance the state of the art that is a tremendous commercial success. I think what they have done with the iPhone is a game changer for user interfaces and possibly the best and most comprehensive example of the cinematic user experience.

So, what’s next? What would you like to see that you haven’t seen yet? What would you like to see more of?
I would like to see a deeper reconsideration of the overall user experience of what we deal with on computers and on mobile devices. On the computer, you are kind of bi-modal. There is the desktop-style user interface, which in spite of all the eye candy and all the little prettiness that even Apple includes, it is the same old thing that is unchanged for 22 years. Then you’ve got the web, which has changed a little bit, but not a lot. RIAs and the scope of the Web 2.0 movement, to some extent, has failed. Web 2.0 is much more about little bits of here-and-there interactivity. I would like to see something where the experience of these sites and the experience of the desktop are reconsidered from the ground up in a way that takes into account what we actually do both with the desktop and the web in an integrated, organic way. In the way that the iPhone reconsidered what it is that we are trying to do with the mobile phone, so I don’t have this bi-modal: you are either making a phone call or you are using the WAP browser. I want to see something where the desktop and the web are consolidated into a whole user experience that takes into account what it is that we are trying to do with this device, and I expect that user interface would be at least as interactive, immersive, and empowering as what you see with the iPhone.