Netflix Animated Feedback

Netflix effectively uses animation to provide feedback in its movie queue interface (IE only). This example shows how a cinematic technique can be integrated into a fairly standard web page. Without the animation, if there were a simple refresh, you need to re-read the list, remembering what you had selected to make sure that your click has had effect. With the animation, you can visually track the change as your selection moves to the top of the list.

State Glue: feedback on state transitions

Martijn van Welie writes about the visual transitions in the iPhone (via informationdesign.org). He breaks down this cinematic experience, demonstrating how the component parts of an animated transitions contribute to your understanding of what the machine is doing on your behalf.

He breaks down the visual transition for deleting an email (with a nice slowmo video in his post) into sub-states which he calls "state glue" which all happens in less than a second:

1. Received the order: button lights up with a glow and fades way when released
2. Preparing the order: the trash can opens up
3. Doing the order: the email wrinkles up and moves into the trash can, trash can closes
4. Finished the order: new email being loaded and network indicator spinning
5. Ready for the next order: new email fully displayed

Inspire Developers

Peter writes Never Say 'No' to Designers which is one of our core values at Laszlo and reflects the attitude which has made OpenLaszlo a flexible platform for building great apps. The corollary to that principle is "inspire developers".

It is a challenge to develop great applications. Engineers need to make the features work and perform well, plus pay attention to all of the little details required to make a design hang together. For the cinematic experience, the details really matter. A short animation can effectively guide the user to understand what is happening behind the scenes, but sometimes it has to bounce to draw your attention or fade out at the right moment, and if it is too long or choppy because something else is hogging the CPU, it'll just suck. The people who write the code have to understand what they are working to achieve and why. They need to be motivated and inspired. The designer plays a critical role in creating this dynamic.

Tips for Inspiring Developers
1) Create beautiful, intuitive designs.
2) Listen to ideas from the engineers. Acknowledge good ideas, even if you have to explain why they wouldn't work in this particular case.
3) No matter how awesome your design is, design documents are boring. Do why you can to make them readable (more on that below).
4) No flying cars. Understand the technology. It's fine to push the boundaries, but if you regularly present designs that are impossible to implement, you will lose credibility.

Understanding the "Users" too
So, like everything else we've got this agile thing going, and as a good designer should, Peter puts together the description of a "persona" to kick off our Calendar project. This is included in a 28 page PRD:

Angie, the representative “end user” needs and uses Calendars to manage both her personal and work schedules. She has not found a good solution that integrates both, so she relies on the IT configured Outlook and Exchange solution for her work schedule. She has been told numerous times about Outlook’s features, and even attended a half-day seminar, but finds it confusing and mostly relies on email to manage her team. She uses Google calendar infrequently to track events in her personal life. She has no clean way to integrate both personal and work calendars or to share calendar events between these calendars. She is also limited by the fact that her boyfriend uses Yahoo Calendar, a system she swore she would never employ after being laid off from Yahoo in 2000. She and her boyfriend, therefore are rarely in sync, a symptom of calendaring woes, and perhaps of the veiled animosity already evident in their 6 month old relationship.

If you just skimmed over that quote, go back and read the last two sentences. It's so like someone I might know to not using software from a company they don't like. The truth is that with a lot of competitive free web products out there, people can afford to be picky. The story is memorable, and because it is funny the engineers are laughing about it at lunch and the guys who skimmed over that part go back and read it in the afternoon.

The magic of using humor in this way is that it doesn't distract, it makes Angie more real. We are seeking to create an experience that bridges an arbitrarily complex technical landscape, as well as human idiosyncrasies, and the persona description captures that. You are designing the application. You are making a hundred key decisions, but you better believe that the engineers writing the code are making hundreds of other decisions that can either support your design or ruin it. The more they understand who the app is for and why they are building it, the more that their decisions will be aligned with yours. Creating stories that feel real allows your development team to emotionally connect with the target customer, enabling them to easily make quick decisions that support the right design.

A Lively Design Spec
Onward to the design spec. According to our process, different projects get different levels of detail for design documentation. Sometimes an image and a live meeting is good enough to sketch it out. When it is a small feature and the designer and engineer have worked closely together for a while, we don't need everything spelled out. However, with a new product, short schedules and some contractors involved, it is worthwhile getting it all down on paper.

While creating a design spec is challenging. Reading all of the words (rather than just flipping through the pictures) requires some fortitude. Here are some snippets from a recent design spec by Peter that include a bit of gratuitous humor that keep people reading:

Creating invitations within the Calendar application is a simple and intuitive process that makes the user feel strong and in control. The invitation process can be initiated when ... Modifying event attributes prompts the user to resend (or not resend) the invitation. All in all, it’s a fun thing to do in your spare time.

When declining an invitation, a message is sent to the author expressing disdain and/or horror. The UI in the message indicates that the event was declined, but the Choice UI remains, allowing the user to respond differently after they have had time to get over the whole whiskey-induced incident.

These are just some ways our design team manages to inspire engineers to read the design docs and make the design come alive in our imaginations.

"Never say 'No' to Designers"

The core of the Cinematic User Experience is animation. The term animation, until recently, has been bound and gagged by negative connotations when applied to the Internet. In their nascence, Flash-based animations were simply longer, more complicated, and often painfully slow branding messages. Despite being quite often beautiful, they served no greater utility than the animated GIFs that preceded them. The "Skip Intro" stigma and the simple, frame-based tools for creation proved to be measurable obstacles for the advancement of the user experience online. After many forays into this troubled landscape, and endless battles fought with dull blades, Laszlo was founded on the belief that despite the impediments, the potential of Rich Internet Applications (RIAs) was worth pursuing.

In answer to the limitations we had encountered trying to build RIAs, Laszlo began by creating a new architecture for enabling such things. Before we had a Cinematic User Experience, Bret Simister (Cofounder and Chief Interface Architect at Laszlo) authored the core animation classes that would support the LZX language, and facilitate the design of powerful web-based applications. As an engineer, mathematician and designer, with a long history of client application work in animation, Bret was in the unique position of knowing what tools designers of applications were missing, and how to create them.

"Never say 'no' to designers." That was the lofty goal Bret offered that lured me to Laszlo. After years of being stymied by the technical limitations of the Web, such a proclamation was musical. The promise that non-linear animation support offered to interaction design was paramount to this tenet. I won't pretend that this has been a simple path, but from the very first application I designed with Laszlo many moons ago, I can honestly say that I have always had this level of support; not only from the technology, but also from all Laszlo engineers.

Without doubt, having an evolved, stable technology and a team of wickedly smart engineers is a great advantage to design and designers, but how does it inform and improve user experience? What does it mean to utilize the Cinematic User Experience as a designer? The simplest answer is: Use your powers for good. Regardless of the infinitely cool, dramatic or swoopilicious potential afforded, the end result should always be a better user experience.

While a better experience may seem logical, it is important to remember that this environment produces a far greater number of variables than a static display of HTML might. Conventional page-based Web sites require skills similar to the book design they are metaphorically linked to: Content hierarchy becomes navigation, layout determines page flow, and typography, photography and illustration work in concert with the whole to fashion style. With the introduction of the Cinematic User Experience, a convergence of graphic design and product design occurs. How information is presented to the user requires the same care, but is enhanced by considerations of rhythm, sequence, timing and the strata of events and data. How an affordance is rendered, what level of interactivity is surfaced, what feedback is provided, and the behavior that results from an action all become integral to the design.

The Cinematic User Experience is interactive, immersive and multi-layered. The management of the parts, and their multi-threaded intersection is essential in creating a functional and, more importantly, usable application. This is, of course, a complicated path for designer and engineer, but the result can be a more powerful, elegant, and useful product. In the near future, I will examine specific design patterns and examples of the pinnacles and pitfalls inherent to this realm of design.

origins of the cinematic user experience

An interview with David Temkin,
founder and CTO of Laszlo Systems
by Sarah Allen, February 15, 2008

What is the “cinematic user experience?”
When this term originated we were in a world where most Web interfaces were static screen design. The user would be presented with a screen, then would click, and get another screen. Each screen was discrete. This was retrogressive over what was previously in place. What was previously in place was the graphical user interface, which is what attracted me to computing in the first place. This was an interface which included not just pictures of things, but objects that you could directly manipulate. You could drag an icon; you could drag or resize a window. There was an organic connection between your actions and what was happening on the screen. Because the graphical user interface had these visual objects that you could directly manipulate, there were an enormous amount of assumptions that a user could make about the objects on the screen. The metaphor of interacting with objects in the real world allowed the user to learn things that you couldn’t possible teach through another metaphor like command line or for that matter the web.

So first and foremost, getting back to what I just saying, it was about getting past this retrogressive phase of a user interface as a series of static pages. The second thing was getting past even the things that we were trying to get back to. If you think of the web as a discrete user interface, meaning I have discrete pages. We’re getting past that into something that is continuous, moving from discrete to continuous. However, both “discrete” and “continuous” aren’t that interesting -- those are geeky words. I don’t know how I originally arrived at the word “cinematic” but it seemed to encapsulate the motion and the interactivity of a kind of forward-looking user interface that took the best of the desktop, namely direct manipulation and the motion and interactivity and welded that to the branding and creative visual design that came out of the web.

The other hint of cinematic is you have drama. A lot of people said with our very early demos that this is what Hollywood has always said user interfaces are going to look like. Every day they look a little more like SciFi movies. We now live in Sci Fi from the perspective of the 1980s.

Can you describe the elements that make something a cinematic user experience?
An interface where two things are going on: one is that you have objects on the screen that are manipulable and respond to commands with motion; and the second piece is that motion actually tells a story about the role of those objects within the overall application and the role of those objects and those transitions within the task that you are trying to complete. Just as in a movie, a director seeks to cue the viewer from what was going on to what is happening next. In a movie, the camera might move, with a pan, or a zoom, you might have all sorts of other effects whose names I don’t know, but you probably do. The cinematic user experience is about using those effects and motion that are tied almost directly to our perceptual system to convey certain ideas and instantly infuse lots of assumptions that are bound to be correct about the application, the environment and the task. That’s what it is about, besides the cool, Hollywood connotation. And by the way, contrary to the most obvious interpretation of the term it is not about video.

When did you first start thinking about using animation in UI?
I’ve been involved in user interface design and advancing beyond desktop UI since the late 80s. I worked IRIS, a research lab at Brown University, where we created a system called Intermedia where there were documents with links within them that linked to another document that could contain different types of media objects. At the time it sounded like purely theoretical academic gobbledy-gook, but it was very similar to the web, which came out only a few years later. We used animation as part of that, just a little bit, but it came a lot more into play I my next job where I spent five years working on the Apple Newton PDA, or Personal Digital Assistant. There was a lot of animation within that interface, again mostly to convey the idea that you were dealing with a physical object. We had this notion of a drawer. You would click it and a drawer would slide out, it would make a drawer sliding noise, along with an animation. These little things would give the user all sorts of ideas and notions, way beyond that individual effect, about the remainder of the system. We also had a central metaphor of a scrolling notepad. To scroll the notepad, you would click the button down and it would go “fsst, fsst” like the noise of a piece of paper sliding along with the animation. Again it was a physical metaphor for what the machine would do. There were all sorts of effects like that in the Newton, but I was always attracted to taking it to the next level and creating a whole system, a visual, physical reactive system where the more you could leverage people’s instincts and intuitions about how the real world works and real-world objects work within the world of the digital, the better you would be doing in terms of actually conveying what that system might actually do.

Do you think that the ultimate end point of all this is a virtual reality system where we operate with real actual objects using some device of the future?
In 1989, I wrote a paper making the argument that the desktop made a huge advancement in usability for most people, mostly because things became visual and directly manipulable, but it stopped at some point. The desktop user interface never represented, for example, the network. There is no visual representation of the network, the people in the network, shared files or anything else. I argued that in order to get beyond the desktop you need something that is more completely visualized, more directly manipulable, but with a sense of magic, because if it directly mirrors the physical word than why bother having a computer at all.

But yes, that’s what I thought at the time. You know, we’ll all put on our gloves and our funny 3D glasses. I did actually get involved with graphics for 3D glasses and I was very excited by that. Since then I’ve kind of become disillusioned with that kind of visual presentation. I believe in 2D, two-and-a-half-D, animation, direct manipulation and in zooming interfaces. I’m not sure that I believe in immersive environments for anything other than video games. Now that may change. We have interesting alternatives to VR gloves. Now we have little webcams that can actually sense your physical position and do object recognition on the position of your hands and you don’t have to wear any funny stuff. Who knows? It could evolve there. We are probably five, ten steps in evolution from that, if it even goes there. But I think, in the here and now, we have barely begun to explore what can be done on a flat screen so I think there is a huge amount of head room within the input and output devices that we now have. I am very excited by touch screens, high resolution touch screens like what you see on the iPhone. The potential for more expressiveness visually, more direct manipulation, more natural interaction than graphic user interface, menus, mouse, pointer.

So do you remember when you actually coined the term?
It was probably somewhere around 2001 or 2002. It’s a marketing term. We were trying to come up with a way to summarize what we had that got people intrigued. We started doing what we were doing at Laszlo, and you remember this, Sarah, because at the time you were working at Macromedia and we came by to demo to you and to Jonathan Gay, who is the guy who invented Flash. It’s the kind of stuff that would get people incredibly excited and yet they couldn’t put a term on it. I’ll tell you two stories about this.

One time I had lunch with Norm Meyrowitz, who at one point was President of Macromedia, and at a later point was on a temporary leave. At that time, I had lunch with him and I showed him the Calendar and Dashboard demo applications. And he says, “this isn’t rich media… these are rich Internet applications.” To my knowledge that was the first time that phrase had ever crossed anybody’s lips -- that was his thing and you can imagine that got taken back to Macromedia and that became the RIA term you still hear today. But that is kind of a dull term – it does not get people excited.

We were struggling to find a good way to describe what is this thing? We didn’t want to say rich media. Rich media is tainted. It makes you think you are going to get some movies. The word “rich” is also tainted. You say rich and you think you are going to get some chocolately desserts that are heavy and expensive – 5MB downloads of chocolately desserts. Who wants that? I want my application.

Cinematic user experience conveys that first of all that you aren’t just watching you are interacting – the “user experience” part of it. We thought cinematic was an intriguing term, that had a non-techie spin to it. It was something that when non-techies saw it, they instantly understood this is a completely different type of product category and yet industry insiders would look at it and say things are moving on the screen, maybe you have a different technical architecture. They couldn’t see from the perspective of the common man or woman what this thing would really look like. We needed a term for that and that is what we ended up with.

What do you think is the best example of the cinematic user experience?
I’ll talk about the best example which has absolutely nothing to do with Laszlo’s technology, but I believe does have to with Laszlo aesthetic and its influence over time – the iPhone interface. They have all sorts of panels in the mobile industry where people talk about the iPhone and you listen to these executives and they say, “I don’t get it, my phone is faster, has more memory and its got 3G and it has MMS and it has all these features that the iPhone doesn’t and why doesn’t it sell more than the iPhone?” “It’s the user experience, stupid.” That’s what it is about.

The iPhone is a tactile experience where you have an almost organic connection between your hand and the objects behind the screen. You have a feeling of reality – those objects behind the screen exist. Of course, they don’t exist. They are pixels that are being drawn by software. But that’s the amazing thing about it. It’s almost a theatrical thing. The software is presenting something that is so real as to feel to the end user that they are dealing with actual objects. The cinematic user experience is the animation, the motion, the way that you actually move things around on the screen, the way that everything is presented as a visual. Everything has a little visual equivalent and is not ten menus deep. It’s a from-scratch design of a user interface that is appropriate to its form factor. The iPhone is a testament to the fact that at least in some quadrants of the industry a lot of hard work is being done to advance the state of the art that is a tremendous commercial success. I think what they have done with the iPhone is a game changer for user interfaces and possibly the best and most comprehensive example of the cinematic user experience.

So, what’s next? What would you like to see that you haven’t seen yet? What would you like to see more of?
I would like to see a deeper reconsideration of the overall user experience of what we deal with on computers and on mobile devices. On the computer, you are kind of bi-modal. There is the desktop-style user interface, which in spite of all the eye candy and all the little prettiness that even Apple includes, it is the same old thing that is unchanged for 22 years. Then you’ve got the web, which has changed a little bit, but not a lot. RIAs and the scope of the Web 2.0 movement, to some extent, has failed. Web 2.0 is much more about little bits of here-and-there interactivity. I would like to see something where the experience of these sites and the experience of the desktop are reconsidered from the ground up in a way that takes into account what we actually do both with the desktop and the web in an integrated, organic way. In the way that the iPhone reconsidered what it is that we are trying to do with the mobile phone, so I don’t have this bi-modal: you are either making a phone call or you are using the WAP browser. I want to see something where the desktop and the web are consolidated into a whole user experience that takes into account what it is that we are trying to do with this device, and I expect that user interface would be at least as interactive, immersive, and empowering as what you see with the iPhone.