UK And German Bank Sites Lead As France, Italy, And Spain Lag
London, May 27, 2002 . . . Flawed navigation and customer service frustrate today’s Web banking users, but low-cost design fixes and an ongoing focus on user experience will help firms win sought-after online transactions, according to a new report by Forrester Research (Nasdaq: FORR).
To uncover how European banking sites can improve, Forrester graded the sites of the 20 largest European retail banks according to its user experience review methodology, which evaluates how well sites help users achieve their goals. Significantly, even the best sites don’t pass overall usability, and the top-rated Halifax, Credit Suisse, and Deutsche Bank sites didn’t reach an overall passing grade. British and German bank sites beat Italian, French, and Spanish banks. Read the rest of this entry »
11月3日, 由英国UiGarden网站组织策划、本地化易用性研究和交互设计组织ChinaHCI网站协办的世界可用性日北京站活动正式在北京翠宫饭店拉开帷幕，本次可用性日的主题为“Make it easy”。来自IBM中国研究中心、中国科学院软件所等众多单位的可用性和用户研究方面的从业人事以及高校的师生们共聚一堂，共同关注并探讨出现在城市公共设施中的可用性问题。（ChinaHCI.Org：何潇 北京报道）
“Of all the courses and workshops I’ve attended over the last three years, Adaptive Path’s
are the best! I want all of my colleagues to attend.”
— Esha Bhatia, The Children’s Hospital of Philadelphia
We’ll be heading to Washington, D.C. with Jesse James Garrett, Jeffrey Veen, Lane Becker, Janice Fraser, Peter Merholz, along with guest presenters Jared Spool, Marc Rettig, Nate Bolt, and others.
Agenda Day 1, Monday, August 22nd
To attend Day 1, please register for a Full-Week Pass.
Single-Day Registration for Day 1 is sold out. A Whole New Internet
Time Activity
9:00-9:45am Introduction to A Whole New Internet
Janice Fraser, CEO, Adaptive Path
Ten years ago, Netscape’s IPO indicated the launch of the internet into mainstream culture. The subsequent boom was followed by a harrowing bust, where companies retrenched and avoided risk. But change is in the air, as companies and individuals return to the internet’s original promise and develop products that truly improve our users’ experiences.
9:45-11:00am Introduction to Ajax/New Web Technology
Jesse James Garrett, Director of User Experience Strategy, Adaptive Path
When Jesse wrote his essay describing the suite of new web technologies he termed Ajax, we had no idea that this idea would take the internet by storm. It’s been covered by everything from CNET to the Wall Street Journal. Unfortunately, the discussion focuses on the technology. In this talk, Jesse will describe the current state of new web technologies, and their implications on design.
11:00-11:30am Break
11:30am-12:30pm Flickr Case Study
Eric Costello, Ludicorp/Yahoo!
The photo-sharing site Flickr has emerged as one of the most influential of the new breed of Web applications. Countless small-scale projects and major online offerings alike have been inspired by elements of Flickr’s design. In this case study, Flickr interface developer Eric Costello takes you inside Ludicorp’s design process to see the thinking that went into the creation of Flickr’s innovative interface.
12:30-1:30pm Lunch
1:30-2:15pm Mass Amateurization
Jeff Veen, Director of Product Design, Adaptive Path
Blogs, wikis, social software, RSS, open APIs and more are leading to placing the tools of content, and even application, generation in the hands of your users. What does the mass amateurization mean for your organization? In this session, Jeff will discuss the mass amateurization landscape, and its implications for enterprises of all sizes.
2:15-3:00pm What the Whole New Internet Means for Business
Janice Fraser, CEO, Adaptive Path
The technologies of the “whole new internet” will drastically affect how you do business – from changing the look and feel of your web presence to impacting your knowledge management solutions and intranet. Within two years, businesses who don’t adapt will be left behind. Janice will discuss how these groundbreaking and exciting new technologies should fit into your two- and five-year web development and business plans.
3:00-3:30pm Break
3:30-5:00pm Ajax Case Study
Jeff Veen, Director of Product Design, Adaptive Path
Adaptive Path has been working on a variety of projects that utilize the capabilities in the Whole New Internet. Jeff will walk through a case study of how he built a product based on these philosophies in this session.
5:30pm Cocktails at location TBD
Day 2, Tuesday, August 23rd Content and Information Architecture
Time Activity
9:00-9:10am Introduction to Information Architecture
Peter Merholz, Director of Practice Development, Adaptive Path
9:10-9:30am The IA of Everyday Things
Jesse James Garrett, Director of User Experience Strategy, Adaptive Path
Jesse shows us the information architecture all around us – and how it can help us in our work.
9:30-10:30am Brand Driven Information Architecture
Jesse James Garrett, Director of User Experience Strategy, Adaptive Path
Jesse explains the similarities between brand positioning and information architecture, and how one benefits from the other.
10:30-11:00am Break
Content Strategy and Effectiveness
11:00am-12:30pm Content Architecture
Peter Merholz, Director of Practice Development, Adaptive Path
When your users approach you, what they are there for can be summarized in four words: “It’s the content, stupid.” A well-built structure is meaningless without good content. Sadly, content crafting and presentation never gets its due in typical user experience processes. We’ll address that by providing frameworks for thinking through the design of content for optimal understanding, use, and effectiveness.
12:30-1:30pm Lunch
1:30-3:00pm WellsFargo.com Case Study
Peter Merholz, Director of Practice Development, Adaptive Path
Melanie Arens, Senior Information Architect, Wells Fargo
In 2003, Adaptive Path performed an extensive content strategy project for WellsFargo.com, interviewing nineteen people about their content needs when researching financial products and services. The research led to a series of models, metrics, and recommendations for the website. In the ensuing two years, Wells Fargo has taken this work and rolled it into a Content Effectiveness Program. Melanie Arens from Wells Fargo will explain how the content strategy work has evolved in the organization.
3:00-3:30pm Break
3:30-4:00pm WellsFargo.com Case Study continued
Future of IA
4:00-4:30pm Algorithmic IA
Jesse James Garrett, Director of User Experience Strategy, Adaptive Path
Increasingly, we’re seeing website information architectures evolve as they adapt to the use of their users. Jesse will describe the underpinnings of such systems.
4:30-5:00pm Metadata for the Masses
Peter Merholz, Director of Practice Development, Adaptive Path
A lot of buzz has been generated on the topic of user-generated tagging and folksonomy. How can your organization take advantage of your users’ ability to tag content in a way that’s meaningful to them? We’ll discuss strategies for marrying this bottom-up approach with more standard classifications.
5:30pm Cocktails at location TBD
Day 3, Wednesday, August 24th
New User Research Methods
Time Activity
9:00-9:30am Adapting Research Methods to the Web
Lane Becker, Director of Professional Services, Adaptive Path
When people think of user research on the Web, they usually think of post-design usability testing. But there’s a lot more to it. Though usability is a tried-and-true research practice for software development, Web development differs from traditional application development, and new approaches to provide a much richer range of research about a site’s users, by taking advantage of the networked environment.
9:30-10:30am Remote User Research: National Gallery of Art
Lane Becker, Director of Professional Services, Adaptive Path
Nate Bolt, Bolt Peters
Typical usability practices are insufficient for truly understanding visitors’ experiences with networked products and services. Having users come into a lab and run through a pre-determined set of tasks might have been fine when testing software 20 years ago, but it doesn’t address the reality of context and content that are essential to the user experience today.
A new method for truly appreciating your visitors experience is remote usability. Through a form of web conferencing, you observe actual visitors to your website, “following” them around as they attempt to get things done. Since the visitor is in their natural environment (home or office), engaging in a passionate task, you get a truer impression of their experience.
In this session, we’ll be joined by Nate Bolt, who worked with us on conducting remote usability for the National Gallery of Art. We’ll walk you through the process, and make it clear how you can begin using this innovative approach the moment you return to your office.
10:30-11:00am Break
11:00am-12:30pm Princess Cruises: Research Findings
Lane Becker, Director of Professional Services, Adaptive Path
Andy Crow, Princess Cruises
Adaptive Path is known for its rigorous task analysis and mental model process, which allows deep customer insight to drive strategy and design. However, many organizations don’t often have the time or resources to engage in such deep work.
In this discussion, we’ll present a case study of a more expedient approach to analyzing customer data, and how it worked for Princess Cruises. We will also share with you a new approach for presenting research findings, the Vision Prototype. Though they look like wireframes, they’re designed to capture the essence of the research in such a way as to help people see its design implications.
The discussion will end with Andy Crow from Princess talking about how they were able to take the research and extend it throughout their organization.
12:30-1:30pm Lunch
1:30-3:00pm Jared’s Method
Jared M. Spool, User Interface Engineering
Jared Spool probably needs no introduction. The principal researcher at User Interface Engineering, Jared has been watching how people use technology for more than 15 years. Recently, he’s extended standard usability methods to capture a deeper and wider variety of interesting detail about customer’s experiences.
In this discussion, Jared will present his new approaches for uncovering user’s needs, desires, and capabilities, and he’ll talk about how to get these approaches inside your organizations.
3:00-3:30pm Break
3:30-4:15pm Field Research - Strategies for Achieving It
Peter Merholz, Director of Practice Development, Adaptive Path
Jesse James Garrett, Director of User Experience Strategy, Adaptive Path
As the products we work on become more integrated in our users’ lives, it’s increasingly important to deeply understand the contexts of use. The best way to do is with field research – going out to where your users are and observing them.
However, for many teams, field research is a distant dream. We’ll discuss strategies for selling and conducting field research.
4:15-5:00pm Documenting Research
Peter Merholz, Director of Practice Development, Adaptive Path
Lane Becker, Director of Professional Services, Adaptive Path
One of the biggest challenges we face in our research is sharing our findings and recommendations with others. How do we help them understand what we saw, and appreciate what needs to change? Peter and Lane will provide guidelines for presenting research findings so that they have the appropriate impact in your organization.
Day 4, Thursday, August 25th
Time Activity
9:00-9:30am Intro to Web 2010
We look back over the prior three days, and discuss their implications for the future. What will the Web look like? How will it be integrated with other parts of the user experience? How will technology help and hinder us?
9:30-10:30am Designing for Experience: Frameworks and Project Stories
Marc Rettig, Founder, Fit Associates
This talk is intended to show what it really looks like when a team designs and builds for human experience. Marc will share frameworks and stories from projects that took a broad view of “experience,” effecting interfaces, systems, environments, training, and communication. These stories illustrate how good research was translated into good designs, which enabled pleasant experiences for the people who use those designs. Along the way he will show actual work artifacts, taking time to suggest how they could be generalized for use by any team.
Among the projects Marc discusses are: discovering opportunities for improving experiences in the household, medical software that became both a learning aid and a communication tool, and a redesign of the entire human interface to a public library (featuring special guest Aradhana Goel of MAYA Design, a member of the library project team).
10:30-11:00am Break
11:00am-12:30pm Designing for Experience: Frameworks and Project Stories
continued
12:30-1:30pm Lunch
1:30pm Field Trip!
You can’t possibly spend an entire week sitting in a workshop. Last year’s curator-led tour was such a hit, we’re heading back for more. We’ve made arrangements for a private tour of the National Building Museum. (Museum entry is on us!)
微软MSN Search部门的用户体验经理de los Reyes认为：“微软始终走在技术和文化改革的风口浪尖，而且他认为下一步应该是设计和创意引导工程和开发。”
Of soufflé and software
August de los Reyes, a user experience manager for MSN search at Microsoft, once spent a month of Sundays trying to make the perfect soufflé Furstenberg—a complicated dish referenced only in a Truman Capote novel. “Just before the soufflé sets in the oven—you inject egg yolks into it. At serving time, the first cut should send yolk spilling over all the different layers,” August says. “It only worked once.”
This kind of vision and perseverance typify August’s approach to design and life. He attended chef school when he was 14, then majored in New Media Design and Creative Writing at Bennington College in Vermont, and was the youngest ever contributing associate at the Harvard Review.
In 1995, August and six other seniors from Bennington produced a CD-ROM multimedia enrollment tool that caught Apple’s eye. They asked the students to present the tool at Mac World Expo ’95 and the exposure brought many job offers. He worked on a series of technologies, from organizing a system for launching Eastman Kodak’s website in 35 countries to researching ambient intelligence at Philips in Holland. August has always thrived at the cutting edge of human-computer interaction, which is what brought him to Microsoft in February 2003. Microsoft is on the cusp of a cultural change,” August says. “I think the next step is design-led engineering.”
Today, August is the liaison between product design and marketing for MSN branding, provides creative direction for MSN branding collateral (photo shoots, CD mass-mailings, and print ads), and owns the look of My MSN and MSN Homepage portals.
He has also worked with user researchers from Windows to revolutionize the way emotion and desirability are measured. It’s no longer good enough that users complete tasks, they should also feel good about the experience—which is how August felt when he finally made that perfect soufflé.
By Gerd Waloszek, User Productivity, SAP AG – 05/25/2005
In this report, I present a few personal notes from the CHI 2005 conference, which took place in Portland, Oregon from April 02 - 07, 2005. As this was my sixth CHI conference, I apologize for not being as enthusiastic as many of my colleagues whom I talked with about it. There is definitely a difference between whether you listen to an interesting presentation for the first or for a third or fourth time. My comments maybe somewhat biased due to my previous CHI and HCI experiences.
On the other hand, I attend the CHI conferences to find out how the HCI field changes over time and whether we experience “progress” or not. Each CHI conference that I visited marked a specific point in time and helped me to spot trends and to integrate them into my personal picture – or puzzle – of our still emerging field. Apart from the pure HCI trends, there are also organizational and business trends that cannot be overlooked. Currently there are heated debates on:
Cost-justifying usability using measures, such as ROI (return on investment) or TCO (total cost of ownership)
Reducing costs for usability activities by outsourcing and offshoring
The growing pace of company reorganizations and how they affect the HCI profession
The self-image of the HCI community as a profession (science, engineering discipline, craft, art …)
As the CHI 2005 conference reflected these topics, I will focus on them in my report.
Outsourcing and Offshoring
Certain changes go unnoticed even though you know that they are going on. One of these is outsourcing usability work. Currently, the vast majority of it is outsourced to India, followed by China, Russia, Canada, Ireland, and Mexico. Whether outsourcing pays off, is still an open question. A cost factor of nine for outsourcing has been claimed: You can hire three people at a third of the cost offshore. Other sources claim that the economical gain is only marginal – and will become increasingly less attractive in the future because the problems can be immense and the wages are rising “offshore.” Thus, often it is more important to be close to relevant markets than saving money by outsourcing.
The panelists (center: Jon Innes, SAP Labs, Palo Alto)
Companies, such as Human Factors International (HFI) or Computer Associates (CA) have been successfully practicing offshoring usability work for years already. But to be successful, you need a deep understanding of the cultural differences and of which tasks can be easily outsourced and which cannot. Typically, routine tasks can be outsourced while more open-ended or conceptual tasks cannot.
Is ROI an Effective Approach for Persuading Decision-Makers of the Value of User-Centered Design?
To me, this panel looked a bit like a put-up affair: Three of the four panelists seemed to agree in beforehand that ROI (return on investment) should be dismissed as a valid measure. Solely Dennis Wixon from Microsoft’s game division put forth the ROI case. Actually, he put forth the case of his RITE (rapid interactive and evaluative testing) method. His contribution would have been better called “RITE – A Commercial,” instead of “Strategic ROI – A Commercial.”
The Panelists (left: Dan Rosenberg, SAP AG)
While IBM’s Jennifer Lai, who stood in for Clare-Marie Karat, claimed that she never had to justify user-centered design using ROI statements, David Siegel characterized ROI as a tactical measure that undermines risky business decisions, such as introducing new technologies or moving into new business sectors. He contrasted it with strategic thinking, which has a long-range focus and treats risk as inherent. In his opinion, ROI can be potentially self-destructive for the whole HCI profession because of its shortsightedness.
SAP’s Dan Rosenberg repeated his story of the myths that ROI is based on and offered TCO (total cost of ownership) as an alternative measure – among others, because he, too, regards TCO as strategic. TCO was introduced by the Gartner group. It recalls the “ecological point balance,” and is plagued in my opinion with a bunch of problems of its own. For example, if you look at a TCO model it is hard to find the places, where usability comes in – which may lead some people (such as managers) to conclude that it isn’t important at all. And with both, ROI and TCO, you are never sure whether you can really attribute certain outcomes to certain actions, such as the effort invested in a user-centered design process.
Corporate Re-Orgs: Poison or Catalyst to HCI?
I do not believe that this panel really answered the question “Poison or Catalyst to HCI?” Instead, we got more personal (typical American?) recommendations, such as: Do a good job, be flexible, and so on. Re-orgs come from “heaven” (that is, from the board) and you had better adapt to them and try to make the best of it for yourself in order to survive. Kelly Brown from eBay at least hinted at the fact that there might be a contradiction between business (or company) and professional goals. Years ago, Don Norman proclaimed that usability people need to “infiltrate” the management in order to support their profession. I would like to add: this is probably the most promising way to ensure re-orgs, which are catalysts for HCI.
The panelists
The Great Debate: Can Usability Scale Up?
What can you expect from such a debate? I did not know Eric Shaffer from HFI but I have known Jared Spool since 1998. So, I expected some sort of “show event” – and, to a certain degree, it was. I learned that Jared Spool can cite wrong numbers, be accused of this by someone from the audience, but in the end nobody cares. Somehow, this is symptomatic of the current state of the profession because it touches on the question of what type of discipline usability is: user interface design, user experience, or whatever label you put on it? In my article “User Interface Design – Is it a Science, an Art, or a Craft?”, I came up with the conclusion that it’s a craft, and this is what Jared in the end also was trying to make clear. So, even despite the shakiness of Jared’s arguments, I am basically “with him.”
The debaters: Eric Shaffer (left) and Jared Spool (right)
As a physicist who moved into the realms of psychology, I had to recognize that there is a difference between “hard” and “soft” sciences. So, I am always skeptical whenever someone tries to make us believe that UI methodology can lead to predictable, reliable results (in the end expressed in numbers, such as ROI or TCO). As evidence, Jared cited Rolf Molich’s CUE4 investigation, in which Molich showed that contrary to the Nielsen-Landauer “law,” the more errors were found in the user interface, the more teams tested it. And more disturbingly: There was little overlap between the results of the different teams.
According to Jared, “craft is based on the skills of individuals, while engineering tries to achieve repeatable results independent of the engineer’s talents.” I also agree with his statement: “Evidence suggests that usability practice is more craft than engineering, but we sell it as an engineering practice.” Eric Shaffer and others do this selling and make good money with it. But in my opinion, we are still in the “age of the gurus.” People, such as Karen Holtzblatt, Alan Cooper, Jakob Nielsen, or Bruce Tognazzini – and I would also add Jared Spool to this list – support my hypothesis by declaring that solely their own approach is the right one (in science, there is – I know I am generalizing a bit – only approach, the scientific). At least, I am in line with Eric Shaffer when he sees the importance of gurus diminishing in the future.
A tough question for the debaters …
Let me conclude this section with three of Jared Spool’s introductory statements:
High investment in user-centered design doesn’t result in more usable designs
In fact, it usually results in less usable design
Usability practice doesn’t scale up (Eric believes that it is ready to scale up right now)
Generally, I agree with him. But the correct conclusion is not to not invest in user-centered design (UCD) or only halfheartedly. In my opinion, Jared just described the current state of affairs, which reflects an insufficient integration of UCD into the development process as well as most software companies’ general view of what is important and what is not.
Looking Forward by Looking Back: Early Gestural Interfaces for Live Electronic Music Composing and Performance
The closing plenary was introduced by a minute’s silence in commemoration of Jef Raskin, the “father” of the Macintosh user interface, who passed away in February.
Michel Waisvisz demonstrates one of his musical instruments
The main act was a presentation by Michel Michel Waisvisz, who reported the history of his self-built electronic music instruments, which are commanded by gestures and buttons. And, of course, he demonstrated some of them. Waisvisz’ musical installations can also be used in group settings, or even distributed over different locations.
So, what is the point of such a presentation at an HCI conference – apart from its entertainment value? After having reviewed a couple of books on design, especially on the relationship between technology and user experience, my answer would be the following: Designers and artists highlight the fact that human experience is not a “prefabricated” thing that you can design into something by using certain techniques – as some UI companies and experts claim. Instead, they claim that people are creative and use their creativity to make up their own, open-ended experiences. It is the open-endedness of human-experience that the designers strive to remind the HCI community of again and again – an idea that they feel seems to have been forgotten by this community.
Wixon, D. (2005). Is ROI an Effective Approach for Persuading Decisionmakers on the Value of User-Centered Design? Panel Discussion at Conference on Human Factors in Computing Systems (CHI 2005).
Download Slides (2.6 MB)
Games User Research at Microsoft Game Studios: www.mgsuserresearch.com
Daniel Rosenberg, Seven myths of usability ROI: www.baychi.org/calendar/20031014
Debate
User Interface Engineering (Company of Jared Spool)
Human Factors International (HFI) (Company of Eric Shaffer)
Aaron Marcus and Associates (Company of Aaron Marcus)
User Interface Design - Is it a Science, an Art, or a Craft? (SAP Design Guild)
Closing Plenary
Crackle.org: www.crackle.org/index.php
Bio Michel Waisvisz: http://www.crackle.org/whoami.php
All photos by Gerd Waloszek (taken with Minolta Dimage A200).
Daniel Rosenberg, Oracle
Meeting report by Nerija Sinkeviciute-Titus ntitus@baychi.org
Daniel Rosenberg began his talk by confessing that he doesn’t believe in usability Return on Investment (ROI). Having spent 30 years in the field of User Experience (UE), and never having been asked to justify usability by its ROI, Rosenberg raises a question: Why are we still discussing this topic?
A Google search for “usability ROI” brought up a limited set of original studies (there were many more cross-citations than articles) which are, in Daniel’s words, “all crap!” Specifically, most are ambiguous, with incomplete data and without related business variables. Daniel did blame some of these shortcomings on lawyers, who often stand in the way of publishing the data in full. (”Lawyers are like beavers. They try to get into the mainstream of progress and dam things up.”)
Rosenberg expressed his opinion that the current ROI models are inadequate and “it is only fair that no CEO would believe them.” Conventional usability ROI theory becomes hard to prove when the data is unreliable, so it’s not clear whether good usability increases sales, market share, customer satisfaction, and profitability. And it is not obvious that poor usability leads to higher training costs, higher support costs, and longer schedules.
Rosenberg concluded that the limited case studies perpetuate a set of myths about usability ROI and went on to present them in a very lively and controversial manner.
Myth #1: Generalization is Valid
He read a citation from Tom Landauer’s book The Trouble with Computers: “Without User Centered Design, a user interface typically has around 40 flaws that can slow users and lead to errors.” Rosenberg pointed out that this statement leaves too much information out. We don’t know whether Landauer is talking about hardware or software, and it isn’t clear whether it is a web storefront, a packaged application, or an internal IT project. Rosenberg laughed and admitted that he would be happy if a product only had 40 usability problems! This kind of statement “doesn’t cut it with executive-class business leaders,” he said.
Myth #2: Calculation of ROI from the Producer Perspective
“Research by Gartner Group … reveals that in corporate practice, the average annual bill for supporting a single PC is $13,000″ (Gibbs, Taking Computers to Task, Scientific American, 1997). According to Rosenberg, it’s a mistake to calculate cost for the producer rather than for the consumer, because it is a fallacy that the producer bears the cost. The company cares about shipping the product fast and gaining market share, but it could care less about reducing the cost for the consumer. Moreover, extra support cost is someone else’s revenue or employment opportunity!
Myth #3: You Can Ignore the Other Factors
“Revenues for one DEC product that was developed using UCD techniques increased 80% for the new version … and usability was cited as the second most significant improvement” (Wixon & Jones, Usability for Fun and Profit, 1995). Rosenberg immediately raises questions: What was the number-one reason? Doubling the size of the sales force? Increasing their commission? Reducing the price of the product 75%? Rosenberg is certain that these kinds of assertions diminish the credibility of the person making them, making it more difficult to get funding.
Myth #4: Analog Comparisons are Not Required
“Cost of bad web design: Loss of approximately 50% of potential sales from the site as people can’t find stuff.” (Jakob Nielsen, Alert Box, 1998, cited by Forrester).
Rosenberg gets agitated here and stresses that we shouldn’t expect a consumer to buy a product on-line 50% of the time. Do you buy something in 50% of the brick and mortar stores you go into when shopping? Can you find a part at Home Depot 50% of the time? People often don’t know what they are looking for. His criticism here is the absence of analog benchmarks in the literature.
Myth #5: All Usability Dollars are Spent Effectively
Rosenberg admits he thinks this is a joke, because he believes that as a profession we are not that effective in communicating our value and delivering value to the corporation or as consultants. Anecdotal evidence would also suggest that we are not as effective commercially as, say, marketing professionals.
Myth #6: Executives will Believe Voodoo Economics
“There are one billion users on the internet, and half of them could come to your site. If the average cost of an abandoned shopping cart is $20, you will lose $10 billion a year in sales of your designer pet food.” (Rosenberg, 2003, Parody of J. Nielsen).
Statements like these will get you coverage in Newsweek and may then be cited as fact by Gartner and Forrester Research groups, jokes Rosenberg. But executives know better, and they won’t fall for this.
Myth #7: UE Resources will Reduce the Software Schedule
“With a $13,000 investment in UE, overall project costs are reduced by $8,000, and total time on the project is reduced by four weeks” (Friedland and Innes, UPA workshop, 2003). Rosenberg pointed out that in 30 years, he had never seen a product ship on time.
After finishing his seven myths of Usability ROI, Rosenberg tried to be a little more positive. He presented three “laws of gravity” affecting usability ROI.
It is cheaper to fix problems early in the design process.
Automation reduces complexity faster and in larger increments than UI design. For example, after home networks got automated (DHCP), the average person can set it up now.
Globalization reduces labor costs. Things that were expensive are now much cheaper. Most of the complex work can be done in India or other places like that. Most of the calculations for the case studies were done in the U.S. several years ago, and are not accurate anymore.
Finally, Rosenberg offered to look for a more strategic approach and to replace the myths by defining usability value by contribution to the customer’s success—Total Cost of Ownership (TCO)—not the producer’s ROI. That is, we should calculate what it costs for the client to be successful.
Rosenberg doesn’t suggest looking at the sales numbers, because they can’t correlate with a single factor like usability. A useful measure is win/loss data as well as Common Industry Format (CIF) data for the sales cycle. As an example, Oracle releases all the bug data to the customer. Customers get a chance to look at the usability reports and can suggest tasks that the UE team didn’t test, adding value to the customer. Another resource is company’s customer support team, which has their own usability methods and metrics.
He stressed again that in the business product ecosystem, usability is just one variable. There has to be good product design (features, performance, cost, reliability, and usability) as well as good execution (manufacturing, distribution, sales, and marketing) to produce enough profit to stay in business by gaining successful customers who can pay you.
Rosenberg concluded with his practical rule of the relevance of software product ROI (not just usability ROI): 10% of the world’s software generates 90% of the software industry revenue. Therefore, if your product is not in that 10%, there is no return!
Original Announcement
The debate and discussion on methods to calculate a return on investment (ROI) for usability work has persisted for over fifteen years. In addition, the upswing in popularity of the internet, in conjunction with the sustained downturn in the economy, has increased the focus many user experience professionals have on providing a cost justification of their work. Many practitioners, it appears, must constantly defend their value in the development process just to retain their employment or client base.
However, a review of the current literature and approach to usability ROI indicates several flaws which, in the opinion of the speaker, work to the detriment of the profession. The current approach is based on a set of myths, perpetuated within the user experience community, that any CEO or business executive would plainly see through. These myths have led to an ineffective, tactical approach to quantifying the value that user experience professionals bring to the design process.
This talk is intended to debunk these myths and suggest a more strategic approach that has proved effective at a number of leading high-tech companies in the valley.
———— Daniel Rosenberg is vice president of development for usability and interface design at Oracle Corporation. He is responsible for the UI design of Oracle Server, Tools, and Applications product lines. His 65-person usability and interface design group at Oracle is also responsible for the development of the Oracle UI standards for Java and HTML, as well as applied UI research on advanced user interfaces.
Prior to joining Oracle, Daniel was the user interface architect for Borland International, and earlier he held the same role at Ashton-Tate. His book with William Cushman, Human Factors in Product Design (ISBN 0444890319, Elsevier, 1991), was the first to formally address the ergonomics of consumer products. His many publications include chapters in the Handbook of Human Computer Interaction (ISBN 0444886737, Elsevier, 1988) and Usability in Practice: How Companies Develop User-Friendly Products (ISBN 0127512500, Academic Press, 1994) as well as numerous journal and magazine articles.
Daniel has been an active member of the CHI community for 25 years, having taught several CHI tutorials, participated in panels and workshops, and reviewed papers. He was also one of the founding editors of ACM’s NetWorker magazine.
Blog 
Daniel Rosenberg is vice president of development for usability and interface design at Oracle Corporation. He is responsible for the UI design of Oracle Server, Tools, and Applications product lines. His 65-person usability and interface design group at Oracle is also responsible for the development of the Oracle UI standards for Java and HTML, as well as applied UI research on advanced user interfaces.
