2013 MeasuringtheUserExperienceSecon

• (Tullis & Albert, 2013) ⇒ Thomas Tullis, and William Albert. (2013). “Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics.” Morgan Kaufmann Publishers Inc.. ISBN:0124157815, 9780124157811

Subject Headings: A/B Tests, Click-through rates, Likert scale, Net Promoter Score, Usability, Usability Study, Usability Test.

Cited By

• http://scholar.google.com/scholar?q=%222013%22+Measuring+the+User+Experience%2C+Second+Edition%3A+Collecting%2C+Analyzing%2C+and+Presenting+Usability+Metrics
• http://dl.acm.org/citation.cfm?id=2526258&preflayout=flat#citedby

Quotes

Table of Contents

 1 Introduction p.1
 2 Background p.15
 3 Planning p.41
 4 Performance Metrics p.63
 5 IssueBased Metrics p.99
 6 SelfReported Metrics p.121
 7 Behavioral and Physiological Metrics p.163
 8 Combined and Comparative Metrics p.187
 9 Special Topics p.209
10 Case Studies p.237
11 Ten Keys to Success p.279

Abstract

Measuring the User Experience was the first book that focused on how to quantify the user experience. Now in the second edition, the authors include new material on how recent technologies have made it easier and more effective to collect a broader range of data about the user experience. As more UX and web professionals need to justify their design decisions with solid, reliable data, Measuring the User Experience provides the quantitative analysis training that these professionals need. The second edition presents new metrics such as emotional engagement, personas, keystroke analysis, and net promoter score. It also examines how new technologies coming from neuro-marketing and online market research can refine user experience measurement, helping usability and user experience practitioners make business cases to stakeholders. The book also contains new research and updated examples, including tips on writing online survey questions, six new case studies, and examples using the most recent version of Excel. Learn which metrics to select for every case, including behavioral, physiological, emotional, aesthetic, gestural, verbal, and physical, as well as more specialized metrics such as eye-tracking and clickstream data. Find a vendor-neutral examination of how to measure the user experience with web sites, digital products, and virtually any other type of product or system. Discover in-depth global case studies showing how organizations have successfully used metrics and the information they revealed. Companion site, http://www.measuringux.com, includes articles, tools, spreadsheets, presentations, and other resources to help you effectively measure the user experience

References

• 1. Bradner, E. (2010,). Recommending Net Promoter. Retrieved on 23.10.2011 from DUX: Designing the User Experience at Autodesk ¿http://dux.typepad.com/dux/2010/11/recommending-net-promoter.html¿.
• 2. Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review. Owen, R., & Brooks, L. (2008). Answering the Ultimate Question. San Francisco: Jossey-Bass.
• 3. Sauro, J. (2011). Usability and Net Promoter Benchmarks for Consumer Software. Retrieved on 23.10.2011 from Measuring Usability ¿http://www.measuringusability.com/softwarebenchmarks.php¿.
• 4. Sauro, J., & Kindlund, E. (2005). Using a Single Usability Metric (SUM) to Compare the Usability of Competing Products. In Proceeding of the Human Computer Interaction International Conference (HCII 2005), Las Vegas, NV.
• 5. Haiying Guan, Mary F. Theofanos, Yee-Yin Choong, Brian Stanton, Real-time Feedback for Usable Fingerprint Systems, Proceedings of the 2011 International Joint Conference on Biometrics, p.1-8, October 11-13, 2011
• 6. International Organization for Standards (ISO)(1998). 9241-11 Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs). Part 11. Guidance on Usability. Geneva, Switzerland.
• 7. Stanton, B., Theofanos, M., Steves, M., Chisnell, D., & Wald, H. (2012). Fingerprint Scanner Affordances, NISTIR (to Be Published), National Institute of Standards and Technology, Gaithersburg, MD.
• 8. Tabassi, E., Wilson, C., & Watson, C. (2004). Fingerprint Image Quality. NISTIR 7151, National Institute of Standards and Technology, Gaithersburg, MD ¿http://www. Nist.gov/customcf/get_pdf.cfm?pub_id=905710¿.
• 9. Theofanos, M., Stanton, B., Sheppard, C., Micheals, R., Zhang, N. F., Wydler, J., Et Al. (2008). Usability Testing of Height and Angles of Ten-print Fingerprint Capture, NISTIR 7504, National Institute of Standards and Technology, Gaithersburg, MD.
• 10. Bergstrom, J., Olmsted-Hawala, E., Chen, J., & Murphy, E. (2011). Conducting Iterative Usability Testing on a Web Site: Challenges and Benefits. Journal of Usability Studies, 7(1), 9-30.
• 11. ISO 9241-124. Ergonomics of Human-system Interaction.
• 12. Jeff Sauro, Joseph S. Dumas, Comparison of Three One-question, Post-task Usability Questionnaires, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 04-09, 2009, Boston, MA, USA
• 13. Jeff Sauro, James R. Lewis, Quantifying the User Experience: Practical Statistics for User Research, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2012
• 14. Jarrett, C., & Roddis, I. (2002). How to Obtain Maximum Insight by Cross-referring Site Statistics, Focus Groups and Usability Techniques. Web based Surveys and Usability Testing. San Francisco, CA: Institute for International Research.
• 15. John Pruitt, Tamara Adlin, The Persona Lifecycle: Keeping People in Mind Throughout Product Design, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 16. Carol M. Barnum, Laura A. Palmer, More Than a Feeling: Understanding the Desirability Factor in User Experience, CHI '10 Extended Abstracts on Human Factors in Computing Systems, April 10-15, 2010, Atlanta, Georgia, USA
• 17. Benedek, J., & Miner, T. (2002). Measuring Desirability: New Methods for Evaluating Desirability in a Usability Lab Setting. Proceedings of Usability Professionals Association, 8-12.
• 18. Picard, R. (2010). Emotion Research by the People, for the People. Emotion Review, 2, 250-254.
• 19. Cornelia Setz, Bert Arnrich, Johannes Schumm, Roberto La Marca, Gerhard Tröster, Ulrike Ehlert, Discriminating Stress from Cognitive Load Using a Wearable EDA Device, IEEE Transactions on Information Technology in Biomedicine, v.14 n.2, p.410-417, March 2010
• 20. Albert, W., & Dixon, E. (2003). Is This What You Expected? The Use of Expectation Measures in Usability Testing. Proceedings of Usability Professionals Association 2003 Conference, Scottsdale, AZ.
• 21. Albert, W., Gribbons, W., & Almadas, J. (2009). Pre-conscious Assessment of Trust: A Case Study of Financial and Health Care Web Sites. Human Factors and Ergonomics Society Annual Meeting Proceedings, 53, 449-453. Also ¿http://www.measuringux.com/Albert_Gribbons_Preconsciousness.pdf¿.
• 22. Albert, W., & Tedesco, D. (2010). Reliability of Self-reported Awareness Measures based on Eye Tracking. Journal of Usability Studies, 5(2), 50-64.
• 23. Aldenderfer, M., & Blashfield, R. (1984). Cluster Analysis (quantitative Applications in the Social Sciences). Beverly Hills, CA: Sage Publications, Inc.
• 24. American Institutes for Research. (2001). Windows XP Home Edition Vs. Windows Millennium Edition (ME) Public Report. New England Research Center, Concord, MA. Available at ¿http://download.microsoft.com/download/d/8/1/d810ce49-d481-4a55-ae63- 3fe2800cbabd/ME_Public.doc¿.
• 25. Andre, A. (2003). When Every Minute Counts, all Automatic External Defibrillators Are Not Created Equal. Published in June, 2003 by Interface Analysis Associates ¿http://www. Usernomics.com/iaa_aed_2003.pdf¿.
• 26. Bangor, A., Kortum, P., & Miller, J. A. (2009). Determining What Individual SUS Scores Mean: Adding An Adjective Rating Scale. Journal of Usability Studies, 4, 3.
• 27. Javier A. Bargas-Avila, Kasper Hornbæk, Old Wine in New Bottles Or Novel Challenges: A Critical Analysis of Empirical Studies of User Experience, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 07-12, 2011, Vancouver, BC, Canada
• 28. Barnum, C., Bevan, N., Cockton, G., Nielsen, J., Spool, J., & Wixon, D. (2003). The "magic Number 5": Is It Enough for Web Testing? 2003, April 5-10, Ft. Lauderdale, FL: CHI.
• 29. Benedek, J., & Miner, T. (2002). Measuring Desirability: New Methods for Evaluating Desirability in a Usability Lab Setting. Usability Professionals Association 2002 Conference, Orlando, FL, July 8-12. Also Available at ¿http://www.microsoft.com/usability/UEPostings/DesirabilityToolkit.doc¿. Also See the Appendix Listing the Product Reaction Cards at ¿http://www.microsoft.com/usability/UEPostings/ProductReactionCards.doc¿.
• 30. Randolph G. Bias, Deborah J. Mayhew, Cost-Justifying Usability: An Update for the Internet Age, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 31. Birns, J., Joffre, K., Leclerc, J., & Paulsen, C. A. (2002). Getting the Whole Picture: Collecting Usability Data Using Two Methods - Concurrent Think Aloud and Retrospective Probing. Proceedings of the 2002 Usability Professionals' Association Conference, Orlando, FL. Available from ¿http://concordevaluation.com/papers/paulsen_thinkaloud_ 2002.pdf¿.
• 32. Breyfogle, F. (1999). Implementing Six Sigma: Smarter Solutions Using Statistical Methods. New York: John Wiley and Sons.
• 33. Brooke, J. (1996). SUS: A Quick and Dirty Usability Scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester & I. L. McClelland (Eds.), Usability Evaluation in Industry. London: Taylor & Francis.
• 34. Jason Burby, Shane Atchison, Actionable Web Analytics: Using Data to Make Smart Business Decisions, SYBEX Inc., Alameda, CA, 2007
• 35. Stuart K. Card, Allen Newell, Thomas P. Moran, The Psychology of Human-Computer Interaction, L. Erlbaum Associates Inc., Hillsdale, NJ, 1983
• 36. Catani, M., & Biers, D. (1998). Usability Evaluation and Prototype Fidelity. In Proceedings of the Human Factors and Ergonomic Society.
• 37. Ann Chadwick-Dias, Michelle McNulty, Tom Tullis, Web Usability and Age: How Design Changes Can Improve Performance, Proceedings of the 2003 Conference on Universal Usability, November 10-11, 2003, Vancouver, British Columbia, Canada
• 38. John P. Chin, Virginia A. Diehl, Kent L. Norman, Development of An Instrument Measuring User Satisfaction of the Human-computer Interface, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p.213-218, May 15-19, 1988, Washington, D.C., USA
• 39. Brian Clifton, Advanced Web Metrics with Google Analytics, 2nd Edition, SYBEX Inc., Alameda, CA, 2010
• 40. Cockton, G., & Woolrych, A. (2001). Understanding Inspection Methods: Lessons from An Assessment of Heuristic Evaluation. Joint Proceedings of HCI and IHM: People and Computers, XV.
• 41. Cox, E. P. (1980). The Optimal Number of Response Alternatives for a Scale: A Review. Journal of Marketing Research, 17(4), 407-422.
• 42. Cunningham, K. (2012). The Accessibility Handbook. Sebastopol, CA: O'Reilly Media.
• 43. Dennerlein, J., Becker, T., Johnson, P., Reynolds, C. J., & Picard, R. W. (2003). Frustrating Computer Users Increases Exposure to Physical Factors. In Proceedings of the International Ergonomics Association, August 24-29, Seoul.
• 44. Dillman, D. A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Et Al. (2008). Response Rate and Measurement Differences in Mixed Mode Surveys Using Mail, Telephone, Interactive Voice Response, and the Internet. Available at ¿http://www.sesrc. Wsu.edu/dillman/papers/2008/ResponseRateandMeasurement.pdf¿.
• 45. Ekman, P., & Friesen, W. (1975). Unmasking the Face. Englewood Cliffs, NJ: Prentice-Hall.
• 46. Everett, S. P., Byrne, M. D., & Greene, K. K. (2006). Measuring the Usability of Paper Ballots: Efficiency, Effectiveness, and Satisfaction. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.
• 47. Stephen Few, Information Dashboard Design: The Effective Visual Communication of Data, O'Reilly Media, Inc., 2006
• 48. Stephen Few, Now You See It: Simple Visualization Techniques for Quantitative Analysis, Analytics Press, 2009
• 49. Stephen Few, Show Me the Numbers: Designing Tables and Graphs to Enlighten, Analytics Press, 2012
• 50. Finstad, K. (2010). Response Interpolation and Scale Sensitivity: Evidence Against 5-point Scales. Journal of Usability Studies, 5(3), 104-110.
• 51. B. J. Fogg, Jonathan Marshall, Othman Laraki, Alex Osipovich, Chris Varma, Nicholas Fang, Jyoti Paul, Akshay Rangnekar, John Shon, Preeti Swani, Marissa Treinen, What Makes Web Sites Credible?: A Report on a Large Quantitative Study, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p.61-68, March 2001, Seattle, Washington, USA
• 52. Foraker. (2010). Usability ROI Case Study: Breastcancer.org Discussion Forums. Retrieved 4/18/2013 from ¿http://www.usabilityfirst.com/documents/U1st_BCO_CaseStudy. Pdf¿.
• 53. Foresee. (2012). ACSI E-government Satisfaction Index (Q4 2012). ¿http://www.foreseeresults. Com/research-white-papers/_downloads/acsi-egov-q4-2012-foresee.pdf¿.
• 54. Friedman, H. H., & Friedman, L. W. (1986). On the Danger of Using Too Few Points in a Rating Scale: A Test of Validity. Journal of Data Collection, 26(2), 60-63.
• 55. Garland, R. (1991). The Mid-point on a Rating Scale: Is It Desirable? Marketing Bulletin(2), 66-70. Research Note 3.
• 56. Zhiwei Guan, Shirley Lee, Elisabeth Cuddihy, Judith Ramey, The Validity of the Stimulated Retrospective Think-aloud Method As Measured by Eye Tracking, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 22-27, 2006, Montréal, Québec, Canada
• 57. Jacek Gwizdka, Ian Spence, Implicit Measures of Lostness and Success in Web Navigation, Interacting with Computers, v.19 n.3, p.357-369, May, 2007
• 58. Hart, T. (2004). Designing "senior Friendly" Websites: Do Guidelines Help? Usability News, 6.1. ¿http://psychology.wichita.edu/surl/usabilitynews/61/older_adults-withexp.htm¿.
• 59. Henry, S. L. (2007). Just Ask: Integrating Accessibility Throughout Design. Raleigh, NC: Lulu. Com.
• 60. Morten Hertzum, Niels Ebbe Jacobsen, Rolf Molich, Usability Inspections by Groups of Specialists: Perceived Agreement in Spite of Disparate Observations, CHI '02 Extended Abstracts on Human Factors in Computing Systems, April 20-25, 2002, Minneapolis, Minnesota, USA
• 61. T T Hewett, The Role of Iterative Evaluation in Designing Systems for Usability, Proceedings of the Second Conference of the British Computer Society, Human Computer Interaction Specialist Group on People and Computers: Designing for Usability, p.196-214, December 1986, York, United Kingdom
• 62. Holland, A. (2012a). Ecommerce Button Copy Test: Did 'Personalize Now' Or 'Customize It' Get 48% More Revenue Per Visitor? Retrieved on 4/18/2013 from ¿http://whichtestwon. Com/archives/14511¿.
• 63. Holland, A. (2012b). Online Newspaper Layout Test: Should Photos Alternate Sides Or Always Appear to the Right of Stories? Retrieved on 4/18/2013 from ¿https://whichtestwon. Com/archives/18744¿.
• 64. Hornbæk, K., & Frøkjær, E. (2008). A Study of the Evaluator Effect in Usability Testing. Human-Computer Interaction, 23(3), 251-277.
• 65. Human Factors International. (2002). HFI Helps Staples.com Boost Repeat Customers by 67%. Retrieved 4/18/2013 from ¿http://www.humanfactors.com/downloads/documents/staples.pdf¿.
• 66. Hyman, I. E., Boss, S. M., Wise, B. M., McKenzie, K. E., & Caggiano, J. M. (2010). Did You See the Unicycling Clown? Inattentional Blindness While Walking and Talking on a Cell Phone. Applied Cognitive Psychology, 24, 597-607.
• 67. ISO/IEC 25062 (2006). Software Engineering - Software Product Quality Requirements and Evaluation (SQuaRE) - Common Industry Format (CIF) for Usability Test Reports.
• 68. Jacobsen, N., Hertzum, M., & John, B. (1998). The Evaluator Effect in Usability Studies: Problem Detection and Severity Judgments. In Proceedings of the Human Factors and Ergonomics Society.
• 69. Kapoor, A., Mota, S., & Picard, R. (2001). Towards a Learning Companion That Recognizes Affect. AAAI Fall Symposium, November, North Falmouth, MA.
• 70. Avinash Kaushik, Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity, SYBEX Inc., Alameda, CA, 2009
• 71. Andrew Kirkpatrick, Richard Rutter, Christian Heilmann, Jim Thatcher, Cynthia Waddell, Web Accessibility: Web Standards and Regulatory Compliance, Friends of ED, 2006
• 72. Kohavi, R., Crook, T., & Longbotham, R. (2009). Online Experimentation at Microsoft, Third Workshop on Data Mining Case Studies and Practice. Retrieved on 4/18/2013 from ¿http://robotics.stanford.edu/~ronnyk/ExP_DMCaseStudies.pdf¿.
• 73. Ron Kohavi, Alex Deng, Brian Frasca, Roger Longbotham, Toby Walker, Ya Xu, Trustworthy Online Controlled Experiments: Five Puzzling Outcomes Explained, Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 12-16, 2012, Beijing, China
• 74. Kohavi, R., & Round, M. (2004). Front Line Internet Analytics at Amazon.com. Presentation at Emetrics Summit 2004. Retrieved on 4/18/2013 from ¿http://ai.stanford. Edu/~ronnyk/emetricsAmazon.pdf¿.
• 75. Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (Eds.), (2000). Committee on Quality of Health Care in America, Institute of Medicine. “To Err is Human: Building a Safer Health System.". Washington, DC: National Academies Press.
• 76. Kruskal, J., & Wish, M. (2006). Multidimensional Scaling (quantitative Applications in the Social Sciences). Beverly Hills, CA: Sage Publications, Inc..
• 77. Mike Kuniavsky, Observing the User Experience: A Practitioner's Guide to User Research (Morgan Kaufmann Series in Interactive Technologies) (The Morgan Kaufmann Series in Interactive Technologies), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2003
• 78. LeDoux, L., Mangan, E., & Tullis, T. (2005). Extreme Makeover: UI Edition. Presentation at Usability Professionals Association (UPA) 2005 Annual Conference, Montreal, QUE, Canada. Available from ¿http://www.upassoc.org/usability_resources/conference/2005/ledoux-UPA2005-Extreme.pdf¿.
• 79. Lewis, J. (1994). Sample Sizes for Usability Studies: Additional Considerations. Human Factors, 36, 368-378.
• 80. James R. Lewis, Psychometric Evaluation of An After-scenario Questionnaire for Computer Usability Studies: The ASQ, ACM SIGCHI Bulletin, v.23 n.1, p.78-81, Jan. 1991
• 81. James R. Lewis, IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use, International Journal of Human-Computer Interaction, v.7 n.1, p.57-78, Jan.-March 1995
• 82. James R. Lewis, Jeff Sauro, The Factor Structure of the System Usability Scale, Proceedings of the 1st International Conference on Human Centered Design: Held As Part of HCI International 2009, July 19-24, 2009, San Diego, CA
• 83. Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology, 140, 55.
• 84. Tao Lin, Masaki Omata, Wanhua Hu, Atsumi Imamiya, Do Physiological Data Relate to Traditional Usability Indexes?, Proceedings of the 17th Australia Conference on Computer-Human Interaction: Citizens Online: Considerations for Today and the Future, November 21-25, 2005, Canberra, Australia
• 85. Gitte Lindgaard, Jarinee Chattratichart, Usability Testing: What Have We Overlooked?, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 28-May 03, 2007, San Jose, California, USA
• 86. Lindgaard, G., Fernandes, G., Dudek, C., & Brown, J. (2006). Attention Web Designers: You Have 50 Milliseconds to Make a Good First Impression!. Behaviour & Information Technology, 25, 115-126.
• 87. Lund, A. (2001). Measuring Usability with the USE Questionnaire. Usability and User Experience Newsletter of the STC Usability SIG. See ¿http://www.stcsig.org/usability/newsletter/0110_measuring_with_use.html¿.
• 88. Martin, P., & Bateson, P. (1993). Measuring Behaviour (2nd Ed.). Cambridge, UK, and New York: Cambridge University Press.
• 89. Maurer, D., & Warfel, T. (2004). Card Sorting: A Definitive Guide. Boxes and Arrows, April 2004. Retrieved on 4/18/2013 from ¿http://boxesandarrows.com/card-sorting-a-definitive-guide/¿.
• 90. Randolph G. Bias, Deborah J. Mayhew, Cost-justifying Usability, Academic Press, Inc., Orlando, FL, 1994
• 91. McGee, M. (2003). Usability Magnitude Estimation. Proceedings of Human Factors and Ergonomics Society Annual Meeting, Denver, CO.
• 92. McLellan, S., Muddimer, A., & Peres, S. C. (2012). The Effect of Experience on System Usability Scale Ratings. Journal of Usability Studies, 7(2), 56-67. ¿http://www.upassoc. Org/upa_publications/jus/2012february/JUS_McLellan_February_2012.pdf¿.
• 93. Gary Miner, John Elder, Thomas Hill, Robert Nisbet, Dursun Delen, Andrew Fast, Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications, Academic Press, 2012
• 94. Molich, R. (2011). CUE-9: The Evaluator Effect. ¿http://www.dialogdesign.dk/CUE-9.html¿.
• 95. Molich, R., Bevan, N., Butler, S., Curson, I., Kindlund, E., Kirakowski, J., Et Al., (1998). Comparative Evaluation of Usability Tests. Usability Professionals Association 1998 Conference, 22-26 June 1998 Washington, DC: Usability Professionals Association, Pp. 189-200.
• 96. Rolf Molich, Joseph S. Dumas, Comparative Usability Evaluation (CUE-4), Behaviour & Information Technology, v.27 n.3, p.263-281, May 2008
• 97. Rolf Molich, Meghan R. Ede, Klaus Kaasgaard, Barbara Karyukin, Comparative Usability Evaluation, Behaviour & Information Technology, v.23 n.1, p.65-74, January-February 2004
• 98. Molich, R., Jeffries, R., & Dumas, J. (2007). Making Usability Recommendations Useful and Usable. Journal of Usability Studies, 2(4), 162-179. Available at ¿http://www. Upassoc.org/upa_publications/jus/2007august/useful-usable.pdf¿.
• 99. John Mueller, Accessibility for Everybody: Understanding the Section 508 Accessibility Requirements, APress L. P., 2003
• 100. Nancarrow, C., & Brace, I. (2000). Saying the "right Thing": Coping with Social Desirability Bias in Marketing Research. Bristol Business School Teaching and Research Review(Summer), 3.
• 101. Jakob Nielsen, Usability Engineering, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 102. Nielsen, J. (2000). Why You Only Need to Test with 5 Users. AlertBox, March 19. Available at ¿http://www.useit.com/alertbox/20000319.html¿.
• 103. Nielsen, J. (2001). Beyond Accessibility: Treating Users with Disabilities As People. AlertBox, November 11, 2001. Retrieved on 4/18/2013, from ¿http://www.nngroup.com/articles/beyond-accessibility-treating-users-with-disabilities-as-people/¿.
• 104. Nielsen, J. (2005). Medical Usability: How to Kill Patients through Bad Design, Alertbox, April 11, 2005 ¿http://www.nngroup.com/articles/medical-usability/¿.
• 105. Nielsen, J., Berger, J., Gilutz, S., & Whitenton, K. (2008). Return on Investment (ROI) for Usability (4th Ed). Freemont, CA: Nielsen Norman Group.
• 106. Jakob Nielsen, Thomas K. Landauer, A Mathematical Model of the Finding of Usability Problems, Proceedings of the INTERCHI '93 Conference on Human Factors in Computing Systems, p.206-213, May 1993, Amsterdam, The Netherlands
• 107. Mie Nørgaard, Kasper Hornbæk, What Do Usability Evaluators Do in Practice?: An Explorative Study of Think-aloud Testing, Proceedings of the 6th Conference on Designing Interactive Systems, June 26-28, 2006, University Park, PA, USA
• 108. Osgood, C. E., Suci, G., & Tannenbaum, P. (1957). The Measurement of Meaning. Urbana, IL: University of Illinois Press.
• 109. Otter, M., & Johnson, H. (2000). Lost in Hyperspace: Metrics and Mental Models. Interacting with Computers, 13, 1-40.
• 110. Helen Petrie, John Precious, Measuring User Experience of Websites: Think Aloud Protocols and An Emotion Word Prompt List, CHI '10 Extended Abstracts on Human Factors in Computing Systems, April 10-15, 2010, Atlanta, Georgia, USA
• 111. Reichheld, F. F. (2003). One Number You Need to Grow. Harvard Business Review, December 2003.
• 112. Carson Jonathan Reynolds, Rosalind Picard, Adversarial Uses of Affective Computing and Ethical Implications, Massachusetts Institute of Technology, Cambridge, MA, 2005
• 113. Sangster, R. L., Willits, F. K., Saltiel, J., Lorenz, F. O., & Rockwood, T. H. (2001). The Effects of Numerical Labels on Response Scales. Retrieved on 3/30/2013 from ¿http://www.bls. Gov/osmr/pdf/st010120.pdf¿.
• 114. Jeff Sauro, Estimating Productivity: Composite Operators for Keystroke Level Modeling, Proceedings of the 13th International Conference on Human-Computer Interaction. Part I: New Trends, July 19-24, 2009, San Diego, CA
• 115. Sauro, J. (2010). Does Better Usability Increase Customer Loyalty? The Net Promoter Score and the System Usability Scale (SUS). Retrieved on 4/1/2013 from ¿http://www.measuringusability. Com/usability-loyalty.php¿.
• 116. Jeff Sauro, Joseph S. Dumas, Comparison of Three One-question, Post-task Usability Questionnaires, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 04-09, 2009, Boston, MA, USA
• 117. Jeff Sauro, Erika Kindlund, A Method to Standardize Usability Metrics Into a Single Score, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, April 02-07, 2005, Portland, Oregon, USA
• 118. Sauro, J., & Lewis, J. (2005). Estimating Completion Rates from Small Samples Using Binomial Confidence Intervals: Comparisons and Recommendations. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Orlando, FL.
• 119. Jeff Sauro, James R. Lewis, When Designing Usability Questionnaires, Does It Hurt to Be Positive?, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, May 07-12, 2011, Vancouver, BC, Canada
• 120. Schwarz, N., Knäuper, B., Hippler, H. J., Noelle-Neumann, E., & Clark, F. (1991). Rating Scales: Numeric Values May Change the Meaning of Scale Labels. Public Opinion Quarterly, 55, 570-582.
• 121. Section 508. (1998). Workforce Investment Act of 1998, Pub. L. No. 105-220, 112 Stat. 936 (August 7). Codified at 29 U. S.C. § 794d.
• 122. Shaikh, A., Baker, J., & Russell, M. (2004). What's the Skinny on Weight Loss Websites? Usability News, 6.1, 2004. Available at ¿http://psychology.wichita.edu/surl/usabilitynews/61/diet_domain.htm¿.
• 123. Pauline A. Smith, Towards a Practical Measure of Hypertext Usability, Interacting with Computers, v.8 n.4, p.365-381, December, 1996
• 124. Snyder, C. (2006). Bias in Usability Testing. Boston Mini-UPA Conference, March 3, Natick, MA.
• 125. Pedro Sostre, Jennifer LeClaire, Web Analytics For Dummies (For Dummies (Computers)), For Dummies, 2007
• 126. Spencer, D. (2009). Card Sorting: Designing Usable Categories. Brooklyn, NY: Rosenfeld Media.
• 127. Jared Spool, Will Schroeder, Testing Web Sites: Five Users is Nowhere Near Enough, CHI '01 Extended Abstracts on Human Factors in Computing Systems, March 31-April 05, 2001, Seattle, Washington
• 128. Stover, A., Coyne, K., & Nielsen, J. (2002). Designing Usable Site Maps for Websites. Available from ¿http://www.nngroup.com/reports/sitemaps/¿.
• 129. Diane Tang, Ashish Agarwal, Deirdre O'Brien, Mike Meyer, Overlapping Experiment Infrastructure: More, Better, Faster Experimentation, Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, July 25-28, 2010, Washington, DC, USA
• 130. Ross Teague, Katherine De Jesus, Marcos Nunes Ueno, Concurrent Vs. Post-task Usability Test Ratings, CHI '01 Extended Abstracts on Human Factors in Computing Systems, March 31-April 05, 2001, Seattle, Washington
• 131. Ross Teague, Katherine De Jesus, Marcos Nunes Ueno, Concurrent Vs. Post-task Usability Test Ratings, CHI '01 Extended Abstracts on Human Factors in Computing Systems, March 31-April 05, 2001, Seattle, Washington
• 132. Tedesco, D., & Tullis, T. (2006). A Comparison of Methods for Eliciting Post-task Subjective Ratings in Usability Testing. Usability Professionals Association (UPA) 2006 Annual Conference, Broomfield, CO, June 12-16.
• 133. Trimmel, M., Meixner-Pendleton, M., & Haring, S. (2003). Stress Response Caused by System Response Time When Searching for Information on the Internet: Psychophysiology in Ergonomics. Human Factors, 45(4), 615-621.
• 134. Edward Tufte, Envisioning Information, Graphics Press, Cheshire, CT, 1990
• 135. Edward R. Tufte, Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire, CT, 1997
• 136. Tufte, E. R. (2001). The Visual Display of Quantitative Information (2nd Ed). Chesire, CT: Graphics Press.
• 137. Edward R. Tufte, Beautiful Evidence, Graphis Pr, 2006
• 138. Thomas S. Tullis, Designing a Menu-based Interface to An Operating System, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p.79-84, April 1985, San Francisco, California, USA
• 139. Thomas S. Tullis, A Method for Evaluating Web Page Design Concepts, CHI 98 Cconference Summary on Human Factors in Computing Systems, p.323-324, April 18-23, 1998, Los Angeles, California, USA
• 140. Tullis, T. S. (2007). Using Closed Card-sorting to Evaluate Information Architectures. Usability Professionals Association (UPA) 2007 Conference, Austin, TX. Retrieved on 4/18/2013 from ¿http://www.eastonmass.net/tullis/presentations/ClosedCardSorting.pdf¿.
• 141. Tullis, T. S. (2008a). SUS Scores from 129 Conditions in 50 Studies. Retrieved on 3/30/2013 from ¿http://www.measuringux.com/SUS-scores.xls¿.
• 142. Tullis, T. S. (2008b). Results of Online Usability Study of Apollo Program Websites. ¿http://www. Measuringux.com/apollo/¿.
• 143. Tullis, T. S. (2011). Worst Usability Issue. Posted July 4, 2011. ¿http://www.measuringux. Com/WorstUsabilityIssue/¿.
• 144. Tullis, T. S., Mangan, E. C., & Rosenbaum, R. (2007). An Empirical Comparison of on-screen Keyboards. Human Factors and Ergonomics Society 51st Annual Meeting, October 1-5, Baltimore. Available from ¿http://www.measuringux.com/OnScreenKeyboards/index.htm¿.
• 145. Tullis, T. S., & Stetson, J.. (2004). A Comparison of Questionnaires for Assessing Website Usability. Usability Professionals Association (UPA) 2004 Conference, June 7-11, Minneapolis, MN. Paper Available from ¿http://home.comcast.net/~tomtullis/publications/UPA2004TullisStetson.pdf¿. Slides: ¿http://www.upassoc.org/usability_ Resources/conference/2004/UPA-2004-TullisStetson.pdf¿.
• 146. Tullis, T. S., & Tullis, C. (2007). Statistical Analyses of E-commerce Websites: Can a Site Be Usable and Beautiful? Proceedings of HCI International 2007 Conference, Beijing, China.
• 147. Tullis, T. S., & Wood, L. (2004). How Many Users Are Enough for a Card-sorting Study? Proceedings of Usability Professionals Association Conference, June 7-11, Minneapolis, MN. Available from http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf.
• 148. Van Den Haak, M. J., De Jong, M. D. T., & Schellens, P. J. (2004). Employing Think-aloud Protocols and Constructive Interaction to Test the Usability of Online Library Catalogues: A Methodological Comparison. Interacting with Computers, 16, 1153-1170.
• 149. Vermeern, A., Van Kesteren, I., & Bekker, M. (2003). Measuring the Evaluator Effect in User Testing. In M. Rauterber Et Al. (Eds.), Human-computer Interaction-INTERACT'03. Pp. 647-654. Published by IOS Press, (c)IFIP.
• 150. Robert A. Virzi, Refining the Test Phase of Usability Evaluation: How Many Subjects is Enough?, Human Factors, v.34 n.4, p.457-468, Aug. 1992
• 151. Vividence Corp. (2001). Moving on Up: Move.com Improves Customer Experience. Retrieved October 15, 2001, from ¿http://www.vividence.com/public/solutions/our+clients/success+stories/movecom.htm¿.
• 152. R. D. Ward, P. H. Marsden, Physiological Responses to Different WEB Page Designs, International Journal of Human-Computer Studies, v.59 n.1-2, p.199-212, July 2003
• 153. Chauncey Wilson, Kara Pernice Coyne, The Whiteboard: Tracking Usability Issues: To Bug Or Not to Bug?, Interactions, v.8 n.3, p.15-19, May/June 2001
• 154. Withrow, J., Brinck, T., & Speredelozzi, A. (2000). Comparative Usability Evaluation for An E-government Portal. Diamond Bullet Design Report, #U1-00-2, Ann Arbor, MI., December. Available at ¿http://www.simplytom.com/research/U1-00-2-egovportal.pdf¿.
• 155. Wixon, D., & Jones, S. (1992). Usability for Fun and Profit: A Case Study of the Design of DEC RALLY, Version 2. Digital Equipment Corporation.
• 156. Wong, D. (2010). The Wall Street Journal Guide to Information Graphics: The Do's and Don'ts of Presenting Data, Facts, and Figures. New York, NY: W. W. Norton & Company.
• 157. Woolrych, A., & Cockton, G. (2001). Why and When Five Test Users Aren't Enough. In Proceedings of IHM-HCI2001, 2, Pp. 105-108. Toulouse, France: Cépadue's-Editions.

}};

	Author	volume	Date Value	title	type	journal	titleUrl	doi	note	year
2013 MeasuringtheUserExperienceSecon	Thomas Tullis William Albert			Measuring the User Experience, Second Edition: Collecting, Analyzing, and Presenting Usability Metrics						2013