Chapter 15: Analytical evaluation Aims: Describe inspection methods. Show how heuristic evaluation can be adapted to evaluate different products. Explain how to do doing heuristic evaluation and walkthroughs. Describe how to perform GOMS and Fitts Law, and when to use them. Discuss the advantages and disadvantages of analytical
evaluation. Inspections Several kinds. Experts use their knowledge of users & technology to review software usability. Expert critiques (crits) can be formal or informal reports. Heuristic evaluation is a review guided by a set of heuristics. Walkthroughs involve stepping through a pre-planned scenario noting potential problems.
Heuristic evaluation Developed Jacob Nielsen in the early 1990s. Based on heuristics distilled from an empirical analysis of 249 usability problems. These heuristics have been revised for current technology. Heuristics being developed for mobile devices, wearables, virtual worlds, etc. Design guidelines form a basis for developing heuristics.
Nielsens heuristics Visibility of system status. Match between system and real world.
User control and freedom. Consistency and standards. Error prevention. Recognition rather than recall. Flexibility and efficiency of use. Aesthetic and minimalist design. Help users recognize, diagnose, recover from errors. Help and documentation. Discount evaluation Heuristic evaluation is referred to as discount evaluation when 5 evaluators
are used. Empirical evidence suggests that on average 5 evaluators identify 75-80% of usability problems. No. of evaluators & problems 3 stages for doing heuristic evaluation Briefing session to tell experts what to do. Evaluation period of 1-2 hours in
which: Each expert works separately; Take one pass to get a feel for the product; Take a second pass to focus on specific features. Debriefing session in which experts work together to prioritize problems. Advantages and problems Few ethical & practical issues to consider because users not involved. Can be difficult & expensive to find
experts. Best experts have knowledge of application domain & users. Biggest problems: Important problems may get missed; Many trivial problems are often identified; Experts have biases. Cognitive walkthroughs Focus on ease of learning. Designer presents an aspect of the design & usage scenarios. Expert is told the assumptions
about user population, context of use, task details. One of more experts walk through the design prototype with the scenario. Experts are guided by 3 questions. The 3 questions Will the correct action be sufficiently evident to the user? Will the user notice that the correct action is available? Will the user associate and
interpret the response from the action correctly? As the experts work through the scenario they note problems. Pluralistic walkthrough Variation on the cognitive walkthrough theme. Performed by a carefully managed team. The panel of experts begins by working separately. Then there is managed discussion that leads to agreed decisions.
The approach lends itself well to participatory design. A project for you http://www.id-book.com/catherb/ provides heuristics and a template so that you can evaluate different kinds of systems. More information about this is provided in the interactivities section of the id-book.com website.
Predictive models Provide a way of evaluating products or designs without directly involving users. Less expensive than user testing. Usefulness limited to systems with predictable tasks - e.g., telephone answering systems, mobiles, cell phones, etc. Based on expert error-free behavior. GOMS
Goals - the state the user wants to achieve e.g., find a website. Operators - the cognitive processes & physical actions needed to attain the goals, e.g., decide which search engine to use. Methods - the procedures for accomplishing the goals, e.g., drag mouse over field, type in keywords, press the go button. Selection rules - decide which method to select when there is more than one. Keystroke level model
GOMS has also been developed to provide a quantitative model - the keystroke level model. The keystroke model allows predictions to be made about how long it takes an expert user to perform a task. Response times for keystroke level operators (Card et al., 1983) Operator K
P P1 H M R(t) Description Pressing a single key or button Average skilled typist (55 wpm) Average non-skilled typist (40 wpm) Pressing shift or control key Typist unfamiliar with the keyboard
Pointing with a mouse or other device on a display to select an object. This value is derived from Fitts Law which is discussed below. Clicking the mouse or similar device Bring home hands on the keyboard or other device Mentally prepare/respond The response time is counted only if it causes the user to wait. Time (sec) 0.22 0.28
0.08 1.20 0.40 0.20 0.40 1.35 t Fitts Law (Fitts, 1954) Fitts Law predicts that the time to
point at an object using a device is a function of the distance from the target object & the objects size. The further away & the smaller the object, the longer the time to locate it and point to it. Fitts Law is useful for evaluating systems for which the time to locate an object is important, e.g., a cell phone, a handheld devices. A project for you Use the web and other resources to
research claims that heuristic evaluation often identifies problems that are not serious and may not even be problems. Decide whether you agree or disagree. Write a brief statement arguing your position. Provide practical evidence & evidence from the literature to support your position. A Project for you Fitts Law Visit Togs website and do
Togs quiz, designed to give you fitts! http://www.asktog.com/ columns/ 022DesignedToGiveFitts.html Key points Expert evaluation: heuristic & walkthroughs. Relatively inexpensive because no users. Heuristic evaluation relatively easy to learn. May miss key problems & identify false ones.
Predictive models are used to evaluate systems with predictable tasks such as telephones. GOMS, Keystroke Level Model, & Fitts Law predict expert, error-free performance.
One member of the pair yields an impartial comparative and the other one a committed comparative. We can't say (John is an excellent tennis player, but he is worse than Tom)because if someone is excellent, he can't be called (worse)...
New elective Units: BSBWHS 407A Assist with claims management, rehabilitation and return-to-work programs p84 BSBWHS 408A Assist with effective WHS management of contractors p87 BSBWHS 410A Contribute to work-related health and safety measures and initiatives p94 Overview of WHS Units...
C. Strongly recommended for implementation and supported by some experimental, clinical, or epidemiologic studies, and by a strong theoretical rationale. - This is the definition of a Category 1B. D. Strongly recommended for implementation and strongly supported by well-designed experimental,...
EE C245 - ME C218 Introduction to MEMS Design Fall 2003 Roger Howe and Thara Srinivasan Lecture 1 Course Overview Lecture 1 Introduction to MEMS Lectures 2-4 Microfabrication Fundamentals Lectures 5-13 Forces, Mechanics, and Transduction Lectures 14-18 Microsystem Fabrication Processes...
Otter Creek. Using Otter Creek model (2015) as an example for this presentation. There are currently other EC/SAR. modeling efforts in progress. Both other EC/SAR models being used for salinity, but important to note we also use models for this...
The Children & Families Act 2014. Services across education, health and care to be jointly commissioned. Education Health and Care plans (EHC plans) to replace statements & Learning Difficulty Assessments (LDAs) with the option of a Personal Budget for families...
Post-Colonial Literary Theory (lens)Until lions tell their stories, tales of hunting will glorify the hunter. A note about lenses: No single lens always gives the clearest view; all have strengths and weaknesses. Applying different lenses to the same text can...
Ready to download the document? Go ahead and hit continue!