-
Notifications
You must be signed in to change notification settings - Fork 0
/
r7.html
56 lines (51 loc) · 10.1 KB
/
r7.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>Katie Soldau - Reading 7</title>
<link href="style.css" rel="stylesheet" type="text/css" />
</head>
<body class="page_readings">
<div class="container">
<div class="content">
<div id="top"></div>
<div class="name">
<p>K<span class="smaller">ATIE</span> <span class= "taller">S</span><span class="smaller">OLDAU - IS4300</span> </p>
<p class="email"> [email protected] </p>
</div>
<div class="navigation_bar_container">
<div class="navigation_bar">
<ul class="nav_options">
<li><a href="index.html">about</a></li>
<li><a href="readings.html" class="current">readings</a></li>
<li><a href="homework.html">homework</a></li>
<li><a href="team_project.html">team project</a>
<li>
</ul>
</div>
</div>
<!-- for navigation -->
<div class="readings">
<!-- R2 -->
<div>
<h1 >
<div class="h1_text">Reading 7 -- Collecting and Analyzing Data </div>
</h1>
<p>
The first article, <a href="http://www.usability.gov/templates/docs/short_test_rep.doc" class="citation">The Usability Report Template</a>, from Usability.gov gives a description of how to write a usability report. It says that at the beginning of the report you should list the date of the report, date of the test, location of the test, information about who it was prepared for, and information about who it was prepared by. Then, it said, an Executive Summary should be written where the main goal and rationale of the study are described. Overall trends should also be discussed here and the reader should get an overall impression of what is in the report. The next section is titled Methodology. Here, the people who were tested are described and relevant information about them is presented. Then, what the participants did, how long they met with a study facilitator, and how many tasks they completed is described. Next, the data that was collected is presented. Major Findings and Recommendations is the following section. Major issues are listed, trends and issues found in user testing are spotted, and recommendations are made. This section is just an overview of the major stumbling blocks and is not a list of every single problem. Detailed Findings and Recommendations follows. First, introductory questions and tasks are presented by listing questions and following that with a summary of responses. Second, different scenarios are detailed, along with their findings and recommendations. Lastly, exit questions and user impressions are given. This can be done by putting responses in a table or by giving an overall summary. </p><p>
The second article, <a href="http://cacm.acm.org/blogs/blog-cacm/107125-stats-were-doing-it-wrong/fulltext" class="citation">The Stats: We’re Doing It Wrong article</a>, by Judy Robertson details how researchers typically utilize the wrong statistical techniques to analyze likert-type style questionnaires. The big problems are that a majority of papers use parametric stats and small sample sizes. This is an issue because Likert scales give original data, ex: “strongly agree” is better than “agree, but it’s not interval data because the distance between “agree” and “strongly agree” isn’t necessarily the same distance as between “neutral” and “agree.” When conducting statistical analysis, ordinal data should use non-parametric statistical tests since these do not assume a normal distribution of data. To help solve these issues, there are a number of things a researcher could do. For one, robust modern statistical methods can be utilized. These tests are not vulnerable to problems that classic parametric tests stumble on with real world data and are at least as powerful as classic parametric tests. Effect size can and must also be considered. APA style guidelines require authors to publish significant results and effect size. Effect size is helpful to know because it tells you how big the difference in mean was between groups. However, this can get tricky when reporting with non-parametric data. Still, this is important to report because it’s a valuable statistic that some people say is so important that if you withhold it, it’s like withholding evidence. </p><p>
The third article, <a href="https://www.ccs.neu.edu/course/is4300sp13/ssl/articles/carifio.pdf" class="citation">Resolving the 50-year debate aroundusing and misusing Likert scales</a>, delved into discussion about how to appropriately use and analyze the scales. It said there are two major views. One view, which Jamieson agrees with, is that Likert scales are ordinal in character, produce rank order data, and that they must be analyzed using non-parametric statistics. The other view, which Pell agrees with, is that they are not ordinal, but in fact are interval in nature and so may be analyzed parametrically. It matters whether they are parametric or not because parametric statistics are more sensitive and powerful than non-parametric ones. Parametric statistics are more likely to catch weaker or emerging findings. Based on empirical evidence that’s been gathered, the intervalist view prevails and so it seems that Likert scales may be analyzed parametrically. This means that presenting the means and standard deviations of the data are encouraged and that parametric techniques can be used.
</p><p>
The fourth article, <a href="https://www.ccs.neu.edu/course/is4300sp13/ssl/articles/p373-nielsen.pdf" class="citation">How to Conduct a Heuristic Evaluation</a>, by Jakob Nielsen does exactly as the title says. It details how this type of evaluation involves a small set of evaluators who examine an interface and then judge its compliance with heuristics, or recognized usability principles. He says that heuristic evaluation is a usability engineering method to find usability problems in a user interface. This is a hard job for a single person to do because there are so many aspects to examine. Nielsen recommends that three or five evaluators be used. When multiple evaluators are working on an interface, they inspect it alone then later come together with their findings. In a user test situation, the user’s actions are interpreted by an observer in a test situation to see how those relate to usability issues. However, in a heuristic evaluation session, the evaluator is the one responsible for analyzing the user interface. Also, unlike in traditional user testing, the evaluator’s questions about the domain can be answered and given help once it is clear they are in trouble and have already commented on the usability problem in question. A session for an individual heuristic evaluator normally lasts between one and two hours. During this time, the evaluator goes through the interface several time and inspects different aspects to compare them with a list of recognized usability principles, or heuristics. Since evaluators don’t exactly use the system to perform a real task, it’s possible to do this with paper prototypes. Once a heuristic evaluation is done a list of usability problems is created. Problems are described and noted separately. Although this type of evaluation doesn’t provide fixes, it’s normally easy to create redesigns since each usability problem is explained in reference to usability principles. Heuristic evaluation is very efficient and considered a discount usability engineering. It is not guaranteed to find every usability problem in an interface, but is a good start and worth the time and effort.
</p><p>
The fifth article, <a href="http://www.ccs.neu.edu/course/is4300sp13/ssl/articles/gould.pdf" class="citation">The 1984 Olympic Message System: A Test of Behavioral Principles of System Design</a>, evaluates the 1984 Olympic Message System (OMS) design. OMS was a voice mail system developed according to three behavioral principles. It was built on the IBM Audio Distribution system and allowed Olympians to send and receive voice messages amongst themselves. Three principles of system design were recommended by the paper: early focus on users and tasks, empirical measurement, and iterative design. They were able to follow these principles because they had good tools. The fact that they built something designed for iterative design and that the user interface and function were separated helped as well. Also, the people that worked on the project were critical and were of the right skill mix. This project demonstrated it was possible to successfully follow behavioral principles and that following the principles actually sped up the design process.
</p><p>
In class I’d like to talk more about how heuristic evaluators look at and examine a system. I understand the general idea and know that they’re looking for heuristics, but it might be interesting to go into more specifics. I’d like to see how these evaluators know where to look and exactly what to look for. Practice and repetition must play some role, and being extremely knowledgeable about all of the heuristics must be a must, but I still wonder exactly what they’re investigating. Do they click all the buttons? Do they try to complete various tasks? Do they simply just look at the design? All of the above? I thought the first reading, the one that described the template, looked extremely useful for the occasion where you’d need to write a usability report. A lot of the second and third articles regarding statistics and Likert scales went right over my head because I’ve never really dealt with statistics. I think I would have gotten more out of those articles had I had more prior knowledge. There were also a lot of acronyms that I didn’t know and were never explained which made it confusing. I found the fifth article, the one about OMS, to be the most interesting because it described the necessities of following user based design and certain principles. I found the topic engaging overall, so reading about how they designed and implemented the system was interesting too.
</p>
</div>
<!-- end of R2 -->
<!-- for content -->
</div>
<!-- for container -->
</body>
</html>