Advanced Methods in HCI

Spring 2019

*Note that the course schedule is tentative.* Lecture slides will be posted after each class.

COMPSCI 690A, 3 Credits, Spring 2019, Time: TueThu 10:00am – 11:15am

Location: Room 140, Computer Science Building

Instructor: Narges Mahyar
Office: Room 322, Computer Science Building
Office Hours: Thursdays 3:20-4:20 pm (* no office hours on Feb 21)

TA: Pooya Khaloo
Office hours: Weds 1-2:30pm, CS 207


This is an advanced course in Human-Computer Interaction. This course will provide a deeper treatment of some topics that are typically found in an undergraduate HCI course. For example, design methodologies, evaluation methodologies (both quantitative and qualitative), human information processing, cognition, and perception. This course will also introduce students to research frontiers in HCI. The course will cover topics of Universal Usability, CSCW, Digital Civics and fundamentals of designing interactive technology for people.

Course Overview
People are increasingly surrounded by interactive computational technology systems that are integral to their everyday life. However, poorly designed systems are common, and they can lead to negative outcomes such as frustration, lost time, and errors. The role of design is more crucial than ever before for crafting appropriate systems that truly meet people’s needs, abilities, and expectations. This course covers the theories and concepts important for all professionals and researchers that design interactive technology for human use. This course will build common ground across students from a range of backgrounds, so they will have a shared vocabulary and methods to bring into other components of the Designing for People. Designing for People means designing for human experience, abilities, and fallibilities, which requires in-depth engagement of people throughout the design process in order to develop interactive technologies that fit human needs and capabilities. More specifically, the course adopts a human-centered design (HCD) approach and teaches a highly iterative process called design thinking. This process draws heavily on fundamental human-computer interaction (HCI) methods. Students will have a chance to practice and hone their abilities through weekly homework in the context of a project, in-class activities, and discussions.

Course Origins
This course was originally developed and taught by Prof. Joanna McGerener and Dr. Leila Aflattony at University of British Columbia  (UBC) as a new graduate course in HCI on the Fundamentals of Designing Interactive Computational Technology for People (DFP). The course draws on Prof. McGerener’s many years of teaching HCI courses at UBC and also borrows materials from Prof. Karon MacLean and Jessica Dawson.

Survey and research articles will be the primary text for the course, chosen from a collection of readings. There is no textbook required.

Students are expected to have taken an HCI course prior to taking this course. While there are no other formal prerequisites, the ability to do basic computer programming will be an asset for the prototyping part of the course. Alternate tools that require minimal programming will, however, be possible. Further, there will be some coverage of experimental design and analysis, which relies on some basic statistical knowledge.

University policies and information

Accommodation Statement
The University of Massachusetts Amherst is committed to providing an equal educational opportunity for all students. If you have a documented physical, psychological, or learning disability on file with Disability Services (DS), you may be eligible for reasonable academic accommodations to help you succeed in this course. If you have a documented disability that requires an accommodation, please notify me within the first two weeks of the semester so that we may make appropriate arrangements.

Academic Honesty Statement
Since the integrity of the academic enterprise of any institution of higher education requires honesty in scholarship and research, academic honesty is required of all students at the University of Massachusetts Amherst. Academic dishonesty is prohibited in all programs of the University. Academic dishonesty includes but is not limited to: cheating, fabrication, plagiarism, and facilitating dishonesty. Appropriate sanctions may be imposed on any student who has committed an act of academic dishonesty. Instructors should take reasonable steps to address academic misconduct. Any person who has reason to believe that a student has committed academic dishonesty should bring such information to the attention of the appropriate course instructor as soon as possible. Instances of academic dishonesty not related to a specific course should be brought to the attention of the appropriate department Head or Chair. Since students are expected to be familiar with this policy and the commonly accepted standards of academic integrity, ignorance of such standards is not normally sufficient evidence of lack of intent (

Inclusive Statemnet
In this course, each voice in the classroom has something of value to contribute. Please take care to respect the different experiences, beliefs, and values expressed by the students, faculty, and staff involved in this course. My colleagues and I support UMass’s commitment to diversity, and welcome individuals regardless of age, background, citizenship, disability, sex, education, ethnicity, family status, gender, gender identity, geographical origin, language, military experience, political views, race, religion, sexual orientation, socioeconomic status, and work experience (

WeekDateTopics and ReadingsLecturesDeliverables
1 Tues- Jan 22

Design Thinking

Norman, Donald A. The Design of Everyday Things. Basic Books, 01/01/2013. Web. 31 Aug. 2017.

Reading #1. Chapter 1. The psychopathology of everyday things.

Reading #2. Chapter 6. Design Thinking. [Read p. 217-236]

Lecture 1Research Journal
 Thurs- Jan 24

Human-centered design and HCI

Reading #3. Mackay, W. E. (1995). Ethics, lies and videotape…. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Denver, Colorado, United States, May 07 – 11, 1995). CHI ’95. ACM, New York, NY, 138-145. doi pdf

Lecture 2

Research Journal

CITI Training-Group2 (ethics training): Please complete the training and post your certificate/s to Gradescope by Monday (1//28) midnightLink

2 Tues- Jan 29

Field Studies – Observations

Reading #4. Blomberg, J., Burrell, M., and Guest, G. (2003). An ethnographic approach to design. Chapter 50.  In Jacko, J. and Sears, A. (Eds.) The Human Computer Interaction Handbook (pp. 964-986). Mahwah, NJ: Lawrence Erlbaum Associates. [Read up to last column on p. 973] doi  pdf

Reading #5. Porcheron, M., Fischer, J. E., and Sharples, S. (2016) Using mobile phones in pub talk. Proceedings of ACM CSCW ’16, 1649-166. doi pdf

Lecture 3 

Research Journal

Read project overview

Form teams

  Thurs- Jan 31

Field Studies – Interviews

Reading #6. Fontana, A. and James F. (1994). Interviewing: The Art of Science. In Denzin, N. and Lincoln, Y. (Eds.) The Handbook of Qualitative Research (pp. 361-76). Thousand Oaks: Sage Publications. [Read up to first column on p. 368] pdf

Reading #7. Allison Woodruff. (2014). Necessary, unpleasant, and disempowering: reputation management in the internet age. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 149-158. pdf

Lecture 4Research Journal
3 Tues- Feb 5

Field Studies – Surveys/questionnaires 

Reading #8. Hochheiser, H., Feng, J. H., & Lazar, J. (2017). Surveys. Chapter 5. Research methods in human computer interaction (pp. 109-133). Elsevier Science. [Read: p.105-128; skip examples if needed] 

Reading #9. Malin Eiband, Mohamed Khamis, Emanuel von Zezschwitz, Heinrich Hussmann, and Florian Alt. 2017. Understanding Shoulder Surfing in the Wild: Stories from Users and Observers. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA, 4254-4265. doi pdf

Lecture 5

Research Journal

Team Contract

First interim-milestone: unstructured observation in public place

  Thurs- Feb7

Qualitative Data Analysis

Reading #10. Guest, G., MacQueen, K. M. & Namey, E. E. (2012). Applied Thematic Analysis. Chapter 1. Introduction to Applied thematic analysis (pp.2-21). Thousand Oaks, CA: SAGE Publications Ltd. [Read p. 7-18] doi pdf

Reading #11. Guest, G., MacQueen, K. M. & Namey, E. E. (2012). Themes and Codes. Chapter 3Applied thematic analysis (pp.49-78). Thousand Oaks, CA: SAGE Publications Ltd. 

Optional Reading. Virginia Braun & Victoria Clarke (2006) Using thematic analysis in psychology, Qualitative Research in Psychology, 3:2, 77-101. pdf link  

Lecture 6Research Journal
 4 Tues- Feb 12

Affinity Diagrams

Reading #12. Holtzblatt, K., and Beyer, H. (2017). Contextual Design: Design for Life. Chapter 6The Affinity Diagram (pp. 127-146). Elsevier Inc. pdf

Reading #13. Nouwens, M., Griggio, C., and Mackay, W., (2017). WhatsApp is for family; Messenger is for friends”:Communication Places in App Ecosystems. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA, 727-735.  doi pdf

Reading #13 (alternative). Vitale, F., Janzen, I., and McGrenere, J. (2018). Hoarding and Minimalism: Tendencies in Digital Data Preservation. Proceedings of the CHI 2018. Paper 587, 12 pages. pdf doi

Lecture 7 Research Journal
  Thurs- Feb 14Presentation (Empathize)
No Reading
 Project Milestone-Empathize 
5 Tues- Feb 19

No class-Monday Class Schedule

  Thurs- Feb 21


Reading #14. Cooper, Alan, et al. (2014) About Face : The Essentials of Interaction Design, Chapter 3Modeling Users: Personas and Goal (pp. 61-99) John Wiley & Sons. [Read p. 81-97] pdf

Reading #15. John Pruitt, Jonathan Grudin (2003). Personas: Practice and Theory. In Proceedings of the 2003 CHI Conference on Human Factors in Computing Systems (CHI ’03). ACM, New York, NY, USA, 1-15. doi pdf

Optional Reading. Lene Nielsen, Kira Storgaard Hansen. (2014). Personas is Applicable – A Study on the Use of Personas In Denmark. In Proceedings of the 2014 CHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 1665-1674. doi pdf

Optional Reading. Tara Matthews, Tejinder Judge, Stephen Whittaker (2012). How Do Designers and User Experience Professionals Actually Perceive and Use Personas? In Proceedings of the 2012 CHI Conference on Human Factors in Computing Systems (CHI ’14). ACM, New York, NY, USA, 1219-1228. doi pdf

Lecture 8 
 6 Tues- Feb 26

Working Class

No Reading

  Thurs- Feb 28

 Establishing Requirements

Reading #16. Rogers, Y., Sharp, H. Preece, J. (2011) Interaction Design: Beyond Human-Computer Interaction. Chapter 10Task Description, Task Analysis (pp.373-388). Chichester, West Sussex : Wiley. [skip activities/assignments] pdf  Link

Optional Reading. Task Examples. [Recommended to read] Link

Lecture 9 Project Milestone-Define  
 7 Tues- March 5

Conceptual Models and Design

Reading #17. Jeff Johnson and Austin Henderson. (2002). Conceptual models: begin by designing what to design. Interactions 9, 1 (January 2002), 25-32. doi pdf

Reading #18. Greenberg, S., and Buxton, B. (2008). Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the 2008 CHI Conference on Human Factors in Computing Systems (CHI ’08). ACM, New York, NY, USA, 111-120. doi pdf

***Bonus reading (optional)Supporting Communication and Coordination in Collaborative Sensemaking. Narges Mahyar, and Melanie Tory. IEEE Transactions on Visualization and Computer Graphics (VAST 2014), pp. 1633-1642, 2014. [Paper] [Video]

Lecture 10

  Thurs- March 7

Human Abilities and Sketching

Reading #19. Buxton, B. (2007). Sketching user experiences: Getting the design right and the right design. Chapter 13-17. Sketching interaction (pp.135-155), Morgan Kaufmann Publishers Inc.  [Read p. 135-151] Link pdf

Reading #20. Greenberg, S. (2011). Sketching user experiences: The workbook, Section 4. Snapshots in Time: The Visual Narrative (pp. 145-177), Morgan Kaufmann Publishers Inc. Link pdf

Lecture 11 
 8 Tues- March 12No Class-Spring Recess  
 Thurs- March 14No Class-Spring Recess  

 Tues- March 19

 Presentation (Ideate) 

***Bonus reading (optional)CommunityCrit: Inviting the Public to Improve and Evaluate Urban Design Ideas through Micro-ActivitiesNarges Mahyar, Michael R. James, Michelle M. Ng, Reginal. A. Wu, Steven P. Dow,  ACM Human Factors in Computing Systems (CHI 2018), 14 pages. [Paper]

 Project Milestone-Ideate 
  Thurs- March 21


Reading #21. Buxton, B. (2007). Sketching user experiences: Getting the design right and the right design. Chapter 35Interacting with Paper (pp.371-391), Morgan Kaufmann Publishers Inc. [Read p. 381-391; skim-through the rest]  Link pdf

Reading #22. Tohidi, M., Buxton, W., Baecker, R., Sellen, A. (2006). Getting the right design and the design right. In Proceedings of the 2006 CHI Conference on Human Factors in Computing Systems (CHI ‘6). ACM, New York, NY, USA, 1243-1252. doi pdf

Lecture 12 
 10 Tues- March 26

Evaluation of Prototypes – Discount Methods

Reading #23. Wilson, C. (2013). User Interface Inspection Methods, Chapter 4.cognitive Walkthrough (pp. 65-79), Elsevier Inc. [Read p. 66-74] Link pdf

Reading #24. Wilson, C. (2013). User Interface Inspection Methods, Chapter 1. Heuristic Evaluation (pp. 1-31), Elsevier Inc.  [Read p. 14-29] Link pdf

Lecture 13 
  Thurs- March 28

Evaluation of Prototypes – Usability Testing

Reading # 25. Dix, A. et al. (2004). Human-Computer Interaction, Chapter 9. Evaluation techniques (pp. 318-364), Pearson. [Read: p.343-362] pdf

Reading # 26. Oram, L., McLean, K., Kruchten, P., Forster, B. (2014). Crafting Diversity in Radiology Image Stack Scrolling: Control and Annotations. In Proceedings of the 2014 DIS Conference on Designing interactive systems (DIS ’14). ACM, New York, NY, USA, 567-576. pdf

Lecture 14

CLIP evaluation

 11 Tues- April 2

Experiments I – Experimental Design

Reading #27. Hochheiser, H., Feng, J. H., & Lazar, J. (2017). Experimental research. Chapter 2Research methods in human computer interaction (pp. 25-44). Elsevier Science. [Read: p.25-41]  

Reading #28. Hochheiser, H., Feng, J. H., & Lazar, J. (2017). Experimental design. Chapter 3Research methods in human computer interaction (pp. 45-69). Elsevier Science.  [Read: p.45-67] 

Lecture 15 Project Milestone-Prototype  
  Thurs- April 4

 Working Class and Prototype Review

***Bonus reading (optional)UD Co-Spaces: A Table-Centred Multi-Display Environment for Public Engagement in Urban Design Charrettes. Narges Mahyar, Kelly Burke, Siyi Meng, Jialiang Xiang, Kellogg S. Booth, Cynthia Girling, and Ronald Kellett, Interactive Surfaces and Spaces (ISS ’16), pp. 109-118, November 6-9, 2016. [Paper]

 12 Tues- April 9 Guest Lecture-Pooya Khaloo Interim-milestone: Proposed goal(s) of experiment 
  Thurs- April 11

 Experiments II – Statistical Analysis

Reading #29. Hochheiser, H., Feng, J. H., & Lazar, J. (2017). Statistical analysis. Chapter 4Research methods in human computer interaction (pp. 71-104). Elsevier Science. 

Lecture 16Project Milestone-Test 1 
 13 Tues- April 16

Working Class and Prototype Review

Reading #30. Findlater, L., Moffatt, K., McGrenere, J., and Dawson, J. (2009). Ephemeral adaptation: the use of gradual onset to improve menu selection performance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’09). ACM, New York, NY, USA, 1655-1664. DOI pdf

  Thurs- April 18

Presentation (Test 1) 

No Reading

 14  Tues- April 23

Experiments III – Threats to Validity

Reading #31. Baecker, R. Grudin, J., Buxton, W., Greenberg, S., (1995) Methodology Matters: Doing research in the behavioral and social sciences. Human Computer Interaction: Toward the year 2000 (pp.152-169), Morgan Kaufmann. pdf

Lecture 17 
  Thurs- April 25

 Working Class and Final Prototype Review

No Reading

 Project Milestone-Test 2
15  Tues- April 30


No Reading

 Project Milestone-Test 3

Researcher journals are an important part of the deliverables in this course.  

Instructions on how to post your entries in your researcher journal are found further below.

Journal entry: Reflections on the assigned readings

Due: by 9:00 PM the night before readings are covered in class. This is a strict cutoff. (Note: we reserve the right to change this time within the term)

Students must do the core readings before the classes. By the due date/time, students must create an entry in their shared journal with a question/comment about the material being covered that day, one note per required reading (typically 2 per class, but can be more or less). We encourage you to also to have a copy of these questions/comments with you in class and use them as a springboard for discussion. 

Your questions/comments should be thoughtful, and clearly show that you have done the reading and reflected on it. They do not all have to be phrased in the form of a question, a comment is fine. If you are genuinely confused by some aspect of the reading, then it is useful and legitimate to ask for clarification. However, simply asking something that you could trivially look up yourself is not a good question. As with any written work that you submit, we expect correct grammar and spelling.

The goal of the shared journal is not only for you to reflect on the readings yourself, but also to learn about your classmates reflections on those same readings. For this reason, the journal entries are open to the full class. This will lead to richer in-class discussions.

Course staff will grade some of your questions/comments (random selection). Those that are graded will be graded on a 3-point scale of {good-3, okay-2, poor-1}, and we will sometimes provide feedback, especially at the outset of term. 

Each question/comment should be at most one short to medium length paragraph. Often two or three sentences will suffice. The goal is to have short, crisp questions/comments. 

We will drop each student’s lowest 3 grades. Thus, you can essentially choose not to do this assignment for 3 of the readings. Choose these wisely.

Unless otherwise specifically arranged with the course staff, your questions/comments will not be graded after the due date/time (as timestamped on the journal entry).

Submitting your Researcher Journal entries

Researcher Journals are found as a module in Pizza. Each reading is set up as a Discussion item with the label that matches the reading number on the course schedule page. For example, to submit your journal entry for the first reading simply “Reply” to the discussion topic “Reading #1”.

Four important things to note:

  1. The researcher journals are open/visible to the entire class. Thus, all students in the class as well as the instructors will be able to see your entry (but only grading for their own entries).
  2. You will not be able to see any entries for a given reading from other students until you have posted/submitted your own entry for that reading.
  3. You are not able to edit an entry, after submission.
  4. You can optionally reply directly to other students’ entries — again, your reply will be visible to all.

Tab Content

Project title: Designing a Human-centred Interactive Computational Technology


Throughout the course, we will explore and apply different methods that are appropriate for designing and evaluating an interactive computational technology that closely meets human needs. Examples of potential technologies are interactive 2D interfaces, interactive 3D devices, wearables, robots, and so on. Your team will choose a topic from the list provided below which will seed your project. You will identify a clear problem to be addressed (or potential design opportunity) by investigating people’s behaviors, activities, and interactions, and will then create a working prototype that meets their needs, which you will evaluate. Following the design thinking process, there are approximately 6 project milestones, which may include a small final design showcase. These will be scheduled approximately every other week and there will be deliverables for each. For approximately half of these milestones, your team will need to be prepared to “present” your work during class time in the form of a “design crit” for which you will be marked and receive constructive feedback. Meeting the project milestones is crucial in completing the project successfully. 

Team formation:  

The first step in the research project is to form a team with 3 other people. There is not a lot of time for this, so you will need to move fast. You are free to work with whomever you choose, but you should strive for as multi-disciplinary a team as possible. Your team will choose a project topic listed in the project description. Groups will be set up for each team in Piazza. Once your team is formed, you will need to complete a team contract and submit it to gradescope (pdf). The due dates for these steps are noted in deliverables on the Schedule page.


Please see course schedule for tentative dates for each milestone. A draft outline of what might be required for each of the milestones is given below. These will be refined and provided to you as you approach each milestone.

  1. Empathize You will employ different data collection techniques (interview, observation, and questionnaire) to gather data around your chosen topic and will synthesize the ethnographic data and preliminary findings. You will need to transcribe the data, highlight the key findings, and submit them as part of the deliverable. 
  2. Define In this stage, you will craft a meaningful and actionable problem statement or design focus through analyzing of the information gathered about user needs and context. In addition, you will develop a persona (or personas), which is a model of a user that focuses on the individual’s characteristics and goals when using an artifact. The personas should be based on thoughtful analysis of data you’ve collected through research you’ve completed with your participant groups.
  3. Ideate You will develop a conceptual design of your potential interactive computational system, considering your participant group requirements. You need to submit a design requirement document with detailed description of a system to be developed. This stage provides source material for building prototypes and innovative solution to the problem. Ideation is about incorporating volume and variety in concept generation through visual representations. So you need to sketch some ideas that represent the interactive computational system visually.
  4. Prototype Prototype creation requires an iterative process and can be created for the early exploration phase (low-fidelity artifact) or the final phase (high-fidelity artifact). At this stage, you will create a working prototype or prototypes (first iteration) of a computational technology according to your concept. The prototype(s) of your concept needs to detail how the concept will be experienced and used. The medium of the mock-up depends on the solution and may show an interface or a physical/tangible 3D mock-up. 
  5. Test There are two main parts to the Test milestone. Part I is done with your team, and Part II is done individually. Part I: You will test your prototype(s) with participants who are representative of the group of people you are designing for. There will be relatively informal usability testing, after which you may improve and refine further the prototype, followed by more formal experimental design, running the experiment, and doing the analysis. Part II: you will individually write up the experiment report, which includes the analysis, discussion and conclusions. 

Grading (tentative)                                                            

Team  Deliverables 1, 2, 3, 4,5 – Part I50%
Individual Deliverable 5 – Part II 20%


Details provided in individual milestone descriptions.


Potential design problems/opportunities/situations, which are deliberately vaguely specified: 

  • Waiting time for paying/ordering food can be long at restaurants.
  • Texting while walking is dangerous, but people cannot stop.
  • Train/metro platforms can be jammed with people. It is frustrating and can be dangerous.
  • Hiking alone in unknown areas can be dangerous. Getting lost is a common problem for solo hikers. 
  • Managing simple physical tasks at home such as turning on/of lights can be challenging for older adults.
    • Need access to older adults
  • Houses can be broken into when no one is at home. Police advise people to make their houses look occupied when they are away.
    • Need access to participants who have homes
  • Babies scream, as they can’t say actual words when they try to get their parents attention. It is challenging for parents to understand their baby’s needs.
    • Need access to participants with babies.
  • Educating children (ages 2-4) can be challenging as they can’t communicate, read and write.
  • Studying effectively has never been more challenging for university students. The demands on their attention have continued to rise.
  • Planning some types of events can be clunky (e.g., potlucks where there is flexibility of date, location, and numbers)
  • Personal safety when walking alone on campus late at night can be a concern.
  • Services like Craigslist work well for resale, but services that support a loaning economy are scarce.
  • Planning a trip among friends and family who are not co-located can be a challenge.
  • Collaborating over files has never been better supported with services such as Google docs, Google Drive, and Dropbox. But these services can make managing files more complex.
  • People have become addicted to their phones. Well-being is being compromised.
  • Children do not play outside as much as they used to, and are therefore not getting the same amount of exercise as they used to.
    • Need access to parents of small children
  • Too much energy is being wasted with lights being left on in homes unnecessarily.
  • Easy access to digital photography and the low cost of storage has led people to lose track of what they capture.
  • Services like Yelp provide crowd-sourced reviews for local business, but reliable services to find and review local musicians and bands are practically non-existent.
  • With the amount of work graduate students do (courses, TAing, research), they generate a lot of files which can be hard to manage.
  • The number of photos people save on their phones is huge, they are generated by several apps (Camera, WhatsApp, Snapchat) and for several purposes (personal, reminders, scans). Phones have some capability to help users manage this  but they don’t really work.
  • Finding a place to rent can be challenging, especially for students who are often new to a city. There are different web portals and also ads spread throughout neighbourhoods. Renters also face stress, potential scams, and may not know if the place is being advertised at a fair price.
  • MOOC courses/e-learning has opened the door for all the students to have a good education. But it’s missing the most critical factor of the educational system: student-teacher real-time interaction. 
  • Some people go to the gym regularly and keep it as a hobby while for others, it can be challenging to make a reasonably flexible workout plan and follow it.
  • Mobile phones are collecting and storing more private user data, sometimes with little awareness on the user’s part. This exposes privacy risk. 
  • Some adults consume a number of over-the-counter and prescription medications; a tool that can assist them in tracking the daily consumption and long-term side effects would be beneficial.
  • New international students can struggle with important non-academic activities such as banking, transportation, and housing. They would benefit from  some guidance.
  • Some people spend a lot of time on international flights and can often get bored, fatigued, and stressed. Airport and in-flight experiences could be made more eventful and less stressful.


  • some of these would be difficult to observe in situ
  • some would involve participants who might be more difficult to access