Don’t just ask a question, then ask the next question. Try and build connection between the questions.
“Following up, instead of asking questions, is a mindset that can really help.” “Trying to ask follow-ups is a way to get out of question-asking and a way to get more connected.”
Are there better ways to ask questions?
- Gathering context and collecting details
- Probe for what has been unsaid
- Questions to build for mental models.
There are a bunch of different questions types for each of these topics. Check out Interviewing Users for suggestions.
Ask follow-up questions!! Never expect a question to generate the answer. “You can’t come in with the perfect question. You need to adapt … know how to follow up.”
What are the top three biggest mistakes that you’ve seen companies make when conducting user research?
- Where people put the answer in the question. What are the things you like to do when you go out to the movies? Do you like the food, catching up with your partner, hanging out with your friends … etc.
People want to do a really good job for you during an interview. “It’s even worse than leading questions. You’re handing them the answers”.
Be ok with silence. Ask the question and then do a hard stop. Sit with the pause.
- Capture audio, don’t take notes and just rely on those notes. People try to take notes in the field. You can’t capture what someone says. “We end up with these fairly biased documents”. Capturing the audio is preferable.
- Debriefing instead of analysing. Go back to them with an analytical mindset during the analysis phase. The debrief process is important to do. But then in the analysis session go back to the data. “The debrief is necessary but it’s not sufficient”.
What’s the one thing that made me successful with what you’re doing today?
Steve kind of hates this question, and talks about why he’s struggling to answer this question.
A solid piece, with evidence and arguments against using focus groups for product design exploration/validation.
My notes from chapter 2 of Practical Ethnography: A Guide to Doing Ethnography in the Private Sector, by Sam Ladner.
“Becoming a good private-sector ethnographer means you must understand your research method and be able to explain it to your clients and stakeholders. Practically speaking, this means you must know how to do ethnography but also how to think about ethnography.”
What is truth, anyway?
In the private sector truth is seen to reside in facts. This is a ‘positivist’ or ‘factist’ perspective about truth. There are generally numbers involved in these facts. The factist approach also usually assumes that future events can be predicted. In a corporate environment you’re going to find yourself surrounding by positivists.
Ethnographers start with an ‘interpretivist’ view of truth. Interpretivists seek to understand how people ascribe meaning to their world. Sociologist Max Weber used the German word verstehen to describe this goal – “to understand a single phenomena as representative or indicative of a wider system”.
Anthropologist Clifford Geertz argues that culture is “the meaning people ascribe to:
- and institutions.”
It’s the ethnographer’s job to uncover these meanings. For the people we are studying, truth lies in these perceptions, not in the absolutes.
My notes: When we talk about user research, or we’re reading articles about qualitative user research methods etc, we’ll often hit some version of this statement: “qualitative research helps us understand the why”. These theoretical approaches gives us the chance to frame those whys through different lenses.
Our identities and the system of meaning in which we find ourselves creates our truth
“A product’s meaning is a function of a consumer’s perception of two broad concepts: 1) his own identity; and 2) the system of meaning in which he finds himself.”
Identities are not fixed. They’re often in flux. What does it mean to be a ‘woman’ or an ‘Asian’? These aren’t really categories that determine our behaviour. They’re roles that we, individually, interpret. Sam Ladner gives a couple of examples here: not all women wear make-up, not all Asian people living in a western country will speak a language other than English.
My notes: You often hear certain expected behaviour being ascribed to a specific identity. One example that drives me nuts is the ‘your mum’ identity when talking about technology. “Imagine your mum trying to use this product”. (Well, my mum works for NASA so she’d probably be ok with your stupid app.)
Erving Goffman, a sociologist, argued that all social life is theatre. In The Presentation of the Self in Everyday Life he argued that we perform different roles in different ‘fronts’. We wear different costumes, act in different ways and use different scripts depending on which front we find ourselves in: work, home, a shop, a party ….
“We regularly and normally engage in “impression management” to control who sees what, and when.” And we “strive to keep our fronts separate.”
My notes: Except that systems like facebook mean those fronts all kind of crossover, which can sometimes be embarrassing.
There are three key aspects of identity that private sector ethnographers will need to grapple with: gender, economic class, race or ethnicity.
“These roles affect (but do not determine) the decision to purchase a product, how consumers user or display the product, and how they influence others to purchase the product in turn.”
Some stuff about gender
- Gender is often seen in terms of two binary states: male and female, but we know (we know this, right) that it’s much more complicated than that.
2. Judith Butler argued, in 1999, that gender is fundamentally a performance.
How this is relevant for private sector researchers? People often grapple with an ideal role of woman or man and purchasing a product can play into this.
Some stuff about economic class
- Karl Marx’s idea that wealth is not a fact “but a socially constructed meaning that is debated, negotiated and resisted”.
2. Cultural capital – the knowledge about which brands to buy and what they signify is very different to actual economic capital.
Some stuff about race and ethnicity
- “Race is more of a social identity than it is a biological phenomenon.”
All of these identity roles intersect.
Cultcha! Or our broader systems of meaning that we find ourselves in.
Identities play out within contexts.
“It is the ethnographer that provides insights into the influences culture has on the individual”.
While the factist approach believes people have all the answers about their context, the interpritivist view understands that we often don’t understand or fully appreciate the impacts our culture has upon us.
Why that’s important to understand: You’ll often be asked to ask interviewees for their opinion about social trends or product trends or similiar. “Consumers are just as unaware of these trends … ”
“We can operationalize culture as values, beliefs, and behaviours.”
Values generally fall somewhere along five axes, as Kluckholm and Seeley put forward in the 1950s.
These axes are:
- Time orientation – is looking forward or looking back more important to the organisation?
- Activity – what activity is correct?
- Human relations – hierarchies or collaboration? Competition?
- Human nature – are we born good, bad or neutral?
- Human to nature – do we dominate nature, live within it, or submit to it?
Values tend to cluster.
What do people believe about a particular topic?
Linton (1936), in his book The Study of Man, argues that there are four types of behaviours.
Sam Lander gives this example:
|Universals||What everyone does||Wear shoes|
|Specialties||What some roles do – e.g. women, managers||Wear dress shoes, wear high heels|
|Alternatives||In the realm of personal taste||Wear hot pink Doc Martens|
|Peculiarities||What only “strange” people do||Go barefoot|
Sum all of this typing up in one sentence
Perception is reality.
And just to be 100% clear, except for the bits I noted as mine, everything here is from Sam Ladner’s book. Go buy the book! It’s good.
User interviews are a great way of gaining insights at the start of a design cycle.
User interviews help us understand how people think, their opinions, their attitudes and the way they are.
User interviews that take place in someone’s office, at their workplace, out on a client visit, in context, let us observe other information too.
I work on an application used by the construction industry. When we visit our users on site, it’s always cold, dusty and there’s alway an issue with the wifi.
It’s a world away from my office, where it’s warm, there’s beer taps in the kitchen (seriously) and me and most of my colleagues are tapped into wireless on at least 2 or 3 different devices.
But using interviews to test an actual design is not quite the right technique. Once you’ve taken in all of the information you can, through interviews and other methods, added this information into documents, onto sticky notes, into your design artefacts, and start designing your prototype, then you need to test the way people behave, not the way they think.
These two things are very different. I’ll give you an example:
If you interviewed me about my attitudes to exercise, and how important I think it is, and how much I prioritise it, I’m going to tell you “yes, yes, very important, very prioritised”.
If you come back and show me your prototype, I’m going to say “great, looks cool, nice colours, I don’t like the green though” and you’ll make those changes and release and you’d be fairly confident I’m going to be a user of your product.
Except if you had asked me to use a prototype of your application for a week, and tracked my behaviour with the prototype and how much I used it when I was exercising, or even how much I exercise full-stop, then you’re going to find out that I am a: a little bit not-honest with myself about how much I exercise, I don’t really use tracking tools when I do exercise and I didn’t want to hurt your feelings because you’ve worked hard and done a good job with your product. I’m nowhere near a target market for your app.
So I dunno about the article about asking people’s opinions about a new design, and whether online or offline testing is going to make much of a difference (except that people will be nicer to your face).
No, not if you’re designing for enterprise. They’re not. They’re really necessary because the people using your software are so different to you that you need personas in a way that people designing consumer products may never know.
But don’t treat them like glossy objet d’arts either
I agree with the warning about treating personas as glossy, canonised artefacts. Don’t do that. New insights about our users should be surfacing all the time, if you’re doing it right.
I am going to try the 2 1/2 D approach with my team. But I really, really I liked this bit: “you still need to have done the user research — you’re not going to brainstorm a persona out of thin air”.
I have actually participated in what I think of as creative writing exercises where personas were produced that were free of all facts. No facts to encumber them at all, those personas could be whatever we wanted them to be.
Don’t do that.
At some point in the ux journey the org ‘gets’ ux and then we all argue about style guides and we get a bit less ‘prove your value’ and a bit more ‘consolidate your styles and drive this ship’.
That’s when specialists happen, I guess. We’re all 40 now, at least mentally, and we start to specialise and we get a bit more like “another jquery plugin, another development framework, another design tool to learn, kill me” and maybe a little more “I will not execute against your flimsy rationale” so at that point we specialise and we make the rationales by using the research, and then ultimately we’ve specialised ourselves into extinction and we come back as consultants. But not yet.
This is a long-winded way of saying I am transitioning into a more research-focused role and I’m excited about it, and also about never, ever participating in a meeting ever again when everyone’s feelings about one particular icon is discussed for over an hour. (This is not design. This is your younger self coming and sitting next to you and saying something like “oh really. this is what you ended up doing?” )