1. Design

Design

The objective of the Design phase is to establish your StoryEngine users and purposes, identify participants and other stakeholders, and finalize your question set. It assumes that you have already had a kickoff meeting to launch the project.

1. Clarify objectives, users, and purposes

Set up a series of design workshops, initially with the core team and then expanding to include other stakeholders as you identify and engage them. The idea is to co-create with the people who use the assets and the insights that will be generated — and ensure that they meet their needs. At the end of this step, you'll want to be clear about...

__ Your StoryEngine purpose(s) and learning objectives — Why are you investing in StoryEngine? What do you want to learn or create? Define what will be generated and what will be different. It may be helpful to review the users in the Introduction, as well as the examples in the Use Cases. If you are going to be using StoryEngine as part of a larger innovation or design thinking process, we suggest that you identify the framework you're going to use at this point (examples: British Design Council's Double Diamond, IDEO's 3-phase framework, or the Stanford D.School's 5 phases).

__ Audiences and key stakeholders — StoryEngine will generate assets, followed by insights. Who within your team will be transforming and/or using the assets you generate along the way? Likewise, who will use the final analytic outputs and recommendations? — Leadership? Your innovation team? Program designers? — Identify and engage all of these stakeholders as early as possible. (Otherwise, you may find that no one is interested in all of the wonderful stories and learning you've produced!)

__ Participants — Who will you ask to participate in the interview process? How might the assets that you generate be useful to them — and what might you do to ensure that they reap the benefits? StoryEngine is designed to be non-extractive, meaning that we strive to create value for participants. The selection of participants is basically your sampling strategy. It's okay if it is not representative, as long as you're clear about your rationale and it serves your objectives. For example, during the StoryEngine pilot, we used a purposeful sample of individuals from different Mozilla Foundation program areas, with careful attention to background (ranging from volunteers to Executive Directors), geography, and gender. Be sure to include different voices in your sample, and be aware of the costs to participating, especially for people struggling with social and economic challenges.

__ Team roles and responsibilities — In short, make sure everyone's on the same page with who is doing what. It can be helpful to create a basic spreadsheet with names, roles, and contact information. And we love using Trello to keep track of who's doing what.

2. Draft and test your question set

Create, test, repeat

Don't overthink your initial question set. Use the outcomes of your design workshops to come up with something good enough, test it on each other (or with friends and family), refine, and then test with two or three participants. Are you getting the information you need? If not, re-write and test again. An iterative process is best.

NOTE: You'll also need to consider what kind of metadata or "descriptors" you'd like to collect for each participant and include a way to collect that, either through questions or email. A list of potential descriptors is on the Data Structure / Descriptors tab of the StoryEngine Data Structure & Analysis Test Results spreadsheet.

You may have heard people say, "Well, that's anecdotal evidence." But what we are looking for is exactly that: anecdotes. Specific and detailed examples drawn from experience. So ensure that your questions are open-ended and that they elicit concrete responses. A formulation we like to use is _"Tell me about a time when..." _Some participants might find this difficult, and will respond a with high-level, general statement — rather than a particular moment in time. For example, in the Mozilla pilot, we asked "Tell me about a time when you felt a sense of success." Of course, you may also want to include questions that surface knowledge, attitudes, and perceptions. An example of this is "What, for you, is a healthy internet?"

EXAMPLE: Humans of the Internet / MozFest 2017

HUMANS OF THE INTERNET QUESTION SET

INTRODUCE yourself and give a general overview of the purpose of the interview and how long it will take + BUILD RAPPORT by asking about something personal not related to your research (friends, family, food, an interest)

01. Can you tell me a bit about your work? Perhaps starting with a broad overview, then highlighting what you’re working on now? Adjust for students and others, for example you may ask about what "they do" versus work.

02. Thinking about your work, tell me about a time when you felt a sense of success. EXAMPLES: breakthrough, insight, interaction, approach — sense of accomplishment or pride. PROBE for a specific example/anecdote if response is too broad: That’s a great example of a type of success — Can you give me a specific anecdote or example related to that?

03. How about an example of a challenge? EXAMPLES: Persistent or recurring challenge, current issue with work. PROBE for how they’ve approached addressing this challenge.

04. This space is called “Humans of the Internet”… What — for you — is a “healthy internet”? NOTE: Some people find it difficult to answer such a broad question. Emphasize you’re not looking for a “right” answer but their own words. You may need to allow for some silence.

05. Can you point to something that makes you feel optimistic for the future of the internet?

06. How about something that worries you about the internet?

07. What led you to be here at MozFest today? PROBE for pathways, what is valued, highlights so far.

08. Anything more you want to tell me? Anything you want to ask? (Give folks a chance to tell you about something they want to talk about or ask you some questions.)

CLOSE the interview by thanking the participant for their time and explaining next steps.

NOTE that with a typical question set we let participants know up front that we will use the notes/transcript to create a story that they can then change any way they want. This is a key element of the StoryEngine methodology — giving people time to reflect and ensure that their story reflects what they want to say. This also helps folks who are more introverted or who find it challenging to articulate their thoughts and experiences. This is not present in the Humans of the Internet question set because it was set up as a podcast lounge at at event — all participants were briefed that their interview would be published immediately during the event.

Ask participants about their experience & check responses with users

After you test the questions with participants, step back and ask them how they felt about the interview. Were they comfortable? Able to express themselves? Which points were challenging or frustrating? What did they like? See if they have any suggestions on how the question set might be improved.

Then, check the results / responses from the initial tests with your key stakeholders and ask for feedback. Ask specifically whether or not the questions are surfacing the types of information they're looking for, and providing the right level of detail. (Be careful — this is not about the substance of the response; focus on the general type of content.)

3. Finalize your interview guide

Based on all of this feedback it's time to finalize your questions. For the StoryEngine pilot, we posted the questions on the project website. This way, we were able to point participants to the question set when they felt nervous or wondered how to prepare for their interview. We are okay with sharing questions in advance because our goal is to create space for reflection.

Create an interview guide. This template also includes guidance for staff and others acting as interviewers.

Finally... it's okay to update your questions as you learn. This will present some challenges in the analysis phase; it's possible to work through them. Serving your stakeholders by producing usable assets and insights is the priority — not being rigid and ensuring that the analysis is as easy as possible.

Now that you have your set of questions and your interview guide, you’re ready to do some deep listening!

Design tools & templates

Interview Guide Template

Data Structure / Descriptors tab — StoryEngine Data Structure & Analysis Test Results (spreadsheet)