Quantifying Qualitative Decisions

Have you ever been impacted by the decision making of someone else? Their decision can seem unfair, to be lacking transparency, or just plain wrong.

In organizations, with teams, managers, budgets, timelines, and many competing priorities, the way we make decisions can make all the difference.

Quantitative decisions are the easy ones. You’ve got a set of information and an action to take. Open rates are down 10%. Only 3% of our customers return. Our users bounce from the homepage 85% of the time. In all of these cases, the decision is pretty clear. We need to do something.

Qualitative decisions, though, are more complex. How will you improve the email open rates? How will you reel your customers back in? What can you do on your homepage to decrease that bounce rate? Whatever your answer to those questions (or which job to choose, which city to live in, which school to send your kids to), there may be a lot of competing factors. Your decision may affect multiple stakeholders, with their own goals, ideas, and different ideas of success.

Often it can seem like these big decisions are made on a whim, despite any emotion and uncertainty behind the scenes.

In this seminar, Lynn Boyden, an information architect, proposes a model for quantifying qualitative decisions—the big ones—more easily. As an exercise in transparency and participatory design, we can come to an outcome we can feel good about even if we don’t like it.

She outlines a process for taking big decisions—in business or in life—and breaking them down into micro-decisions. Perhaps your team is hiring a new employee and you’ve got a job description to write, candidates to source, and interviews to do. In this process of quantifying a qualitative decision, we’re going to first gather all the stakeholders up and down the org chart who would be part of the decision committee. Then, everyone is going to share their qualifications for this role, such as speaks two languages, is courteous to customers, has experience in campaign building, is willing to join the company kickball league.

Now that you’ve identified all the qualities, you determine how important they are to all of the decision makers with a 0-1-2 scale. Is the new hire being bilingual not important at all (0), a nice to have (1) or a must have (2)? This gives qualities that are more important to more stakeholders more weight in the final outcome.

In the post-interview process, the decision makers evaluate the candidate against the weighted list of qualifications again on a 0-1-2 scale: not sufficient, good enough, great. After the hiring team reviews all of the qualities and qualifications you tally up a raw score for each candidate and then multiply by the qualities’ weight. After multiplying and adding down the graph, it becomes very clear which candidate is best suited for the position.

What a good decision-making process looks like

• It’s inclusive from the beginning by including all the stakeholders and taking into account what’s important to them.
• The process is adaptable from decision to decision, team to team, organization to organization.
• It can be tracked so that those who participate and who are impacted understand the process and reasoning.

How to keep it simple

• Use a 0-1-2 scale to force a commitment. Something is either good, middling, or it's bad.
• Narrowing evaluative choice helps to prevent decision fatigue. Besides, what’s the difference between a three and four on a five-point scale?
• Statistically, the mean and median of the range are the same, giving more interpretable information.

How to improve decision making transparency

• Balance the decision making participation between the typical decision-making managers and those who are impacted by this decision. If working with different discipline teams (like marketing, product, engineering) they need equal representation, too.
• Keep the scoring visible with shared Google sheets, projecting the tally, whiteboards, and sticky notes.
• Document your process.

When we quantify qualitative decisions in the way Lynn Boyden describes, we go a long way toward being more inclusive, ethical, and transparent in our work.