The O’Reilly Design Podcast: Quickly test ideas like a design thinker.

In this week’s Design Podcast, I sit down with Julie Stanford, founder and principal of user experience agency Sliced Bread Design. We talk about how to get in the rapid experimentation mindset, the design thinking process, and how to get started with rapid experimentation at your company. Hint: start small.

Here are some highlights:

What is rapid experimentation?

Rapid experimentation is a technique for figuring out if you have a good idea. As you're going around designing things, running a company, being an entrepreneur—whatever it is that you do in your work life or your daily life—you probably have all kinds of ideas that you're really excited about, and you're probably thinking, ‘Hey, is there some way I could tell early on without investing a lot of time and energy and resources if this is actually an idea that has legs for people?’

... Rapid experimentation is a technique for creating experiments that actually test the use of, or engagement with, your idea in action. It's a process for creating experiences really quickly that are representative of an aspect of some idea you're excited about. Then, you can see how people are actually engaging with this thing or service or process  that you're trying to design.

Why it’s so hard to get in the rapid experimentation mindset

Starting at a young age, we're conditioned that when people ask you questions that are factual and have answers, you, as a reasonable, smart person are proving your worth and your knowledge by knowing the answers to those questions. Rapid experimentation is the opposite of that. It starts from a place of, ‘I don't know the answer to this and I'm never going to find out by sitting at my desk and thinking really hard,’ or even, ‘I'm never going to find out by hanging out with some other really smart people at my work and discussing it for a really long time. I'm still not going to get the answer.’

It's admitting that the only way you're going to get the answer is by running an experiment that might fail. It's in the word. It's an experiment. It could be true; it could be false. Who knows. We'll see. We have a hypothesis about how it might turn out, but a lot of hypotheses are disproven, and this may be a situation like that. Even if we learn that this idea in its current form is not a good idea, we're going to learn something about why it's not working as-is, and that's going to lead to an even better idea.

The 5-second definition of design thinking

It’s a cyclical process of understanding from users, having that inform some design that you're doing, and then going back out to users and getting feedback on it. In a nutshell, that’s probably the quickest description of design thinking I have from a process-oriented perspective.

How to start using rapid experimentation at your company

Start small—especially if you don't have a culture at your work that is okay with failure or that gives you time to do this. Pick a small thing that's on your plate, come up with a few ideas, and see if you can run a very small test that maybe only you're privy to. Then once you get the results—and I guarantee you, it's going to be super interesting—advertise the results. People will get really intrigued and wonder how they might be able to get that kind of data on their own.

The O’Reilly Design Podcast: Designing for the “six minds,” the importance of talking like a human, and the future of predictive AI.

In this week’s Design Podcast, I sit down with John Whalen, chief experience officer at 10 Pearls, a digital development company focused on mobile and web apps, enterprise solutions, cyber security, big data, IoT,  and cloud and dev ops. We talk about the “six minds” that underlie each human experience, why it’s important for designers to understand brain science, and what people really look for in a voice assistant.

Here are some highlights:

Why it's important for product designers to understand how the brain works

I think that by knowing a little bit more about the brain—what draws your attention, how you hold things in memory, how you make decisions, and how emotions can cloud those decisions...the constellation of all these different pieces helps us make sure we're thinking like our audience and trying to discover their frame of...literally their frame of mind when they're picking a product or service and using it.

The “six minds” that underlie each human experience

One is vision and attention. The second is memory and all your preconceived ideas and the ways you think the world works. The third is wayfinding—that's your ability to move around in space, in this case, move around a virtual world. The fourth is language, so the ability to have different linguistic terms. Associated with that is the emotional content there. And, finally, all of that is in service of helping you make decisions and solve problems in your world.

What we look for in a voice-based assistant

We studied how a diverse group of people use Siri, Cortana, Alexa, and Google Assistant, and then we asked, "Well, which one would be your favorite to take home? Which was your personal preference?" A lot of people did pick Google Assistant, which made all kinds of sense because that one did the best at answering questions. But then the second most popular by a wide margin was Alexa from Amazon's Echo—despite actually being the least successful at answering questions. So, that was intriguing to us and we kind of wondered why.

It turns out that the folks who picked Google Assistant often described what they were looking for from these systems as things like, "I just want the answer fast, just the facts. Give me the answer; I just want to know what's happening." And some of the people who preferred Alexa said things like, "Well, it answered the question the way I asked it." Or, "I like that I can converse back and forth with it. It makes me feel like I'm speaking to a human." So, there are really humanistic qualities they gravitated to with Alexa.   

...We can't just go out and test our systems to be “percent correct” accurate, we also need to think about this human component. I think that's the thing I wasn't necessarily expecting to find from our study. We were curious about this humanistic quality, but we didn't know how important it was.

How predictive should AI systems be...when does it become creepy?

In our study, we asked questions like, “How much would you like this to know about you?” For example, Amazon knows how often you've bought toothpaste, so it could probably predict if you're running low on toothpaste. It could ask on a random Tuesday, "Gosh, Nikki, would you like some more toothpaste?" And you're thinking, "How did it know? And where is it looking? And did it have a camera? And who else is in the room?" There are mathematical models that can predict these things quite well.

...There can be all kinds of ways that devices can augment your cognition—and we already do this; we're already, in some ways, cyborgs, every time we use Google Maps or every time we Google a price to make a decision on choosing something. There are a lot of ways this works, and we are very comfortable with it now. Finding out the weather in advance is actually augmenting what we know, helping us make better decisions.

It can keep doing this; it's just that we're not used to it doing it in space and time, and we're not used to it being as predictive. We're used to asking it a question and then receiving the answer as opposed to it anticipating that you might need an answer.

The O'Reilly Design Podcast: Designing in secret, designing for voice, and why improv is an essential design skill.

In this week’s Design Podcast, I sit down with Cheryl Platz, senior designer at Microsoft for the Azure Portal and Marketplaces. We talk about the challenges of working on a top-secret design project, the research behind Amazon's Echo Look, the skills you need to start designing for voice, and how studying improv can make you a better designer.

Here are some highlights:

The challenges of designing secret projects

The Windows Automotive project I worked on at Microsoft wasn't fully 'tented,' but it was kind of hush-hush, so I thought I was prepared when I went to work at Amazon on the Echo Look. But this was...I had never experienced anything truly this secretive. As a designer, being cut off and unable to talk about what you're working on cuts off a part of your process in a way that is a little disorienting. You cannot openly go to customers and ask them questions. You cannot openly go to other designers and ask them questions, and sometimes you can't even ask general questions without causing some kind of curiosity or alarm. ... You really need to get connected with whatever intrinsically motivates you because you're not going to be able to go to critiques and get validation or support or insights from other designers, for the most part. You're going to have to find other ways to gut check yourself to make sure you're considering all perspectives to make sure you don't have any blind spots. It certainly makes things a lot more challenging, and it creates a weird sort of career tension where you know you're on the cutting edge of something and you can’t...tell...anyone.

Why voice assistants are all women

There's some really good research that takes into account the way our gender perception influences our perception of digital assistants. ... The fascinating thing was that there was cognitive dissonance when a gendered voice talked about a subject that was not perceived to be in that gender's area of strength. For example, if you had a woman talking about the inventory at Home Depot, there was cognitive dissonance. If you had a man talking about fashion, that was cognitive dissonance. So, if you're a company and you're trying to release a product that's going to be very disruptive and cause privacy concerns—and I did not work on the initial release of the Echo, so this is me talking on behalf of myself and not on behalf of any company—but my guess is that if you're a company looking to make this really disruptive wave, you have to minimize cognitive dissonance elsewhere to get people to open their minds to a microphone in their home. For the American market—and the features that they planned for Alexa—Amazon knew Alexa would be used largely in the kitchen. Kitchen timers are super popular. Alarms. Household management stuff. I wish that we were just super gender neutral, but the fact of the market here is that cognitive dissonance exists. It's real. So, if you have a home-oriented product in America, you kind of have to start with a female voice.

How studying improv makes you a better designer

Doing improv has a direct impact on how you handle conversations, how you approach problem solving, how you approach question-and-answer sessions at conferences. The more you study improv, the more you learn that the world is not full of right and wrong answers. There are a lot of different ways to answer a question. For example, when I'm at conferences and I get tough Q&A, that improv training, where there are just a number of ways to handle a situation—none of them are wrong. There is an answer. Just have faith in your ability to find it. And to listen. That's the other thing. A lot of improv training is about listening to people, starting to understand their motivation—that's a very valuable skill, and I will be the first to admit that early in my career, I was not great at listening. I wanted to be right. I was as guilty as the next person of waiting for the other person to finish speaking so I could speak. Improv helps you get past that.

The O’Reilly Design Podcast: The sombrero-shaped designer, leading design teams, and designing for retail.

This week, I sit down with Cynthia Savard Saucier, director of design at Shopify and author of Tragic Design. Saucier also is keynoting at Velocity in New York, October 1-4, 2017. We talk about moving from working in design to leading designers, the real and sometimes negative impact that design decisions can have on users, and how design is organized at Shopify.

Here are some highlights:

Design at Shopify

At Shopify, we have more than 2,000 employees, so we're starting to become quite large. We don't have a design department per say; we have a UX umbrella, and under that UX umbrella, we have designers, content strategists, UX researchers, and front-end developers. So, it’s slightly different than some other companies, where front-end developers are working within the UX team. We try to have someone from all disciplines on every project.

Some projects require many designs; for example, when we designed the checkout experience at Shopify, we needed a lot of designers because it's customer facing, and there are different pieces that had to tie together.

Sombrero-shaped designers

We often hear of 'T' shapes; I use something I call a 'sombrero shape,' or the Hershey Kiss shape. I want someone who is really good at one thing, but can be stretched at doing two or three other skills. These are my favorite types of employees. Once you have a bunch of sombreros, they usually cover the whole spectrum of design, and that's perfect.

Another thing we always ask ourselves is, what would this new candidate add to our culture, add to our team? We actively want people who are not like us.

Tragic design

We tend to think design has an impact, but we only think about the positive impact. Our book outlines how designers need to ask the right questions when they're designing.

In design school, we're taught to make things look beautiful or create desirable experiences. We are never once exposed to the consequences of terrible design decisions. For example, some designers design with a single type of user in mind, or they forget that they're designing for someone other than themselves; this leads to injustice and exclusion.

As a designer you have to think of all the different situations and make sure you try to prevent any mistakes from happening.

The O’Reilly Design Podcast: What makes healthy teams healthy, being customer obsessed, and design and research at Microsoft.

This week, I sit down with Travis Lowdermilk senior UX designer at Microsoft, and Jessica Rich, UX researcher at Microsoft; Lowdermilk and Rich are also co-authors of the Customer Driven Playbook. We talk about why failing fast is not always a good approach, sensemaking, and never losing track of the customer’s voice.

Microsoft’s customer focus

Travis: Over the past few years, Microsoft reemphasized its mission to connect and learn from customers, so we're seeing a sort of Renaissance period at the company where there's this kind of recommitment to being customer obsessed.

This isn't something that's unique to Microsoft; you see this with other companies as well, that folks aregetting hip to the idea that in order to make great products, you’ve got to listen to your customers and you’ve got to do it in a procedural way—you can't just comb the feedback forums and come up with ideas; there has to be a process.

We have the desire to be Lean and Agile, but I think what's unique to Microsoft and other big companies is we also have a kind of unique responsibility. It's great to want to be startup-y and embody those fail-fast type philosophies, but we also have to make sure we keep our customers’ best interest in mind. It's a hard ideology to swallow when this company's responsible for software that spans countries and cultures, we have these software products that militaries rely on, software that helps first responders respond in a disaster situation. The gravity of what we work on can't always be a fail-fast model.

That being said, the challenge for us and anybody in UX, is to find ways to help them operate in a way that aligns with the responsibility we have, but still allows them to respond quickly. Quite frankly, to not lose the customers’ voice along the way. This is a big company, and we have big divisions. We're trying to do things as one Microsoft across the company that involves everything from Windows to Office to Skype, moving in a concerted effort, but then there are things our individual teams are trying to do. It can be easy to lose the customers’ voice in all that.

That's why we have whole dedicated sections in our book to an activity called sensemaking. It's the idea that you need to periodically step back from your work and look at the bigger picture, to identify those patterns, and that's something that's really resonated here at Microsoft. We have a huge insider program with the Windows product, where we have hundreds of thousands of customers, millions actually, giving us hourly feedback. How do we step back from that and make sense of what do we do with the data we're collecting?

Design and UX research at Microsoft

Jessica: Travis and I are in the cloud and enterprise division, and we work on Visual Studio. Our UX team is both, as he mentioned, design and research, and we support and partner with our product teams, which include engineering and product managers. The interesting thing about our group is that it doesn't matter what role in the organization you’re in; everyone is customer focused. Our entire team is involved in customer development, and we all use different types of mixed methodologies. We use things like A/B testing, analytics, surveys, focus groups—the list goes on and on. The idea is that we want to learn as much as we can from our customers and make products that suit their needs. We share our results with everyone in our organization, so if a particular team is having conversations with a certain type of target customer, they share it with our entire organization so we can all have a shared understanding of our customer. The idea is that we've framed this as raising our organization’s IQ about our customer, customer IQ. Everybody's learning from these experiences so we can build on the learnings we have from all of our customer engagements, whether it's qualitative or quantitative.

Healthy teams: Stepping outside your role

Travis: The teams I enjoy working with are folks who have a mutual respect for each other and a desire for learning. Like Jessica was saying, they check their ego and their role at the door, andthey're hungry to learn more, not just from the outside world but from each other. I'm a designer who works with a bunch of researchers, but the researchers don't make me feel like, ‘oh, well, you're just the designer—you can't do the research work.’ There's no element of that.

I think it’s critically important that we can go beyond our roles and say, ‘Yes I'm a product manager, but I want to do some research, and I want to try this hat on, and I want to talk to customers and do it in a procedural way.’ Or, ‘I'm a dev and I want to step outside and try a design thinking activity and explore some ideas.’ I think the best teams are the ones that are able to do that effectively, and also that they're willing to build off each others’ ideas and share knowledge with one another. That's not always easy at a company like Microsoft—or any other company where, especially in a large organization, it pays to stand out and be recognized as an individual. We're getting better at that, but it's still something each company struggles with.

To be a member of a great team, you’ve got to want to serve or to assist the team and help others succeed. The best teams understand that, yes, we all have our personal ambitions and our own individual goals but the team, as a cohesive unit, is going to work better if we're all willing to assist and share what we're learning and also be willing to learn from others. I learn from a design perspective; I'm open and receptive to learn something that I can add to my ‘design toolbox’ from a product manager or an engineer. That happens because I'm open and receptive to it.

The O’Reilly Design Podcast: The connective nature of product management, “no work above, no work below,” and the importance of talking to people who aren’t your customers.

This week, I sit down with Matt LeMay, product coach, consultant, and author of Product Management in Practice. We talk about the four guiding principles of product management, what he has learned about himself as a product manager, and how to conduct meaningful research.

Defining product management

To me, being a product manager is all about being the connective tissue, the glue that connects whatever the different roles are within your organization. The specific organizational roles might vary, depending on where you are. You might be working more closely with technical people. You might be working more closely with marketing people, but whoever those different players are, your job as product manager is to be the aligner in chief or translator in chief, the person who is ultimately responsible and accountable for everybody having a shared language and a shared sense of purpose.

CORE product management skills

The four guiding principles came out of the four CORE skills, which is an acronym for communication, organization, research, and execution. I wrote a piece on Medium a few years ago, which was my attempt to challenge the traditional three-way Venn diagram of product management with business, technology, and UX. Having worked at a lot of enterprises and companies where people might not actually be that close to the technology side or might not be thinking about user experience as a day-to-day concern, I felt like those three areas captured a common set of subject matter knowledge that product managers will encounter, but not the actual skills they'll need to connect between those different subject matter ideas. Some people commented and rightly pointed out that something seemed to be missing from it.

That thing seemed to be an element of research, or the ability to actually glean information from the outside world. Erika Hall, in the book Just Enough Research, says that, "Research is just applied critical thinking," which I love as a way of defining research. I like using the word ‘research’ because it also makes it clear that it's not just about being smart; it's about actually doing the work of seeking out alternate perspectives, and explanations, and ideas. These four skills—communication, organization, research, and execution—each one comes with a guiding principle, and I stand by these four guiding principles. For communication, the guiding principal is clarity over comfort, which is really going back to what I was talking about earlier, about this idea that there are times as a product manager when you will have to state things that might seem painfully obvious or ask questions that you know are wading into really difficult political challenges for the organization, but if there is not absolute clarity in your team and in your organization about what people are working on and why, then you cannot succeed as a product manager.

If people don't know what they're doing and why they're doing it, and know that really clearly, then it doesn't matter how good the thing is that you ship or how quickly you ship it; the team will eventually start to fragment and fall apart because that understanding is so fragile and so susceptible to miscommunication and to tomfoolery by people who are trying to steer the product direction one way or another.

For the organization principle, we have ‘change the rules, don't break the rules.’ This was another one that took me a long time to understand. I come from music. I am not a process person. I think a lot of folks who start out as product managers are like, "Yeah. All this stuff is stupid. We shouldn't have 800 steps to do everything. We'll just work really fast. We'll move fast and break stuff, and it'll be awesome," but there's a downside to that, which is that when the rules don't work and people work around the rules, you're basically incentivizing rule breakers and people who are not communicating well. The people who figured out how to game the system accomplish the most, and the people who are trying to go through the system are dinged for not shipping enough software or not being performant enough in whatever way.

For research we have to live in the user's reality, which is pretty straightforward, but also very difficult. When you work in an organization, you live in that organization's reality. That is your day to day. You believe the things people in that organization believe, and it's shockingly easy to become fundamentally misaligned with the reality of your customer, especially when the metrics are telling you you're doing an okay job, but your customers are actually not that engaged. Living in your customer's reality is about getting beyond just looking at isolated metrics, particularly vanity metrics, to understand your customers and really understand their perspective, their world view, how it's changing, how it's evolving, so you can continue to meet their needs as they change and evolve, rather than getting stuck in the way things have always been and the status quo of your organization.

Finally, for execution, this is one one my favorite ones: no work above, no work below. This means that as a product manager, you have to do whatever it takes for your team to succeed. It's pretty well documented that there can be no work below you or beneath you as a product manager. Right? If you have to bring coffee and donuts to the team, that's what you do. If you have to learn how to do something that isn't super fun and exciting to you, that's what you do. Product managers who say, ‘That's not my job,’ or, ‘That's not something I like to do,’ do not generally succeed.

Living in your user’s reality

I'm a firm believer in qualitative research generally, but within that set of qualitative research, I'm a firm believer in talking to people who are not your best customers. I'm a firm believer in talking to people who are considered casual users or users who abandoned your product. There's a tendency, when companies do qualitative research, to over index on the power users and the good customers and to just keep building things for them, but when you talk about living in your user's reality, you're really talking about living in multiple realities for multiple users. In a lot of cases, the people you're talking to need to be the people you're most afraid to hear from or who you initially feel have the most tenuous and least passionate understanding of your project, because those are often the people who are going to make or break your product's success and who are going to be where your growth opportunities come from.

When I talk about living in your user's reality, a lot of that has to do with getting outside of the closed feedback loop of looking for the vanity metrics that support that you're doing a good job and talking to the good customers who will tell you how much they love your product and also have a million product ideas. It's the people who don't really have any product ideas who are just like, ‘Yeah. I don't know. It's fine. Sometimes I use it. Sometimes I don't’—those are the people whose perspective you really need to understand the most because their perspective is probably the farthest away from yours. Not taking those people seriously, not considering them, is a very dangerous thing that I've seen a lot of product organizations do and fall into.

It's funny. I was at a training with a financial services company a few weeks ago. We were walking through some qualitative research, and people were getting very tense, ‘Well, I'm talking to somebody, but they went totally off into left field, and they're not talking about my product anymore. They're talking about their life.’ I get that concern. Right? Because you're there to do a job, but there's an element, and this feels sort of esoteric, but I think it's true, there's an element of faith that goes into those kinds of conversations, where if you really trust and follow somebody's own line of thinking, there will be value in it, but if you go in trying to steer a conversation back to your assumptions or the things that you want to be true, that is exactly where the conversation will go.

Related resources:

• Product Management in Practice—live online training course by Matt LeMay
• Product Management for the Enterprise—online video tutorial by Blair Reeves

The O’Reilly Design Podcast: Leadership, the design of product teams, and hiring optimists.

This week, I sit down with Nate Walkingshaw, chief experience officer of Pluralsite and co-author of Product Leadership. We talk about hard and soft leadership skills, building cross-disciplinary product teams, and why it’s important to use the layover test when hiring.

Here are some highlights:

Hard and soft skills for product leadership

The two different paths are hard skills and soft skills. From a soft skills perspective, don't be a jerk. That's the first thing. Aggregated data wins a lot of debates around the workplace. So, come prepared. Kindness, humility, living in reality—I think those are simple things that continually come to the forefront that need to be restated all the time when you're working in a cross-functional environment.

The way people experience you and the way you experience others really has to do with two things: conflict versus context. I think great leaders have an ability to back away, look at how you work on the business instead of in it, and then really find a way to collaborate with others to come up with the best outcome for the company, the users, the customers, and everyone that's working together.

We spend a lot of time around the soft skills because it's the nature of product development work, design work, and engineering work, requiring those teams to be cross-functional. We can't ship an experience without all of those teams working in concert together. We spend a lot of cycles investing in people, investing in the culture, investing in the soft skills of individuals.

From a hard skills perspective, it comes down to you needing to have the hard skills in product management and design out of the box.

The design of product teams

The perspective we want to share is that we have a great mission and vision, and people are connected to it. From an organizational design, what's the strategy that lays underneath the foundation of that? That each individual who drives into work every day can see their role, the strategy, the mission, and the vision of the company, and they feel really connected to it. The reason they feel connected to it is because of the way we designed the teams.

Everyone who sits inside the experience organization—which is the product management, user experience design, and content teams—could recite democratizing professional technology learning or closing the technology skills gap to you. The goal, the mission behind that is really creating progress through technology that lifts the human condition.

Using the layover test when hiring

The first thing that comes to the front of my mind is attitude and eternal optimism. Really great leaders, great product managers, look at a problem as an opportunity to unlock something they have never discovered before. When you interview folks or when you work on teams, you can smell that attribute pretty quickly, even in the interview process. You get a pretty good sense for it right out of the gate.

The other thing is the old layover test. Does this person pass the layover test? That is, if I got stuck in the airport with Mary Treseler, after 24 hours would we still be friends? The layover test is a big deal. Is this someone I could hang out with, someone I would want to go out to drinks with and socialize with? Would I want to introduce this person to my family? Would I be proud to work with this person? All of those things really matter. It goes back to the fact that we spend more time at work with these people than we do with our families. Are you fun to work with? From a cognitive perspective, can you unlock complicated problems? Are you creative? Can you come up with creative solutions? I think those things really matter a lot.

The O’Reilly Design Podcast: Asking the right questions, conducting research in an agile environment, and conscious confidence.

In this week’s Design Podcast, I sit down with David Farkas, associate director of user experience at EPAM and co-author of the book UX Research. We talk about his book, why everyone should learn to conduct research, and how to open up your mind to ask the right questions.

Farkas and his co-author Brad Nunnally also are teaching a series of online courses:

• Learning UX Research: Understanding Methods and Techniques—May 8 or July 10, 2017.
• Learning UX Research: Analyzing Data and Sharing Results—May 22 or July 24, 2017.

Here are a few highlights from our conversation:

Asking the right questions

The best way to understand if your product or service is resonating with the customers is through some sort of observation. Regardless of your role within an organization, I think everyone should have some awareness of the research going on and, at the very least, be observing it, if not directing and driving it themselves.

I think there are two main areas people struggle with when asking questions, especially in the context of research. The first way is trying to craft questions so they can sound smart and really knowledgeable about the domain. What we always have to remember when we're conducting research is: this isn't about showing off my skills as an interviewer or researcher. It's about trying to learn something new. There's this discomfort in a lot of places—and I’ve felt it too and I still feel it, depending on the project domain—where we want to be able to go into a research session knowing everything we need to know to ask the smartest questions. That's counterintuitive in a lot of ways to what research is—the whole reason we conduct research is because we don't know something.

The first challenge is trying to sound smart when conducting research, when the whole point of conducting research is to fill a knowledge gap. The second challenge is probably a little bit bigger: when conducting research, you really want to elevate the participant. It sort of comes second to wanting to look smart. You have to put that ego down and make sure the person you're conducting research with is able to look and feel like a rockstar. This is particularly hard when doing any type of product testing or product validation, where the participant might feel frustrated about not being able to accomplish a task. It’s important to make sure they know this is not about them or their skill sets. It's about their opinions, and putting them in the best light possible is a really hard challenge for even good researchers, but it’s an important skill newer researchers need to learn and practice.

Agile and UX research: Debunking research misconceptions

There's a bit of a misconception that thorough means long, time consuming, and expensive. Thorough really just means, in my mind, that it's ongoing. I think the real lesson in any type of Agile or Lean environment is that any research is better than no research, and to start as small as you possibly need to. If that's gorilla research, taking out sketches to a coffee shop and having a conversation there, it gets the ball rolling, it gets the conversation started. There's a lot of risk involved in doing gorilla research like that as opposed to doing something a little more formal and properly sourcing your participants, but any research is better than no research.

There's another misconception that research has to happen at the beginning of a project, and if we missed the research, the ship has sailed. Really, we can do research at any time, and we should do research at any time. When we're discovering the problem, defining the solution, and validating the solution, all of those things can happen very quickly and become part of a sprint cycle and part of our design iterations.

One of the most common misconceptions is that you need to be a researcher to do research. I've been on projects where everyone—business analysts, project managers, account managers, etc.—has conducted the research with me. With a five- or 10-minute conversation beforehand, they've been able to understand what our area of inquiry is and learn some best practices in terms of the participant dynamics between the researcher, moderator, and note taker. Really, with just a little bit of training and preparation, anyone can conduct research.

The conscious confidence matrix

The idea of the ‘conscious confidence matrix’ starts with actually being unconsciously incompetent. It's the idea that you start out not knowing what you don't know, and then you move into knowing what you don't know; knowing what you know; and, finally, subconsciously knowing what you do know. It becomes ingrained in you. That only happens through research, so the idea that you start not knowing what you don't know is like this: ‘David, you're going to be doing a project on a large health care application. Okay, I don't know anything about health care, I don't know where to start.’ That's actually the first question of research—then I can start to understand that the project is about these areas of health care, and I know I don't know about these three of the four areas, so let me explore that. Then, I learn about those areas and, on a very conscious level, I can pull the different pieces of knowledge out of my memory bank. Then, ultimately, by the end of the project, the knowledge is so ingrained in my brain that I don't have to think about what the answers are when people ask me about acronyms or workflow or process; it just becomes a natural part of my conversation and something I'm able to speak about naturally.

The O’Reilly Design Podcast: Design ethics and value systems, and what the Ford Pinto can teach us about the importance of human-centered design.

In this week’s Design Podcast, I sit down with Jonathan Shariat, senior interaction designer at Intuit and co-author of the forthcoming book Tragic Design. We talk about his new book and survey some use cases that point a spotlight on the importance of ethical standards in design.

Here are a few highlights from our conversation:

The Ford Pinto

This was the '60s, and they were trying to make a car that was very cheap, very light, and the market was very competitive. And, again, the reason we chose to put this story in, is that there are so many facets that apply directly to designers’ everyday work. Try to empathize with the business as well—even just an extra $25 dollars per car; the cars cost $2,000 at that time, so charging an extra $25 dollars per car could price you out of the market. The sensitivity on price was really big. And one thing that they were trying to do was make this really big trunk for the car, but what that meant is they had to move the gas tank underneath the car. And what that ended up doing was, when you got rear ended, even at speeds as low as 20 to 30 miles per hour, the car would crumple, and the gas would spill out. In many cases, of course, in a car accident, there's going to be sparks. So, in a lot of cases, there were sparks, and then you have all this gas, and the car would erupt in flames. To make matters worse, at slightly higher speeds, the doors would jam.

So, you have this terrible situation where people are locked inside and the car explodes very easily at 20, 30 miles per hour in a very simple rear-ended accident. The engineers knew this, and they brought this issue up; they even came up with a solution of adding some rubber bumpers at the back that would really reduce the impact of this issue. But the price of it was a little too much—I think it was an extra $25 bucks, around there.

The business did a calculation. And the calculation was this: they had a price for the vehicles, the extra price of this little extra bumper, and then the price of people's lives, which was actually given to them by the traffic administration. It's pretty sobering to know there's a price for heads, you know? And they calculated what it would cost to add all this to all these cars, and then they compared that to: okay, we estimate there's going to be about 180 deaths and 180 serious burns, and what would it cost to pay for those deaths and burns for being sued and whatnot?’ And through that calculation, they came up with, oh, it's cheaper to not add this safety feature to the cars.

When you have a set of ethical values that you aren't willing to move against, to stretch, or to break, then you’re willing to challenge yourself to think of better solutions, and you're willing to challenge the situation and prod and poke and figure out: where can these numbers move, where can these numbers change? How can we do something that's going to match our ethical values? And just to reiterate that, once they did release the car, there was actually a worse number of deaths and the cost of the lawsuits was even higher—the CEO even said that it was the situation that almost bankrupt Ford. So, Ford might not even be around today if this had been even a little bit worse.

Of course, they scrambled to fix it, and the engineers, within a few extra weeks, came up with a much cheaper solution that fixed the car. So, if they’d had a value system in place where they listened to their engineers and said, no this is a big problem we're fixing it; that's too much—can you do better? Can you do better, we won't release this car unless we can do this in a relatively safe way. Then these engineers would have had the possibility of pushing a little bit harder and figuring out a solution that would have been both cheaper and saved lives, and I think that's the most important thing.

At the end of the day, your business has to make money. But, if your business also doesn't have these ethical values in place, you as a designer don't have ethical values in place, then these justifications, these calculations, which are somewhat valid in their own rite, lull you into thinking that it's okay to let go of some of the things that you hold dear in your ethical standards.

The hidden cost of bad design decisions

Oftentimes, dark patterns and harmful things really have a hidden cost. In the Pinto example, there was this hidden cost that people didn't trust Fords for a long time after that. It almost broke the company. In this example, yes the initial amount of income came up, but it was hidden. There were actually cancellations, the brand was getting hurt, and things like that. So, it really just involved trying to figure out what it took to change that person's mind, but in other times when there's no data, it's just a matter of being vocal and being consistent and saying, ‘hey, these are my ethical values’ and just making a fuss over and over again.

You'll be surprised how often being consistent and making a fuss works. It's been proven over and over again, right? All the wonderful protests and things that have gone on, being consistent and being vocal. If you're at your company and there are things that you disagree with, first of all, you have to decide what scale do you disagree with it? Is it something that's in your ethical gray area? Is it past the red line for you? This is something I think each designer has to really decide for themselves, and a lot of times, you don't think about it.

That's another aspect of the book—we challenge designers at the end to deeply think about it. Where are your ethical values? Because a lot of times we feel uncomfortable, but we do it anyway because we're unsure how we feel about it, and then we regret that later on. Another really important aspect is deciding what's a gray area for you that you're going push back on really hard, and then what's just a red-line subject for you. Another example is right when I joined Intuit, I sent out an email to the co-founder and the CEO, Brad Smith, and they both replied and they said yes. I just asked them, ‘hey, can I talk to you about design and ethical design?’ And they said yes.

So, this is a tens of thousands of people company, and all it took was sending out an email and just asking. It was the same way with my previous companies, too. I worked at a 300-person startup and just emailed the CEO. He had been at the company for two days, he was a new CEO, and he was willing to meet as well, and, of course, many of the other managers and staff that I've worked with have been more than willing to have an open ear.

I think once people know that someone disagrees with a particular thing, like if you say, ‘you guys, I don't think this is ethical’ or ‘I don't think this is to our brand,’ or what have you, once that's been vocalized, people are much more willing to start backing away from those practices. You'd be surprised.

Maps reveal gaps

I think, again, for both designers who are just coming into the field, but even some veteran designers, really take stock of your standard of ethics. Read through some of the examples or search on Medium for these types of things. There's a lot of stories out there. Decide for yourself, what are the things that I'm okay with, and what are the things that are a red line for me, and what are the things that are a gray area for me? What am I going to do about those? Make a plan.

I think for young designers especially, it's something that you want to know beforehand, because like I said, when you're in the moment, you're not going to have a clear thought; you're going to be kinda confused about what you want to do about it. You don't always have the confidence to push back. But I want to encourage you to do so, and I think that's what you're getting paid for. So, if you're at a job, both paid and unpaid, in the beginning really, it's your job to stand up for your users as a designer.

That's why I really think designers have a unique place in all this—of course, engineers, product managers, CEOs, everyone, we're all responsible for this, and I think they'll all have something to learn from the book. But designers especially are in a unique place in the process, where it's literally our job to understand and advocate for the user. I mean, it's everyone's job at the company, but oftentimes, the responsibility of the designer to make sure that happens. So, for young designers, make sure you have your standards in place and that you're willing and capable of pushing back on that and being vocal.

The O’Reilly Design Podcast: The importance of intentional thinking, user-centered data visualizations, and separating functionality from implementation.

In this week’s Design Podcast, I sit down with Noah Iliinsky, senior UX architect at Amazon’s AWS group, co-author of Designing Data Visualizations, and co-editor of Beautiful Visualization. We talk about how design is organized at Amazon, 17 keys to success, and why being intentional will ensure you are working on the right problems.

Here are a few highlights from our conversation:

Design at Amazon

Typically, design is organized at Amazon in one of two ways. The first category is, as a designer, you would probably work with the storage team, database team, and networking team. You're sort of the go-to designer for a variety of the product offerings that that larger class of technology has. So, you might be working on two or three products at once for a couple of weeks at a time. Get a new feature out, get a revision out, and then switch to some other product within that class. Those designers tend to sit with the big design pool, all on the same floor. All hanging out together.

The other way that the designers typically work is, there are some groups that are large enough that they have a couple of designers working on a single product. So, for example, Quicksight has a couple of designers. And when a product is large enough or has enough UI work that there's more than one designer working on it full-time, they tend to sit with the technology and product managers, sort of dedicated full-time within that group.

We're a young organization when it comes to really building a lot of these UI-based products, but it's a very exciting time to be here. We have really good, really supportive leadership in the UX space; there's a real mandate for us to build high-quality interfaces here. So, the designers are part of the very early conversations with product management and with the technology leadership. Typically, it's the product manager who writes the spec of the product we're building. But we have processes in place where that spec gets reviewed by designers who get to talk about whether this is the right product, is this going in the right direction, that's going to be interesting to implement, why are we doing it this way. So, it's a very exciting time to be here as these changes and these innovations are coming along, and how we do this work and really bring designers to the table in the conversation as products are being not just implemented but conceived.

Data visualizations

I don't think you need to be able to code or you need to know statistics to create good visualizations. Although absolutely it's a help, I wouldn't say, ‘Hey kid, you can't design visualizations till you learn R.’ That's just simply not true. There's plenty of tools that you can use to do it.

The biggest flaw really, I think, with visualizations and how it's done today is that people lose sight of the user-centered design aspect. So, this is a thing that I have been writing about and teaching ever since I started talking about it and ever since I started writing about it 10 years ago when I was working on my Masters thesis. It's really bringing the user-centered design approach and this notion of what problems am I really solving with this visualization,  what questions am I answering? And the quick-and-easy approach is, we have some data; we're going to graph it. Done. And that's a really incomplete process because it doesn't actually take into account the customer and what are their needs right now and what questions do they need to have answered right now.

I have a pinned Tweet on my Twitter account because a thousand people have said, ‘OK, but what graph should I use?’ This is like asking, ‘What car should I buy?’ I need to know a little bit more about your situation. What shoes should I buy? I need to know more about you. So, I thought, OK, I can compress this entire conversation into one Tweet, which I did.

The Tweet says, ‘Step 5. What graph do I use? 4. What data matters. 3. What questions need answering? 2. What actions do I need to inform? 1. What do I care about?’ And the fun part, the cool part is playing with the graph types, so people want to start with Step 4 or Step 5—I've got some data. Let's graph it.

Maps reveal gaps

If I were to pick one thing I really wanted designers to do differently, I would say to be really intentional about the problems on which you're choosing to spend your life efforts.

In terms a little less pessimistic and a little bit more grounded in day-to-day, there are two things I think everybody should do more of and would benefit more from. One is, draw your architecture maps. Draw your flow maps. Draw your swim lanes. Draw a diagram or a map of some kind because you can see where you don't understand the problem when you don't know how to draw the map. And if you skip the map phase and go right to the interface, I guarantee you it's going to be a mess. I've watched that happen. Even at the level where there's an icky part in the architecture map and I say, ‘What's going on there?’ They reply, ‘Oh yeah, yesterday, one of our customers called the PM and said they needed the feature to work this way, so the PM told us we had to do it. So we had to change it.’ And I say, ‘That icky part in the map means you don't actually understand what the requirement is because the PM doesn't actually understand what the requirement is because they got it from the customer and they just threw it over the wall.’

The maps reveal where the gaps in your thinking are and if you think you can draw a good interface when you don't have those requirements defined, it's just never going to work out. So, use the map as a thinking tool. Use a diagram, it's crucial.

And the other is one of the ones I stumbled in to accidentally. I accidentally took a class in college. It wasn't the class I thought it was, but it was in the industrial engineering school at the University of Washington and it ended up being largely about this approach called axiomatic design, which is a theory put forward by Nam Suh at MIT. It's essentially this notion of separating the required functionality from the implementation—and we're super bad at that. We go to implementation right away. When I say implementation, I mean if you say a list, now you're talking about implementations instead of function. And really, really knowing how those are separate gives you so much more flexibility and room for so much more creativity and for the solution you're designing because as soon as you say list or window or screen or app, all of those things tie you then to this particular notion of implementation, and they remove from your imagination all the other possibilities of the way this particular problem could be solved.

The O’Reilly Design Podcast: Build measure learn, the One Metric That Matters, and balancing hubris and humility.

In this week’s Design Podcast, I sit down with Ben Yoskovitz, investor, entrepreneur, and former VP of product at VarageSale and at GoInstant. We talk about using metrics in product development and why anyone building anything new needs to have both hubris and intellectual honesty. Yoskovitz is co-author of Lean Analytics, and is teaching a two-day course on product strategy as part of the upcoming O'Reilly Design Conference.

Here are some highlights from our conversation:

Lean Analytics and Lean Startup

I think many people will have had this experience where they have an idea, they think it's a great idea. They go out and build something, and they invest heavily in that, from a people perspective, from an hours, from a dollars perspective. They launch it, and nobody cares, or not enough people care, let's say. You realize, wait a second, I've made all these mistakes through that, going through this exercise, going through the motions of building something. Now what do I do?

Lean Startup is designed to solve for that, to get you to understand the risks in advance and ‘de-risk’ those things—applying some scientific methodology, or the scientific method, to building products. Lean Analytics is a way of measuring your progress through that process. You need the combination of a little bit of the theory and, well, how do I go about building things? How do I understand what a problem is or how to validate it? How do I do customer interviews? This sort of tactical stuff.

Then, Lean Analytics is really about: how do I measure my progress through this so that I know I'm doing it successfully? Not for the sake of just doing Lean Start Up, but so that I can ultimately build something that my users or customers want.

Build, measure, learn is at the core of Lean Start Up. When you see it, visually, it's a little cycle. Build, measure, learn—it goes around and around. It's an iterative cycle. We'll say, in Lean Start Up parlance, that you're trying iterate through build, measure, learn as quickly as you possibly can, as frequency as you can to get to the ultimate success.

We’re all liars

I'll start with ‘we're all liars.’ I say that almost every single time I present. Partly because it's kind of funny, or I'd like to think it is. Partly because it's true. Often, I'm speaking to entrepreneurs, where I think it's particularly true. The reason is, because I think you actually, as an entrepreneur, have to be a bit of a liar.

There's a number of reasons for that. One is, you're creating something that doesn't yet exist. You're selling it, whether you're actually selling it or not, but you have to convince others that your vision is real and important. That might be for recruiting people. It might be users. It might be customers.

In many ways, you're saying, I believe this thing is true. I'm going to create something to solve this problem or realize this vision. You have to believe me. Let's all agree, that we don't really know if the vision is true. That's where intellectual honesty comes in. Frankly, we say this a lot about the Lean Analytics book, which is, it's not about exclusively using data. It's not about being so wholly data driven that you ignore your gut or insights or anything else. There’s a little bit of lying there. Let's call it a little bit of sizzle before the steak. There's other ways of paraphrasing it.

That's one reason. The other, I think, is that entrepreneurs need what I call a reality distortion field. We need to surround ourselves, and I put myself in this bucket as an entrepreneur, because being an entrepreneur is hard. We've all heard that before. It's 100% true. Early-stage employees, I think, would be put into the same bucket, the first handful of employees. You get into an early-stage company and all the risk is there. You have to get up every morning and fight the good fight for what you believe to be true and for the vision that you're working toward achieving. You have to surround yourself in this reality distortion field.

Now, having said that, I think where the risk comes is when that reality distortion field gets so strong to the point where you're deluding yourself. That's when, as an entrepreneur, you're running a 100 miles an hour.

Another way of thinking about this is ego. Entrepreneurs need ego in order to survive. I believe that to be true. But you can have so much, when you're ego is so big or so strong or so overpowering that you stop listening to other people. You stop recognizing when you're making mistakes. Then, you're going to fail. There's a balance there between ‘we're all liars,’ the reality distortion field, and intellectual honesty, which I believe is so important for entrepreneurs because it's so easy to delude ourselves into believing things that are ultimately not true.