Over the past six years, as I’ve built a UX team from a solo practice to a team of 11, I’ve seen how design research can greatly influence product development. Early on, we had little time for customer interviews or usability tests. We were mostly shooting from the hip, listening to customer support, and revising on the fly.
We now do scores of usability tests, user interviews, and competitive analysis, and we create detailed reports summarizing our findings. But this brought us to a new problem: without a way to preserve and combine our results, our insights quickly slipped into the hazy distance as documents got lost on a hard drive, or ignored by someone in a different department.
We ended up living in a Groundhog Day research loop, asking the same questions and rarely building upon what we already knew.
Now we need connections—a way to pull together disparate data points, qualitative and quantitative data, and long histories of research into a central clearinghouse that can be shared, searched, and maintained by different teams. After years on a research treadmill, that’s exactly what we’ve started doing at MailChimp—and far from being just a data solution, open access to this information has strengthened the connections between teams, and supported a general culture of inquiry.
It all started with a personal crisis.

A moment of crisis

Customer feedback streams into my inbox in spades from a form on the MailChimp website. Hundreds of emails offer ideas for new features or ways to make things better. I love reading them, but last summer I started to feel overwhelmed. I was reading hundreds of emails daily, many of which had useful feedback, but weren’t worthy of adding to our roadmap. Maybe down the road an issue would reach critical mass, but until then they sat in limbo.
It was choking my productivity, and making my head spin. A friend of mine who’s helped many people tame their inbox and prioritize their work life recommended I simply nuke all of the emails and shut down the form. “If you can’t process the information, then stop wasting your time!” But my gut told me there was value in the feedback; I just wasn’t sure how to use it.
In Gmail I starred the feedback that was worthy of consideration later, and set up a script to forward it to an email address associated with an Evernote account. It was now curated and preserved in a searchable database, which cleared my inbox—but I still had no plan for how I’d use the data.
Months went by. While studying trends in email automation to inform some new designs, Ben Chestnut, co-founder and CEO of MailChimp, sent me an email late one afternoon asking what we knew about how customers use RSS-to-Email, a related feature. We were making plans for our roadmap, and needed insights that would help us rethink both features to better serve customers. With little time for a detailed study, I turned to my bucket of feedback to see if a search would reveal anything.
Simply searching for “RSS to Email” returned 45 extremely helpful pieces of feedback on the topic, each with an email address provided when users filled out the feedback form. Normally, when conducting a study, recruiting users for interviews was like fishing with a big net. We’d post something on Twitter, or maybe even use a recruiting tool to find users that meet our criteria. It can be very time consuming. But with a database of feedback with email addresses, recruiting was like spear fishing. We found just the right people to speak to within seconds.
Patterns emerged in hours, not weeks. I could see what people struggled with, and how we could make simple changes to improve usability. I followed up by email with a few customers to learn more, and was quickly able to create a plan for how we could make this part of our app much better.
It was a lightbulb moment that left me wondering what patterns might emerge if we had more data to search.

More data please

I shared my story with my colleagues Gregg Bernstein, Jenn Downs, and Fernando Godina, each of whom had data to contribute. Gregg moved user interview transcripts and notes into Evernote. Jenn added usability test findings, and Fernando, God bless him, read through more than 10,000 account closing surveys to find the ones that could tell us the most about why people close a MailChimp account. We were amassing a nice little collection of data of various types.
Ben once again sent us an email asking for insights about a feature. “It seems like more people are asking for an easier way to embed YouTube videos in an email. Do you guys know if there’s a trend here?”
Normally a question like this would be left unanswered, because it’s not significant enough to warrant deep investigation—but now we could find quantitative answers in seconds. Gregg searched our big bucket of data, and sure enough there was a small trend emerging. “11,” Gregg told Ben. “11?” Ben replied. “Yup, that’s how many people have mentioned YouTube video embedding in feedback, account closings, usability tests, and customer interviews.”
We were already blown away by this success. But we knew our data pool still wasn’t showing the whole picture. What pockets of data were out there in other corners of the company?

Data avalanche

We talked to our support team, engineers, data scientists, analytics folks, social media people, and the email delivery team, all of whom have lots of valuable data. Soon large amounts of diverse data streamed in, and the ownership of the data pool shifted from the UX team to the entire company. The support team shared patterns they’d seen in emails and live chats. Engineers wrote scripts that grabbed aggregate data about popular pathways in the app and industry demographics of our users, and emailed them into the database weekly. Because Google Analytics lets you schedule emails with custom reports, we were able to stream in mobile device usage data and completion rates of important workflow funnels. Tweets, Facebook, and blog comments streamed in with even more customer feedback. Survey data, email delivery stats, industry research, and notes from each of our app releases—we added everything we could to the data pool to gain an even broader perspective.

Bringing teams together

As we’ve opened our data up between teams, interesting things have happened. People who wouldn’t normally have occasion for conversation are meeting regularly to compare notes and share what they’ve learned. Our “data nerds” now get together for lunch, and share stories of the projects they’re working on. Data sharing is leading to new collaborations we never would have imagined.
Data nerd lunch
Design researchers collaborated with the DesignLab to turn a months-long user study into a series of beautiful posters illustrating MailChimp’s customer archetypes. After a recent major redesign of the app, support and design research teams collected feedback from customers, printed it out, and tacked the notes underneath each persona poster—helping us triage issues from different customers and devise solutions quickly.
Personas
Each persona has a name and a detailed series of traits. When we do customer interviews, we store the transcripts and notes in our big database, and tag each with the name of a persona so we can see patterns in user types. What does “Andre” have to say about coding HTML email templates? How is “Ada” reacting to the redesign? Now we can answer questions like this by searching with both tags and keywords.
Though we’re not a company that struggles with political drama, close collaboration between teams certainly helps bring our people together and builds respect for the work we’re all doing.
We’ve found that when people are given the opportunity and the platform to share their data or do something new with existing data, they feel pride knowing their work is valuable to others. It feels good to see different areas of the company benefiting from the work you’re doing. Everyone wants their work to be valued and appreciated.

Everyone is a researcher

Sharing the Evernote account with everyone in the company who wants access has encouraged ambient learning scenarios, in which people in all departments are browsing through our research and stumbling upon insights they never knew existed. As he was working on a new iOS interface, Stephen Martin, a designer in our MobileLab team, was curious which stats are most important to customers looking at campaign reports. He did a quick search in Evernote, and stumbled upon a chart from our 2013 survey sent to thousands of customers. He could see a clear ranking of the stats that customers use the most. He used this data to create a poster showing the hierarchy, which helped him make smarter design decisions driven by user research.
Stephen Martin
Mobile designer Stephen Martin turned discovered data into informed design.
Now that everyone has access to the data, everyone is a researcher.

Regular insight outputs

Even though the power of Connected UX is amazing, it’s easy for insights to languish in obscurity unless you’re regularly pulling them out and sharing them with your team.
Rather than wait for a project to provide this motivation, every other week we compose an email that goes to the entire company sharing interesting stats and broad trends. We rotate authorship to make sure many perspectives are represented. Each email concludes with an invitation to start contributing data—or simply browse out of curiosity.

It’s not the tool that matters

We tried a lot of different solutions before landing on Evernote as our data hub. Wikis and custom databases always seemed too technical, and were bound to alienate some people who would love to contribute or just lurk. Your organization might find a simple Drupal install or a custom database works best. The storage tool really doesn’t matter, so long as it helps you adhere to these basic principles:

EASY IN, EASY OUT

Any hindrance, no matter how small, preventing anyone from contributing or browsing data will kill the process. People shouldn’t have to learn new systems to be involved. Contributing data via email is perfect, because it requires no additional learning. Using a consumer software solution is also advantageous because many people will have experience with it. Eliminate all barriers to participation to get lots of people involved.

UBIQUITOUS ACCESS

We all know that mobile devices are outselling PCs and extending the desktop experience into every part of our lives. By making your data accessible across multiple devices, you’ll find insights will happen more routinely—in the line at the grocery store, in meetings, or on the couch in the evening. Ubiquity of access makes ambient learning easier.

DATA FOR EVERYONE AND EVERYONE’S DATA

Give everyone in your company access to the data and diligently invite contributions. It’s important that the data is open and shared so teams are encouraged to collaborate. From this collaboration you’ll find the most mind-boggling insights you would’ve otherwise never discovered.

Silo busting

By connecting disparate data, you’ll discover trends in seemingly disconnected things. That’s exactly what has us so excited. We’re finding patterns between departments, and among customers. We’re breaking down the silos that separate data streams and the teams that manage them.
From the insights we’re gleaning come new, research-driven strategies for our company. That’s new for us, and it’s completely changed the way we work in just a few short months. It’s made us a smarter company and has helped us create more informed design strategies. We no longer lose research, and we’re all more aware of the collective knowledge we possess. What we’re building is more than just connected data—it’s a connected company.
The experience has left me wondering, are other companies also taking a connected approach to research? What insights are you discovering, and what tools are you using? Do you have a story to share? I’d love to hear from you.