Since its inception, half a billion guests have checked in at Airbnb listings worldwide. Currently, they have over 7 million listings on the platform.
What drives their massive growth? Perhaps it’s the company’s unshakeable focus on their guests’ and hosts’ experience, which becomes really challenging at Airbnb’s scale. And that's where data comes in.
“Data represents the voice of our users at scale. So we use data as a proxy to understand how guests and hosts can be best served by our product. We translate data into a voice that our employees can understand, whether it's through insights or through machine learning algorithms that use the data to improve user experience,” says Jeff Feng, Head of Data, ML, & AI at Airbnb.
Jeff and his team essentially help to empower the entire company to make data-informed decisions. They also empower their products to be able to have machine learning and AI-powered experiences. Their mission is to build leverage for Airbnb through trustworthy data. They aren’t just a data-informed company, but a data-powered one as well.
However, things were quite different when Jeff joined Airbnb. He realized that only 24% of the company used data on a weekly basis. That’s when he decided to build an in-house Data University to empower every employee within Airbnb to make data-informed decisions by providing education that scaled by different roles in different teams.
To help you learn from Airbnb's growth and data culture, we invited Jeff to share tips and strategies on how to build a data-informed corporate culture. Here are some of the key takeaways from Episode 2 of our Secrets and Stories webinar series.
People often confuse data-informed with data-driven culture, however, the two are quite different concepts.
Data-driven culture actually has a connotation around making decisions based on data alone. So, a data-driven company kind of misses the context around the data, as well as the user experience when making data-driven decisions. For example, you launch an A/B test regardless of how positive or negative the metrics are.
On the other hand, a data-informed decision looks beyond that and takes into account the context of what the experiment results are, the user experience you want people to have, as well as how the decision may impact both.
Airbnb started from a place where more people in the company were making gut-based decisions. There was a very strong emphasis on user experience and user journey. One interesting thing about Airbnb is its concept of snow-white frames.
“So our founders were really big fans of Disney. And one of the things they did was creating frames of the entire guest and host journey and what the ideal user experience would be at each of those points,” says Jeff.
However, there was one challenge. Airbnb wasn’t really strong at using data to inform their decisions. So they tried to take the middle ground of being foreign about considering the user experience and leveraging data to provide anecdotal evidence about how they're actually impacting key metrics.
Want your company to have a data-informed culture similar to Airbnb? Follow these 7 tips:
Building early data foundations help you to champion development within the company. The capabilities and tools you develop early on help you make good data-informed decisions later on.
For instance, Riley Newman was one of Airbnb’s first employees. He was originally the head of analytics and became head of data science later on. He really focused on building out a strong data science and analytics team, which also included data engineering.
According to Jeff, “This was so impactful because we are a consumer company and we have many different guests and hosts. It’s really hard to be able to understand what their feedback is on the product without leveraging data effectively.”
Over time, Airbnb built out a solid data foundation, including data warehouse and data processing and visualization tools. Their data team also developed key capabilities, robust A/B testing methods, and machine learning algorithms.
The data culture helped Airbnb establish data as one of the key foundations for strategic meetings within the company. They were able to create effective dashboards to show their key company metrics and how they're trending on a week over week and month over month basis.
It’s possible to instill a data culture later on, however, you need strong support coming from the top management. You require executive support as well as ideas about how to infuse data within the key meetings. Moreover, you need to develop an OKR-driven system.
OKR stands for Objectives and Key Results. It’s a goal-setting technique that was originally developed by Intel and later adopted by Google. You can use OKRs as a way to align yourself on certain goals that you're trying to reach, whereas the key results are the metrics that you are using to measure your progress.
But how do you convince your management to build a data culture? Jeff believes the best way is to share anecdotal evidence. He says, “Within Airbnb, we were able to show the impact by bringing forth examples of how things really helped us to drive our top-line metrics.”
If everyone in the company doesn’t have sufficient data skills, data science can actually become a bottleneck instead of supporting your business.
For instance, the team at Airbnb wasn't initially scaling as people were having trouble understanding data. Their product teams would come to the data scientists to seek answers for even basic questions such as why the experiment failed or how many listings they had in a certain region. These ad-hoc queries were taking time away from the data team to perform deeper analysis.
Therefore, it’s important that everyone in the company is able to self-serve and feel empowered enough to answer their own questions rather than going to data scientists and analysts for everything. Moreover, it also reduces the feedback loop, which is actually the time lost in making decisions and can bog down your business performance.
If you want all non-technical employees to be able to use data and tools effectively, you need to lower the floor for working with data and remove that friction.
When introducing your employees to a new tool or technology, it’s important to walk them through and hold their hand to actually get to that first point where they realize value.
Moreover, you need to inspire them about the impact of data-informed decision-making – not just on the company, but their own careers as well.
“We shared case studies where data-informed decisions made an outsized impact on the company and also shared examples where employees, who were once struggling with data, actually moved into other roles,” explains Jeff.
Another tip is to build data fluency expectations into different job roles. For instance, within the product management team, what are the skills and expectations that you have for product managers in terms of working with data?
This will not only help you ensure that you're hiring the right talent, but will also set expectations internally about what things project management will do compared to what data sites will do. This helps provide clarity in terms of roles within the organization.
To truly empower people, you need to train them on how to use the tools effectively to be able to ask the right questions. However, it can be challenging if you learn just enough to be able to use the tools, but not understand the concept completely.
For this, you can create support groups or identify people within the organization who have more expertise in these particular data domains. Have them work together with people who are new or in the initial stages of empowering themselves with data.
With the ever-changing needs of different users, you need to continuously update various things to fulfill those needs. You can start with the core data set that captures your most important data elements and then move on to focus on the end product and metrics.
“At Airbnb, we created something called data doctor, which gave our people a safe forum to get their queries answered both around data analysis and how to use the tools effectively,” says Jeff.
“We also had a formalized mentorship program. We had a platform called Mentor Cloud where different employees, generally those that were a bit more tenured, would volunteer to provide mentorship on various topics, such as leadership, management, or even more technical skills,” he adds.
There are different ways you can measure the impact of your data training, some of which are quantitative while others are anecdotal.
On an individual level, you can track how well people use the information learned in training sessions. For instance, you can measure how data-informed they are in terms of running A/B tests to drive some of your core metrics.
Another way is to see whether you’re building smarter and more intelligent products. According to Jeff, “From the time I started, we had maybe several dozen machine learning models. Today, there are probably close to a couple of hundred. We've definitely seen tremendous growth in terms of how we're using machine learning and AI to personalize the product.”
This is one of the brightest signs that your data training program is working.
While focusing on driving greater adoption of data usage throughout the company, you also need to focus on building and rebuilding great user experiences with your products. After all, a 2% increase in user retention is the same to profits as cutting costs by 10%.
“If there’s anything that has held us even more accountable, it’s building great products for users,” mentions Jeff.
At Enki, we believe that every professional should have a baseline set of data skills, and that every company that wants to grow needs to be exploring how to make that a reality.
Airbnb had the foresight and resources to create Data University, an initiative that has served them incredibly well. The reality, though, is that not every company can simply create their own Data University. It’s expensive, resource intensive, and technologically complex...
Until now, that is!
Enki’s enterprise platform combines intuitive software, customized curriculum, and on-demand mentorship to give companies a cost and time-efficient path to improve employee data literacy.
If you’d like to learn more, here’s schedule a meeting with a member of our team.
Artem Kubatkin
Head of Growth
https://www.linkedin.com/in/artemkubatkin
