About this role
Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.
The Community You Will Join:
Airbnb is a mission-driven company dedicated to helping create a world where anyone can belong anywhere. It takes a unified team committed to our core values to achieve this goal. Airbnb's various functions embody the company's innovative spirit and our fast-moving team is committed to leading as a 21st century company.
The Difference You Will Make:
We are looking for a talented Staff Software Engineer to join our Data Infrastructure organization and lead our efforts in building Data Catalog infrastructure products across Airbnb’s Data Ecosystem. In this role, you will work with cross-functional teams to develop and build infrastructure, tooling, standards, and processes to ensure all data assets at Airbnb are cataloged, discoverable and governed.
A Typical Day:
• Develop and implement data catalog products to support Airbnb data users on data discovery, metadata management, data quality, data lineage.
• Develop and implement metadata infrastructure to support data governance and processes, including ownership management, data classification, data privacy, access control and data retention.
• Build industry leading end to end data lineage product across online, offline, machine learning, metrics, visualizations and third party datasets.
• Collaborate with cross-functional teams to ensure all datasets at Airbnb are cataloged, governanced and meeting business goals and regulatory requirements. Conduct programs to define and ensure metadata quality and metadata governance requirements.
• Work with various data infrastructure framework teams to identify, design and build metadata integrations to provide data catalog products for each data system.
• Build infrastructure toolings and processes for scalable metadata onboarding and integrations.
• Develop and implement metadata driven data policies and procedures to support data governance requirements.
Your Expertise:
• BS/MS/PhD in Computer Science or a related field, or equivalent work experience.
• 9+ years of experience in software engineering, with a focus on data infrastructure.
• Experience working with data storage and distributed processing technologies, such as Hive, Spark, Trino, Flink or SQL databases.
• Strong programming skills in one or more of the following languages: Java, Python, or Scala.
• Experience with data catalog products – metadata infrastructure, metadata management, metadata integration framework and tooling, metadata-driven data management.
• Experience with workflow orchestration solutions (e.g. Apache Airflow, Prefect, Kubeflow, etc..)
• Str