Principal Data Engineer (Datalab)

Please Note: The application deadline for this job has now passed.

Job Introduction

BBC audiences expect the best content to be available to them in a single place, personalised to their preferences and interests. At the moment this is difficult for us to achieve since our content and audience data is distributed across systems that are hard to connect. We’re also missing metadata about lots of our programmes, which makes them difficult to discover. We’re currently unable to properly engage the next generation of TV license fee payers, many of whom already have less affinity with the BBC than the rest of the UK population.

Datalab was formed to address these issues, by creating a simpler way to discover content. We are doing this by bringing all of our data together into one place, and by using machine learning to enrich it. As we do this, we become able to match our programming with individuals’ interests and context. Our approach is to build a data platform that can be extended by other BBC teams, and which allows many different products to create consistent and relevant experiences for audiences.

Individuals within the Principal Data Engineer role in Datalab lead the development of data pipelines that are scalable, repeatable, and secure, and can serve multiple users within the BBC. They help facilitate getting data from a variety of different sources, getting it in the right formats, assuring that it adheres to data quality standards, and assuring that downstream users can get that data quickly. This role usually functions as a leading member of an agile team.

Over the past year, we launched the first in-house personalised recommendation engine for BBC Sounds as well as the first ML-driven recommendation engine for BBC Sport and News short-form videos. We have many more exciting projects on the horizon, working across the BBC product portfolio.

Our team objectives are:

  1. Make it easy for BBC teams to rapidly develop and deploy Machine Learning engines
  2. Provide great recommendations across multiple BBC products

Role Responsibility

We are aiming high and have an open brief to define what works best for our audience. We want to stay lean and move quickly to build, test and learn as we go so your contribution will make a difference from day one. We want everyone to feel responsible for our collective success.

You will help us create a data and machine learning environment that can scale to millions of users. You will help integrate new data sources and ensure that the code we write is robust and scalable. You have a keen interest in machine learning (but not necessarily previous experience). You are excited and knowledgeable about a tech stack that includes Google Cloud Platform, Python and Kubernetes with a commitment to micro-services and infrastructure as code.

You’ll engage with engineers working on other BBC apps and services, tapping into the wealth of knowledge and experience of an organisation already serving a vast global audience. Learning is an important part of the role, and you’ll have access to BBC Academy training programmes, along with the opportunity to attend technology conferences and use other resources to progress.

The Ideal Candidate

We would expect you to have significant experience in establishing data applications into production (ideally in a cloud environment) for millions of users, and also in coaching and managing more junior team members.

You will demonstrate having wide exposure to different data storage systems and/or machine learning algorithms. Experience with model management / algorithmic lifecycle management and involvement in the data engineering community would be strong positives.

We are looking for the following skillsets:

  • Ability to communicate with and provide leadership within multi-functional teams
  • Passion for development and data best practices
  • Track record of delivering production-ready code
  • Significant demonstrable experience of writing Python and using associate frameworks
  • Significant experience developing APIs (if possible with async experience)
  • Test-driven development approach, the experience of writing unit and functional tests.
  • Significant experience of cloud-based development, with AWS or GCP experience being most beneficial
  • Working knowledge of machine learning systems

Also desireable:

  • Effectively able to pair programme with junior and senior developers
  • Data streaming systems
  • Document source-to-target mapping
  • Build data streaming systems
  • Experience developing and optimising ETL pipelines
  • Effective data preparation for analysis
  • Implement machine learning & deploy similarity metrics (such as nearest neighbours) at scale
  • An understanding of how software design relates to overall system architecture

Package Description

Band: D
Contract type: 12 month fixed term contract/attachment
Location: London (w12) or Glasgow

We’re happy to discuss flexible working. Please indicate your choice under the flexible working question in the application. There is no obligation to raise this at the application stage but if you wish to do so, you are welcome to. Flexible working will be part of the discussion at offer stage.

  • Excellent career progression – the BBC offers great opportunities for employees to seek new challenges and work in different areas of the organisation.
  • Unrivalled training and development opportunities – our in-house Academy hosts a wide range of internal and external courses and certification.
  • Benefits- We offer a competitive salary package, a flexible 35-hour working week for work-life balance and 26 days (1 of which is a corporation day) with the option to buy an extra 5 days, a defined pension scheme and discounted dental, health care, gym and much more.

The situation regarding the coronavirus outbreak is developing quickly and the BBC is keen to continue to ensure the safety and wellbeing of people across the BBC, while continuing to protect our services. To reduce the risk access to BBC buildings is limited to those essential to our broadcast output. From Wednesday 18th March until further notice all assessments and interviews will be conducted remotely. For more information go to

Mae’r sefyllfa gyda’r coronafeirws yn datblygu’n gyflym, ac mae’r BBC yn awyddus i barhau i sicrhau diogelwch a lles pobl ar draws y BBC, gan barhau i warchod ein gwasanaethau hefyd. I leihau’r risg, dim ond y bobl sy’n hanfodol i’n hallbwn darlledu fydd yn cael mynediad i adeiladau’r BBC. O ddydd Mercher 18fed Mawrth ymlaen, bydd pob asesiad a chyfweliad yn cael ei gynnal o bell, nes rhoddir gwybod yn wahanol. I gael mwy o wybodaeth, ewch i

About the Company

We don’t focus simply on what we do – we also care how we do it. Our values and the way we behave are important to us. Please make sure you’ve read about our values and behaviours in the document attached below.

Diversity matters at the BBC. We have a working environment where we value and respect every individual's unique contribution, enabling all of our employees to thrive and achieve their full potential.

We want to attract the broadest range of talented people to be part of the BBC – whether that’s to contribute to our programming or our wide range of non-production roles. The more diverse our workforce, the better able we are to respond to and reflect our audiences in all their diversity. 

We are committed to equality of opportunity and welcome applications from individuals, regardless of age, gender, ethnicity, disability, sexual orientation, gender identity, socio-economic background, religion and/or belief. We will consider flexible working requests for all roles, unless operational requirements prevent otherwise.

To find out more about Diversity and Inclusion at the BBC, please click here

This website is using cookies to improve your browsing experience. If you navigate to another page without changing the settings below you consent to this. Read more about cookies.


This website is using cookies to improve your browsing experience. If you navigate to another page without changing the settings below you consent to this. Read more about cookies.