Senior Data Engineer
Audience Platform is one of the BBC’s biggest and most ambitious online projects. Its aim is to create technology solutions and organise data to shape a personal BBC for each of our users.
Our products and services include the BBC account, personalised recommendations, notifications, data warehousing and insights, and participation services such as comments, moderation and voting.
You will be part of a large team who support a platform that powers BBC services reaching millions of people. We work with iPlayer TV and Radio, News, Sport, Children’s, Weather and many more.
The Research & Measurement Data (RMD) team, which is delivered as part of the BBC Audience Platform, is responsible for the ingestion, transformation and provision of research data. The data is utilised for insights, analytics, and personalisation of the audience experience, direct audience engagement and data innovation, and supporting projects require data extract, transform and load.
This role will be central to establishing a better understanding of different datasets from external and internal, to set out data acquisition approaches from different sources, to support insight teams to accelerate the data needed to move from idea to implementation, from data to intelligence, and to build up data management knowledge and best practices to enable the BBC to use data to drive a new and deeper relationship with our audiences.
- Works on multiple initiatives as a technical team member and lead driving user story analysis and elaboration, design and development of software applications, testing, and builds automation tools
- Defines data sourcing options and solutions, specifically the identification of data availability gaps, and identify the preferred data sourcing approach for each dataset requirement
- Extracting, transforming and loading data to support prototyping and trialling of new services and BI dashboards
- Producing ad-hoc data extracts and reports on data from a wide variety of systems using tools such as Hadoop, Hive, Pig, Impala, Python, Java, R, RedShift and Tableau
- Develops profiles of data to provide visibility of data usability, and to identify data quality issues for remediation
- Supports various data analytical solutions includes real-time data, and builds automation processes and tools
- Works with the architecture team to define conceptual and logical data models
- Reviews database changes according to database design standards and principles, and trains new joiners for existing solutions and processes
- Works with other data engineers in the team to make sure that all data solutions are consistent, and as well as fitting into existing architecture design
Are you the right candidate?
- Bachelor's degree in computer science, computer engineering, other technical discipline, or equivalent work experience such as data engineering, data science or software engineering
- Experience working in agile environment such as SCRUM and Kanban
- Experience of conducting analytics with large datasets structured and unstructured
- Experience of conducting data analytics with cloud based technologies (preferably AWS or Google Cloud)
- Experience of using tools such as Hadoop, Hive, Pig, Impala, Python, Java, R and SQL
- Deep experience of doing data analytics against relational databases and a good understanding of relational database management concepts
- Good knowledge of data modelling and understanding of different data structures and their benefits and limitations under particular use cases
- Proven experience of working within environments with a diverse data landscape
- Proven experience of working with business and data analysts to translate data requirements into functional specifications to enable the building of data platforms
- Experience working with business analysts to understand and document data integration and data quality requirements
- Experience of prototype-enabled delivery, allowing for discovery/innovation based delivery approaches as part of overall delivery lifecycles
- Experience of working in BI/MI projects and using tools such as Tableau or Google Data Studio or similar
- Experience of using Amazon Redshift, Lambda, S3, EMR, Athena, Spectrum
- Ability to communicate at all levels and to translate technical jargon for non-technical staff
- Ability to apply analytical and structured thinking to resolve programming problems
- Experience of using data analytics platform such as Alteryx, Dataiku and similar
- Experience of working in broadcast, production, start-up or media sector
Excellent career progression – the BBC offers great opportunities for ambitious software engineers to seek new challenges and work on many different and varied products.
Unrivalled training and development opportunities – our in-house Academy hosts a wide range of internal and external courses and certification.
A vibrant digital community – we have a strong focus on continuous improvement and engineering excellence. We host regular talks from internal and external speakers, and provide generous opportunities to attend industry events and conferences.
A variety of technical challenges – we work across a wide range of platforms, languages and technologies; from the latest frameworks and open standards, to the newest devices and modern cloud platforms.
Benefits - We offer a competitive salary package, a flexible 35-hour working week for work-life balance and 26 days holiday with the option to buy an extra 5 days, a defined pension scheme and discounted dental, healthcare, gym and much more.
About the BBC
We don’t focus simply on what we do – we also care how we do it. Our values and the way we behave are important to us. Please make sure you’ve read about our values and behaviours in the document attached below. You’ll be asked questions relating to them as part of your application for this role.
The BBC is committed to building a culturally diverse workforce and therefore strongly encourages applications from underrepresented groups. We are committed to equality of opportunity and welcome applications from individuals, regardless of their background.