Principal Data Engineer

The Role

The team

Millions of people use the products we build. We're customer focused, informed by data and have a product mindset, using state-of-the-art cloud-first technology. Iterating and evolving daily. In a fast-changing world, we not only make sure our systems work effectively but our Technology teams are spearheading the transformation to a digital-first business. There’s never been a more exciting time to join us.

As part of the Advanced Advertising Technology team, the focus is on data analytics, monetisation and financial governance. When you join us, you enter a fun working environment with opportunities to learn, to grow and make a real difference. Small enough that your impact’s felt in the business, but big enough that your impact reaches millions of people.

The role

As a Principal Data Engineer you will be known as a technical authority and champion of data engineering delivery. You will identify, define and nurture the how and the what with regards to our data engineering process. You will own and support initiatives which lift our general level of competency and working culture, and be excited about helping people achieve their full potential. You will be expected to work with a good degree of independence and autonomy.

You are responsible for guiding and leading your team in the adoption of CT&C’s engineering best practices ways of working. You will engage with, and collaborate with, cross-functional teams to ensure your own team can implement scalable and secure data solutions.

This is a leadership, people oriented role with the potential for some hands-on work and carries reporting line duties for a small team of data engineers. You will be a significant contributor to the ongoing refinement process of adapting and improving CT&C’s engineering best practices.

The role will entail but not be limited to:

• Management - Recruiting, building and maintaining a strong data engineering capability. Responsible for mentoring and career development, training/up-skilling; technical leadership; productivity; reviews and performance assessment of all team members
• Direct key technical choices of related platforms, services, tools, development languages and other tools, and facilitate their passage through technical governance
• Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability. Seeking out and identify issues impacting our ability to deliver efficiently and promote solutions
• An authority available to the wider ITV community specialising on the underlying data sets within the data products delivered by your team. Promoting knowledge and skills via the ITV guilds
• Support CT&C data engineering budget/forecasting process
• Ownership and delivery of scalable and secure data pipelines built by your team
• Lead the collaboration with the ITV commercial teams and data product owners to define data product requirements and visualisation needs
• Engage with and lead ITV group/central architectural activity and processes on behalf of CT&C
• Introduce design standards, patterns and review processes
• Define and resolve design and technological issues
• Monitor and optimise data pipelines and workflows to ensure high performance and reliability.
• Work with the QA team to define automated quality processes to measure data correctness and report outcomes through our observability infrastructure
• Organise and execute CT&C’s data governance practices and ensure compliance with ITV data privacy and security regulations
• Ensure all data products conform to all observability requirements and suitable dashboards are in place. Promote a long term design philosophy to proactively reduce incidents and firefighting
• Support and ensure the data engineering team fulfils its ways of working duties

Skills you’ll need (minimum criteria)

• Strong team leadership skills in a data engineering environment
• Management experience: career development, delivery management & skills assessment
• Skilled in designing and building Databricks/Hadoop (Cloudera/HortonWorks) /Spark data products
• Skilled in owning, designing and implementing data pipelines ingesting enterprise levels of data volume. In-depth knowledge of data engineering concepts, data integration, and ETL processes
• Ability to write production-grade code including automated testing
• Experience with the following
  • Deploying code via CI/CD platforms (e.g. Github Actions, Jenkins)
  • Working with distributed computing frameworks, such as Apache Spark
  • Cloud platforms (e.g., AWS, Azure, Google Cloud) and associated services (e.g., S3, Redshift, BigQuery)
  • Familiarity with data modelling, database systems, and SQL optimisation techniques

Other things we’re looking for (key criteria)

• Knowledge of the UK broadcast industry & broadcast/OTT advertising market
• Knowledge of digital marketing and advertising industry
• Ability to lead, inspire and influence others to work towards a common goal
• Confident, methodical, decision maker & problem solver with strong communication and collaboration skills
• Inquisitive & self motivated, staying up to date with industry knowledge and trends
• Familiar with:
  • Data design techniques for classification and regression as well as knowledge of A/B testing, experimental design, and general statistical modelling
  • Architecture disciplines such as Data Mesh Architecture, Data Architecture, BI Architecture and Enterprise Architecture
  • Understanding of data governance, data privacy, and security principles.
• Knowledge of tools across other data management capabilities – data engineering, data governance, data quality, metadata management and master data management
• Knowledge of designing and building out data visualisation solutions based on technology such as: Tableau, AWS QuickSight, Looker, ThoughtSpot