Senior Data Engineer

Hybrid - within commuting distance of one of our campuses in London, Birmingham, Blackburn, Doncaster, Bradford, Redcar or Bristol

We are seeking a motivated and enthusiastic Senior Data Engineer to join our TED team and play an important role in building and maintaining the technical infrastructure that supports the NIoT and TED.

The National Institute of Teaching (NIoT) has an unswerving commitment to high-quality, evidence-informed teacher education, and is on a mission to improve teacher and leader development across the education system.

A key part of that mission is our Teacher Education Dataset (TED), an ambitious data and research initiative focused on building better evidence about teacher development and what makes teaching impactful. By bringing together education, research and digital infrastructure, TED will support new insight into how teachers develop, improve and thrive across their careers.

About the role

We are seeking a motivated and enthusiastic Senior Data Engineer to join our TED team and play an important role in building and maintaining the technical infrastructure that supports the NIoT and TED. Working at the intersection of data engineering, data and research, the postholder will help develop secure, reliable and scalable systems that enable robust analysis and contribute to high-quality, evidence-informed improvement across the sector.

Corporate responsibilities

• To ensure that the responsibilities of the role are carried out in a way which reflects the mission and the values of the NIoT.

• To be aware of and observe all policies, procedures, working practices and regulations, and in particular to comply with policies relating to Equal Opportunities, Health and Safety, Confidentiality, Data Protection and Financial Regulations, reporting any concerns to an appropriate person.

• To comply with all reasonable management requests.

Key Responsibilities

• Design, build and maintain Azure-based data pipelines to support the ingestion, transformation and curation of TED datasets.

• Develop and maintain Azure SQL databases and research-facing schemas across raw, transformed and curated data layers.

• Write efficient, secure and well-structured T-SQL to support ingestion, validation, transformation and optimisation.

• Contribute to the development and ongoing refinement of a Common Data Model across multiple education data providers.

• Integrate and harmonise data from school MIS and EdTech platforms, including Bromcom, Arbor and Wonde.

• Build and maintain robust validation, logging, auditing and reconciliation processes to improve data quality and trust.

• Support secure data handling, including pseudonymisation, access control and compliance with data-sharing agreements and security requirements.

• Work closely with analysts, researchers, architects and partners to deliver scalable, analytics-ready data solutions.

• Contribute to agile planning, technical design, documentation and operational handover.

Essential criteria

• Proven experience in data engineering, including building and maintaining production-grade data pipelines.

• Strong experience with Azure Data Factory and Azure SQL Server.

• Strong T-SQL skills, including schema design, stored procedures and query optimisation.

• Experience implementing validation, monitoring, logging and reconciliation controls within data pipelines.

• Strong understanding of secure data architecture and the handling of sensitive personal data.

• Experience with Azure DevOps, version control and CI/CD practices.

• Strong analytical and problem-solving skills, with excellent attention to detail.

• Ability to communicate effectively and work successfully in a multidisciplinary team.

Desirable criteria

• Experience with education datasets, school MIS systems or EdTech platforms.

• Experience with common data models, longitudinal datasets or research-oriented data infrastructure.

• Experience with Power BI, Fabric or related analytics technologies.

• Experience in a public sector, education or high-governance data environment.

• Interest in applying data engineering to improve education and teacher development.

Key benefits

• Generous Annual Leave 27 days holiday a year (plus 8 bank holidays).

• Flexible Working we offer flexible start and end working times, with hybrid working in place for all roles.

• Pension - Entry to the Local Government Pension Scheme.

• Salary Sacrifice Schemes we offer salary sacrifice schemes for bikes, cars and tech!

• Lifestyle Benefits We offer discounts on gyms, cinema, retail and much more!

• Parenthood Leave We offer above the statutory minimum for maternity, adoption and paternity leave.

• Working Environment We have a stimulating, supportive and rewarding working environment with a dedicated team of likeminded professionals.

• Support: Our Employee Assistance Programme (EAP) provides confidential support for employees on personal or work-related matters.

• Development: We fully support the development of our staff and ensure that you have high level of continuous professional development.

We think a wide range of different work and educational experiences could support you to be successful in this role. We encourage applications from all backgrounds, communities and industries, and are committed to employing a team that has diverse skills, experiences and abilities.

If you have any queries about this role, please contact our recruitment team.

For more information and to apply, please visit our vacancies page.

Closing date: 10.00am on Monday, 1 June 2026.

We reserve the right to close this vacancy early if we receive a high volume of applications.

This post requires a satisfactory enhanced disclosure from the Disclosure and Barring Service (DBS) with a Children s Barred List Check and Occupational Health Check is required as a condition of employment.