Job Role
We are seeking a Senior Databricks Data Engineer to lead delivery of data engineering workstreams within large-scale data platform and migration programmes.
This role combines hands-on technical expertise with consulting leadership, ensuring the successful design, delivery and operation of modern data platforms while working closely with client stakeholders.
You will play a key role in shaping architecture decisions, mentoring engineers and driving delivery across complex engagements.Key Responsibilities
- Lead design and implementation of enterprise-scale data pipelines and lakehouse architectures
- Provide technical leadership across Databricks, Spark and cloud data platforms
- Drive delivery of data engineering workstreams within Agile programmes
- Engage with client stakeholders to translate business objectives into technical solutions
- Mentor engineers and establish engineering best practices
- Lead architecture and design discussions with architects and platform teams
- Oversee performance optimisation and platform reliability
- Support data migration strategy and execution
- Ensure governance, security and compliance standards are met
- Produce architecture documentation, design artefacts and delivery roadmaps
Key Requirements
- 7+ years experience in data engineering or data platform development
- Strong experience with Databricks, Spark and distributed data processing
- Experience with Unity Catalog, Delta Lake and streaming architectures
- Advanced Python and Microsoft SQL Server development
- Experience designing enterprise data architectures
- Proven experience leading engineering teams or workstreams
- Strong stakeholder engagement and consulting skills
- Working knowledge of Hadoop
- Experience with test-driven development and CI/CD
- REST APIs and Kafka experience
Nice to Have
- Experience leading data platform migrations
- Cloud architecture expertise (AWS preferred)
- Experience managing delivery in large transformation programmes
- Experience delivering projects in regulated environments
