Data Engineer Senior/ Lead - Azure

Job Description

We need a dynamic and multi-skilled individual who is passionate about technology to lead the development of services in our Data Platform, with a specific focus on our cloud (Microsoft Azure and Databricks) based services and technologies. The role alsohas responsibility for helping to lead the skills growth of our Data Engineers.The Data Platform team is part of a growing Data Technology team, and is responsible for developing, supporting and promoting the Data Platform, an enterprise-wide collection of services and technologies used by all Technology teams for the deployment of datasolutions.It will best suit a full stack developer with experience of Azure cloud technologies, such as Data Factory Pipelines, Event Grids, Data Lake Storage, Azure Functions, SQL Pools, and API Management. The role will be embedded in a multi-disciplinary and self-organisingagile team, managing the cloud services and developing in-house micro-services and APIs to enhance the platform. The emphasis is on providing self-service capabilities to other technology teams to deploy and run their own solutions on the Data Platform.The Data Platform Engineering Lead is expected to be involved in technical decisions, following best DevOps engineering practices as well as focusing on hands-on development activities. It is likely to have experience across different industries, and althoughprevious Insurance industry experience will be beneficial, it is not essential.We are not a siloed organization; we work collaboratively with other teams for the benefit of the Firm.Key Responsibilities* Design, develop and deliver services and solutions to enhance the Data Platform (Ingestion, Storage, Events/Messages, API Management)* Install, design and configuration of new technologies* Be hands-on; this is an active development role.* Lead the growth and development of other engineers in the Data Platform team specifically, through coaching, training and mentoring. Also work with other Engineering Leads to develop the skills of the Data Technology team generally.* Build relationships within Technology and across the wider Business through delivery. Work in a rapid proof-of-concept style, engage stakeholders and adopt a fail-fast approach.* Multi-task - Data Technology is a dynamic environment; a flexible, can-do attitude always yields results.* Collaborate with the other Engineering Leads and Product Owners to execute our IT strategy.* Provide Data Platform consultation and other services to IT products and teams* Develop to house standards; ensure code is clear, functional, and easy to support and that designs make sense in the wider IT arena. Adherence to best practice for security configuration of Data Technologys Platform* Test driven development approach is essential.Candidate RequirementsTechnical Knowledge and Experience:* Preferred candidate will likely be educated to BSc Honours Degree Level in mathematical, computing or engineering.* Minimum of 5 years actively working with programming languages. * Strong experience with technologies and methodologies (especially cloud) used to support Data-centric processes and applications, such as ETL, ELT, Data Vault (2.0), Pipelines, Data Lakes and Warehouses, etc. would be highly advantageous.* Prior knowledge of insurance industry not essential but will be advantageous.System Skills:* Experience of Azure is essential, as we design and develop cloud-based solutions using services like Data Lake Storage, Synapse, Event Grid, Data Factory, Spark Pools, Functions, Logic Apps etc.* Strong experience with object orientated programming languages such as C#, Python or Java.* Experience working with APIs (REST, GraphQL and equivalents).* Experience with automated deployments in a cloud environment, using tools and technologies such as Bicep, ARM and TerraForm.* Experience implementing automated test-driven development approach and continuous testing.* Experience working with a variety of database technology such as SQL, NoSQL and file storage would be a strong advantage. Additionally, database design knowledge & ability to solve data problems using established techniques.* Experience with source control and automation tools like GitHub* Scrum or Kanban experience would be advantageous.Robert Walters Operations Limited is an employment business and employment agency and welcomes applications from all candidates