Senior Data Engineer (flexible, remote working)

Job Description

Our client is an Arms Length Government Body and they are seeking a Senior Data Engineer.The organisation we are representing has the desire to be data led. The senior data engineer is a critical role in the growing Data and Analytics team that is responsible for delivering the data transformation programme.They have secured funding for a 3-year programme to work with a Data Partner to become data led in its decision making. They are presently 12 months into a large-scale data transformation programme they are looking to:1) Establish a new data technology stack that includes a Data Lakehouse Data analytics Platform and Enterprise BI platform.2) Significantly evolve our organisation’s ability to use data to inform and enable more effective actionable decision making that adds value to the travel and tourism sector.3) Onboard data assets on a use case basis to release value to the business.4) Upskill the organisation from both a downstream usability and also operational ownership perspective.To work with the data science lead and other data engineers as part of the central data and analytics team. The senior data engineer will be responsiblefor delivering high-quality data outcomes via modern data sourcing, integration, and modelling techniques utilising our new cloud data platforms. You will be working with a small team that is responsible for managing the performance, integrity, and usabilityof our data, supporting our new Data Lake, designing, and delivering new data pipelines that bring new data sources into our business from both existing on prem locations and external/third party sources.Experience needed:

• Python, PySpark & SQL - The ability to design and implement well structured code whilst also reviewing and feeding back to others.
• Data Modelling - the ability to design well structured data sets that are at the correct grain and structured for widespread use. (e.g using Kimball modelling.)
• Handson experience of DevSecOps, CI/CD working practices and data orchestration
• Experience in using data lakehouse platforms such as Databricks in data engineering and structuring across all different dimensional hierarchies to make data available for wider business case.
• Experience in working in a modern enterprise cloud stack - Azure preferred, but other platforms include including GCP, AWS will be considered.
• Experience in designing key reference depoyment templates using ARM/BICEP/TERAFORM.

Personal Specification:

• 2+ years experience working as a Data Engineer
• 2 years hands on experience with cloud technology platforms with azure preferred GCP and AWS will be considered
• 3+ years + experience in the following programming languages - including Python, PySpark, SQL. DAX and R desirable.
• Good understanding of data management and government concepts.
• Previous experience of working with multi-disciplinary data and analytics teams including senior stakeholders across different divisions and countries, as well as functional SME’s (data scientists, platform architects, BI architects etc).
• 2+ years’ experience across key foundational products including ARM/BICEP templates, designing HLD’s and structuring key platform tenants to enable business usage.

Required skills

• Data
• Data Management
• cloud technology