Senior Data Engineer

Job Description

Job title: Senior Data EngineerLocation: London, E14,Salary: £65,000 - £75,000 depending on experienceContract type:PermanentHours:37.5, MondayFridayWFH policy:Employees are required to attend the office 2 days/weekFlexible working: Informal flexible work patterns subject to line manager discretion including a 9-day fortnight. Role Summary:As Senior Data Engineer, you will have a specific focus on data and integration, ensuring that they drive single version of the truth by developing and maintaining reliable solutions to store and manage key data.In role, you will source, model, and provide data in a form which is ready for report / dashboard building. You will take ownership for the design, development, and maintenance of LCCCs databases and data warehouse, as well as ensure all data systems conformto the data architecture and strategy expectations.As Senior Data Engineer, you will drive better data governance through the creation and embedding of principles and processes. You will also build and maintain key data governance and management artefacts such as the data model, data dictionary and KPI catalogue.Key Responsibilities:Own, design, develop and maintain databases and ensuring reliable data across the organisation.Source, model and provide reliable data in a form that is ready for report / dashboard building.Drive better data governance through the creation and embedding of principles and processes.Build and maintain key data governance and management artefacts e.g., Logical Data Model & Flow Diagram, Data Dictionary, KPI Catalogue.Define overall approach and data flow for Extract, Transform and Load; and application of this for a given deployment/project.Identify patterns, anomalies, and structure of data in preparation for Extract Transformation and Load.Design and implement data models within a data warehouse based on business requirements.Write ETL data validation and data reconciliation queries.Define high level and low-level design of a data warehouse to ensure robustness e.g., restorability, traceability, ease to support, etc.Define, embed, and drive master data management approaches, principles, and processes.Identify data quality issues through data profiling, analysis, and stakeholder engagement.Design and maintain the Microsoft Azure modern architecture, implementation, and support.Skills Knowledge and Expertise:Essential:At least 4 years experience in Microsoft Azure Datawarehouse architectureand Data Warehousing tools.Experience of ETL and ELT processes, working with database architecture and business intelligence tools.Experience with database administration.Able to develop and optimise queries in SQL.Experience in creating data pipelines using Azure Data Factory.Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure. Functions, Serverless Architecture, ARM Templates.Experience in writing PowerShell scripts.Able to work with large datasets and extrapolate conclusions.Highly desirable:Experience in building and maintaining reliable and scalable ETL on big data platforms as well as experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark and column-oriented databases.Data Engineering experience in Microsoft stack.

Keyskills :
Senior Data Engineer