Data Engineer Remote Stoke on Trent

Job Description

At Dee Set we are undergoing an exciting digital transformation that includes building a strong Data Science and Analytics team to drive data integration, modelling and inference utilising the best-in-class Data Analytics, Engineering and Machine Learningtools and techniques.The role will require working within the Data Science and Analytics team and with a range of business functional teams, including Tech and Change. You will focus on development and maintenance of the end-to-end data pipelines and data warehouse, implementationsof systems and standards for service quality, maintenance of code repositories. You will be a strong independent problem solver capable of working through complex problems with minimal guidance. You will build and manage the data infrastructure and environmentsand implement best practice for the storage and processing of large-scale data sets, enabling the team to succeed.Key Accountabilities and Responsibilities:

• Ingestion of data from source systems, identifying and resolving data quality issues
• Design and develop ETL processes to create robust data assets
• Design, build and maintain the end-to-end data pipelines and data models to serve analytics solutions
• Automation of pipelines
• Support to the business in the implementation and usage of Data Engineering applications and tools
• Taking ownership of our data lake and warehouse platform
• Take responsibility for the quality of the code base ensuring that the data is reliable and matches the user needs
• Collecting and analysing user requests and estimating required effort
• Be a strong advocate for quality, ensuring that data and common repositories of code are delivered and maintained to a high standard

WHAT YOULL NEEDWe are looking for an outstanding problem solver with an inquisitive mind, good eye for details and real hands-on experience. You will love working with data and documentation. Having self-motivation to learn and keep pace with numerous analytics and softwaretools with an ambition to become an expert in applying such technologies is essential.Knowledge, Skills and Experience required:Knowledge of the processes, tools and techniques for extracting, joining, cleaning and shaping data from multiple sources for subsequent analysis, ability to create robust data models to help drive business insights.

• Collect and translate the business requirements for analytics solutions into required data structures
• Implement and operate multiple types of data structures and troubleshoot complex problems associated with these structures
• Anticipate underlying problems and limitations in the data structures
• Design and develop the data engineering steps required to provide the necessary data using complex queries and transforms (MS-SQL, Python, Snowflake, Azure services: Blob storage, Cosmos DB, Elasticsearch, etc.)
• Orchestrate data pipeline steps into logical executable sequence (preferably using Matillion)
• Identify reasonable timescales and data owner(s) input need for data transformation exercise
• Utilise a systematic approach to projects and tasks by using the most efficient means of generating consistent and optimum results
• Optimise the use of compute and storage in data engineering platform
• Optimise query, design efficient code, perform code reviews, utilise version control and collaborative development (GitLab, Snowflake, Azure, Python)
• Incorporate design, build, test and defect resolution methodology and operational documentation at the Development stage

Desirable Skills and Competencies:

• Experience building and maintaining a live cloud data lake and data warehouse
• Star-Schema warehouse design and implementation
• Experience using the Matillion, Snowflake and Azure ecosystems
• CI/CD experience
• Machine learning model deployment experience is a plus

WHY USBring your expertise and realise your potential to further your career with a world-class team. This is a brilliant opportunity to join the market leading and award-winning Dee Set Group and play a key role in delivering the digital transformation. We offerflexible and remote working, opportunities to learn and grow professionally and personally, access to money-saving perks via our Colleague Hub and a competitive pension plan.With an AI-powered recruitment process we give you the best possible application experience and provide personalised feedback. We are proud of our fairness and equali