Senior Data Engineer

Job Description

Department Overview The Technology division runs and supports all aspect of technology delivery across the Bank, from maintaining critical systems that underpin the UK economy, to providing data analytics, implementing comprehensive cyber security programmes, and introducing creative solutions in support of a digital workplace. Data Platforms is the domain that develops, supports and maintains the data platform services used across the organisation. We are responsible for designing & building our data platforms and then working with our business users to engineer data to meet their use cases. Job description The use of data within Central Banking is approaching a new phase in its evolution. The dual effect of increasingly granular data being collected through financial regulation and huge advances seen within the data technology space are presenting an exciting opportunity! Data Platforms Domain is right at the centre of this transformation. Through your daily work, you will be responsible for ensuring that we have the right tools, standards and frameworks in place so that our broader data engineering team are able to run efficiently. This would involve installation and review of potential new tools and feeding into the Domains strategic roadmap. As the senior data engineer, you may be expected to act as a technical lead, guiding a delivery team through the design, implementation and support of the solutions and services we deliver. In these instances, you will have accountability for the resolution of the technical issues on the team, ensuring that specialist support is engaged where appropriate. It is a fantastic opportunity if you are driven, proactive and intent upon ensuring the continuing success of this fast-growing Technology Domain and its part in underpinning some of the core responsibilities that the Bank holds around monetary, financial stability and regulatory supervision. Role Requirements You would be expected to have a good understanding of all elements of the software development lifecycle. Experience of on-premise data platforms (HDP) is required, but a broader interest and knowledge of cloud data platforms would be beneficial. Part of your role will also involve the evaluation of new applications and services to allow you to feed into the future roadmap for the domains platforms and technologies. Therefore skills that allow you to install, configure and trouble shoot applications on Linux based platforms will be a real advantage. You should enjoy being able to play and understand new technologies, and be able to showcase benefits of these technologies will be essential. We build our pipelines by making use of a third party tool to wrap around PySpark code, so it is essential to have experience of using PySpark. Exposure to Hive is necessary, alongside best practice in how to design data pipelines and store data in a cost effective and performant manner. You should also have good communication skills, being able to have conversations with business users and being able to summarise for senior technology colleagues is a must for this role. You will need to be an excellent teammate, acting as a technical guide, train and mentor to the other members of the team, whilst being able to drive a high level of quality from your colleagues. Excellent problem solving skills will enable you to help the team tackle the complex challenges you will face. It is a challenging role at an exciting time in the Bank of Englands data journey, so do you think you are up to the challenge Minimum Essential Criteria The following are critical entry criteria for this role:

Technical leadership experience and demonstrable skills in mentoring less experienced or technical people

An enthusiasm for learning new technologies and skills

Data Engineering design skills, incorporating security and service considerations

Experience of delivering Data Pipelines on Hortonworks / Cloudera installations

Extensive experience in Python and Spark

Data modelling & thorough understanding of the benefits of distributed computing

Detailed understanding in best practices for use of source control, preferably with experience of GIT and Jira

Linux or Unix navigation skills

Knowledge of industry wide analytical and visualisation tools

Desirable Criteria It would also be helpful for you to have some experience in:

Experience in one or more of the main cloud providers for delivery of Data Pipelines

Apache Airflow

Yarn and Ambari

Apache Atlas and Data Plane Services

Experience in R

Experience of Enterprise Linux or Unix systems administration

Configuration management automation experience (e.g., via Ansible, Terraform or similar technology)

An Understanding and enthusiasm of/for DevOps tools (preferably Jenkins, Git, Nexus, Artifactory), principles and practices

This specific role offers a base salary of up to £76,000subject to experience and skillset) The total package also includes:

A non-contributory, career average pension giving you a guaranteed retirement benefit of 1/95th of your annual salary for every year worked. There is the option to increase your pension (to 1/50th) or decrease (to 1/120th) in exchange for salary through our flexible benefits programme each year. The current cost to the Bank of providing this pension is 35% of your salary.

A discretionary performance award based on a current award pool and target award of 10%, with a maximum potential award of 25%

A 7% benefits allowance with the option to take as salary or purchase a wide range of flexible benefits

25 days annual leave with option to buy up to 13 additional days through flexible benefits

Private medical insurance and income protection insurance

We anonymise applications so hiring managers will not be able to see your personal information, your original CV or any covering letters when reviewing your submission. Please complete work history and the application form questions as requested becuase incomplete submissions will not be reviewed. The closing date for applications is 30/12/2020Please apply online, ensuring that you answer the application questions, complete the work history and submit your CV.