Data Engineering Lead

Job Description

Ideas | People | TrustWe’re BDO. An accountancy and business advisory firm, providing the advice and solutions entrepreneurial organisations need to navigate today’s changing world.We work with the companies that are Britain’s economic engine - ambitious, entrepreneurially-spirited and high-growth businesses that fuel the economy - and directly advise the owners and management teams that lead them.We’ll broaden your horizons To ensure our services and applications are fit for the modern market, our IT team collaborates with every department. They develop, they explore and they implement the new ideas helping us to change the future of accounting, tax and business consulting. But,just as importantly, they maintain the tech that keeps us advancing. By testing and adopting the future of financial technical solutions, they find new and exciting ways to drive us forward. And you could too. In an IT role at BDO, you’ll become part of ateam that act as the backbone for our business. No matter who you are or what your skillset is, we’ll give you the training and support you need to achieve whatever you put your mind to.We’ll help you succeed Leading organisations trust us because of the quality of our advice. That quality grows from a thorough understanding of their business, and that understanding comes from working closely with them and building long-lasting relationships.You’ll be someone who is both comfortable working proactively and managing your own tasks, as well as confident collaborating with others and communicating regularly with senior managers, directors, and BDO’s partners to help businesses effectively. You’llbe encouraged to identify and draw attention to opportunities for enhancing our delivery and providing additional services to organisations we work with.The Data Engineering Lead role is part of the Chief Data Office function and reports into our Data Engineering Manager. It is a critical role to deliver on our vision which is TO EMPOWER EVERY COLLEAGUE AND CLIENT OF BDO WITH DATA. You will be working as partof a multi-disciplinary team of data and analytics managers, data analysts and fellow data engineers to develop and manage data products that can empower our people to make decisions from data to drive growth activities and bring efficiencies to our processes.You will be working on a greenfield cloud platform and will play a key role in shaping and developing our engineering standards as we build out our enterprise data platform. We have some exciting ambitions and plans around building out a data marketplace soour data products can be democratised for use right across the firm.You’ll also:

• Architect, build and maintain data pipelines from data sources through to integration and consumption.
• Take the lead in the design, development, testing and deployment of data engineering solutions including promoting best practices, performing code reviews , maintaining version control and supporting data products. (hypercare, post go-live support etc.)
• Work closely with the other teams in the data office, architecture, and IT security, to deliver data content through configuration, de-duplication, consolidation and conforming data from multiple data sources.
• Evaluate data engineering development needs and determine between delivering point solutions or strategic projects depending on complexity, effort and time constraints.
• Be responsible for using innovative and modern tools, techniques, and architectures to automate the most-common, repeatable, and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity.
• Ensure technology solutions are in alignment with the data strategy and target state data design.
• Work with IT teams, business analysts, and data analytics teams to understand data consumers’ needs and to develop solutions.
• Support ETL, common data structures and business intelligence architectures.
• Understand, organise and extract value from both structured and unstructured data sets housed within the disparate BDO data storage systems.
• Promote data governance through collaboration with the data governance team, and through documentation, observation and controls.

You’ll be someone with:

• Proven experience of working as a Data Engineer.
• Extensive hands-on experience with Microsoft Azure data engineering techniques.
• Extensive experience developing analytics using Azure Data Factory, Databricks, Azure Synapse Analytics, Azure Logic Apps, Azure Functions, Power BI.
• Strong experience in data extraction and processing languages including SQL, Python/Spark.
• Experience of working with cloud data storage such as Data lakes, Data Warehouses, SQL and No SQL (Cosmos DB).
• Hands on experience of agile methodologies and capable of applying DevOps and increasingly DataOps principles to data analytics to improve the communication, integration, reuse and automation of data flows between data managers and consumers across an organization
• Experience of managing internal technical staff as well as external consultants.
• Experience with designing, building and managing data pipelines for data structures encompassing data transformation, data models, schemas, metadata and workload management.
• Knowledge of optimizing data pipelines, pipeline architectures and integrated datasets using traditional data integration technologies. These should include ETL/ELT, data replication/CDC, message-oriented data movement and API design.
• An understanding of data security and information governance guidance and best practice.
• A track record in making data solutions accessible for end-users.
• Experience of working with stakeholders to deliver a data-driven approach to decision making.
• Experience in working with both IT and business users translating business requirements into cohesive data solutions.
• Expert knowledge of star schema/snowflake modelling including dimensional data modelling, the identification of dimensions, facts, measures and hierarchies.

You’ll be able to be yourself; we’ll recognise and value you for who you are and celebrate and reward your contributions to the business. We’re committed to agile working, and we offer every colleague the opportunity to work in ways that suit you, your teams,and the task at hand.At BDO, we’ll help you achieve your personal goals and career ambitions, and we have programmes, resources, and frameworks that provide clarity and structure around career development.We’re in it togetherMutual support and respect is one of BDO’s core values a