Big Data Engineer

Job Description

My client is looking for a Big Data Engineer to be based remotely due to the current situation, although they do have offices in both Ipswich and Bristol.This is for an initial 3 month contract with the potential to extend.DevOps summary: Performing a multi-skilled role youll need to have experience building, deploying and managing data ingestion pipelines using Cloudera (HDFS, Spark, Kafka, Hive, Impala, HBase) and Elasticsearch technologies. You should be familiar with using tools like Puppet, Jenkins and with ETL tools such as NiFi. Knowledge of version control e.g. GitLab and Linux essential. Experience in automated testing and performance testing would also be a bonus. The role will also involve some software development, so development skills in Java/Scala and scripting languages such as Python or bash scripting would be desirable. Experience of security protocols such as TLS, SAML2 and Kerberos would be advantageous. You should have an analytical and problem solving mindset. Youll need to be capable of quickly understanding new technologies and establishing best practice for their use. Youll work with users to understand their requirements and then design, develop and configure solutions using the appropriate architectural components. You may be required to integrate and configure COTS (Commercial Off-The Shelf) products. You may need to support out of hours software deployments. Youll also provide in-life support for investigation and resolution of application problems. As an experienced DevOps practitioner you will be expected to provide an analytical and innovative input to projects, within a clear framework of accountability. As well as being competent in your own area, you will also assist in the definition of strategy and process improvements. You will feed into best practices to be followed by the team in order to have a positive impact on the business. You may be required to oversee and direct the work of others. Youll ensure that all work conforms to BT software engineering standards. Essential skills and experience *Cloudera - HDFS, Spark, Kafka, Hive, Impala, HBASE *Elasticsearch/Kibana *Pipeline release automation using Puppet, Jenkins, GITLAB or similar *Experience working in a Linux environment (including system administration fundamentals) Desirable *Core software engineering skills (demonstratable industrial experience) *Scripting skills in bash and python *Programming languages e.g. Java/Scala (demonstratable industrial experience) *Experience with automated test frameworks e.g. JUnit, Cucumber *Security technologies e.g. SSO integration, SAML2, Kerberos *Containerisation (e.g. Docker, Kubernetes) LA International Computer Consultants Ltd is an HMG Approved Consultancy and operates as an IT & Engineering Consultancy or as an Employment Business & Agency, depending upon the precise nature of the work, for security cleared jobs or non-clearance vacancies, we welcome applications from all sections of the community and from people with diverse experience and backgrounds.Award Winning LA International Computer Consultants Ltd (Recruiter Awards for Excellence - Best IT, Best Public Sector & Gold Awards) and the most prestigious award that any business can receive The Queens Award for Enterprise: International Trade 2015.