Specialist Support Engineer: DataOps

Job Experience & Skills Required:

• 3+ years’ experience working in Big data environment, optimising and building big data pipelines, architectures and data sets with e.g. Java, Scala, Python, Hadoop, Apache Spark and Kafka
  

• Minimum one year experience with Scala programming language
  

• Minimum one year experience managing a team
  

• Cross domain knowledge
  

• Familiarity with Hadoop ecosystem and its components
  

• Good knowledge of the concepts of Hadoop
  

• Solid experience in a working environment in Big Data development utilising SQL or Python
  

• Experience in Big Data development using Spark
  

• Experience in Hadoop, HDFS and MapReduce
  

• Experience in database design, development and data modelling
  

• Manage an assigned team through day-to-day support tasks
  

• Oversee development plans for the team and provide mentorship to the team
  

• Provide guidance and peer review
  

• Support pipelines end to end
  

• Build and deploy enhancements and new developments or new data pipelines
  

• Identify and drive optimisation opportunities across the environment
  

• Manage the handover of new applications ensuring that required standards and practices are met
  

• Improvement on recovery time in case of prod failures
  

• Test prototypes and oversee handover to the Data Operations teams
  

• Attend and contribute to regular team and User meetings
  

• Responsible for the actual coding or programming of Hadoop applications • High-speed querying