Data Engineer- Python, Apache Spark

£85,000 +Bonus +Benefits +Flexible Working

Location: London

Job reference: 441574


Job Summary

The Data Engineer role will sit within a brand new Data Engineering team. The Data Engineering team is responsible for providing infrastructure, tools and support to multiple data specialists across the business. The role will be defined depending on personal experience as more senior candidates will be able to take on more of a leadership role.

Role Responsibilities

  • Build, maintain and improve new and existing data infrastructures
  • Monitor and adapt with the evolving data engineering landscape, suggesting and implementing new tools and methodologies
  • Build services for real time data preparation, conversion, and storage using Python, Apache Parquet and Bash
  • Implement new data transformation pipelines utilising Python, Apache Spark and Postgres
  • Extend the data platform by integrating AWS into the exisiting workflow
  • Implement and support notebook infrastructures i.e. Jupyter and Apache Zeppelin.
  • Work on and enhance trading data analysis applications that run on the current and new data platforms
  • Collaborate with both internal and external Data Scientists to provide support for machine learning models

The Ideal Candidate

  • Solid experience of working in Data Engineering with technologies like Spark, Parquet, Airflow, Kafka
  • Be confident in building stable and scalable largescale data pipelines
  • Familiarity with cloud services such as AWS and Azure
  • A proven ability to take on board and adapt work styles using new technologies
  • Experience with Python, Spark, SQL and Git

 

jobs@detail2recruitment.com 0333 023 0045

 

Detail2Digital acts as an employment agency in respect of this position. We will process your CV and personal information to assess your suitability for the role. If your application is taken forward we will record your personal information and contact you to discuss the role. We may consider you for other relevant roles we have that we believe will be of interest to you. Your personal information will be held securely. For more information please refer to www.detail2recruitment.com/privacy

 
Apply for this job Email to myself Close listing
 

Latest Jobs BACK TO SEARCH

Data Engineer

(International eCommerce Business )

Hemel Hempstead

£55k

View job

Python Developer X2

(Analytics and SaaS Consultancy)

London

£80k

View job

Sales Executive

(B2C Print Specialist )

Manchester

£23k - £25k

View job

Fullstack Developer

(SAAS/Online Platforms)

London

£60k

View job

Experienced Recruitment Consultant

((Digital) )

Altrincham

£20k - £30k

View job

Account Manager

(Radio Advertising )

Leeds

£25k - £30k

View job

Frontend Developer- Python, Apache Spark

(Analytics and SaaS Consultancy)

London

£60k

View job

Junior Python Developer

(Analytics and SaaS Consultancy)

London

£55k - £60k

View job

Reporting Analyst

(Energy/Utilities )

Watford

£30k - £35k

View job

IVA Lead Verifier

(Financial Services )

Bury

£18k - £20k

View job

 

Enter your details or

 
 

Browse for your CV (doc / pdf / rtf files only)