Connecting...

W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9yzwqty29tbwvyy2uvanbnl2jhbm5lci1kzwzhdwx0lwvulmpwzyjdxq

PySpark Developer (Inside IR35)

Location: London, England Salary: Negotiable
Sector: Consultancy Type: Contract
Reference #: CR/080963_1624438379
  • Position :- PySpark Developer
  • Start :-ASAP
  • End Date :-6 Months + possible extension
  • Location :- Right now remote, after pandemic on-site (London, UK)
  • Contract is Inside IR-35

Key responsibilities:

  • Fundamentals of Spark using the Dataframe API
  • Understanding partitioning of data
  • Analysing and performance tuning Spark queries e.g. looking at the DAG
  • Knowledge of Hadoop and its ecosystem of technologies especially Hive Python, OOP concepts using Python
  • Knowledge of Conditional Statements & Loops: If-else Control Structures, For/While Loops
  • Demonstrate a comprehensive understanding of Complex Data Types: Shallow & Deep Copies, Working with Lists & Tuples, Dictionaries & Sets
  • Understand Fundamental Data Structures & their Implementation
  • Good knowledge of Exceptions & Command Line Arguments
  • Contributes to quality assurance by writing unit and functional tests.
  • Ensures development happens for all Software Components in accordance with Detailed Software Requirements specification, the functional design and the technical design document.
  • Basic knowledge of UNIX
  • Demonstrate source control knowledge (preferably GIT)
  • Ability to analyse databases directly using query language tools such as SQL
  • Experience on ETL process on Big Data
  • Have an understanding of data relationships, normalization