Connecting...

W1siziisimnvbxbpbgvkx3rozw1lx2fzc2v0cy9yzwqty29tbwvyy2uvanbnl2jhbm5lci1kzwzhdwx0lwvulmpwzyjdxq

Data Engineer

Location: Belgium Salary: €400 - €420 per day
Sector: Consultancy Type: Contract
Reference #: CR/076633_1610387576

Data Engineer / Belgium (Remote during pandemic) / 6 months / Start ASAP

Tasks:

* Conceive and build data architectures
* Participate in the short/mid/long term vision of the overall system
* Execute ETL (extract/transform/load) processes from complex and/or large data sets
* Ensure data are easily accessible and that their exploitation is performing as requested, even in highly scalable circumstances
* Participate to the architecture and planning of the big data platform to optimize the ecosystem's performances
* Create large data warehouses fit for further reporting or advanced analytics
* Collaborate with machine learning engineers for the implementation and deployment of different solutions
* Ensure robust CI/CD processes are in place

Requirements:

We are looking for strong candidates with the following academic and professional experiences:

* A Master in Informatics, Engineering, Mathematics, or related field
* Demonstrable experience with big data platforms (Hadoop, Cloudera, EMR, Databricks)
* Technical knowledge in: o Data pipeline management
o Cluster management
o Workflow management (Oozie, Airflow)
o Database management of SQL and noSQL databases
o Large file storage (HDFS, Data Lake, S3, Blob storage)
* Strong knowledge of Hadoop ecosystem: Hortonworks/Cloudera/EMR
* Strong knowledge of Java/Scala and Python
* Strong knowledge of Spark (Scala and Pyspark)
* Strong knowledge of CI/CD concepts
* Stream processing such as Kafka, Kinesis, Elisticsearch
* Good knowledge of a cloud environment
* High level understanding of data science concepts
* Knowledge of Data Visualisation framework like QlikSense is a plus

Candidate Profile:

* You are multi-disciplined ability to work with divers APIs and understand multiple languages well enough to work with them
* Experience in telecom and/or financial sector is a plus
* Experience with an agile way of working is a plus
* Languages: English (very good in reading, writing, speaking) is a must