Data Engineer

Location: Manchester, Greater Manchester Salary: £300 - £325 per day
Sector: Consultancy Type: Contract
Reference #: CR/076087_1607685524

Data Engineer / Manchester / 4-6 months / Start ASAP


* Design, Develop, Test and maintain efficient, scalable data pipelines and ETL process
* Translate customer requirements to develop effective, robust data solutions in line with customer Data Engineering standards
* Work with other Data Engineers and Architects in tuning and optimising the data pipelines/ETL processes
* Effective stakeholder engagement at all stages of the Data Engineering project lifecycle, from pre-feasibility through to delivery
* Implement processes to monitor performance and data quality

Key Skills/Knowledge:

* Azure Data Factory, Azure Databricks, Azure SQL Dataware house / Azure Synapse

Experience required:

* Must have experience in developing, building and deploying solutions based on Microsoft Azure SQLDW, TSQL, Databricks, Data Lake, Azure Data Factory, Blob Storage PySpark/Spark SQL
* Demonstrable experience and understanding of end to end data delivery techniques including data modelling, ETL building, Quality Assurance and performance tuning
* Experience in designing and building cloud based ETL/ELT pipelines and data migrations and dealing with high volume systems
* Good understanding of monitoring and tuning data pipeline workloads, preferably in Azure Synapse
* Experience in building pipelines and loading data into Azure SQL Data Warehouse / Azure Synapse or SQL Server BI Toolset (SRSS, SSIS, SSAS)
* An in-depth understanding of data management (e.g. permissions, recovery, security and monitoring) and capable of choosing the right data stores/data processing for the job (relational and non-relational stores, batch, and Real Time processing)
* Experience in dealing with different data schemas (including Microsoft SQL Server i.e. MSSQL, NoSQL, API Based) is required
* Experience of working with MI / BI Analysts / Visualisers to understand their requirements
* Experience in agile delivery methodologies, CI/CD practices
* Any knowledge of the following is desirable: Teradata, Azure pipelines, Azure DevOps and Linux