Azure Data Engineer

Location: India Salary: US$100 - US$150 per day + fully remote
Sector: IT Service Provider Type: Contract
Reference #: CR/077724_1613564632

Are you an experienced Azure Data Engineer with a passion for excellence who is looking for their next opportunity? Then we may have the perfect opportunity for you!

A client of RED's, a global consulting firm, are currently seeking two Azure Data Engineers on a fully remote basis to work with a new customer and supporting a team who strive for excellence.

What you will do:

  • Meet with stakeholders and individual contributors, including Business and Technology, to understand and document current-state enterprise data & process architecture.
  • Understand application data and how it is used across the enterprise to define data archive standards and implement a data archive solution across global database estate.
  • To provide creative solutions for data-related problems at scale by mixing in house and open-source technologies.
  • Help build robust data pipelines and ETL using cloud tools like Azure DATA FACTORY and Azure Data Lake.
  • Enable effective decision making by retrieving and aggregating data from multiple sources and extract information from complex data sets.
  • Mine and analyze data sets across various touchpoints (files, web, app, marketing platforms) to provide actionable insights for business teams to make data-driven decisions.
  • Responsible for building real-time and batch data ingestion pipelines integrating different data sources like RDBMS, data warehouses, plant legacy systems, historian systems, file storage systems etc.

Skills required

  • We are looking for an experienced Data Engineer to build the data and feature engineering pipelines powering the next generation of products at our company. Data engineers will interact with the product managers and internal customers to understand the data landscape and how to acquire the product's data.
  • You will work within the Power Platform and Data engineering, which is accountable for delivering value to business groups. The team does this by using modern techniques, tooling, and behaviors to deliver software as quickly as possible.
  • You will be asked to accomplish a wide range of tasks; all focused on using modern technology to build scalable, robust data ingestion and staging pipelines.
  • We are looking for an experienced Data Engineer who is passionate about Microsoft technology and has skills in ETL technologies such as Azure Data Factory, Azure Synapse, Azure SQL Database, SQL Server ORACLE Database.
  • Overall, 8+ years of Data initiative experience.
  • 6+ years of hands-on experience with ETL developments is a must.
  • 4+ years of hands-on experience with Oracle PL/SQL and SQL Server T-SQL Programming is a must.
  • 3+ years of hands-on experience with Cloud data engineering and data pipelines using Azure tools.
  • Good exposure with Oracle PL/SQL and SQL Server T-SQL Programming.
  • Must be proficient in writing Stored Procedures, Triggers, Functions and Views using PL/SQL and T-SQL.
  • Expertise with the Microsoft cloud ETL tools Azure Data Factory, Azure Synapse (DWH) and Azure Data Bricks.
  • Expert level exposure with SQL Server Integration Services (SSIS).
  • Expertise with ADLS v2 for Data Lake design, importing data and administration using both command line and GUI interface.
  • Must have experience in using Azure Cognitive Services to process structured & unstructured data.
  • Design, develop and maintain appropriately scaled, effective ETL solutions to support data loading, processing, reports generation and reconciliation using Microsoft SQL Server and Oracle databases.
  • Experienced with the Azure stack and Microsoft PaaS offerings like Azure SQL Database, Azure Data Factory, Azure Synapse (DWH), Azure Data Bricks, Logic Apps, Azure Functions and Azure Flow.
  • Expertise in developing ETL/ELT workflows.
  • Experience in Cloud platform - Azure eco-system.
  • Experience working with different data storage options including Azure BLOB storage, ADLS Gen 1 and Gen 2 etc.
  • Experience in building Modern ETL process using Azure Data Factory, Azure SSIS IR, Azure File Sync Agent, Data Lake, Synapse, Azure SQL Database, U-SQL, Databricks, Azure Functions, SSIS and Azure Storage.
  • Good to have experience in Python, SQL, PowerShell
  • Professional experience designing, building, and maintaining data ingestion pipelines.
  • Experience with CI\CD using Azure DevOps and Azure Git.

If this role could be of interest, and you would like more details, please get in touch with your up to date CV!