Senior Data Engineer - Microsoft Fabric

Job Description

• We are looking for an experienced Senior Data Engineer with strong hands-on experience in Microsoft Fabric, data engineering and software development. The successful candidate must be comfortable writing code and have practical programming experience using C# and Python.
• The candidate must have a strong understanding of modern data engineering principles and be able to design, develop, optimise and support scalable data solutions using Microsoft Fabric.

Responsibilities

• Design, develop and maintain modern data engineering solutions using Microsoft Fabric.
• Develop and maintain data ingestion, transformation and integration pipelines.
• Work with OneLake, Lakehouse and Fabric Warehouse.
• Develop and optimise Data Pipelines and Dataflows Gen2.
• Write and maintain production-quality code using Python and C#.
• Develop and maintain Shell scripts to automate data processing, file transfers, job scheduling and operational tasks.
• Use SQL and, where applicable, PySpark/Spark for data processing and transformation.
• Develop data models and support Power BI and semantic models.
• Monitor, troubleshoot and optimise data pipelines and data workloads.
• Investigate and resolve data quality, performance and integration issues.
• Ensure data solutions are reliable, scalable, secure and fit for purpose.
• Translate business requirements into technical data solutions.
• Collaborate with Data Engineers, BI Developers, Architects and business stakeholders.
• Support the implementation of best practices across data engineering, development and deployment.

Requirements

• 5+ years' experience in Data Engineering or a related technical data role.
• 3+ years' strong hands-on experience with Microsoft Fabric.
• Valid Microsoft Certified: Fabric Data Engineer Associate DP-700 certification – Essential.
• Strong programming and software development experience.
• Proven hands-on experience writing code using C# and Python.
• Hands-on experience with Shell Scripting.
• Strong SQL skills.
• Experience with OneLake, Lakehouse and Fabric Warehouse.
• Experience developing Fabric Data Pipelines and Dataflows Gen2.
• Strong understanding of ETL/ELT, data modelling and data warehousing.
• Experience with Power BI and semantic models.
• Experience with Azure/cloud-based data platforms.
• Strong troubleshooting and problem-solving skills.
• Ability to work independently and engage effectively with technical and business stakeholders.

Certification – Essential

• Microsoft Certified: Fabric Data Engineer Associate DP-700 – Valid certification required.

Advantageous

• Azure Data Engineering experience.
• PySpark/Spark.
• Azure Data Factory, ADLS or Synapse.
• CI/CD and DevOps experience.
• Data governance, security and data quality.
• Experience working in Agile environments.