Lead Data Engineer

Shoprite Group of Companies · Cape Town, Western Cape

Stop applying one at a time.

JobAlertsZA auto-applies to South African jobs like this one for you, overnight. Upload your CV once — we do the applying.

Start free — we apply for you →

About the role

  • We're looking for an experienced Lead Data Engineer to lead the design, development, and evolution of enterprise-scale data platforms that enable data-driven decision-making across the Group.
  • This role combines deep technical expertise with leadership responsibilities, overseeing advanced data engineering solutions, guiding engineering teams, and helping shape the future data strategy within a modern cloud and big data environment.
  • As a senior technical leader, you will drive architecture decisions, establish engineering best practices, mentor team members, and ensure the delivery of scalable, secure, and high-performing data solutions.

What you'll do

Data Engineering & Architecture

  • Design, build, and optimise complex data engineering solutions and platforms.
  • Define and maintain data architecture standards and technical roadmaps.
  • Develop scalable data pipelines and big data solutions using modern cloud technologies.
  • Ensure technical documentation remains current, accurate, and aligned with architectural standards.
  • Drive innovation and continuous improvement across the data engineering landscape.

Technical Leadership

  • Provide technical guidance and mentorship across data engineering teams.
  • Establish and enforce engineering standards, development patterns, and best practices.
  • Lead DevSecOps adoption and ensure quality engineering practices are consistently applied.
  • Share knowledge through communities of practice, technical presentations, and mentoring initiatives.
  • Stay current with emerging technologies, tools, and industry trends.

Agile Delivery & Stakeholder Engagement

  • Facilitate planning, backlog grooming, sprint reviews, and team alignment activities.
  • Translate business requirements into scalable technical solutions.
  • Partner with Business Analysts, product owners, architects, and stakeholders to drive successful outcomes.
  • Provide project, delivery, and incident updates to leadership and key stakeholders.
  • Maintain alignment between delivery plans and strategic objectives.

Team Development

  • Support performance management, development planning, and capability growth within the team.
  • Identify skills gaps and contribute to learning and development initiatives.
  • Promote collaboration, engagement, and a high-performance engineering culture.
  • Facilitate innovation initiatives, hackathons, and knowledge-sharing activities.

What you bring

Qualifications

  • Degree or Diploma in Computer Science, Mathematics, Engineering, Information Systems, or a related field NQF 6 or higher.
  • Postgraduate qualification in a relevant discipline would be advantageous.
  • Equivalent practical experience will be considered where formal qualifications are not held.

Experience

  • 5+ years' experience in Data Engineering, Software Development, BI, or a related data discipline.
  • Demonstrated experience leading or mentoring Data Engineering teams.
  • Proven experience delivering enterprise-scale data solutions.
  • Experience within retail, FMCG, or similarly complex environments is advantageous.

Technical Skills

  • Advanced Python and PySpark development.
  • Strong SQL skills, particularly in large-scale data environments.
  • Big Data technologies including Hadoop and Spark.

AWS services such as

  • EMR
  • S3
  • Athena
  • RDS
  • Lambda
  • API Gateway
  • Experience with Azure, GCP, and Snowflake.
  • Data warehouse and Business Intelligence concepts.
  • Solution architecture and technical design.
  • Software Development Life Cycle SDLC.
  • DevOps and Infrastructure Management practices.

Closing Date

  • 2026/09/10
Auto-apply to this jobView original posting ↗
Lead Data Engineer at Shoprite Group of Companies — Cape Town, Western Cape · JobAlertsZA