Back to Work

Case Study

WoolworthsGroup(ASX:WOW)

Data Team Lead · WooliesX Everyday Rewards

Project Timeline

August 2017

Data Team Lead Role

Joined WooliesX Everyday Rewards team as Data Team Lead, focusing on big data solutions on AWS.

November 2017

Tequila Decision Engine Phase-1

Led architectural development of big data decision engine for customer segmentation based on spending patterns.

March 2018

Marketing Transformation

Supported WooliesX journey from campaign-focused to customer-focused targeted marketing, achieving 18% higher email clicks.

July 2018

Data Platform Security Remediation

Led security improvements following KPMG audit, refactoring 6% of code across GitHub repositories.

October 2018

Proximity Project

Developed competitor advantage system offering location-based intelligence for better customer targeting.

January 2019

Everyday Rewards Apple Wallet

Enabled Apple Wallet integration for Everyday Rewards program, improving customer convenience.

May 2019

Data Lake Optimization

Reduced AWS Redshift cluster cost from $200k/month to $80k/month through optimization and data pipeline redesign.

Project Overview

As Data Team Lead (Contract) on the WooliesX Everyday Rewards team, I led architectural and technical development of big data solutions — coaching the team and collaborating with cross-functional partners across Woolworths retail, BWS, Dan Murphy, and Big-W. On the technical side I designed and developed data solutions on AWS, processing billions of transactions from online and brick-and-mortar stores to enable sophisticated customer segmentation and targeted marketing.

My primary project, Tequila (Decision Engine Phase 1), used Salesforce Marketing Cloud and behavioural personalisation algorithms to transform WooliesX from campaign-focused to customer-focused targeted marketing. I also led security remediation following a KPMG audit, significantly reduced AWS infrastructure costs, and delivered location intelligence and digital wallet capabilities that extended the Everyday Rewards programme.

Key Achievements

  • Led Tequila Decision Engine — PySpark customer segmentation on EMR processing billions of transactions; achieved 18% higher email click-through and 3% higher conversion, shifting WooliesX to customer-focused targeted marketing
  • Reduced AWS Redshift cluster cost from $200k/month to $80k/month through data pipeline redesign, Informatica IICS optimisation, sort/distribution key tuning, and Parquet conversion to S3
  • Led security remediation following KPMG audit: refactored 6% of code across GitHub repos, replaced exposed credentials and unsecured GPG keys with IAM roles and Credstash — received green light from auditors
  • Delivered Proximity — location intelligence system providing competitive advantage through better, location-aware customer offers
  • Delivered Everyday Rewards Apple Wallet integration, improving digital convenience and engagement for millions of Rewards members

Technologies Used

Apache Spark / PySpark on EMRInformatica IICSAWS (Kinesis, S3, Lambda, Aurora, DynamoDB, Redshift, ECS)PythonJenkinsSalesforce Marketing CloudPostgreSQLMySQLHiveCredstashIAMGitHubCI/CD pipelines

Results & Impact

The Tequila Decision Engine transformed WooliesX's marketing from broadcast campaigns to precision customer targeting — delivering measurably higher engagement and conversion. Redshift cost optimisation saved $120k per month in AWS infrastructure spend. The KPMG remediation secured the Rewards platform from known vulnerabilities and passed audit. Proximity and Apple Wallet extended the programme's digital reach and competitive positioning.

Want to Discuss This Further?

Happy to detail the Tequila decision engine architecture, the Redshift cost reduction approach, or the KPMG security remediation work.

Shivendra Singh — Head of Data, Information & AI