Global QSR Strengthens GCP Migration with Independent Validation and Testing Support

Quick Overview

Client: A global QSR

⦿ Cloud Platform – AWS, Google Cloud Platform (GCP)

⦿ Data Warehouse -Amazon Redshift, Google BigQuery

⦿ Data Integration – Talend

⦿ Data Processing & Transformation – Databricks, dbt

⦿ Validation Framework – Python-based reconciliation scripts, DVT

⦿ Reporting & BI – Tableau, MicroStrategy, Alteryx

Objective: Validate the architectural integrity, data accuracy, and functional equivalence of the AWS to GCP data platform migration to ensure the migrated platform meets design, performance, and business requirements.
Challenge: Address data inconsistencies, schema mismatches, and reporting gaps during migration
Solution: Wavicle implement an automated validation framework to verify data, schema, reports, and environment readiness throughout the migration process. Standardized architecture validation ensured secure, production-ready deployment.
Results: Wavicle’s structured validation restored stakeholder confidence and ensured the new GCP platform was stable for business operations.

Challenges

As part of a larger cloud modernization effort, this QSR started moving from ETL to ELT pipelines and adopted a modular medallion architecture using dbt. The migration replaced key AWS data components with GCP tools, including Google Cloud Storage (GCS) and BigQuery.

Although the architectural shift offered long-term scalability and self-service capabilities, it also introduced considerable complexity. Data models were being rebuilt, pipelines were restructured, and reporting layers were reconnected to entirely new environments. As workloads transitioned from AWS to GCP, concerns emerged regarding data accuracy, completeness, and consistency.

Beyond data and reporting variances, the program also required consistent architecture-level checkpoints to validate environment configuration, governance standards, and access controls across Dev/Pre‑Prod/Prod.

Post-migration validation uncovered issues that required deeper investigation, including:

Data discrepancies between Redshift and BigQuery
Schema mismatches across tables and columns
Reporting inconsistencies in Tableau, MicroStrategy, and Alteryx
Gaps in testing coverage from the migration vendor
Environment configuration issues affecting pipeline readiness
Repeated regression cycles requiring revalidation
Lack of standardized architecture validation checkpoints and supporting documentation
Limited visibility into pipeline dependencies and orchestration readiness during regression cycles
Security validation gaps across environments (IAM roles, service accounts, and dataset access)

The organization recognized that migration alone was insufficient. To ensure a successful cutover and maintain business continuity, it required an independent validation partner to conduct rigorous end-to-end testing across architecture, data, schema, reporting, and environment layers.

Program Phasing

The QSR engaged Wavicle as an independent validation partner to assess the new GCP environment before full production cutover.

The migration program was divided into two phases to reduce operational risk and ensure controlled validation.

Phase 1 focused on migrating core reporting workloads from Amazon Web Services to Google Cloud Platform. This phase included transitioning data from Amazon Redshift to BigQuery and validating downstream reports to support production cutover. The primary objective was to ensure business continuity and reporting accuracy in the new GCP environment.

Phase 2 extends the transformation by completing the shift from ETL to ELT. Legacy Talend pipelines are being replaced with modular transformations using dbt, aligned with a medallion model. In parallel, existing AWS workloads—including Lambda functions, EMR, and Glue jobs—are being migrated to GCP‑native services such as Dataflow and Cloud Run. All future development will occur natively on GCP to enhance scalability and support long-term modernization.

Solution

Instead of relying only on existing testing tools, Wavicle developed a comprehensive validation framework tailored to the client’s architecture. This approach verified accuracy at each layer of the medallion model, from raw ingestion in Bronze to curated datasets in Gold and downstream reporting tools.

The validation strategy included:

Data validation: Used Google Data Validation Tool (DVT) to reconcile datasets between Redshift and BigQuery using SQL-based comparisons.

Schema validation: Built custom scripts to compare table structures, column definitions, data types, and detect mismatches or missing fields.

Reporting validation: Developed an automated comparison framework to validate report outputs across Tableau, MicroStrategy, and Alteryx.

Architecture governance validation: Standardized architecture review checkpoints to validate platform configuration, naming/metadata consistency, orchestration reliability, and security controls prior to production cutover

Architecture & environment validation: Created automation scripts to verify GCP service configurations, pipeline readiness, and deployment setup.

UAT and regression testing: Conducted end-to-end user acceptance testing, verified fixes, and supported repeated regression cycles to ensure production stability.

The framework consolidated validation across architecture, storage layers, orchestration, transformation standards, and access controls to provide a repeatable sign-off mechanism.

When gaps were identified in the migration vendor’s testing coverage, Wavicle expanded its scope. The team revalidated previously approved components, added checks requested during the project, and supplemented client-side testing tools with manual and automated validation.

By developing the validation suite in-house and aligning it with the client’s medallion architecture, Wavicle ensured that the Wave 1 cutover to GCP was accurate, stable, and ready for business operations.

Result

With Wavicle’s independent validation framework, the client transitioned to production with increased confidence and control. Documented architecture validation improved deployment readiness and gave stakeholders clearer visibility into technical risks, dependencies, and governance compliance before go‑live. The complex migration became a structured, verified process supported by rigorous end-to-end testing.

Wave 1 milestones were achieved as planned, and reporting workloads were successfully migrated to GCP. Business users accessed dashboards and reports in the new environment without disruption.

The engagement delivered:

On-time completion of Wave 1 milestones
Verified data accuracy between Redshift and BigQuery
Consistent and validated reporting across Tableau, MicroStrategy, and Alteryx
Stable GCP environment readiness prior to production cutover
Reduced compute costs for reporting workloads compared to the previous environment, while maintaining performance and reliability
Stronger stakeholder confidence in the migration process
Standardized architecture validation evidence to support production deployment readiness and architecture sign‑off
Improved platform consistency and reduced manual governance review effort through repeatable validation checkpoints

Migration validation impact

Area	Before Wavicle Engagement	Wavicle Validation Approach	After Validation
Data accuracy	Discrepancies between Redshift and BigQuery	Google DVT reconciliation and SQL-based comparisons	Reconciled and validated datasets
Schema consistency	Mismatched tables, columns, and data types	Custom-built schema validation scripts	Fully aligned table structures
Report reliability	Inconsistent outputs across BI tools	Pixel-by-pixel automated report comparison	Verified reporting consistency across Tableau, MicroStrategy, and Alteryx
Environment readiness	Configuration uncertainty in GCP setup	Automated GCP environment validation scripts across Dev/Pre-Prod/Prod	Production-ready cloud configuration
Testing coverage	Gaps in migration vendor testing	End-to-end automated validation framework	Comprehensive validation across data, schema, and reports
Architecture governance & security	Limited standardized architecture checkpoints; inconsistent validation of security and access controls across environments.	Standardized architecture review checkpoints to validate platform configuration, naming/metadata consistency, orchestration reliability, and security controls (IAM roles, service accounts, dataset access, and row-level security) prior to production cutover.	Documented architecture readiness and security controls to support production deployment sign-off

Through independent validation and support across multiple regression cycles, Wavicle ensured that the new GCP platform was operational, accurate, stable, and ready for future development.

Next Steps

With Wave 1 validated, the client is proceeding to Wave 2, which includes full ETL to ELT modernization and migration from AWS Databricks to GCP Databricks. All future development will be built natively on GCP, with Wavicle continuing to provide validation, testing, and quality assurance to ensure long-term platform stability and scalability.

In Wave 2, Wavicle will extend the same validation approach to support the GCP Databricks transition and ensure the new GCP-native platform remains stable and production-ready.