How Resident Advisor Escaped Year-Long ETL Firefighting

The Background

Resident Advisor is the world’s leading platform for electronic music, connecting millions of fans, artists, and event organisers globally. Real-time event data powers both user experiences and revenue-critical reporting, making their data infrastructure business-critical.

Multiple vendors failed to deliver reliable support. Their engineering team spent a year firefighting infrastructure instead of building product, draining resources, slowing development, and eroding confidence in business-critical data.

The team had tried multiple solutions, commercial ETL tools, custom scripts, even in-house CDC pipelines. None had been able to solve the problem.

 

THE CHALLENGE

Engineering Team Under Siege:
Critical engineering capacity was trapped in maintenance instead of strategic development.

“Basically all I knew for a year was firefighting vendor issues, with regular two-week periods where it was all we were thinking about.”

James Kelly, Data Analyst.

 

Vendor was the bottleneck:
Support tickets sat unresolved for eight months with staff who didn’t understand their own product. No confidence in vendor commitments. No path forward when things broke.

Complete Data Confidence Crisis:

• ETL downtime broke user-facing reporting.
• Leadership lost confidence in the data driving strategic decisions. Revenue was directly at risk.

“We couldn’t afford to stick with our previous vendor, our data reliability is too critical.“

Duncan Williams, CTO

 

THE SOLUTION

Resident Advisor chose Matatika’s Mirror Mode migration process, which runs new pipelines in parallel with existing ones, validating outputs before any switch. This zero-risk approach eliminated the fear of downtime or data loss.

Partnership Methodology Direct access to technical experts who solve both product and business challenges – removing the failed account management layers.

Technology Platform

• PostgreSQL replacing RedShift for better performance and flexibility at their scale
• Integrated ETL platform replacing Stitch Data and custom cron solutions
• Performance-based pricing with cost-neutral migration
• Comprehensive monitoring with proper staging environment

“Everyone we spoke to completely understands the product and what we’re trying to do. The consultancy element is your USP, doing this with poor-quality suppliers was almost impossible.”

Duncan Williams, CTO

 

RESULTS

• 99% faster problem resolution with mean time dropping from 8 months to days
• 100% elimination of firefighting freed engineering team from maintenance to strategic work
• Zero downtime migration replaced data failures with predictable operations
• Cost-neutral transformation maintained infrastructure spend while gaining reliability and expert support
• Staging environment implemented enabling safe testing before production releases
• Technical partnership model eliminated vendor dependency with direct access to solution architects

 

KEY SUCCESS FACTORS

• Zero-Risk Migration Mirror Mode eliminated traditional migration risks by running both systems in parallel until superior performance and reliability were proven.
• Expert Partnership Model Direct access to engineers who understand both technology and business challenges, replacing broken vendor support models with genuine technical partnership.
• Technical Flexibility Solution designed for teams technical enough to need control but not staffed to build everything in-house, avoiding the limitations of no-code tools.
• Cost-Neutral Value Infrastructure costs maintained while delivering dramatic
  improvements in reliability, support quality, and engineering efficiency.

 

STRATEGIC IMPACT

RA’s data team went from spending a year firefighting infrastructure problems to focusing entirely on strategic work that drives business growth. The transformation shows that getting ETL right isn’t just about technology, it’s about freeing your best people to work on what matters.

 

ARCHITECTURE & DATA SOURCES

• Data Integration Sources:
  Mixpanel, Kevel, MS SQL Server, PostgreSQL, Google Sheets
• Infrastructure Migration:
  RedShift + Stitch ETL → PostgreSQL + integrated Matatika platform
• Migration Approach:
  Zero-downtime parallel validation ensuring data consistency
  and business continuity

 

Considering an ETL switch?

Download the ETL Escape Plan →

#Case Study