Posts Tagged ‘ETL Tools’
Why marketing data connectors quietly inflate your ETL costs
And how MVF consolidated paid media data without losing insight Marketing data rarely breaks loudly. It degrades quietly. Pipelines keep running. Dashboards still load. Spend gets approved. But somewhere between your tenth and thirtieth connector, the economics stop making sense. Teams often assume marketing connectors are cheap because each one looks small in isolation. A Google Ads connector here. A LinkedIn Ads connector there. Another for TikTok. Another for reporting exports. Each one feels justified. Together, they quietly become the most expensive part of the data stack. MVF learned this the hard way.
Building Data Platforms That Actually Solve Business Problems
Meet Teddy Bernays Teddy is a highly autonomous Freelance Data Engineer and Google Cloud Trainer who focuses on helping startups and mid-sized companies build efficient, scalable, and cost-effective data platforms. He started his career in the complex world of audio engineering before transitioning to IT, where he found a fascination with the mechanics "under the hood" of data systems. Today, he is a firm believer that the solution to data inconsistency isn't always more code, but more clarity. His approach is simple: “If it’s simple, do it simple. You don't need three different tools to solve a one-tool problem.”
MVF Makes ETL 7x Cheaper While Migrating 1B+ Rows Across 60+ Sources
MVF is a global customer generation platform that depends on fast, accurate data pipelines to drive reporting and marketing performance across multiple regions and channels. Their ETL spend had nearly tripled under a new pricing model, creating unsustainable cost pressure. Costs were escalating faster than business value, engineering teams were stretched, and they needed a simpler, more sustainable approach.
Background:
MVF is a global customer generation platform that depends on fast, accurate data pipelines to drive reporting and marketing performance across multiple regions and channels.
Their ETL spend had nearly tripled under a new pricing model, creating unsustainable cost pressure. Costs were escalating faster than business value, engineering teams were stretched, and they needed a simpler, more sustainable approach.
As Andonis Pavlidis, Head of Data at MVF, explained: “Prices were always increasing, SLAs were dropping. It didn’t feel like a partnership anymore, but rather like a commodity.”
Challenge: Spiralling Costs and Deteriorating Service
MVF’s data team was under increasing strain from a fragile and inefficient ETL setup.
- Engineering drag – half a dozen custom pipelines in Airflow consumed valuable analyst time.
- Support bottlenecks – four-day ticket cycles left the team firefighting instead of focusing on growth. As Andonis Pavlidis described it: “When something went down with Fivetran, it was four days of back and forth tickets. If something goes down with Matatika, it’s a Slack message and it’s fixed by the next day.”
- Sensitive data risk – HIPAA/GDPR disposition pipelines relied on fragile infrastructure outside MVF’s control.
- Fragmented pipelines – multiple parallel connectors inflated costs and slowed data delivery.
Solution: Mirror Mode Migration + Custom Connector Enablement
MVF partnered with Matatika to deliver a risk-free migration strategy that cut costs and simplified infrastructure.
- Mirror Mode migration – replicated all existing pipelines in parallel, ensuring zero disruption during validation and cutover.
- Custom connector replacement – rebuilt and stabilised bespoke Airflow jobs, including Iterable, Five9, Survicate, Everflow, Injixo, Invoca, Airtable and more. New connectors such as Baidu were also developed.
- Sensitive data improvements – re-engineered disposition pipelines from fragile webhooks to robust API calls, giving MVF safer and more controllable data flows.
- Simplified operations – decommissioned Airflow and introduced a dedicated development workspace for safe testing and deployment.
As Andonis Pavlidis noted: “We loved working with the Matatika team as they felt an extension of our team. They shared the same frustrations and reacted as we would react ourselves.”
Results: 7x Cheaper, Zero Risk, Superior Performance
The migration delivered more than savings. MVF gained faster pipelines, stronger reliability, and greater team productivity. Is this better?
- 7 x cheaper than renewal (86% cost reduction) – achieved with Matatika’s performance-based pricing, while migrating 1B+ rows across 60+ sources.
- Risk-free migration – completed with zero downtime or disruption.
- Up to 4x faster pipelines – Bing Ads connector reduced from 1.8 hours to 30 minutes, enabling intraday refresh and reduced Snowflake cost.
- Engineering capacity reclaimed – two days of monthly maintenance eliminated, freeing analysts for higher-value projects.
- Same-day issue resolution – from “four-day ticket cycles” to direct Slack access with Matatika engineers.
- Complete infrastructure modernisation – legacy Airflow retired, fragmented pipelines consolidated, and data architecture strengthened for scalability and compliance.
As Andonis Pavlidis concluded: “We went to Matatika for a migration. We ended up with an improvement of our stack.”
Appendix: Full Connector List
Fivetran connectors migratedGoogle Adwords, GA4 Export, Iterable, MySQL (x12), Facebook Ads, Appwiki, Bing Ads, Webhooks, Outbrain, Taboola, Twitter Ads, TikTok Ads, LinkedIn Ads, Google Sheets, S3 and Dbt Cloud Reporting.
Custom Airflow connectors migrated to Matatika supported connectorsAirtable, Five9, Survicate, Everflow, Injixo, Invoca, Custom S3.
New connector developedBaidu and CallMiner
Appendix: Full Connector List
MVF’s migration spanned a wide range of connectors across marketing, customer interaction, compliance, and operational data sources, together accounting for more than 1B rows per year.
| Category | Example Connectors | Estimated Rows / Year |
| Marketing & Paid Media | Google Adwords, GA4 Export, Facebook Ads, Bing Ads, TikTok Ads, LinkedIn Ads, Outbrain, Taboola | 600M |
| Customer Interaction | Iterable, Five9, Survicate, Everflow, Invoca (real-time + historical) | 250M |
| Sensitive / Compliance Data | Webhooks (disposition bronze), Airtable, Injixo | 120M |
| Operational / Other Sources | MySQL (x12), Google Sheets, S3 Export, Dbt Cloud Reporting, ExpertReview S3 | 30M |
| New Development | BaiduCall MIner | Included above |
Google Sheets for Analysts: How to Eliminate the Data Update Bottleneck
The Ticket Queue That Never Ends
Your analyst sends a Slack message at 9am.
“Hey, can you update the product-to-GL mapping table? Finance added three new categories last week and our revenue reports are off.”
It’s a five-minute job. You know exactly what needs doing. But you’re in the middle of debugging a pipeline issue, and context switching now means you’ll lose an hour getting back to where you were.
“I’ll get to it this afternoon,” you reply.
By 2pm, you’ve forgotten. The analyst follows up at 4pm. You finally do it at 5:30pm, right before you are about to leave. The analyst thanks you, clearly frustrated that a simple update took all day.
The next week, same analyst, different request. Update the territory assignments. Then the customer segment definitions. Then the exchange rates. Then back to the GL mappings because something changed again.
You’re not a data engineer anymore. You’re a ticket processor for data updates that shouldn’t require engineering at all.
Stop Manually Uploading Spreadsheets: 5 High-Impact Use Cases (and How Resident Advisor Fixed It)
If your week still includes exporting CSVs and uploading them into dashboards, you’re paying a “data tax” in delays, context switching, and stale numbers. Here are the top five use cases where teams should replace manual uploads with a real Google Sheets → warehouse pipeline plus a quick look at how Resident Advisor (RA) made this work in production.
Aftermath of the DBT + Fivetran Merger: What It Really Means for Data Teams
Two weeks on from the DBT + Fivetran merger, and the dust hasn’t settled, it’s only just starting to reveal what comes next. The deal didn’t just merge two companies. It merged two very different philosophies about how modern data should move, transform, and deliver value.
Build Strategy First. Choose Technology That Deserves It
Introduction Every organisation has lived this story. A new data platform or AI tool promises to fix everything. Budgets are approved, contracts are signed — and a few months later, the same problems remain. Dashboards don’t align, pipelines still break, and confidence in the numbers keeps slipping. For Dylan Anderson, Head of Data Strategy at Perfusion, the real issue isn’t the technology itself — it’s where teams choose to begin. “Vendors are great at selling the idea that their product will solve all your data needs,” he says. “But strategy isn’t about buying more tools. It’s about helping the business reach its goals using data, technology, and AI in the right way.” In this episode of Data Matas, Dylan joins host Aaron Phethean to explore why strong data strategies always start with purpose, and how honest, outcome-driven technology earns its place in the stack.
How high-performing data teams align business goals, data strategy, and technology to build trust that lasts
The Smart Way to Connect Mixpanel to Your Revenue Data
Mixpanel gives you brilliant product analytics. Funnels, retention, user journeys. You can see exactly what users do in your application. But as teams mature, they start asking questions Mixpanel can't answer on its own. Which users are most valuable? Which marketing campaigns drove high-value customers? How does feature usage correlate with revenue? The most valuable insights come from connecting Mixpanel data to the rest of your business. User behaviour linked to revenue outcomes. Product engagement joined to marketing attribution. Feature usage correlated with customer lifetime value. That's what becomes possible when you extend Mixpanel with a warehouse-first approach.
Three Things Every Data Leader Should Kill Before Building Another Dashboard
Most organisations are drowning in dashboards that no one trusts. In this Data Matas episode, former Worldpay and FIS data leader Phil Thirlwell explains why the key to better decisions isn’t building more it’s stopping first. He breaks down how dashboard sprawl, KPI overload, and service-desk habits create chaos, and how treating dashboards like products can rebuild trust. Phil shares practical ways to simplify metrics, prioritise outcomes, and run data teams with purpose. The takeaway: fewer dashboards, clearer decisions, stronger alignment between data teams and the business.
How to kill dashboard sprawl, service-desk habits, and KPI overload to rebuild trust in data.
From Tool Mastery to Systems Design: How Data Engineers Actually Grow
Many data engineers plateau after mastering tools but struggle to scale because they haven't learned to think in systems. This blog explores how transitioning from query writing to system design is the key to sustainable growth, effective mentorship, and resilient analytics platforms.