Product Update: October

Every day we meet data teams that are stretched to breaking point.  A day without pipeline errors is worth celebrating.  Spending hours moving changes from one place to another is normal.  Working on the actual data and helping their business counterparts analyze business opportunities is a distant dream.  Matatika’s modern dataops platform makes data operations simple – helping you get more done, be more agile, and create trusted data!

TL;DR

News, features, and data

• Matatika is one of the most innovative big data companies 2023 Learn more…
• Be more agile with automated release and rollback
• Create trusted data with tests and transformations
• Get more done with online development environments
• Understand your data better with data lineage
• Released stacks of features
• Squashed heaps of bugs
• Updates to our data sources and plugins

Release and rollback, we’ve got you

Driven by our customers’ needs for better collaboration and coordination of changes, we’ve made some big improvements to our integration with GitHub – the leading source management platform.  We now support private GitHub repositories, automated deployment via GitHub webhooks, and we made it really easy to use GitHub code releases to orchestrate testing and change promotion from dev, to test, to production.  This closer integration with GitHub complements our open and progressive approach to development, No Code all the way to Full Code management if you need it.  Our customers tell us it has never been so easy to manage their changes!

Introduction to change promotion Learn more…

Setup change promotion in the Matatika Platform Read our documentation…

Test your data, transform your results

Together with our customers, we have seen the benefits of providing a small number of high quality and trusted datasets to users who need self-service analytics.  By investing in models that represent the business and tests that reconcile the data view of the world, data teams are able to confidently provide answers to the bulk of the businesses data questions. We now support out of the box test reports and we have added model validation pipelines as a standard pipeline in every workspace.  Coupled with our notifications, we have everything data teams need to get started with automated testing of their data.

Introduction to developing and testing transformations Learn more…

Transforms and testing in the Matatika Platform Read our documentations…

Painless development, from anywhere

The Matatika Platform makes it simple and straightforward for developers to create isolated Workspaces that are near replicas of production.  We have now made it easy to develop changes from anywhere.  All editions of our platform come packaged with standard devcontainers that can be used in VSCode or fully browser based in GitHub Codespaces.  In these development environments, you develop and test your changes before committing them and then test all together in your Matatka Workspace.  Go to your code repository, login, copy your environment from Matatika, press ‘.’ and you are developing.  It’s that simple.

Setup your development environment in the Matatika Platform Read our documentation…

Understand your data, build trust

Users instantly lose trust in your data if it doesn’t pass the sense check.  Furthermore, some users will dig into “what that number really means and where it comes from”.  With the Matatika Platform we now create self-documenting data lineage artefacts to answer these questions instantly.  Drill down into the lineage for any of your high quality, trusted datasets.  Documentation is no longer painful and you strengthen trust in your data with modern dataops!

Introduction to data lineage Learn more…

Features

• Data heroes using the Matatika lab can now watch for errors and all activity on pipelines
  Watch Pipelines how-to guide
• Data heroes can now configure a Custom Email Service Provider
• Data heroes can now configure a domain whitelist allowed to access their instance of the Matatika app
• Data heroes can now see the GitHub commit ID for the deployed version of their workspace
• Data heroes can now view data lineage as Mermaid visualisations, including zoom and pan functionality
• We now support creating Matatika Workspaces from Meltano project repositories
• We now support a new GitHub app for cloning Matatika Workspaces from private GitHub repositories
• We can now create a workspace referencing an existing repository and branch (public and private)
• New workspace task in-progress indicator
• New icon for user tasks
• We added support for Markdown links in table datasets

Fixes and updates

• Fixed a ton of minor mobile styling issues
• Fixed display of channels deployed from repository
• We can now handle switching between up to 99 workspaces
• We now render markdown description from datasets correctly in Lab
• Don’t parse default environment from workspace.yml (now just meltano.yml)
• Relaxed validation of dataset alias through create/update from API
• Export from app returns dataset raw data, if present
• Regression where defined setting order not applied on plugin load
• Improved performance of hub endpoints
• Documented timeout for PipelineML
• Documented missing step to Specify a Custom Auth0 Identity Provider in getting started guide
• Docs improvement to show example configuration in both terminal and docker-compose.yml for Getting Started Community Edition

Data source and plugins

• Instant Insights for Capsule CRM data source
• UK Bank Holidays data source tap-govuk-bank-holidays
• Major rework of our Shopify data source
• Minor update to our Adwords data source to support the latest API version
• Minor config update and fixes to our Capsule CRM data source
• New matatika variant of tap-dbt-artifacts extractor
• New dbt-artifacts plugin
• New analyze-dbt-models plugin
• Add support for Meltano’s plugin requires logic
• We also released a number of security and dependency updates to our Matatika supported data sources. Why not join our Slack group to check out the full release notes.

Finally, if any of the above modern dataops features has piqued your interest, get in touch or consider giving us a star on GitHub!