Product Update: August

TL;DR

• Stability is the priority: data must be reliable

We are delighted with all the feedback from our customers; it helps us build a better product.  Increasingly, as they use our platform to move and transform their data, they steer our direction towards a faster, more efficient product experience. 

This month we bring you four new product features which make your workspaces both flexible AND stable – we have a new way of working that will save you time, effort, and most importantly errors.

Along with many of our partners, we are dissolving obstacles by rewriting the coding rulebook.  “This is how we’ve always done it” is not a phrase you will ever hear from us. 

The best bit is that new customers can also benefit from all this development, without taking months to build their own cloud data platform, particularly when it comes to pipelines.

Read on for the features we have been working on this month

 

Composable data imports and reports

A lot of businesses are still using custom scripts to connect data pipelines.  We are all about making this faster. Now you can add and remove upgradable data components.  As well as build custom actions with native support for Meltano tasks.  The flexibility here is a game changer: from data imports to running scheduled reports, any custom job can be configured and scheduled in no time! 

A new approach to Data Sharing Ops

To share live information with your customers,  you need to be confident that the data is accurate, reliable and dependable.  Making sure the test and live environments are stable is a priority for us and our customers. 

This issue, well known to developers, needed a new solution for data and so we have created configuration in code, with secure credential deployment, for your development, staging and production environments.

Not only does it save time, it also minimises errors.   Our innovative approach means sensitive credentials management is taken care of and your changes are deployed faster and more securely.

New data as code artefacts

The Matatika DataML artefacts enable you to promote stable releases so your customer information sharing is robust and secure.  This month we have released Pipelines in addition to our Datasets, Channels, and Workspace definitions.

Pipelines – share information automatically with your customers or import their information so you share up to date data

Datasets – data sharing and visualisations

Channels – organise the way you present the information and who sees it

Workspace – consistency in branding, terminology and customisation make on and offline environments the same 

https://www.matatika.com/docs/dataml/

Native support for Meltano

Meltano is fast becoming the data extraction platform of choice and has a growing list of plugins besides the 300 data sources.  We are pleased to be an active partner in the Meltano community and have contributed many features and plugins.  For example this month, we have contributed several features that help our platform, and anyone else in the community, to import existing Meltano projects and immediately host these projects.

New

• #6578 Add –no-install option to meltano add — Thanks, @Reuben (Matatika)!
• #6596 Support glob paths in file plugin update extra — Thanks, @Reuben (Matatika)!
• #6581 Add new settings for hub_api_root and hub_url_auth — Thanks, @Daniel Walker (Matatika)!

For more of what Meltano are doing… 

If you want a better look at the technical features we have written then look at the links below.  

• Pipeline YML deploy & hot reload
• Pipeline environment endpoint
• Workspace default environment 
• Lock file deployment 
• Meltano.yml deploy

Watch out for a big announcement next month.  We’ve got something really exciting to show you.