Pipelines
A pipeline is a workspace-specific configuration or definition for extracting, loading and transforming data from a given dataplugin - following the concept of ELT. Pipelines are run as jobs to load data into a workspace, either manually or on a predetermined schedule. Only a single pipeline can be run at any given time.
Objects
Pipeline
Path | Type | Format | Description |
---|---|---|---|
id |
String |
Version 4 UUID | The pipeline ID |
status |
String |
Pipeline Status | |
name |
String |
The pipeline name | |
schedule |
String |
Cron | The interval at which to launch a new job e.g. 0 0 9-17 * * MON-FRI launches a job on the hour nine-to-five weekdays |
timeout |
Integer |
Unsigned | The number of seconds after which the job will terminate - if set to 0 , an implicit default value of 300 seconds is used |
script |
String |
Bash | Custom script to be executed as the pipeline job |
created |
String |
ISO 8601 timestamp | When the pipeline was created |
lastModified |
String |
ISO 8601 timestamp | When the pipeline was last modified |
properties |
Properties |
The properties to run the pipeline with, defined by the pipeline dataplugin settings |
|
dataSource |
String |
The pipeline data source dataplugin name |
|
dataStore |
String |
Version 4 UUID | The pipeline datastore id |
_embedded.dataSource |
DataPlugin |
The pipeline data source dataplugin | |
_embedded.dataStore |
DataStore |
The pipeline data store | |
_embedded.'latest job' |
Job |
The latest job run from the pipeline |
{
"id" : "1dcbb564-53c3-474c-8d18-0ab8b5961144",
"status" : "READY",
"name" : "SIT-generated pipeline (updated)",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:34:15.653809",
"lastModified" : "2022-06-30T15:34:15.65381",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"oauth_credentials.refresh_token" : "refresh_token",
"view_id" : "1234567890",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
},
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"created" : "2022-06-30T15:34:17.35072",
"type" : "PIPELINE_CONFIG",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2022-06-30T15:34:41.313",
"endTime" : "2022-06-30T15:35:50.585",
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
},
"delete job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"type" : "DELETE"
},
"logs" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4/logs?sequence=0",
"type" : "GET"
}
}
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"verify pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/verification",
"type" : "POST"
},
"create job" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "POST"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
}
}
}
Properties
For each setting s
in the dataplugin settings
:
Path | Type | Description |
---|---|---|
s.name |
s.kind |
Refer to s.description |
Formats
Pipeline Status
Value | Description |
---|---|
READY |
The pipeline completed processing resource changes |
PROVISIONING |
The pipeline is processing resource changes |
FAILED |
The pipeline failed to process resource changes |
Requests
- View all pipelines in a workspace
- View a pipeline
- Initialise a pipeline in a workspace
- Create or update a pipeline in a workspace
- Create or update a pipeline as a draft
- Validate a pipeline configuration in a workspace
- Verify a pipeline
- Delete a pipeline
See Also
- View all running or completed jobs for a pipeline
- Create a job from a pipeline
- Subscribe to a pipeline
View all pipelines in a workspace
GET
/api/workspaces/{workspace-id}/pipelines
Returns all configured pipelines in the workspace {workspace-id}
.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Pipeline collection with HAL links.
{
"_embedded" : {
"pipelines" : [ {
"id" : "1dcbb564-53c3-474c-8d18-0ab8b5961144",
"status" : "READY",
"name" : "SIT-generated pipeline (updated)",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:34:15.653809",
"lastModified" : "2022-06-30T15:34:15.65381",
"properties" : { },
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"created" : "2022-06-30T15:34:17.35072",
"type" : "PIPELINE_CONFIG",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2022-06-30T15:34:41.313",
"endTime" : "2022-06-30T15:35:50.585"
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"verify pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/verification",
"type" : "POST"
},
"create job" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "POST"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
}
}
}, {
"id" : "5ff689e1-708e-4007-a2e2-3619edb7ba86",
"status" : "DRAFT",
"name" : "SIT-generated pipeline (updated)",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:31:44.470698",
"lastModified" : "2022-06-30T15:31:44.4707",
"properties" : { },
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "4c6511d0-2fe2-4a4c-b461-e7a1a578803f",
"created" : "2022-06-30T15:31:45.394542",
"type" : "PIPELINE_CONFIG",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2022-06-30T15:32:08.358",
"endTime" : "2022-06-30T15:34:10.237"
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"verify pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/verification",
"type" : "POST"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f"
}
}
} ]
},
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines?page=0&size=20&sort=name,asc"
}
},
"page" : {
"size" : 20,
"totalElements" : 2,
"totalPages" : 1,
"number" : 0
}
}
View a pipeline
GET
/api/pipelines/{pipeline-id}
Returns the pipeline {pipeline-id}
.
Prerequisites
- Pipeline
{pipeline-id}
must exist
Request
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144' -i -X GET \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("GET", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Pipeline with HAL links.
{
"id" : "1dcbb564-53c3-474c-8d18-0ab8b5961144",
"status" : "READY",
"name" : "SIT-generated pipeline (updated)",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:34:15.653809",
"lastModified" : "2022-06-30T15:34:15.65381",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"oauth_credentials.refresh_token" : "refresh_token",
"view_id" : "1234567890",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
},
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"created" : "2022-06-30T15:34:17.35072",
"type" : "PIPELINE_CONFIG",
"exitCode" : 0,
"status" : "COMPLETE",
"startTime" : "2022-06-30T15:34:41.313",
"endTime" : "2022-06-30T15:35:50.585",
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
},
"delete job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"type" : "DELETE"
},
"logs" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4/logs?sequence=0",
"type" : "GET"
}
}
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"verify pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/verification",
"type" : "POST"
},
"create job" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "POST"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
}
}
}
Initialise a pipeline in a workspace
POST
/api/workspaces/{workspace-id}/pipelines
Initialises a new pipeline in the workspace {workspace-id}
.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines' -i -X POST \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Pipeline with HAL links.
{
"id" : "5ff689e1-708e-4007-a2e2-3619edb7ba86",
"status" : "PROVISIONING",
"timeout" : 0,
"created" : "2022-06-30T15:31:43.28017",
"lastModified" : "2022-06-30T15:31:43.280172",
"properties" : { },
"_links" : {
"create pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86",
"type" : "PUT"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/draft",
"type" : "PUT"
},
"validate pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/validation",
"type" : "POST"
}
}
}
Create or update a pipeline in a workspace
PUT
/api/workspaces/{workspace-id}/pipelines/{pipeline-id}
Creates or updates the pipeline {pipeline-id}
in the workspace {workspace-id}
.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Body
Pipeline resource.
{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86' -i -X PUT \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json' \
-d '{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86"
data = {
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK / 201 Created
Pipeline with HAL links.
{
"id" : "5ff689e1-708e-4007-a2e2-3619edb7ba86",
"status" : "PROVISIONING",
"name" : "SIT-generated pipeline",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:31:44.470698",
"lastModified" : "2022-06-30T15:31:44.4707",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"oauth_credentials.refresh_token" : "refresh_token",
"view_id" : "1234567890",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
},
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "4c6511d0-2fe2-4a4c-b461-e7a1a578803f",
"created" : "2022-06-30T15:31:45.394542",
"type" : "PIPELINE_CONFIG",
"status" : "QUEUED",
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f"
},
"delete job" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f",
"type" : "DELETE"
},
"logs" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f/stopped",
"type" : "PUT"
}
}
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/5ff689e1-708e-4007-a2e2-3619edb7ba86/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"withdraw job" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f/stopped",
"type" : "PUT"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/4c6511d0-2fe2-4a4c-b461-e7a1a578803f"
}
}
}
Create or update a pipeline as a draft
PUT
/api/workspaces/{workspace-id}/pipelines/{pipeline-id}/draft
Creates or updates the pipeline {pipeline-id}
in the workspace {workspace-id}
as a draft.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Body
Pipeline resource.
{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft' -i -X PUT \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json' \
-d '{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft"
data = {
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics",
"schedule" : "0 0 0 25 12 ?",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"view_id" : "1234567890",
"oauth_credentials.refresh_token" : "refresh_token",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
}
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("PUT", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK / 201 Created
Pipeline with HAL links.
{
"id" : "1dcbb564-53c3-474c-8d18-0ab8b5961144",
"status" : "PROVISIONING",
"name" : "SIT-generated pipeline",
"schedule" : "0 0 0 25 12 ?",
"timeout" : 0,
"created" : "2022-06-30T15:34:15.653809",
"lastModified" : "2022-06-30T15:34:15.65381",
"properties" : {
"end_date" : "2022-06-30T16:31:44.141753+01:00",
"oauth_credentials.access_token" : "access_token",
"reports" : "reports",
"oauth_credentials.refresh_token" : "refresh_token",
"view_id" : "1234567890",
"oauth_credentials.client_id" : "client_id",
"oauth_credentials.client_secret" : "client_secret",
"start_date" : "2022-05-30T16:31:44.141674+01:00"
},
"dataSource" : "tap-google-analytics",
"dataStore" : "1e39e629-3622-4546-990a-024fd22927b0",
"_embedded" : {
"dataSource" : {
"id" : "67caf756-749f-43ab-b086-d91a59bb2ebc",
"name" : "tap-google-analytics",
"label" : "Google Analytics",
"imageUrl" : "/assets/images/datasource/tap-google-analytics.svg",
"description" : "App and website analytics platform hosted by Google\n\n## Prerequisites\n\nBefore importing data from Google Analytics you or your webmaster will need to have performed some initial setup to grant\nyourself access to the Google Analytics for your website.\n\n[Google Analytics Prerequisites](https://www.matatika.com/docs/data-sources/tap-google-analytics/prerequisites)\n\n## Data source settings\n### View ID\n\nOnce you have connected with your google account, the `View ID` drop down will allow you to choose which view you want to get your data from.\n\n### Start Date\n\nProviding a start date allows you to set how much historical data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### End Date\n\nProviding an end date allows you to limit how much data you wish to import into you workspace. \n\nThis date has to be in the ISO-8601 date format, see the example below.\n\nFormat: `YYYY-MM-DDTHH:MM:SSZ`\n\nExample: `2020-10-01T13:12:11Z`\n\n### Reports\n\nProviding the path to a reports json file allows you to supply the tap with a user defined definition of tables and information you want to sync from Google Analytics.\nThis does require you to commit a file to your workspace repository so that Google Analytics can find it during run time, and may also break the dbt transforms that are required to have our included datasets work.\nFor more information on how to define a custom reports file see the [Meltano Hub Documentation](https://hub.meltano.com/extractors/google-analytics#reports).\n\n## Learn more\n\n[Matatika Google Analytics Data Source Documentation](https://www.matatika.com/docs/data-sources/tap-google-analytics)\n\n[Meltano Hub Google Analytics Documentation](https://hub.meltano.com/extractors/google-analytics)"
},
"dataStore" : {
"id" : "1e39e629-3622-4546-990a-024fd22927b0",
"name" : "Default",
"imageUrl" : "/assets/images/datastore/target-postgres.png",
"loader" : "target-postgres--transferwise"
},
"latest job" : {
"id" : "752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"created" : "2022-06-30T15:34:17.35072",
"type" : "PIPELINE_CONFIG",
"status" : "QUEUED",
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
},
"delete job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4",
"type" : "DELETE"
},
"logs" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4/stopped",
"type" : "PUT"
}
}
}
},
"_links" : {
"update pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "PUT"
},
"delete pipeline" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144",
"type" : "DELETE"
},
"draft pipeline" : {
"href" : "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/draft",
"type" : "PUT"
},
"self" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
},
"jobs" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/jobs",
"type" : "GET"
},
"add subscription" : {
"href" : "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144/subscriptions"
},
"datastore" : {
"href" : "https://catalog.matatika.com/api/datastores/1e39e629-3622-4546-990a-024fd22927b0"
},
"withdraw job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4/stopped",
"type" : "PUT"
},
"latest job" : {
"href" : "https://catalog.matatika.com/api/jobs/752e1887-3198-48a9-98a4-f9bf1f4b76d4"
}
}
}
Validate a pipeline configuration in a workspace
POST
/api/workspaces/{workspace-id}/pipelines/validation
Validates a pipeline configuration in the workspace {workspace-id}
.
Prerequisites
- Workspace
{workspace-id}
must exist
Request
Body
Pipeline resource.
{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics"
}
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/validation' -i -X POST \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json' \
-d '{
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics"
}'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/validation"
data = {
"name" : "SIT-generated pipeline",
"dataSource" : "tap-google-analytics"
}
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers, data=data)
print(response.text.encode('utf8'))
Response
200 OK
No response body provided.
400 Bad Request
Pipeline property validation errors.
{
"timestamp" : "2022-06-30T15:31:44.0997",
"status" : 400,
"error" : "Bad Request",
"message" : "Validation failed for object='resource'. Error count: 4",
"errors" : [ {
"codes" : [ "NotBlank.oauth_credentials.access_token", "NotBlank" ],
"defaultMessage" : "No value given for setting",
"objectName" : "resource",
"field" : "properties.oauth_credentials.access_token",
"bindingFailure" : true,
"code" : "NotBlank"
}, {
"codes" : [ "NotBlank.oauth_credentials.refresh_token", "NotBlank" ],
"defaultMessage" : "No value given for setting",
"objectName" : "resource",
"field" : "properties.oauth_credentials.refresh_token",
"bindingFailure" : true,
"code" : "NotBlank"
}, {
"codes" : [ "NotBlank.view_id", "NotBlank" ],
"defaultMessage" : "No value given for setting",
"objectName" : "resource",
"field" : "properties.view_id",
"bindingFailure" : true,
"code" : "NotBlank"
}, {
"codes" : [ "NotBlank.start_date", "NotBlank" ],
"defaultMessage" : "No value given for setting",
"objectName" : "resource",
"field" : "properties.start_date",
"bindingFailure" : true,
"code" : "NotBlank"
} ],
"path" : "/api/workspaces/53ad82e6-fad5-417b-ad02-4e81cfb845f8/pipelines/validation"
}
Verify a pipeline
POST
/api/pipelines/{pipeline-id}/verification
Verifies the configuration of the pipeline {pipeline-id}
.
Prerequisites
- Pipeline
{pipeline-id}
must exist
Request
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/pipelines/bb9c9375-c67f-4c5b-9b4d-0de3ef770e6d/verification' -i -X POST \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/pipelines/bb9c9375-c67f-4c5b-9b4d-0de3ef770e6d/verification"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("POST", url, headers=headers)
print(response.text.encode('utf8'))
Response
200 OK
Job with HAL links.
{
"id" : "923877ed-2951-48a9-9668-1d7316cfbb2a",
"created" : "2022-06-30T15:27:43.483708",
"type" : "PIPELINE_VERIFY",
"status" : "QUEUED",
"_links" : {
"self" : {
"href" : "https://catalog.matatika.com/api/jobs/923877ed-2951-48a9-9668-1d7316cfbb2a"
},
"delete job" : {
"href" : "https://catalog.matatika.com/api/jobs/923877ed-2951-48a9-9668-1d7316cfbb2a",
"type" : "DELETE"
},
"logs" : {
"href" : "https://catalog.matatika.com/api/jobs/923877ed-2951-48a9-9668-1d7316cfbb2a/logs?sequence=0",
"type" : "GET"
},
"withdraw job" : {
"href" : "https://catalog.matatika.com/api/jobs/923877ed-2951-48a9-9668-1d7316cfbb2a/stopped",
"type" : "PUT"
}
}
}
Delete a pipeline
DELETE
/api/pipelines/{pipeline-id}
Deletes the pipeline {pipeline-id}
.
Prerequisites
- Pipeline
{pipeline-id}
must exist
Request
Example Snippets
- cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144' -i -X DELETE \
-H 'Accept: application/json, application/javascript, text/javascript, text/json' \
-H 'Content-Type: application/json'
- Python (requests)
import requests
url = "https://catalog.matatika.com/api/pipelines/1dcbb564-53c3-474c-8d18-0ab8b5961144"
headers = {
'Authorization': ACCESS_TOKEN
}
response = requests.request("DELETE", url, headers=headers)
print(response.text.encode('utf8'))
Response
204 No Content
No response body provided.