Link Search Menu Expand Document

Datastores

Datastores define a destination for data loaded into a workspace by pipelines. The default datastore for a workspace is its own PostgreSQL database hosted by Matatika, but this can be changed at any time to another datastore with your own credentials (see our supported dataplugins of type LOADER).


Objects

Datastore

Path Type Format Description
id String Version 4 UUID The datastore ID
created String ISO 8601 timestamp The instant at which the datastore was created
lastModified String ISO 8601 timestamp The instant at which the datastore was last modified
name String   The datastore name
loader String   The datastore loader dataplugin name
workspace String Version 4 UUID The datastore workspace id
jdbcUrl String JDBC URL The datastore JDBC URL
properties Properties   The datastore properties

Properties

For each setting s in the dataplugin settings:

Path Type Description
s.name s.kind Refer to s.description

Requests


View all datastores in a workspace

GET

/api/workspaces/{workspace-id}/datastores

Returns the datastores in the workspace {workspace-id}.

Prerequisites

  • Workspace {workspace-id} must exist

Request

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores' -i -X GET \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Datastore collection with HAL links.

{
  "_embedded" : {
    "datastores" : [ {
      "id" : "8139609c-1d05-4165-ab03-4daf5e746147",
      "created" : "2022-11-04T15:30:12.681895",
      "lastModified" : "2022-11-04T15:30:12.681896",
      "name" : "Warehouse",
      "properties" : {
        "password" : "xOQG4i9X_v8VK6bY0_8KBy07he",
        "dbname" : "mhmtpqt",
        "default_target_schema" : "analytics",
        "port" : "5432",
        "host" : "nutty-ugli.db.elephantsql.com",
        "user" : "mhmtpqt"
      },
      "commands" : { },
      "logoUrl" : "/assets/images/datastore/target-postgres.png",
      "loader" : "target-postgres",
      "workspace" : "aa075319-f989-45b1-8854-6669d258d49a",
      "jdbcUrl" : "jdbc:postgresql://nutty-ugli.db.elephantsql.com:5432/mhmtpqt",
      "sqlAlchemyUrl" : "postgresql://mhmtpqt:[email protected]:5432/mhmtpqt?options=-csearch_path%3Dpublic",
      "managed" : true,
      "_links" : {
        "self" : {
          "href" : "https://catalog.matatika.com/api/datastores/8139609c-1d05-4165-ab03-4daf5e746147"
        },
        "make-default" : {
          "href" : "https://catalog.matatika.com/api/datastores/8139609c-1d05-4165-ab03-4daf5e746147/default"
        },
        "dataplugin" : {
          "href" : "https://catalog.matatika.com/api/dataplugins/00f03224-59e4-4606-a667-b367453755fe"
        }
      }
    } ]
  },
  "_links" : {
    "self" : {
      "href" : "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores?page=0&size=20"
    }
  },
  "page" : {
    "size" : 20,
    "totalElements" : 1,
    "totalPages" : 1,
    "number" : 0
  }
}

View a datastore

GET

/api/datastores/{datastore-id}

Returns the datastore {datastore-id}.

Prerequisites

  • Datastore {datastore-id} must exist

Request

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02' -i -X GET \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("GET", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Datastore with HAL links.

{
  "id" : "186cc602-8f97-46cd-b15e-2271dd4b1b02",
  "created" : "2022-11-04T15:30:13.842522",
  "lastModified" : "2022-11-04T15:30:14.047247",
  "name" : "Test datastore (updated)",
  "properties" : {
    "password" : "D8qIu4oexOhM0QTVNMwIXtafcT9Gqp5Q",
    "dbname" : "claaizvh",
    "default_target_schema" : "analytics",
    "host" : "nutty-ugli.db.elephantsql.com",
    "user" : "claaizvh"
  },
  "commands" : { },
  "logoUrl" : "/assets/images/datastore/target-postgres.png",
  "loader" : "target-postgres",
  "workspace" : "aa075319-f989-45b1-8854-6669d258d49a",
  "jdbcUrl" : "jdbc:postgresql://nutty-ugli.db.elephantsql.com:5432/claaizvh",
  "sqlAlchemyUrl" : "postgresql://claaizvh:[email protected]:5432/claaizvh?options=-csearch_path%3Dpublic",
  "managed" : false,
  "_links" : {
    "self" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "update datastore" : {
      "href" : "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "delete datastore" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "make-default" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02/default"
    },
    "dataplugin" : {
      "href" : "https://catalog.matatika.com/api/dataplugins/00f03224-59e4-4606-a667-b367453755fe"
    }
  }
}

Set a datastore as the workspace default

PUT

/api/datastores/{datastore-id}/default

Sets the datastore {datastore-id} as the workspace default.

Prerequisites

  • Datastore {datastore-id} must exist

Request

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02/default' -i -X PUT \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02/default"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

No response body provided.


Initialise a new datastore in a workspace

POST

/api/workspaces/{workspace-id}/datastores

Initialises a new datastore in the workspace {workspace-id}.

Prerequisites

  • Workspace {workspace-id} must exist

Request

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores' -i -X POST \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("POST", url, headers=headers)

print(response.text.encode('utf8'))

Response

200 OK

Datastore with HAL links.

{
  "id" : "186cc602-8f97-46cd-b15e-2271dd4b1b02",
  "created" : "2022-11-04T15:30:13.645592975",
  "lastModified" : "2022-11-04T15:30:13.645593175",
  "properties" : { },
  "commands" : { },
  "workspace" : "aa075319-f989-45b1-8854-6669d258d49a",
  "managed" : false,
  "_links" : {
    "self" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "create datastore" : {
      "href" : "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    }
  }
}

Create or update a datastore in a workspace

PUT

/api/workspaces/{workspace-id}/datastores/{datastore-id}

Creates or updates the datastore {datastore-id} in the workspace {workspace-id}.

Prerequisites

  • Workspace {workspace-id} must exist

Request

Body

Datastore resource.

{
  "name" : "Test datastore",
  "loader" : "target-postgres",
  "properties" : {
    "password" : "D8qIu4oexOhM0QTVNMwIXtafcT9Gqp5Q",
    "dbname" : "claaizvh",
    "default_target_schema" : "analytics",
    "user" : "claaizvh",
    "host" : "nutty-ugli.db.elephantsql.com"
  }
}

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02' -i -X PUT \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/hal+json; charset=ISO-8859-1' \
    -d '{
  "name" : "Test datastore",
  "loader" : "target-postgres",
  "properties" : {
    "password" : "D8qIu4oexOhM0QTVNMwIXtafcT9Gqp5Q",
    "dbname" : "claaizvh",
    "default_target_schema" : "analytics",
    "user" : "claaizvh",
    "host" : "nutty-ugli.db.elephantsql.com"
  }
}'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"

data = {
  "name" : "Test datastore",
  "loader" : "target-postgres",
  "properties" : {
    "password" : "D8qIu4oexOhM0QTVNMwIXtafcT9Gqp5Q",
    "dbname" : "claaizvh",
    "default_target_schema" : "analytics",
    "user" : "claaizvh",
    "host" : "nutty-ugli.db.elephantsql.com"
  }
}
headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("PUT", url, headers=headers, data=data)

print(response.text.encode('utf8'))

Response

200 OK / 201 Created

Datastore with HAL links.

{
  "id" : "186cc602-8f97-46cd-b15e-2271dd4b1b02",
  "created" : "2022-11-04T15:30:13.842521763",
  "lastModified" : "2022-11-04T15:30:13.842522663",
  "name" : "Test datastore",
  "properties" : {
    "password" : "D8qIu4oexOhM0QTVNMwIXtafcT9Gqp5Q",
    "dbname" : "claaizvh",
    "default_target_schema" : "analytics",
    "user" : "claaizvh",
    "host" : "nutty-ugli.db.elephantsql.com"
  },
  "commands" : { },
  "logoUrl" : "/assets/images/datastore/target-postgres.png",
  "loader" : "target-postgres",
  "workspace" : "aa075319-f989-45b1-8854-6669d258d49a",
  "jdbcUrl" : "jdbc:postgresql://nutty-ugli.db.elephantsql.com:5432/claaizvh",
  "sqlAlchemyUrl" : "postgresql://claaizvh:[email protected]:5432/claaizvh?options=-csearch_path%3Dpublic",
  "managed" : false,
  "_links" : {
    "self" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "update datastore" : {
      "href" : "https://catalog.matatika.com/api/workspaces/aa075319-f989-45b1-8854-6669d258d49a/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "delete datastore" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"
    },
    "make-default" : {
      "href" : "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02/default"
    },
    "dataplugin" : {
      "href" : "https://catalog.matatika.com/api/dataplugins/00f03224-59e4-4606-a667-b367453755fe"
    }
  }
}

Delete a datastore

DELETE

/api/datastores/{datastore-id}

Deletes the datastore {datastore-id}.

Prerequisites

  • Datastore {datastore-id} must exist

Request

Example Snippets

  • cURL
curl -H "Authorization: Bearer $ACCESS_TOKEN" 'https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02' -i -X DELETE \
    -H 'Accept: application/json, application/javascript, text/javascript, text/json' \
    -H 'Content-Type: application/json'
  • Python (requests)
import requests

url = "https://catalog.matatika.com/api/datastores/186cc602-8f97-46cd-b15e-2271dd4b1b02"

headers = {
  'Authorization': ACCESS_TOKEN
}

response = requests.request("DELETE", url, headers=headers)

print(response.text.encode('utf8'))

Response

204 No Content

No response body provided.