Create A Data Import Pipeline
Time required: 5 minutes
You must have:
- Access to a workspace.
- Admin permissions for that workspace.
In this guide, we will quickly go through the high-level steps of how to get a data import set up in your workspace.
Each data source for a data import require different settings and configurations to work, and information about these specifics will be displayed on the right of your screen as you set up your data import.
Set Up Steps
- When in your workspace, click the
Labbutton on the menu, select
Pluginsfrom the menu and go the the
- Find or search for the data source you want and click the
INSTALLbutton. You will be moved to the
- Click the
+ PIPELINEbutton next to the your chosen data source plugin.
- At the top in
Nameyou can add a custom name for your pipeline.
- Expand the settings menu(s) and fill in at least all the required settings which are marked with a
Connect to Google.
- In the second section
Clean, transform and organise, decide if you want to use our default data import script by leaving
Default Actionsselected, or supply your own custom actions or script.
- In the third and last section
Automate your import, you can choose how often your data import should run. There are a few example schedules or you can use
Customto create your own.
Save. A green bar will appear at the top of your screen telling you your data import was saved.
- Head back to the
Pipelinesscreen, where for the next 1 to 2 minutes a
config jobwill be running on your data import. This will set everything up and commit your new data imports’s changes to your workspace repository.
- Once the
config jobhas completed you are free to run your data import, or leave it to its schedule.
If you want more control of your pipeline, or don’t want to use a data source plugin, you can create a use a custom pipeline: Create a Custom Pipeline