Setting up Google Cloud

Overview

Connecting your Google Cloud account lets Redbird send datasets into your cloud buckets using the Cloud Storage Output Node.

Set Up

1. After adding the node to the canvas, connect the dataset you want to upload to Google Cloud.
2. You can choose whether to append a timestamp to the uploaded file name; this option is enabled by default.

3. Click Add/Edit Credentials. Then Click Add Credentials in the Google Cloud Storage Section of the modal.

4. From there, you will see a modal pop up where you can give the configuration a name.

Next, you will need to create a Google Service Account to get the Json file that is requested. Follow along the steps in the next section to do this.

Creating a Google Service Account

A service account is a special kind of account that can be granted permission to access Google Products. Redbird will use your service account and the various APIs to access the files and folders you want to process from Google's Platforms. Follow the steps below to complete this process.

1. Log into your Google account.
2. Access the Enable API Wizard by clicking here.
3. Click Create Project.
4. Provide the Project with a name and select your organization.
5. Click Create.
6. Follow the steps to enable access to API (1. Confirm Project; 2 Enable API).
7. Navigate to Google Cloud Console.
8. Go to Menu menu > IAM & Admin > Service Accounts.
9. Click Create Service Account and follow the steps.
10. Click Done.

Next, you will need to create credentials for the service account. The next section in this guide walks you through this process.

Create Credentials for Service Account

You need to obtain credentials to allow Redbird to access your Service Account. Credentials come in the form of a JSON file.

1. Navigate to Google Cloud Console.
2. Go to Menu menu > IAM & Admin > Service Accounts.
3. Select your project.
4. Find your service account and click on the actions menu.
5. Click Manage Keys.
6. Click Add Keys > Create New Key.
7. Select JSON.
8. Click Create. Clicking Create will save a copy of the JSON file to your downloads.
9. Also, make sure to note the email generated for the service account. This will come in handy in the next section.

Once the JSON file has been loaded into Redbird, your configuration modal window should look like below. Click Next.

Now that we have our service account created and the generated email saved, we need to grant permission to the bucket that will be shared with Redbird. Follow along on the next section.

Redbird Permissions for Google Cloud Bucket

1. Go to the Google Cloud Buckets.
2. Go to the bucket you want to access.
3. Go to permissions.
4. Click Grant Access.
5. Now paste the email we saved from the Service Account setup above to the Add principals section.
6. In the Assign Roles section, search for "Storage Object Admin" and select it.
7. Click Save.

❗️

Note:

Based on the permissions you want Redbird to have to your bucket you can assign the role in Step 6 above accordingly.

One of the two levels below would be needed as a minimum:

Storage Object Creator: Can only upload new objects (create/upload). Cannot read or overwrite existing ones.

Storage Object Admin: Can read, write, create, delete any object in the bucket. Full control over objects.

Now that we have the bucket permissions set up, go back to Redbird and specify the name of the bucket like below. You do not have to input anything under directory unless you want to specify down to that level. Click Done.

Now you are all set up to use your new Google Cloud connection within Redbird. Simply select the credentials you would like to use from the dropdown on the main configuration page. See here for the guide on how to push files to a Cloud Storage Platform.