# Google Cloud Storage Data Transfer API

This API provides endpoints to schedule and manage jobs that ingest data from a tenant's external Google Cloud Storage (GCS) bucket into our system. It uses a background job processing system to handle one-time historical transfers and recurring (daily or hourly) transfers.

**Base URL**: `https:/https://api.replen.it.replen.it/ingestion/google-cloud-storage`

***

## 🔐 Authentication

> 🔒 **This endpoint requires authentication**

### Required Header

```http
x-replenit-auth-key: YOUR_BASE64_ACCESS_KEY
```

### 🗝️ How to Get Your Access Key

1. **Log in** to your Replenit admin dashboard
2. **Navigate** to the Data Health and Management
3. **Generate** a new access key or use an existing one
4. **Copy** the key (provided in base64 format)

***

## Prerequisites: Tenant GCS Bucket Permissions 🔑

Before using any of these endpoints, the tenant whose data is being ingested **must** grant permissions to our service account on their GCS bucket. Without this, all transfer jobs will fail with an access denied error.

Please provide the following instructions to your tenant:

* **Principal (Our Service Account):** `your-service-account@your-gcp-project.iam.gserviceaccount.com` *(Replace with given address by the customer support)*
* **Required Role:** `Storage Object Viewer` (`roles/storage.objectViewer`)

**Instructions for the Tenant:**

1. In the Google Cloud Console, navigate to the source GCS bucket.
2. Go to the **Permissions** tab.
3. Click **Grant Access**.
4. In the **New principals** field, paste our service account email address.
5. In the **Assign roles** field, select **Storage Object Viewer**.
6. Click **Save**.

***

## Request Body: `TenantGoogleCloudStorageOptions`

All `POST` endpoints use the following JSON object as the request body.

| Field                        | Type             | Description                                                                   | Required |
| ---------------------------- | ---------------- | ----------------------------------------------------------------------------- | -------- |
| `TenantId`                   | GUID             | The unique identifier for the tenant.                                         | Yes      |
| `BucketName`                 | string           | The name of the tenant's GCS bucket.                                          | Yes      |
| `HistoricalDataFolderPaths`  | Array of strings | A list of folder prefixes to scan for the **historical** transfer.            | No       |
| `DataFolderPaths`            | Array of strings | A list of folder prefixes to scan for **recurring** (daily/hourly) transfers. | No       |
| `HourlyJobStartingMinuteUTC` | integer          | The minute of the hour (0-59) when the recurring hourly job should run.       | No       |
| `DailyJobStartingHourUTC`    | integer          | The hour of the day (0-23, UTC) when the recurring daily job should run.      | No       |
| `DailyJobStartingMinuteUTC`  | integer          | The minute of the hour (0-59) when the recurring daily job should run.        | No       |

***

## Endpoints

### 1. Register Historical Transfer 📂

Registers a one-time, fire-and-forget job to transfer historical data. This job uses the `HistoricalDataFolderPaths` from the request body.

* **Endpoint:** `POST /register/historical-transfer`
* **Request Body Example:**

  ```json
  {
    "BucketName": "my_bucket_replenit",
    "HistoricalDataFolderPaths": [
      "customers/Historical/",
      "products/Historical/"
    ]
  }
  ```
* **Success Response (200 OK):**

  ```json
  {
    "Success": true,
    "Message": "Historical transfer registered.",
    "Data": {
      "JobId": "12345"
    }
  }
  ```
* **Error Response (500 Internal Server Error):**

  ```json
  {
    "Success": false,
    "Message": "Failed to register historical transfer.",
    "Data": {}
  }
  ```

***

### 2. Register Daily Transfer 📅

Registers or updates a recurring job that runs once per day at a specified UTC time. This job uses the `DataFolderPaths` from the request body. The unique ID for this job is `daily-transfer/{TenantId}`.

* **Endpoint:** `POST /register/daily-transfer`
* **Request Body Example:**

  ```json
    "BucketName": "my_bucket_replenit",
    "DataFolderPaths": [
      "customers/",
      "products/"
    ],
    "DailyJobStartingHourUTC": 2,
    "DailyJobStartingMinuteUTC": 30
  }
  ```
* **Success Response (200 OK):**

  ```json
  {
    "Success": true,
    "Message": "Daily transfer registered.",
    "Data": {
      "recurringJobId": "daily-transfer/3fa85f64-5717-4562-b3fc-2c963f66afa6"
    }
  }
  ```

***

### 3. Register Hourly Transfer 🕒

Registers or updates a recurring job that runs every hour at a specified minute. This job uses the `DataFolderPaths` from the request body. The unique ID for this job is `hourly-transfer/{TenantId}`.

* **Endpoint:** `POST /register/hourly-transfer`
* **Request Body Example:**

  ```json
  {
    "BucketName": "my_bucket_replenit",
    "DataFolderPaths": [
      "transactions/"
    ],
    "HourlyJobStartingMinuteUTC": 15
  }
  ```
* **Success Response (200 OK):**

  ```json
  {
    "Success": true,
    "Message": "Hourly transfer registered.",
    "Data": {
      "recurringJobId": "hourly-transfer/3fa85f64-5717-4562-b3fc-2c963f66afa6"
    }
  }
  ```

***

### 4. Unregister Recurring Transfer 🗑️

Removes a recurring (daily or hourly) job schedule. This does not affect any jobs that are already running. You must provide the `recurringJobId` returned from the registration endpoints.

* **Endpoint:** `DELETE /unregister`
* **Query Parameter:**
  * `jobId` (string, required): The unique identifier of the recurring job (e.g., `daily-transfer/3fa85f64-5717-4562-b3fc-2c963f66afa6`).
* **Example URL:**

  ```
  https://api.replen.it/ingestion/google-cloud-storage/unregister?jobId=daily-transfer/3fa85f64-5717-4562-b3fc-2c963f66afa6
  ```
* **Success Response (200 OK):**

  ```json
  {
    "Success": true,
    "Message": "Transfer unregistered.",
    "Data": {}
  }
  ```
* **Error Response (500 Internal Server Error):**

  ```json
  {
    "Success": false,
    "Message": "Failed to unregister transfer.",
    "Data": {}
  }
  ```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://replenit.gitbook.io/replenit-docs/gcs-transfer-scheduler.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.