Ingest data using bulk jobs API | Zuora

Learn the workflow for bulk data ingestion using Zuora Data Loader APIs.

You can perform data ingestion with a CSV file using the Data Loader Bulk Jobs API. It outlines the required actions to ensure a seamless and efficient data ingestion process. In the context of data ingestion, this includes all jobs related to Import, Update, Delete, and Cancel.

Zuora recommends retrieving the following API clusters to validate supported objects and fields before programmatically submitting a job. These steps help ensure data accuracy and proactively validate data before ingestion, reducing errors and preventing failed ingestion jobs.

Normalization rules

Normalize the value for the fields API URL as follows:
- Preserve values that are already hyphenated. e.g., bill-run-batches → bill-run-batches
- Convert spaces to lowercase hyphens. e.g., Accounts and Contacts → accounts-and-contacts
- Convert mixed-case values to lowercase. e.g., Payment → payment
Do not normalize objectType when creating the bulk job. Use it exactly as returned.

Object Type - Retrieve or Get request

First, select the object type for which the data ingestion has to be performed. This step will validate the following information:

Supported Object Types by the API
Supported data ingestion actions on each object - Import, Delete, Cancel, Update
The required permissions and links to the data dictionary for the object

API Request

curl --request GET \
  --url https://rest.zuora.com/data-loader/metadata/csv/objects \
  --header 'Authorization: Bearer YOUR_ACCESS_TOKEN'

Understanding the API Response

{
 /* "metadata": [*/
"objects": [
    {
      "value": "Accounts and Contacts",
      "label": "Accounts and Contacts",
      "objectType": "Account",
      "canDelete": true,
      "canUpdate": true,
      "canImport": true,
      "cancellable": false,
      "createPermissions": ["permission.accountCreate"],
      "updatePermissions": ["permission.ModifyAccount"],
      "deletePermissions": ["permission.AccountDelete"],
      "importDataDictionaryUrl": 
"https://docs.zuora.com/en/zuora-platform/data-management/data-loader/data-dictionary-for-data-loader/import/accounts-and-contacts",
      "updateDataDictionaryUrl": 
"https://docs.zuora.com/en/zuora-platform/data-management/data-loader/data-dictionary-for-data-loader/update/advance-accounts-and-contacts-update"
    }
  ]
}

Key information to use from the API Request:

Object Type - The object for which the data ingestion job has to be performed
Supported Actions - Check canImport, canUpdate, canDelete, and cancellable fields to verify if the action you need is allowed.
Permissions - Ensure you have the right API permissions (createPermissions, updatePermissions, deletePermissions).
Data Dictionary URLs - Refer to the provided links for details on the required fields and data structure.

Object Name - Retrieve or Get Request

Now, retrieve the Object Name for the Object Type selected in Step 1. The objectName is required for ensuring that data is ingested accurately. The API call below will provide a list of supported objectNames along with the actions (import, update, delete) that can be performed on each object.

Use Advanced Accounts and Contact to refer to the Accounts and Contact object.

API Request

curl --request GET \
  --url https://rest.zuora.com/data-loader/metadata/json/objects \
  --header 'Authorization: Bearer YOUR_ACCESS_TOKEN'

Understanding the API response

{
  "metadata": [
    {
      "objectName": "accounting-code",
      "objectType": "AccountingCode",
      "actionsSupported": ["import", "update", "delete"]
    },
    {
      "objectName": "accounting-period",
      "objectType": "AccountingPeriod",
      "actionsSupported": ["import", "delete"]
      
    },
    {
      "objectName": "accounts-and-contacts",
      "objectType": "Account",
      "actionsSupported": ["import", "update", "delete"]
    }
  ]
}

Key information to use from the API Request:

Identify the objectName corresponding to your objectType from the previous step.
Ensure that the action (e.g., import, update, delete) required to perform is supported for that object.
Use objectName when initiating the csv bulk job request.

Retrieve field metadata for Object Name

After Object Name in the previous step, the next step is to fetch the field metadata for that object. Each object has a unique set of fields and supported data types, which vary based on the action type (e.g., Import, Update, Cancel, Delete).

API Request

curl --request GET \
  --url https://rest.zuora.com/data-loader/metadata/csv/objects/accounts-and-contacts/fields?action=Import' \
  --header 'Authorization: Bearer YOUR_ACCESS_TOKEN'

In this example, we are fetching the field metadata for:

objectType: Account
value: Accounts and Contacts. The value of objectName is retrieved from the API response in Step 1.
Action Type: Import

Understanding the API Response

The response provides a detailed structure of the fields, their data types, and validation rules.

Sample response (truncated for clarity):

{
  "name": {
    "type": "string",
    "regex": "(^(?!\\s*$).{1,255})$",
    "required": true
  },
  "billCycleDay": {
    "type": "number",
    "regex": "^(0|1|2|...|31)$",
    "required": true
  },
  "currency": {
    "type": "string",
    "regex": "^(AUD|USD|EUR|GBP|JPY|CAD)$",
    "required": true
  },
  "notes": {
    "type": "string",
    "regex": "^.{0,65535}$",
    "required": false
  },
  "additionalEmailAddresses": [
    {
      "type": "string",
      "regex": "^(|([a-zA-Z0-9_.%+-]+@[a-zA-Z0-9.-]+.[a-zA-Z]{2,}))$",
      "required": false
    }
  ],
  "autoPay": {
    "type": "boolean",
    "regex": "^(true|false)$",
    "required": false
  }
}

Key information to use from the API Request:

Each field has a defined data type:“name”: { “type”: “string” } → Must be a string with max 255 characters.”billCycleDay”: { “type”: “number” } → Must be a number between 0-31.”currency”: { “type”: “string” } → Only allows specific currency codes.
Required vs. Optional Fields:“name” is required, meaning it must be present in every record.”notes” is optional, meaning it can be left out of the record.
Handling Arrays: additionalEmailAddresses is an array of strings, meaning multiple email addresses can be provided.
Boolean Field Format: autoPay must be either “true” or “false”

Creating a Data ingestion job with Bulk Jobs API

When your CSV file is ready with object type and action, create a job with the following API.

API Request

curl --request POST \
  --url 'https://rest.zuora.com/bulk-data/bulk-jobs' \
  --header 'Authorization: Bearer <YOUR_ACCESS_TOKEN>' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "Import Accounts",
    "description": "A test Import file",
    "objectType": "Account",
    "jobType": "Import",
    "fileType": "csv"
}'

Request parameters

Field	Description
name	A name for the job. This is for your reference
description	Optional, provide a description for the job
objectType	The type of object selected for data ingestion, for e.g. “Account”
JobType	The type of job: Import, Update, Cancel, or Delete
file Type	Format of the data file

Sample response

{
  "id": "dd447e38-6d9e-418d-9e61-6b553479727a",
  "uploadRequest": {
    "uri": "https://bulk-data-uploads-prod.s3.us-west-2.amazonaws.com",
    "fields": {
      "key": "dd447e38-6d9e-418d-9e61-6b553479727a",
      "bucket": "bulk-data-uploads-stg",
      "x-amz-algorithm": "AWS4-HMAC-SHA256",
      "x-amz-date": "...",
      "x-amz-credential": "...",
      "x-amz-security-token": "...",
      "x-amz-signature": "...",
      "Policy": "..."
    }
  },
  "status": "Created",
  "jobType": "Import"
}

Key information to use from the API Request:

Id: This is your unique job Id. You’ll need it in later steps to upload your file and track job progress.
uploadRequest: This section provides the pre-signed S3 URL and necessary fields for securely uploading your JSON-lines file. This is a one-time use URL specific to this job.
The uploadRequest.uri is the destination URL for uploading the file, and the fields contain all form fields required for the S3 POST upload.

Upload CSV file

After you’ve successfully created a bulk job using the Create Bulk Data API, the response will contain a pre-signed S3 POST URL and associated fields required to upload your data file. Before submitting the bulk job for processing, you must upload your CSV file to the specified S3 location.

API Request

curl --request POST \
  --url 'https://bulk-data-uploads-prod.s3.us-west-2.amazonaws.com/' \
  --header 'Content-Type: multipart/form-data' \

Sample request

curl --request POST \
  --url 'https://bulk-data-uploads-prod.s3.us-west-2.amazonaws.com/' \
  --header 'Content-Type: multipart/form-data' \
  --form bucket=bulk-data-uploads-stg \
  --form Policy=... \
  --form x-amz-date=... \
  --form x-amz-signature=... \
  --form x-amz-credential=... \
  --form x-amz-algorithm=AWS4-HMAC-SHA256 \
  --form key=... \
  --form x-amz-security-token=... \
  --form file=@/path/to/your/data.csv

Key information to use from the API Request:

Do not modify any of the form fields (bucket, Policy, x-amz-*, key, etc.). These must be used exactly as provided in the Create Bulk Job API response.
Only change the file parameter to point to your actual .csv file on your local machine.
The file format must match the job’s expected input format — in this case, CSV.

Submit bulk job with CSV for processing

Once you have created the bulk job and successfully uploaded the CSV file to S3, the final step is to submit the job for processing. For this, make a POST request to the job submissions endpoint, using the job ID returned in the response from the Creating Data Ingestion job Bulk data API in step 4.

API Request

curl --request POST \
  --url https://rest.zuora.com/bulk-data/bulk-jobs/dd447e38-6d9e-418d-9e61-6b553479727a/submissions \
  --header 'Authorization: Bearer e970cc57b54246c79dc44934fa30bb69'

Key information to use from the API Request:

Replace the job Id in the URL (dd447e38-6d9e-418d-9e61-6b553479727a) with the actual Id from your Create Job response.
Ensure your access token (Bearer <token>) is valid and up to date.

Query the Bulk Data job

After submitting the job, you can monitor its progress by querying the Bulk Job Summary API. Use the job ID returned from the Create Job API in the URL path.

API request

curl --request GET \
  --url https://rest.zuora.com/bulk-data/bulk-jobs/dd447e38-6d9e-418d-9e61-6b553479727a/summary \
  --header 'Authorization: Bearer 1c5e8844694144e0b5034a2a1b3068e6'

Sample request

{
  "id": "dd447e38-6d9e-418d-9e61-6b553479727a",
  "name": "Import Accounts",
  "description": "A test Import file",
  "createdAt": "2025-01-29T03:02:14Z",
  "submittedAt": "2025-01-29T03:02:26Z",
  "status": "Completed",
  "rowCount": 20,
  "rowsSucceeded": 15,
  "rowsFailed": 5,
  "lastUpdatedAt": "2025-01-29T03:02:35Z",
  "objectType": "accounts-and-contacts",
  "jobType": "Import",
  "fileType": "csv"
}

Key information to use from the API Request:

The status field will update as the job progresses (Submitted, In Progress, and eventually Completed).
Poll the endpoint periodically until the job reaches Completed.
Wait a few seconds or minutes between each poll, depending on the size (number of rows) of your job, to avoid unnecessary load.

Download URLs for source, success, and error files

Once the job has completed processing, you can download the result files to review the outcome of your import. These files provide a clear view of which records were successfully imported and which ones failed, along with error details. Replace <JOB_ID> with the ID returned by the Create Job API, and <ACCESS_TOKEN> with your valid OAuth token.

API Request

curl --request GET \
  --url https://rest.zuora.com/bulk-data/bulk-jobs/<JOB_ID>/downloads \
  --header 'Authorization: Bearer <ACCESS_TOKEN>'

Sample Request

{
  "jobId": "dd447e38-6d9e-418d-9e61-6b553479727a",
  "status": "Completed",
  "sourceFile": "https://...source.csv",
  "errorRowsFile": "https://...error_rows.csv",
  "successRowsFile": "https://...success_rows.csv",
  "delimiter": ","
}

Key information to use from the API Request:

sourceFile: URL to download the original source csv file you uploaded.
successRowsFile: csv file containing all records that were successfully processed.
errorRowsFile: csv file containing the records that failed, along with detailed error messages.
delimiter: Specifies the delimiter used if your file was in CSV format.