Twilio Segment logo

Reverse ETL

Reverse ETL allows the use of a database (aka: Segment Warehouse) as a source of data to be connected and sent to supported Segment Destinations. Previously, it was only possible to use a Segment Warehouse as a destination.

Sync schedule JSON configuration

The structure of scheduleConfig varies with the value in scheduleStrategy. All strategies will employ a key/value object with minimum depth of 1.

Strategy: "manual"

When scheduleStrategy is "manual", this field can be excluded altogether. With this strategy, syncs will never be triggered automatically, so they must be requested manually.

Strategy: "periodic"

When scheduleStrategy is "periodic", the only supported key is "interval", which will be a string that is in a format accepted by Go's time.ParseDuration function. For example, "3h" will sync every 3 hours, while "30m" will sync every 30 minutes. The interval must be greater than 10 minutes ("10m") and shorter than 1 week ("168h").

An example config would look like this:

{
  "interval": "1h"
}

Strategy: "specific_days"

When scheduleStrategy is "specific_days", there are only 3 supported keys: "days", "hours" and "timezone".

The "days" field must be an array of numbers, corresponding to the desired days of the week that syncs should run. Each item must be a number from 0 (Sunday) up to 6 (Saturday). There needs to be at least 1 item, but also no more than 7.

For example: [0, 6] would correspond to the weekend (Saturday and Sunday) while [1, 2, 3, 4, 5] would correspond to the weekdays (Monday through Friday).

The "hours" field must be an array of numbers, corresponding to the desired hours of the day that syncs should run. Each item must be a number from 0 (12am or 0:00) up to 23 (11pm or 23:00). There needs to be at least 1 item, but also no more than 24.

For example: [0, 12] would correspond to the running at midnight (12:00am / 0:00) and noon (12:00pm / 12:00).

The "timezone" field must be a valid value from the IANA database used by Golang, which should match the list found here. This allows the "hours" above to reflect your own time zone which may be easier to configure, but you can also choose "UTC" if you want to avoid things like DST (daylight savings).

An example config would look like:

{
  "days": [0, 6],
  "hours": [0, 6, 12, 18],
  "timezone": "America/Los_Angeles"
}

This would run syncs 4 times per day (midnight, 6am, noon, 6pm) but only on weekends (Saturday and Sunday).

Create Reverse ETL Manual Sync

Triggers a manual sync for a Reverse ETL Connection.

In the request body, the subscriptionId is the id that follows after /mappings/ portion in the URL of the sync.

For example, the subscriptionId would be 2 for this sync: https://app.Segment.com/example-workspace/reverse-etl/destinations/example-destination/sources/example-source/instances/1/mappings/2/source-id/3/model-id/4/sync-details

The rate limit for this endpoint is 20 requests per minute, which is lower than the default due to access pattern restrictions. Once reached, this endpoint will respond with the 429 HTTP status code with headers indicating the limit parameters. See Rate Limiting for more information.

Securitytoken
Request
Request Body schema: application/vnd.segment.v1alpha+json
required
sourceId
required
string (sourceId)

The id of the Source.

modelId
required
string (modelId)

The id of the Model.

subscriptionId
required
string (subscriptionId)

The id of the Subscription.

Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

post/reverse-etl-syncs
Request samples
application/vnd.segment.v1alpha+json
{
  • "sourceId": "sourceId",
  • "modelId": "modelId",
  • "subscriptionId": "subscriptionId"
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

Create Reverse Etl Model

Creates a new Reverse ETL Model.

• When called, this endpoint may generate the Model Created event in the audit trail.

Securitytoken
Request
Request Body schema: application/vnd.segment.v1alpha+json
required
sourceId
required
string (sourceId)

Indicates which Source to attach this model to.

name
required
string (name)

A short, human-readable description of the Model.

description
required
string (description)

A longer, more descriptive explanation of the Model.

enabled
required
boolean (enabled)

Indicates whether the Model should have syncs enabled. When disabled, no syncs will be triggered, regardless of the enabled status of the attached destinations/subscriptions.

scheduleStrategy
required
string (scheduleStrategy)

Determines the strategy used for triggering syncs, which will be used in conjunction with scheduleConfig.

Enum: "MANUAL" "PERIODIC" "SPECIFIC_DAYS"
required
object (scheduleConfig)

Depending on the chosen strategy, configures the schedule for this model.

query
required
string (query)

The SQL query that will be executed to extract data from the connected Source.

queryIdentifierColumn
required
string (queryIdentifierColumn)

Indicates the column named in query that should be used to uniquely identify the extracted records.

Responses
201

Created

404

Resource not found

422

Validation failure

429

Too many requests

post/reverse-etl-models
Request samples
application/vnd.segment.v1alpha+json
{
  • "sourceId": "qQEHquLrjRDN9j1ByrChyn",
  • "name": "Model 1",
  • "description": "This model is very useful.",
  • "enabled": true,
  • "query": "SELECT 'user_1' AS id",
  • "queryIdentifierColumn": "id",
  • "scheduleStrategy": "MANUAL",
  • "scheduleConfig": { }
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

List Reverse Etl Models

Returns a list of Reverse ETL Models.

Securitytoken
Request
query Parameters
object (PaginationInput)

Defines the pagination parameters.

This parameter exists in alpha.

Example: pagination=pagination.count=10
Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

get/reverse-etl-models
Request samples 
import { configureApis, unwrap } from '@segment/public-api-sdk-typescript'

const api = configureApis('/* Insert your Public API token here */')

try {
  const result = await unwrap(api.reverseEtl.listReverseEtlModels())
  console.log(JSON.stringify(result))
} catch (e) {
  console.log('ERROR:', e)
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

Delete Reverse Etl Model

Deletes an existing Model.

• When called, this endpoint may generate the Model Deleted event in the audit trail.

Securitytoken
Request
path Parameters
modelId
required
string [ 1 .. 255 ]
Example: fxXMc5bLdKnDfEgBpDbV11
Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

delete/reverse-etl-models/{modelId}
Request samples 
import { configureApis, unwrap } from '@segment/public-api-sdk-typescript'

const api = configureApis('/* Insert your Public API token here */')

try {
  const result = await unwrap(api.reverseEtl.deleteReverseEtlModel('fxXMc5bLdKnDfEgBpDbV11'))
  console.log(JSON.stringify(result))
} catch (e) {
  console.log('ERROR:', e)
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

Get Reverse Etl Model

Returns a Reverse ETL Model by its id.

Securitytoken
Request
path Parameters
modelId
required
string [ 1 .. 255 ]
Example: dY1JLx4HgWHKUNsB76tEU
Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

get/reverse-etl-models/{modelId}
Request samples 
import { configureApis, unwrap } from '@segment/public-api-sdk-typescript'

const api = configureApis('/* Insert your Public API token here */')

try {
  const result = await unwrap(api.reverseEtl.getReverseEtlModel('dY1JLx4HgWHKUNsB76tEU'))
  console.log(JSON.stringify(result))
} catch (e) {
  console.log('ERROR:', e)
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

Update Reverse Etl Model

Updates an existing Reverse ETL Model.

• When called, this endpoint may generate one or more of the following audit trail events:* Model Settings Saved

  • Model State Change Toggled
Securitytoken
Request
path Parameters
modelId
required
string [ 1 .. 255 ]
Example: gFBTkmP2ENiu6BQRmdHBkr
Request Body schema: application/vnd.segment.v1alpha+json
required
name
string (name)

A short, human-readable description of the Model.

description
string (description)

A longer, more descriptive explanation of the Model.

enabled
boolean (enabled)

Indicates whether the Model should have syncs enabled. When disabled, no syncs will be triggered, regardless of the enabled status of the attached destinations/subscriptions.

scheduleStrategy
string (scheduleStrategy)

Determines the strategy used for triggering syncs, which will be used in conjunction with scheduleConfig.

Enum: "MANUAL" "PERIODIC" "SPECIFIC_DAYS"
object (scheduleConfig)

Depending on the chosen strategy, configures the schedule for this model.

query
string (query)

The SQL query that will be executed to extract data from the connected Source.

queryIdentifierColumn
string (queryIdentifierColumn)

Indicates the column named in query that should be used to uniquely identify the extracted records.

Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

patch/reverse-etl-models/{modelId}
Request samples
application/vnd.segment.v1alpha+json
{
  • "name": "My Updated Model",
  • "scheduleStrategy": "MANUAL",
  • "scheduleConfig": { }
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}

Get Reverse ETL Sync Status

Get the sync status for a Reverse ETL sync. The sync status includes all detailed information about the sync - sync status, duration, details about the extract and load phase if applicable, etc.

The rate limit for this endpoint is 250 requests per minute, which is lower than the default due to access pattern restrictions. Once reached, this endpoint will respond with the 429 HTTP status code with headers indicating the limit parameters. See Rate Limiting for more information.

Securitytoken
Request
path Parameters
modelId
required
string [ 1 .. 255 ]
Example: modelId
syncId
required
string [ 1 .. 255 ]
Example: syncId
Responses
200

OK

404

Resource not found

422

Validation failure

429

Too many requests

get/reverse-etl-models/{modelId}/syncs/{syncId}
Request samples 
import { configureApis, unwrap } from '@segment/public-api-sdk-typescript'

const api = configureApis('/* Insert your Public API token here */')

try {
  const result = await unwrap(api.reverseEtl.getReverseETLSyncStatus('modelId', 'syncId'))
  console.log(JSON.stringify(result))
} catch (e) {
  console.log('ERROR:', e)
}
Response samples
application/vnd.segment.v1alpha+json
{
  • "data": {
    }
}
➔ Next to Selective Sync