Creating a Custom Source

A custom source lets you push data from any system into Guru as searchable, citable records. You define the shape of your data once, then run a sync each time you want to refresh it: open the sync, push your records, and close it. This guide walks through the full lifecycle using nothing but the API, with a NetSuite customer sync as the running example.

Reach for a custom source when Guru has no native connector for your system, or when you want full control over how records are structured and searched.

Endpoints

All endpoints below are relative to the API root https://api.getguru.com/api/v1/ and use basic auth with your Guru user and API token.

How a custom source works

These ideas cover almost everything in this guide:

• A source holds object types. Each object type is one kind of record, such as Customer. It is either STRUCTURED, for records you push as JSON in your own shape, or RECORD, for documents and uploaded files that have a title, a link, and a body.
• Records are identified by your externalId. Pushing a record with a new externalId creates it; pushing one that already exists replaces it. There's no separate update call.
• A sync is how Guru removes records. You open a sync, push records, and close it. Closing with COMPLETE removes every record of that object type you didn't push or touch during the sync. Syncs also drive the sync status Guru reports for the source.
• Every source needs trackStatus: true. Set it when you create the source; it can't be changed later. Without it, the push, upload, and sync endpoints in these guides return 404 or 400.

Keep Guru in step with your system

You can sync in one of two ways, and the choice decides how changes in your system reach Guru:

Full syncs are the simplest: every run pushes the complete set, and Guru ends up matching your system exactly. Incremental syncs send less data on each run but need that extra cleanup for deletions. See Incremental syncs.

You can also push a single change outside any sync, for example to reflect an edit as soon as it happens (see Push outside a sync).

For diagrams of each approach, with every step linked to its section, see Custom Source Flows.

How a sync works

A sync is a bracketed operation on one object type:

1. Open the sync with a status call. Guru returns a sync number.
2. Push each record, tagging every push with that sync number.
3. Close the sync with a second status call.

The sync number is how Guru knows which records belong to this run. Closing with COMPLETE removes any record of the object type that wasn't pushed or touched with the current sync number. A sync that opens, pushes nothing, and completes empties the object type.

Step 1: Create the source

Creating a source defines its structure. A STRUCTURED object type, like the one in this example, needs three things: fields describe the pieces of data each record carries, templates describe how Guru builds a record's title, link, and searchable text from those fields, and optional facets expose fields as filters. Each is covered in its own section below.

The example creates a CUSTOM source with one object type, Customer, and the smallest useful field set: a title, a link back to the record, and one JSON field holding everything else. If your system already produces JSON records, you may not need to reshape them at all; see Fields: the shape of a record.

The optional useSourceNativePermissions flag opts the source into mirroring your source system's permission model, and creation is the only time it can be set. Leave it out unless you plan to sync permissions; if you do, read Adding Native Permissions to a Custom Source before creating the source.

Request

curl -X POST "https://api.getguru.com/api/v1/sources" \
  -u $GURU_USER:$GURU_TOKEN \
  -H "Content-Type: application/json" \
  -d '{
    "definition": {
      "type": "CUSTOM"
    },
    "externalId": "netsuite-customers",
    "config": {
      "type": "CUSTOM",
      "name": "Acme NetSuite - Customers",
      "trackStatus": true,
      "specification": {
        "application": {
          "objectTypes": [
            {
              "name": "Customer",
              "sourceDataType": "STRUCTURED",
              "fields": [
                {"name": "title", "contentSelector": "/title", "dataType": "TEXT"},
                {"name": "url", "contentSelector": "/url", "dataType": "TEXT"},
                {"name": "fields", "contentSelector": "/fields", "dataType": "JSON"},
                {"name": "created_at", "contentSelector": "/created_at", "dataType": "DATE_TIME"},
                {"name": "modified_at", "contentSelector": "/modified_at", "dataType": "DATE_TIME"}
              ],
              "facets": [
                {"type": "FIELD", "name": "Created Date", "field": {"name": "created_at"}, "allowMultipleValues": false, "targetType": "CREATED_DATE"},
                {"type": "FIELD", "name": "Modified Date", "field": {"name": "modified_at"}, "allowMultipleValues": false, "targetType": "MODIFIED_DATE"}
              ],
              "templates": {
                "titleTemplate": "${f_title}",
                "externalUrlTemplate": "${f_url}",
                "searchTemplate": "<#if f_fields??><#list f_fields as propName, propValue>${propName}: ${propValue?string}\n</#list></#if>"
              }
            }
          ]
        }
      }
    }
  }'

Response

The response echoes the created source. Capture two ids from it: the source's id, and the object type's id under sourceObjectTypes. You need both for every later call.

{
  "id": "1ceb7083-1049-43ee-a4fa-c75c8d901521",
  "name": "Acme NetSuite - Customers",
  "externalId": "netsuite-customers",
  "config": {"type": "CUSTOM", "name": "Acme NetSuite - Customers", "trackStatus": true},
  "sourceObjectTypes": [
    {
      "name": "Customer",
      "id": "643fed7f-4982-4852-b0a1-b8f9a427fdf0",
      "sourceDataType": "STRUCTURED",
      "fields": [ ... ]
    }
  ]
}

Body fields

The request nests a few levels deep. The top level sets the source definition; config.specification.application.objectTypes is where you define the structure.

Each entry in objectTypes has:

Structured and record object types

Every object type declares a sourceDataType, which tells Guru how to turn what you push into a searchable record. There are two values:

The NetSuite example in this guide is STRUCTURED, and the rest of Step 1 covers that path. For document-style content, declare a RECORD object type with title, url, and content fields, the same three fields Guru's built-in document sources use:

{
  "name": "Article",
  "sourceDataType": "RECORD",
  "fields": [
    {"name": "title", "contentSelector": "/title", "dataType": "TEXT"},
    {"name": "url", "contentSelector": "/url", "dataType": "TEXT"},
    {"name": "content", "contentSelector": "/content", "dataType": "TEXT"}
  ]
}

When you push one of these records in Step 4, the body carries those three keys, with the document's text in content:

{
  "title": "Returns and Refunds Policy",
  "url": "https://intranet.example.com/policies/returns",
  "content": "Customers can return any unused item within 30 days of delivery for a full refund..."
}

A STRUCTURED object type must define titleTemplate, externalUrlTemplate, and searchTemplate, or the create request fails with a 400. A RECORD object type can leave templates out.

If your documents are files such as PDFs or Word documents, you can upload the files themselves and let Guru extract the text. See Pushing Files to a Custom Source.

Fields: the shape of a record

A field is one piece of data Guru extracts from each record you push. It has a name, which is how templates refer to it (as ${f_<name>}), a contentSelector, which is a JSON path into the body you will push in Step 4 (/title reads the top-level title key), and a dataType.

Fields exist so you can push records in the shape your system already produces. If your system returns JSON, push each record as it comes, with no reshaping, and point each field's contentSelector at the value you need. Selectors follow nested objects and array positions, so a raw NetSuite customer record like this one:

{
  "id": "4570",
  "companyName": "Westmen Chemicals Ltd",
  "email": "[email protected]",
  "subsidiary": {"id": "2", "refName": "United Kingdom"},
  "entityStatus": {"id": "13", "refName": "Active"},
  "lastModifiedDate": "2026-09-01T12:00:00Z",
  "links": [{"rel": "self", "href": "https://example.netsuite.com/app/common/entity/custjob.nl?id=4570"}]
}

can be described directly, with fields that reach into it:

[
  {"name": "title", "contentSelector": "/companyName", "dataType": "TEXT"},
  {"name": "url", "contentSelector": "/links/0/href", "dataType": "TEXT"},
  {"name": "subsidiary", "contentSelector": "/subsidiary/refName", "dataType": "TEXT"},
  {"name": "status", "contentSelector": "/entityStatus/refName", "dataType": "TEXT"},
  {"name": "modified_at", "contentSelector": "/lastModifiedDate", "dataType": "DATE_TIME"}
]

Templates then use those fields like any others, for example a searchTemplate of Company: ${f_title}\nSubsidiary: ${f_subsidiary}\nStatus: ${f_status}. Keys you don't declare as fields are stored with the record but don't affect search.

The example in this guide takes a different approach, useful when you want every attribute searchable without listing each one: it declares a single field of type JSON whose selector points at a nested object, and puts the record's attributes inside it. Guru stores the whole object, you can add or remove keys on every push without touching the definition, and the search template can iterate the object (see Templates below). Either way, declare dedicated fields for values you need individually, such as anything a facet or template refers to.

Facets: optional filters

A facet exposes a value as a filter in Guru's search and in Knowledge Agent filters. Facets are optional. A FIELD facet, the kind this section covers, references a declared field by name; a TAG facet references a tag instead (see Fields or tags below). The example source defines none; here is what the Customer object type looks like with a modified date added, both the field and the facet over it:

{
  "fields": [
    {"name": "title", "contentSelector": "/title", "dataType": "TEXT"},
    {"name": "url", "contentSelector": "/url", "dataType": "TEXT"},
    {"name": "fields", "contentSelector": "/fields", "dataType": "JSON"},
    {"name": "modified_at", "contentSelector": "/modified_at", "dataType": "DATE_TIME"}
  ],
  "facets": [
    {"type": "FIELD", "name": "Modified Date", "field": {"name": "modified_at"}, "allowMultipleValues": false, "targetType": "MODIFIED_DATE"}
  ]
}

The targetType tells Guru what the value means. CREATED_DATE and MODIFIED_DATE map the field onto Guru's built-in date filters; CUSTOM creates a filter from any other field, such as a status or category. If you add fields for facets, remember that each one needs a matching key in the bodies you push in Step 4 (here, a top-level modified_at).

Fields or tags

A facet reads its values from one of two places. A field is part of the record's content: Guru reads it from the JSON body you push, at the field's contentSelector. A tag is a value attached to the record alongside its content: you declare it on the object type in tagConfigs and set it on each push with an X-Guru-Object-Tag header. Both can back a facet, so the question is only where the value comes from.

Tags also work on JSON pushes. For plain values such as dates and categories, a field does the same job with one less thing to keep in sync. See Pushing Files to a Custom Source for how to declare tags, facet on them, and set their values.

Templates: title, link, and search text

Templates turn a record's fields into the three things Guru uses from it: a title, a link back to the record in your system, and the text that search and answers draw on. Guru doesn't render a source record as a page; search results show its title, its link, and snippets of its search text. They are FreeMarker templates, and every declared field is available in them as ${f_<name>}.

The searchTemplate is the one to get right: whatever it renders is the record's entire searchable content, the text Guru indexes and draws answers from. A field that no template mentions is stored but invisible to search. The example's template pairs with the JSON field pattern:

<#if f_fields??><#list f_fields as propName, propValue>${propName}: ${propValue?string}
</#list></#if>

Reading it left to right: <#if f_fields??> renders only when the fields object was pushed, <#list f_fields as propName, propValue> iterates its keys, and ${propName}: ${propValue?string} emits one labeled line per key. For the customer pushed in Step 4, it renders:

Company Name: Westmen Chemicals Ltd
Email: [email protected]
Phone: +448450389258
Category: Manufacturing
Subsidiary: United Kingdom
Status: Active

That is what makes the flexible JSON bag searchable: every attribute you push becomes a readable, labeled line without being listed by hand in the definition.

How fields map to the records you push

The contentSelector on each field is the contract between the source definition and the content you push in Step 4. When you push a record, Guru reads each field's value from the path its contentSelector names. If the pushed body has nothing at that path, the field is empty.

The definition above and this push body line up like this:

Keep the paths in the definition and the keys in the pushed body in sync: they are the same contract, described once at creation and fulfilled on every push.

Step 2: Store the source and object type ids

Creating a source is a one-time setup step, not something you repeat on every sync. When you create the source, store the returned source id and object type id in your own system. Every later call in this guide is keyed by those two ids, so persist them once and reuse them on every run.

If you lose the stored ids, the externalId you assigned at creation is the way back to them. The listing endpoint filters on it, and the matching sources include their id and object type ids:

curl "https://api.getguru.com/api/v1/sources?externalId=netsuite-customers" \
  -u $GURU_USER:$GURU_TOKEN

Guru does not enforce uniqueness on externalId, so the filter returns every source carrying that value. As long as your integration assigns each source a distinct id, the lookup resolves to exactly one.

To read a source's sync state on a later run, fetch it by the id you stored with fieldDetail=FULL. Without that parameter, the response leaves the sync state out.

curl "https://api.getguru.com/api/v1/sources/$SOURCE_ID?fieldDetail=FULL" \
  -u $GURU_USER:$GURU_TOKEN

The response adds an objectSyncs list with one entry per object type:

{
  "objectSyncs": [
    {
      "objectType": {
        "name": "Customer",
        "id": "643fed7f-4982-4852-b0a1-b8f9a427fdf0",
        "currentSyncNumber": 3
      },
      "syncStatus": "SYNCING",
      "lastSyncAttempt": "2026-09-30T18:25:09.839+0000",
      "lastSyncedDate": "2026-09-30T18:24:16.399+0000",
      "lastSyncCompleted": "2026-09-30T18:24:18.917+0000",
      "lastSyncStatus": "SYNCED",
      "objectCount": 4
    }
  ],
  ...
}

GET /sources with no filter lists every source you can access, which is useful for discovery or debugging.

Step 3: Open the sync

Open a sync on the object type with a status call. The response returns the sync number as objectType.currentSyncNumber; hold onto it for every push and for the close call.

Choose the statusAction by whether this object type has completed a sync before, which you can tell from lastSyncedDate (Step 2):

• START_INITIAL for the first sync of an object type, while lastSyncedDate is missing.
• START for every sync after that.

The difference matters only while a first sync is open: until it completes, Guru refuses START, but START_INITIAL restarts a first sync that got stuck.

Request

curl -X PUT "https://api.getguru.com/api/v1/sources/$SOURCE_ID/types/$OBJECT_TYPE_ID/status" \
  -u $GURU_USER:$GURU_TOKEN \
  -H "Content-Type: application/json" \
  -d '{
    "statusAction": "START",
    "maxSyncTimeInMinutes": 60,
    "dependentObjectTypeIds": []
  }'

Request fields

Response

{
  "lastSyncAttempt": "2026-09-22T23:56:31.914+0000",
  "objectType": {
    "name": "Customer",
    "id": "643fed7f-4982-4852-b0a1-b8f9a427fdf0",
    "currentSyncNumber": 1
  },
  "syncStatus": "SYNCING"
}

objectType.currentSyncNumber is the sync number for this run. Pass it as syncNumber in Step 4 and Step 5.

Step 4: Push records

With the sync open, push each record's content. $RECORD_EXTERNAL_ID in the path is your system's unique id for the record, 4570 for this customer. It is what makes each record distinct in Guru, so it must vary per record. The syncNumber query parameter ties the push to the open sync.

The body is the JSON your contentSelector paths point into. The fields object here matches the JSON field defined in Step 1.

Request

curl -X PUT "https://api.getguru.com/api/v1/sources/$SOURCE_ID/objecttypes/$OBJECT_TYPE_ID/objects/$RECORD_EXTERNAL_ID/content?syncNumber=$SYNC_NUMBER" \
  -u $GURU_USER:$GURU_TOKEN \
  -H "Content-Type: application/json" \
  -d '{
    "title": "Westmen Chemicals Ltd",
    "url": "https://example.netsuite.com/app/common/entity/custjob.nl?id=4570",
    "fields": {
      "Company Name": "Westmen Chemicals Ltd",
      "Email": "[email protected]",
      "Phone": "+448450389258",
      "Category": "Manufacturing",
      "Subsidiary": "United Kingdom",
      "Status": "Active"
    }
  }'

Repeat this call once per record, each with its own externalId and body, all using the same syncNumber. To upload files instead of JSON, see Pushing Files to a Custom Source.

Push outside a sync

You can push a record without opening a sync by leaving out syncNumber. Guru creates or replaces the record right away, the same as a push inside a sync, which suits integrations that mirror single changes as they happen. Uploading a file works the same way.

A record pushed this way isn't part of any sync, so the next full sync closed with COMPLETE removes it unless that sync pushes or touches it again. If you run full syncs, include every current record in them, even ones you also push individually. A syncNumber from a sync that's already closed fails with a 409.

Step 5: Close the sync

Close the sync with a second status call. Pass the same syncNumber you opened with, in the body.

• COMPLETE runs a full reconciliation: records not pushed with this syncNumber are removed. Use this when the sync pushed the complete set of records.
• COMPLETE_INCREMENTAL applies only the changes you pushed and leaves other records in place. Use this when the sync pushed only the records that changed since the last run.
• FAIL cancels the sync: it closes it without removing anything and marks the object type's sync status as FAILED or INVALID, depending on the statusReason you send. Use this when a run can't finish, for example because your source system stopped responding partway through. Records you already pushed in the run are kept.

Request

curl -X PUT "https://api.getguru.com/api/v1/sources/$SOURCE_ID/types/$OBJECT_TYPE_ID/status" \
  -u $GURU_USER:$GURU_TOKEN \
  -H "Content-Type: application/json" \
  -d '{
    "statusAction": "COMPLETE",
    "syncNumber": '"$SYNC_NUMBER"'
  }'

Once the sync completes, Guru indexes the records. There is a short delay before they are searchable and answerable.

To cancel a run instead, send FAIL with the same syncNumber. You can add a statusReason that says why (see below) and a free-text errorDetails describing the problem:

curl -X PUT "https://api.getguru.com/api/v1/sources/$SOURCE_ID/types/$OBJECT_TYPE_ID/status" \
  -u $GURU_USER:$GURU_TOKEN \
  -H "Content-Type: application/json" \
  -d '{
    "statusAction": "FAIL",
    "syncNumber": '"$SYNC_NUMBER"',
    "statusReason": "API_ERROR",
    "errorDetails": "NetSuite returned 503"
  }'

Status reasons

The statusReason decides which status the sync ends in. Reasons that point to a problem with the integration's setup mark it INVALID, meaning someone needs to fix something before syncs can succeed. Reasons that describe a run that didn't finish mark it FAILED.

Guru also accepts PARSING_ERROR, EXTRACTION_ERROR, HOMEBOARD_IMPORT_FAILED, and NO_ADMIN, which describe Guru's own import jobs; don't use them for a custom source. Any other value fails with a 400.

Either way, the sync is closed: the next run opens a new sync with START as usual.

Incremental syncs

Full syncs are simplest: push every record each run and close with COMPLETE. For large or slow-changing data sets, an incremental sync pushes only what changed and avoids re-sending everything. Plan to pair incremental syncs with a periodic cleanup sync for deletions, unless your source has a single RECORD object type and you delete records directly.

To run one, push only the records modified since the last sync, then close with COMPLETE_INCREMENTAL instead of COMPLETE. Because COMPLETE_INCREMENTAL does not reconcile, records you did not push stay in Guru untouched. That includes records deleted in your system; to remove those, see Delete records between full syncs. Use the object type's lastSyncedDate (see Step 2) as the watermark: push every record modified at or after it. It's the time the last successful sync started, not when it finished, so changes made while that sync was running are picked up by the next one. Closing with COMPLETE_INCREMENTAL moves it to the start of the run you just closed. A run closed with FAIL doesn't move it, so the next incremental run picks up from the last successful one.

Managing the source

To give groups access to the source, rename it, read back what you've pushed, or remove records between full syncs, see Managing a Custom Source.

Deleting a source

To tear a source down completely, delete it. This removes the source and all of its records.

curl -X DELETE "https://api.getguru.com/api/v1/sources/$SOURCE_ID" \
  -u $GURU_USER:$GURU_TOKEN