"lookup" type custom fields are not supported and cause discovery to fail

[nickhamlin]

Zendesk supports custom fields that allow lookups for related objects (for example, one might have a "parent organization" field on the organization that allows lookups for other organization records to indicate situations where one organization is a subsidiary of another). The Zendesk API identifies fields like this as having a datatype of "lookup".

This type is the only one for which this tap doesn't have an explicit mapping here. This means that when one tries to use it for discovery of fields, it is unable to handle the presence of lookup fields and all subsequent discovery (and pipeline runs that depend on it) is blocked.

On some initial exploring, it looks like adding a specific mapping of the lookup type to a string in the JSON linked above allows things to start working again, but I'm not sure whether that's a sustainable solution that will work for all users.

[lukas-gust]

I'm having this same issue (using Meltano) with both the default variant and the singer-io variant.

[lukas-gust]

Doing some research through the ZenDesk documentation it appears that lookups will only ever be custom fields of ID's into Users, Tickets, or Organizations only. This page describes how to retrieve the lookup records: https://developer.zendesk.com/documentation/ticketing/using-the-zendesk-api/retrieving-lookup-relationship-fields-with-the-api/. I think it implies that it would always be of integer type. Making the below modification would suffice.

Source: https://github.com/singer-io/tap-zendesk/blob/15939fa2530923543167e5b3b01302601e01e620/tap_zendesk/streams.py#L24:L33

Change:

CUSTOM_TYPES = {
    'text': 'string',
    'textarea': 'string',
    'date': 'string',
    'regexp': 'string',
    'dropdown': 'string',
    'integer': 'integer',
    'decimal': 'number',
    'checkbox': 'boolean',
    'lookup': 'integer'
}

The custom field nature means that the end user has the context of what it relates to meaning there is no need to run a sub stream. In my case I have organizations mapped to organizations via a lookup field.