Incorrect and duplicated records in Xpath to JSON pointer mapping file

[jlevers]

I'm building a tool to automatically convert the old POST_PRODUCT_DATA and POST_FLAT_FILE_LISTINGS_DATA feed types to the new JSON_LISTINGS_FEED. Both the XML and TSV mapping files available here have missing/incorrect data. I reported the issues with the TSV mapping file in #4264 (and @johnkw reported it in #4318), but have since discovered issues with the XML mapping file too.

There are tags that are mapped to JSON pointers that simply don't exist in the JSON schemas returned by the Product Type Definitions API. For instance, the XML mapping file contains this record:

Message/Product/ProductData/Home/ProductType/FlatSheet/ColorMap,,/attributes/color/0/standardized_values/0

However, the FLAT_SHEET JSON schema returned by the Product Type Definitions API defines the color attribute like so:

{
    "title": "Color",
    "description": "Provide the color of the product",
    "examples": [
        "Cranberry"
    ],
    "type": "array",
    "minItems": 1,
    "minUniqueItems": 1,
    "maxUniqueItems": 1,
    "selectors": [
        "marketplace_id",
        "language_tag"
    ],
    "items": {
        "type": "object",
        "required": [
            "language_tag"
        ],
        "properties": {
            "value": {
                "title": "Color",
                "description": "Provide the color of the product",
                "editable": true,
                "hidden": false,
                "examples": [
                    "Cranberry"
                ],
                "type": "string",
                "minLength": 0,
                "maxLength": 1000,
                "anyOf": [
                    {
                        "type": "string"
                    },
                    {
                        "type": "string",
                        "enum": [
                            "Beige",
                            "Black",
                            "Blue",
                            "Brown",
                            "Cream",
                            "Gold",
                            "Gray",
                            "Green",
                            "Multicolor",
                            "Orange",
                            "Peach",
                            "Pink",
                            "Purple",
                            "Red",
                            "Teal",
                            "White",
                            "Yellow"
                        ],
                        "enumNames": [
                            "Beige",
                            "Black",
                            "Blue",
                            "Brown",
                            "Cream",
                            "Gold",
                            "Gray",
                            "Green",
                            "Multicolor",
                            "Orange",
                            "Peach",
                            "Pink",
                            "Purple",
                            "Red",
                            "Teal",
                            "White",
                            "Yellow"
                        ]
                    }
                ]
            },
            "language_tag": {
                "$ref": "#\/$defs\/language_tag"
            },
            "marketplace_id": {
                "$ref": "#\/$defs\/marketplace_id"
            }
        },
        "additionalProperties": false
    }
}

The JSON path that the XML mapping file points to for the Xpath Message/Product/ProductData/Home/ProductType/FlatSheet/ColorMap, which is /attributes/color/0/standardized_values/0, doesn't exist in that definition. So when I try to map the ColorMap tag to that JSON pointer, the resulting JSON_LISTINGS_FEED fails to validate against the FLAT_SHEET JSON schema. Based on the definition above, the correct JSON pointer would be /attributes/color/0/value.

There are over 1,000 records in the XML mapping file that map the ColorMap tag (for different product types) to this incorrect JSON path, so this one example alone indicates that there are 1,000+ errors in the mapping.

---

The other issue I've found is that many attribute mappings are duplicated, with the only difference between the duplicates being that in one of them, the JSON path has an extra space before it. Check the same Xpath as above, for example:

Message/Product/ProductData/Home/ProductType/FlatSheet/ColorMap,,/attributes/color/0/standardized_values/0
Message/Product/ProductData/Home/ProductType/FlatSheet/ColorMap,, /attributes/color/0/standardized_values/0

This problem is more annoying than anything else, as it doesn't actually prevent the file from being used but means that any tool that uses it needs to deduplicate it first.

I would really, really appreciate it if this could get resolved ASAP – I'm sure it's causing issues for many other developers who are trying to make the switch to the JSON_LISTINGS_FEED before the deadline. I would be happy to collaborate with the SP API developers on making this happen.

CC @chapmanjw @puppsupr

[johnkw]

Yes, until these bugs are fixed, the JSON_LISTINGS_FEED is completely unusable. If POST_FLAT_FILE_LISTINGS_DATA really gets deleted before these bugs are fixed, the only possible fallback plan would be to generate the current feed files automatically as everyone is currently doing, and then manually upload the feed via Seller Central.

The deprecation really needs to be postponed until there's some possible way to programmatically perform the upgrade (and to continue to be able to use the template files, which are only available on SellerCentral in the CSV format.)

[github-actions]

This issue is scheduled to close soon. If you are still encountering problems, please feel free to open a new issue and make a reference to this one.

[johnkw]

Is POST_FLAT_FILE_LISTINGS_DATA still deprecated without any documented way to migrate?

[jlevers]

@puppsupr @MeenaAmz @chapmanjw et al, it would be really, really helpful to get an update on this.

[ritikacd]

@puppsupr @MeenaAmz @chapmanjw et al, it would be really, really helpful to get an update on this.

Hi @jlevers

This is to update you that we are actively working on updating the mapping files. We will update the thread once new versions are available in the developer documentation.

Thanks,
Ritika