Getting a lot of "illegal_argument_exception" when importing data from Magento 2

[jahvi]

Current behavior

When running yarn mage2vs import you get a lot of illegal argument exceptions when trying to update records in ES.

Example

error: uncaughtException: [illegal_argument_exception] Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block] :: {"path":"/vue_storefront_catalog/cms_block/18/_update","query":{},"body":"{\"upsert\":{\"store_id\":[0],\"id\":18,\"identifier\":\"login-data\",\"title\":\"Login Info Block\",\"content\":\"<div class=\\\"message info\\\" style=\\\"margin-top: 50px;\\\">\\n    <p><strong>Try Demo Customer Access</strong></p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Email:</span>[email protected]</p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Password:</span>[email protected]</p>\\n</div>\",\"creation_time\":\"2020-11-01 21:13:57\",\"update_time\":\"2020-11-01 21:13:57\",\"active\":true,\"type\":\"cms_block\",\"tsk\":1604285724877},\"doc\":{\"store_id\":[0],\"id\":18,\"identifier\":\"login-data\",\"title\":\"Login Info Block\",\"content\":\"<div class=\\\"message info\\\" style=\\\"margin-top: 50px;\\\">\\n    <p><strong>Try Demo Customer Access</strong></p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Email:</span>[email protected]</p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Password:</span>[email protected]</p>\\n</div>\",\"creation_time\":\"2020-11-01 21:13:57\",\"update_time\":\"2020-11-01 21:13:57\",\"active\":true,\"type\":\"cms_block\",\"tsk\":1604285724877}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block]\"}],\"type\":\"illegal_argument_exception\",\"reason\":\"Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block]\"},\"status\":400}"} date=Mon Nov 02 2020 02:55:25 GMT+0000 (Greenwich Mean Time), ..., stack=[Error: [illegal_argument_exception] Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block] :: {"path":"/vue_storefront_catalog/cms_block/18/_update","query":{},"body":"{\"upsert\":{\"store_id\":[0],\"id\":18,\"identifier\":\"login-data\",\"title\":\"Login Info Block\",\"content\":\"<div class=\\\"message info\\\" style=\\\"margin-top: 50px;\\\">\\n    <p><strong>Try Demo Customer Access</strong></p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Email:</span>[email protected]</p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Password:</span>[email protected]</p>\\n</div>\",\"creation_time\":\"2020-11-01 21:13:57\",\"update_time\":\"2020-11-01 21:13:57\",\"active\":true,\"type\":\"cms_block\",\"tsk\":1604285724877},\"doc\":{\"store_id\":[0],\"id\":18,\"identifier\":\"login-data\",\"title\":\"Login Info Block\",\"content\":\"<div class=\\\"message info\\\" style=\\\"margin-top: 50px;\\\">\\n    <p><strong>Try Demo Customer Access</strong></p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Email:</span>[email protected]</p>\\n    <p><span style=\\\"display:inline-block; width: 80px; padding-right: 10px;\\\">Password:</span>[email protected]</p>\\n</div>\",\"creation_time\":\"2020-11-01 21:13:57\",\"update_time\":\"2020-11-01 21:13:57\",\"active\":true,\"type\":\"cms_block\",\"tsk\":1604285724877}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block]\"}],\"type\":\"illegal_argument_exception\",\"reason\":\"Rejecting mapping update to [vue_storefront_catalog] as the final mapping would have more than 1 type: [category, cms_block]\"},\"status\":400}"},...]

Expected behavior

Records should be imported into ES without any errors.

Steps to reproduce the issue

1. Get a fresh Magento 2.4.1 installation with sample data.
2. Use the latest develop branch from storefront ui and add integration details.
3. Run yarn mage2vs import

Repository

N/A

Can you handle fixing this bug by yourself?

• YES
• NO

Environment details

• Browser: N/A
• OS: Mac OS Catalina 10.15.7
• Node: 12.18.3
• Code Version: develop

Additional information

This seems related to changes to ES6+ where mapping types were removed so index can only have 1 type.

https://www.elastic.co/guide/en/elasticsearch/reference/6.0/removal-of-types.html

[goldenkat213]

di you find any solution to the issue? same issue here, though I have rewritten putMappings fuctions in my meta/elastic.js like this::

function putMappings(db, indexName, next) {
db.indices.putMapping({
index: indexName,
include_type_name: true,
type: "product",
body: {
properties: {
slug: {type: "keyword"},
url_key: {type: "keyword"},
url_path: {type: "keyword"},
sku: { type: "keyword" },
size: { type: "integer" },
size_options: { type: "integer" },
price: { type: "float" },
originalPriceInclTax : { type: "float" },
has_options: { type: "boolean" },
special_price: { type: "float" },
color: { type: "integer" },
color_options: { type: "integer" },
pattern: { type: "text" },
id: { type: "long" },
status: { type: "integer" },
weight: { type: "integer" },
visibility: { type: "integer" },
created_at: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
updated_at: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
special_from_date: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
special_to_date: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
news_from_date: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
news_to_date: {
type: "date",
format: "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
description: { type: "text" },
name: { type: "text" },
configurable_children: {
properties: {
url_key: { type: "keyword"},
has_options: { type: "boolean"},
price: { type: "float"},
sku: { type: "keyword"},
special_price: { type: "float"}
}
},
configurable_options: {
properties: {
attribute_id: { type: "long" },
default_label: { type: "text"},
label: { type: "text"},
frontend_label: { type: "text"},
store_label: { type: "text"},
values: {
properties: {
default_label: { type: "text"},
label: { type: "text"},
frontend_label: { type: "text"},
store_label: { type: "text"},
value_index: { type: "keyword" }
}
}
}
},
category_ids: { type: "long" },
eco_collection: { type: "integer" },
eco_collection_options: { type: "integer" },
erin_recommends: { type: "integer" },
tax_class_id: { type: "long" }
}
}
},{
index: indexName,
include_type_name: true,
type: "taxrule",
body: {
properties: {
id: { type: "long" },
rates: {
properties: {
rate: { type: "float" }
}
}
}
}
},{
index: indexName,
include_type_name: true,
type: "attribute",
body: {
properties: {
id: { type: "long" },
attribute_id: { type: "long" },
options: {
properties: {
value: { type: "text", "index" : "not_analyzed" }
}
}
}
}
}).then(res1 => {
console.dir(res1, { depth: null, colors: true })
}).catch(err1 => {
console.error(err1)
next(err1)
})
}

and it worked, but i get the same error when indexing the attributes, after indexing the products