Search image with Hybrid method

[thanhtung2693]

Hello,

My use case is I want to search ecommerce products with image and options. For example my query would be: https://example.com/product.jpg with Size S, but with the options (with Size S) will make Marqo download the wrong image URL or wrong result.

Here is my index settings:

{
    "type": "structured",
    "model": "marqo-fashion-siglip-custom-load",
    "modelProperties": {
        "name": "hf-hub:Marqo/marqo-fashionSigLIP",
        "dimensions": 768,
        "type": "open_clip",
    },
    "normalizeEmbeddings": True,
    "allFields": [
        {"name": "thumbnail", "type": "image_pointer"},
        {"name": "options", "type": "array<text>", "features": ["lexical_search"]},
        {"name": "name", "type": "text", "features": ["lexical_search"]},
        {
            "name": "multimodal_image_name",
            "type": "multimodal_combination",
            "dependentFields": {"thumbnail": 0.9, "name": 0.1}
        }
    ],
    "tensorFields": ["multimodal_image_name"],
}

and this is the query:

mq.index(INDEX_NAME).search(
    q="https://example.com/product.jpg with size M",
    search_method="HYBRID",
    limit=10,
)

Thanks.

[papa99do]

Hello @thanhtung2693 . Sorry for the late reply. Marqo supports passing a dictionary as a query. See the second example in this doc for details. Specifically, you can use this query to retrieve the docs (please test the weights to get a better recall on your dataset):

mq.index(INDEX_NAME).search(
    q={"https://example.com/product.jpg": 1.0, "size M": 0.5},
    search_method="HYBRID",
    limit=10,
)

Please let me know if this works for you. Thanks.