qdrant shutting down when performing a search query.

[wammy19]

Hi,

I've been using qdrant for a while now in a system, but am running into an issue all of a sudden. When performing a search query (using the python client), qdrant is terminating and the client is spitting out this error message:

qdrant_openapi_client.exceptions.ResponseHandlingException: can't handle event type ConnectionClosed when role=SERVER and state=SEND_RESPONSE

There's no error coming from the docker container, it's just exiting.

I've not changed any of my search code (which was working just fine yesterday), and I've just updated to the latest version of qdrant.

Any idea's as to why this problem may be occurring? Any help is greatly appreciated.

[wammy19]

I'd like to add that there is no issues with indexing documents.

[generall]

Hi @wammy19 can you please describe steps you are doing to reproduce the issue?
Please also provide information regarding operation system and machine you are using. The issue looks similar to #78 but I though it was fixed in the newest version

[wammy19]

Hi @generall,

Yes the issue does seem similar. Steps for reproducing are.

Creating a collection via python client with an API I've written using FastAPI:

@app.post('/create-collection')
def create_qdrant_collection(collection_name: str):
    """
    :return: None

    Creates qdrant collection.
    """

    # Vector size 512
    qdrant_indexer.create_new_collection(collection_name, int(os.getenv('DISTILBERT_MULT_LANG_VEC_SIZE')))

    return {'message': f'Successfully created collection. {collection_name}'}

I'm indexing documents in batches like so:

def _index_into_qdrant(self, vectors: List[float], payload: List[dict], collection: str) -> None:
    """
    :param vectors: List[float]
    :param payload: List[dict]
    :param collection: str
    :return: None

    Indexes vectors and payload into qdrant storage.
    """

    for i, vector in enumerate(vectors):
        request(
            'POST',
            f'http://{self._ip_address}:{self._port}/collections/{collection}',
            headers={'Content-Type': 'application/json'},
            data=json.dumps({
                "upsert_points": {
                    "points": [
                        {
                            "id": randrange(0, self._qdrant_max_num_of_docs),
                            "vector": vector,
                            "payload": payload[i]
                        }
                    ]
                }
            })
        )

Searching the collection like so:

def search(self, query: str, start_date: str, end_date: str) -> List[NewsDocument]:
    """
    :param query: str
    :param start_date: str
    :param end_date: str
    :return: Iterator[NewsDocument]
    """

    # Convert start ands end date into an int so we can use a range function qdrant provides for filtering via dates.
    start_date_as_int: int = convert_date_to_int(start_date)
    end_date_as_int: int = convert_date_to_int(end_date)

    # Encode data for qdrant.
    encoded_data: List[float] = self._encode_data(query)

    # Search filter.
    search_filter = Filter(
        must=[  # Must be in range of dates.
            FieldCondition(
                key='last_modified_date_as_int',
                range=Range(  # Search within range specified.
                    gte=start_date_as_int,  # gte = greater than and equal too.
                    lte=end_date_as_int  # lte = less than and equal too.
                )
            )
        ]
    )

    # Search qdrant.
    results = self._qdrant_client.http.points_api.search_points(
        name=self._qdrant_collection_name,
        search_request=SearchRequest(
            vector=encoded_data,
            filter=search_filter,
            top=self._num_of_documents_to_retrieve,
            params=None
        )
    )

Doing a GET request of the collection returns:

GET http://127.0.0.1:6333/collections/test_collection

{
    "result": {
        "status": "green",
        "vectors_count": 1261,
        "segments_count": 5,
        "disk_data_size": 0,
        "ram_data_size": 0,
        "config": {
            "params": {
                "vector_size": 512,
                "distance": "Dot"
            },
            "hnsw_config": {
                "m": 16,
                "ef_construct": 100,
                "full_scan_threshold": 10000
            },
            "optimizer_config": {
                "deleted_threshold": 0.2,
                "vacuum_min_vector_number": 1000,
                "max_segment_number": 5,
                "memmap_threshold": 50000,
                "indexing_threshold": 20000,
                "payload_indexing_threshold": 10000,
                "flush_interval_sec": 10
            },
            "wal_config": {
                "wal_capacity_mb": 32,
                "wal_segments_ahead": 0
            }
        },
        "payload_schema": {
            "main_text": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "title": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "source": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "source_link": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "published_date_as_int": {
                "data_type": {
                    "type": "integer"
                },
                "indexed": false
            },
            "language": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "document_type": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "last_modified_date_as_int": {
                "data_type": {
                    "type": "integer"
                },
                "indexed": false
            },
            "published_date": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "last_modified_date": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            },
            "article_id": {
                "data_type": {
                    "type": "keyword"
                },
                "indexed": false
            }
        }
    },
    "status": "ok",
    "time": 0.00003792
}

I'm running qdrant via docker-compose:

qdrant:
  image: generall/qdrant
  volumes:
    - ./qdrant-storage:/qdrant/storage
  ports:
    - ${QDRANT_PORT}:${QDRANT_PORT}  # Port it 6333

qdrant exiting message:

qdrant_1 exited with code 132

My system:

• CPU: AMD Ryzen 7 3700X 8-Core Processor
• GPU: Nvidia 1060
• Ubuntu 20.04.3 LTS
• Docker version 20.10.8, build 3967b7d
• latest pre-built image from DockerHub

Let me know if I've missed anything :)

[generall]

Can you please try again with version v0.4.2

[wammy19]

Everything seems to magically be working again 👍