sos_cmd query hangs in some instances

[DebRez]

I have a sos database with a lot of job_id =0 entries that I wanted to filter out. When I run:
sos_cmd -C host -qS meminfo -X comp_time -V job_id -V component_id -F job_id:ne:0
The system runs very slowly - sending data to screen. Size of database is 28G.
It locked up and I stopped the query after about 3 minutes with nothing showing up. Then I tried again and it took about 10 seconds to find all 9900+ entries.

Is it that the index needs to be built the first time or something?

[tom95858]

You selected the index comp_time that does not contain the attribute job_id, which is what you are filtering on. The logic will iterate through every object in the database in component/time order returning those that have a job_id != 0. It ran faster the 2nd time because you had already cached a big part of the file into memory and so it didn't need to be read from disk.

If you ask to filter on an attribute that is not in the index you've selected, it will examine every single object in the database in index order checking to see if it matches the filter condition.

If you did this:

sos_cmd -C host -qS meminfo -X job_comp_time -V job_id -V component_id -F job_id:ge:1

It will run much faster. Please look at the query carefully, I changed both the index and the filter comparison. NE is not something that is ordered, so it won't use least-upper-bound when searching for matching keys.