Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: review flow compound index and/or add a new index for state and flow.cookie #161

Open
viniarck opened this issue Aug 7, 2023 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@viniarck
Copy link
Member

viniarck commented Aug 7, 2023

As I was fixing issue #160, I just realized that clients querying for cookie_range and state without dpid wouldn't hit the compound index, which is the args/query pattern that telemetry_int will use in the future. So, in the future this should be analyzed if a new index will be needed, currently it's hitting the single IXSCAN { flow.cookie: 1 } index, probably wouldn't impact much, but currently it's not ideal:

v2/stored_flows?cookie_range=12284936249945383243&cookie_range=12284936249945383243&state=installed&state=pending

{
    op: 'query',
    ns: 'napps.flows',
    command: {
      find: 'flows',
      filter: {
        state: { '$in': [ 'installed', 'pending' ] },
        'flow.cookie': {
          '$gte': Decimal128("12284936249945383243"),
          '$lte': Decimal128("12284936249945383243")
        }
      },
      sort: { 'flow.priority': -1, updated_at: 1 },
      projection: { _id: false },
      readConcern: { level: 'majority' },
      lsid: { id: new UUID("0f962b7c-d46a-41bb-9b79-e3c1a9f8b49e") },
      '$clusterTime': {
        clusterTime: Timestamp({ t: 1691417103, i: 2 }),
        signature: {
          hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
          keyId: Long("0")
        }
      },
      '$db': 'napps',
      '$readPreference': { mode: 'primaryPreferred' }
    },
    keysExamined: 0,
    docsExamined: 0,
    hasSortStage: true,
    fromMultiPlanner: true,
    cursorExhausted: true,
    numYield: 0,
    nreturned: 0,
    queryHash: 'C9307D11',
    planCacheKey: 'A24E0674',
    locks: {
      FeatureCompatibilityVersion: { acquireCount: { r: Long("1") } },
      Global: { acquireCount: { r: Long("1") } },
      Mutex: { acquireCount: { r: Long("1") } }
    },
    flowControl: {},
    readConcern: { level: 'majority', provenance: 'clientSupplied' },
    responseLength: 225,
    protocol: 'op_msg',
    millis: 0,
    planSummary: 'IXSCAN { flow.cookie: 1 }',
    execStats: {
      stage: 'SORT',
      nReturned: 0,
      executionTimeMillisEstimate: 0,
      works: 3,
      advanced: 0,
      needTime: 1,
      needYield: 0,
      saveState: 0,
      restoreState: 0,
      isEOF: 1,
      sortPattern: { 'flow.priority': -1, updated_at: 1 },
      memLimit: 104857600,
      type: 'simple',
      totalDataSizeSorted: 0,
      usedDisk: false,
      inputStage: {
        stage: 'PROJECTION_DEFAULT',
        nReturned: 0,
        executionTimeMillisEstimate: 0,
        works: 1,
        advanced: 0,
        needTime: 0,
        needYield: 0,
        saveState: 0,
        restoreState: 0,
        isEOF: 1,
        transformBy: {},
        inputStage: {
          stage: 'FETCH',
          filter: { state: { '$in': [Array] } },
          nReturned: 0,
          executionTimeMillisEstimate: 0,
          works: 1,
          advanced: 0,
          needTime: 0,
          needYield: 0,
          saveState: 0,
          restoreState: 0,
          isEOF: 1,
          docsExamined: 0,
          alreadyHasObj: 0,
          inputStage: {
            stage: 'IXSCAN',
            nReturned: 0,
            executionTimeMillisEstimate: 0,
            works: 1,
            advanced: 0,
            needTime: 0,
            needYield: 0,
            saveState: 0,
            restoreState: 0,
            isEOF: 1,
            keyPattern: { 'flow.cookie': 1 },
            indexName: 'flow.cookie_1',
            isMultiKey: false,
            multiKeyPaths: { 'flow.cookie': [] },
            isUnique: false,
            isSparse: false,
            isPartial: false,
            indexVersion: 2,
            direction: 'forward',
            indexBounds: { 'flow.cookie': [Array] },
            keysExamined: 0,
            seeks: 1,
            dupsTested: 0,
            dupsDropped: 0
          }
        }
      }
    },
    ts: ISODate("2023-08-07T14:05:16.783Z"),
    client: '172.18.0.1',
    allUsers: [ { user: 'napp_user', db: 'napps' } ],
    user: 'napp_user@napps'
  }
@viniarck viniarck added enhancement New feature or request 2023.2 Kytos-ng 2023.2 labels Aug 7, 2023
@viniarck viniarck self-assigned this Aug 7, 2023
@viniarck viniarck changed the title feat: review flow compound index and/or add a new index for state: 1 and flow.cookie feat: review flow compound index and/or add a new index for state and flow.cookie Aug 7, 2023
@viniarck viniarck added 2024.1 and removed 2023.2 Kytos-ng 2023.2 labels Nov 28, 2023
@viniarck viniarck removed the 2024.1 label Feb 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant