[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI #150968

nicpenning · 2023-02-13T03:43:36Z

Describe the bug:
Quite a few O365 rules are showing not installed and the link will navigate the user to: https://kibana/app/integrations/detail/o365-1.3.0/overview

Kibana/Elasticsearch Stack version:
8.6.1
Server OS version:
Windows Server 2016
Browser and Browser OS versions:
Latest Chrome or Edge
Elastic Endpoint version:
N/A
Original install method (e.g. download page, yum, from source, etc.):
Download Page -> Extract -> Execute
Functional Area (e.g. Endpoint management, timelines, resolver, etc.):
Detection Rules
Steps to reproduce:

Have the latest rules and integrations installed
Notice the integrations show as not installed.

Current behavior:
Rules are showing the integrations are missing
Expected behavior:
Rules should show the integrations are installed.
Screenshots (if relevant):

elasticmachine · 2023-02-13T03:43:39Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2023-02-16T15:40:24Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

…ed or enabled when they actually are (#152055) ## Summary Resolves: #142081 #149970 #150968 By adding an initial query for installed integrations and augments the existing `InstalledIntegrationArray` constructed using `PackagePolicy`'s. Also removes `version` from the `packageKey` when calculating installed integrations as there can be mis-matches between different policy versions and the integration itself, and I believe the intended behavior here is to not have multiple `relatedIntegrations` returned for different versions. We may want to expand the response here to include all the different policy versions that exist (and perhaps # of agents assigned the policy). Lastly, updates `getIntegrationsInfoFromPolicy()` to also pull the base `package` details in addition to the policy_template details, as this is what ensure base packages show as `Installed: enabled` if they have an integration policy assigned (vs just showing as `Installed` like when there isn't an integration policy). Note: This PR also adds the `getPackages()` method to the `PackageClient` as it didn't currently exist, and was only available via the fleet API via the `/api/fleet/epm/packages` route. ### Before: <img width="500" src="https://user-images.githubusercontent.com/2946766/221066781-be7aa1c6-1728-4200-98b2-d40946e48bbe.png" /> ### After <img width="500" src="https://user-images.githubusercontent.com/2946766/221323469-e24081f9-0741-41fd-8227-9e319c98b0d3.png" /> --------- Co-authored-by: Georgii Gorbachev <[email protected]>

…ed or enabled when they actually are (elastic#152055) ## Summary Resolves: elastic#142081 elastic#149970 elastic#150968 By adding an initial query for installed integrations and augments the existing `InstalledIntegrationArray` constructed using `PackagePolicy`'s. Also removes `version` from the `packageKey` when calculating installed integrations as there can be mis-matches between different policy versions and the integration itself, and I believe the intended behavior here is to not have multiple `relatedIntegrations` returned for different versions. We may want to expand the response here to include all the different policy versions that exist (and perhaps # of agents assigned the policy). Lastly, updates `getIntegrationsInfoFromPolicy()` to also pull the base `package` details in addition to the policy_template details, as this is what ensure base packages show as `Installed: enabled` if they have an integration policy assigned (vs just showing as `Installed` like when there isn't an integration policy). Note: This PR also adds the `getPackages()` method to the `PackageClient` as it didn't currently exist, and was only available via the fleet API via the `/api/fleet/epm/packages` route. ### Before: <img width="500" src="https://user-images.githubusercontent.com/2946766/221066781-be7aa1c6-1728-4200-98b2-d40946e48bbe.png" /> ### After <img width="500" src="https://user-images.githubusercontent.com/2946766/221323469-e24081f9-0741-41fd-8227-9e319c98b0d3.png" /> --------- Co-authored-by: Georgii Gorbachev <[email protected]> (cherry picked from commit b833b10)

banderror · 2023-03-01T08:09:32Z

Fixed by #152055

ghost · 2023-03-06T05:44:50Z

Hi @banderror

We were trying to regress this issue and on enabling the Microsoft 365 Unusual Volume of File Deletion Rule the related integration under it is showing as not installed and on clicking on it we have to fill in some manade field to install this integration.

so can you please share the what field value to put in in order to enable the related integration and regress this issue.

Rules.-.Kibana.Mozilla.Firefox.2023-03-06.11-09-23.mp4

ghost · 2023-03-06T12:07:38Z

Hi @banderror

Just for update on adding test under the mandate field, we are able to add the integration to the policy and after that we are getting Related integration as installed ✔️ .

banderror · 2023-03-06T13:35:11Z

@karanbirsingh-qasource Great. You can put anything in this field, it doesn't matter for the code that shows related integrations and their installation status.

MadameSheema · 2023-03-06T14:33:25Z

Awesome!! Can we close the ticket @karanbirsingh-qasource @banderror?

banderror · 2023-03-07T11:17:35Z

@karanbirsingh-qasource Please reopen if needed

ghost · 2023-03-09T03:40:02Z

thanks @banderror for the confirmation 👍

…ed or enabled when they actually are (elastic#152055) ## Summary Resolves: elastic#142081 elastic#149970 elastic#150968 By adding an initial query for installed integrations and augments the existing `InstalledIntegrationArray` constructed using `PackagePolicy`'s. Also removes `version` from the `packageKey` when calculating installed integrations as there can be mis-matches between different policy versions and the integration itself, and I believe the intended behavior here is to not have multiple `relatedIntegrations` returned for different versions. We may want to expand the response here to include all the different policy versions that exist (and perhaps # of agents assigned the policy). Lastly, updates `getIntegrationsInfoFromPolicy()` to also pull the base `package` details in addition to the policy_template details, as this is what ensure base packages show as `Installed: enabled` if they have an integration policy assigned (vs just showing as `Installed` like when there isn't an integration policy). Note: This PR also adds the `getPackages()` method to the `PackageClient` as it didn't currently exist, and was only available via the fleet API via the `/api/fleet/epm/packages` route. ### Before: <img width="500" src="https://user-images.githubusercontent.com/2946766/221066781-be7aa1c6-1728-4200-98b2-d40946e48bbe.png" /> ### After <img width="500" src="https://user-images.githubusercontent.com/2946766/221323469-e24081f9-0741-41fd-8227-9e319c98b0d3.png" /> --------- Co-authored-by: Georgii Gorbachev <[email protected]>

nicpenning · 2023-08-23T21:43:05Z

FYI, the 0/1 integrations installed is still a problem when navigating to the rules.

Inside of the rules and hovering it shows it is installed but not at a glance.

nicpenning · 2023-08-23T21:49:04Z

@karanbirsingh-qasource Pinging for visibility.

nicpenning · 2023-08-23T21:52:05Z

FYI - Running version 8.9.0 and latest version of the integration 1.17.1

banderror · 2023-08-24T10:08:10Z

Thanks @nicpenning, and sorry for that. Would you mind updating the description with the fresh info, such as the versions you mentioned and anything else you think could be helpful? For instance, it could be useful to know how many integration policies you have created for Microsoft 365, how many agent policies include these integration policies, how many agents are assigned to these policies, etc. Any screenshots or screen recordings, etc.

nicpenning · 2023-08-24T18:12:41Z

Current state:
Version 8.9.0 of the stack however the 1 agent running the integration is 8.8.2 with the Microsoft 365 integration (1.20.1)
This same host has 19 other integrations on it.

Upon upgrading, it also states that there are 0 agents with a policy that has this integration. This has been an issue also for quite some time (we have never seen this work on some integrations).

After upgrade, proof we have this integration in a policy

This shows the M365 Integration installed.

But another issue with this is that the assets do not show under the assets tab either. This is an increasing issue across our integrations.

ebonybubbles · 2024-01-04T17:22:45Z

Issue with UI where an integration is installed and I believe to be enabled because it is assigned to an agent and successfully ingesting data, however it is still only showing up as "installed" and the integration ticker is still only showing "0/1". I have attached a screen grab as well.

jkelas · 2024-12-27T12:59:10Z

Hi @nicpenning
I couldn't reproduce the issue on the latest main today.
Please see the screenshots below.

My procedure was:

Install All Rules
Filter Rules by 365
Click one of them, and from the Rule Details Page click the Microsoft Office 365 Integration and install it (fully, not just assets, however without adding to agent)
The Microsoft Office 365 Integrations shows as Enabled, both on the Rule Details Page, and on the Installed Rules page

Note: I am doing this on Mac, not Windows. I don't think this plays a role here, however I am not 100% sure.

@nicpenning Please check if you can reproduce the error. This bug was reported in Feb 2023.

FYI @banderror

nicpenning · 2024-12-29T19:11:31Z

I will take a look.

I think the main issue was that the integrations previously were version specific. So if you were running 1.12.0 for example, the integration was looking for 1.3.0 specifically as mentioned in my initial post.

jkelas · 2024-12-30T12:02:01Z

@nicpenning Thank you. Looking forward to seeing your observations on this one.

jkelas · 2024-12-30T12:15:52Z

@banderror When working on ticket 152408 I noticed that the ticket originated from a PR 152055 which declares to be fixing this issue. I thought that this might be the reason why I am not able to reproduce it today, buy I discovered that in this PR 178295 we stopped using this endpoint /fleet/integrations/installed and introduced a new one, /fleet/integrations/all. The implementation is different which might explain why the bug is no longer reproducible. Please take a look.

nicpenning added bug Fixes for quality problems that affect the customer experience Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. triage_needed labels Feb 13, 2023

MadameSheema assigned banderror Feb 16, 2023

MadameSheema added Team:Detections and Resp Security Detection Response Team Team:Detection Rule Management Security Detection Rule Management Team labels Feb 16, 2023

nicpenning mentioned this issue Feb 17, 2023

[Security Solution] Fixes certain Related Integrations showing as not installed even when they are #149646

Merged

spong assigned spong and unassigned banderror Feb 20, 2023

spong added v8.7.0 Feature:Related Integrations Security Solution Detection Rules Related Integrations feature and removed triage_needed labels Feb 20, 2023

banderror added impact:medium Addressing this issue will have a medium level of impact on the quality/strength of our product. Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area labels Feb 22, 2023

This was referenced Feb 22, 2023

[Security Solution] Related Integration shows as not installed even when it is #149644

Closed

[Security Solution] Fixes Related Integrations showing as not installed or enabled when they actually are #152055

Merged

banderror mentioned this issue Mar 1, 2023

[Security Solution] Prebuilt rules' Related Integrations: Azure and GCP are shown as not installed #142081

Closed

banderror added the fixed label Mar 1, 2023

banderror added the QA:Validated Issue has been validated by QA label Mar 7, 2023

banderror closed this as completed Mar 7, 2023

banderror reopened this Aug 24, 2023

banderror unassigned spong Aug 24, 2023

banderror removed fixed QA:Validated Issue has been validated by QA labels Aug 24, 2023

nicpenning changed the title ~~[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed~~ [Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI Aug 24, 2023

banderror added sdh-linked 8.14 candidate labels Jan 30, 2024

banderror removed the 8.14 candidate label Jun 7, 2024

banderror assigned jkelas Nov 18, 2024

banderror added the 8.18 candidate label Nov 18, 2024

toby-sutor mentioned this issue Nov 25, 2024

[Security Solution] Rule integrations are displayed disabled while being enabled #200167

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI #150968

[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI #150968

nicpenning commented Feb 13, 2023

elasticmachine commented Feb 13, 2023

elasticmachine commented Feb 16, 2023

banderror commented Mar 1, 2023

ghost commented Mar 6, 2023

ghost commented Mar 6, 2023

banderror commented Mar 6, 2023

MadameSheema commented Mar 6, 2023

banderror commented Mar 7, 2023

ghost commented Mar 9, 2023

nicpenning commented Aug 23, 2023

nicpenning commented Aug 23, 2023

nicpenning commented Aug 23, 2023

banderror commented Aug 24, 2023

nicpenning commented Aug 24, 2023

ebonybubbles commented Jan 4, 2024

jkelas commented Dec 27, 2024

nicpenning commented Dec 29, 2024

jkelas commented Dec 30, 2024

jkelas commented Dec 30, 2024 •

edited

Loading

[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI #150968

[Security Solution] Prebuilt rules' Related Integrations: Microsoft Office 365 are shown as not installed (0/1) in Rules UI #150968

Comments

nicpenning commented Feb 13, 2023

elasticmachine commented Feb 13, 2023

elasticmachine commented Feb 16, 2023

banderror commented Mar 1, 2023

ghost commented Mar 6, 2023

ghost commented Mar 6, 2023

banderror commented Mar 6, 2023

MadameSheema commented Mar 6, 2023

banderror commented Mar 7, 2023

ghost commented Mar 9, 2023

nicpenning commented Aug 23, 2023

nicpenning commented Aug 23, 2023

nicpenning commented Aug 23, 2023

banderror commented Aug 24, 2023

nicpenning commented Aug 24, 2023

ebonybubbles commented Jan 4, 2024

jkelas commented Dec 27, 2024

nicpenning commented Dec 29, 2024

jkelas commented Dec 30, 2024

jkelas commented Dec 30, 2024 • edited Loading

jkelas commented Dec 30, 2024 •

edited

Loading