Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create basic integration API prototype #160003

Closed

Conversation

Kerry350
Copy link
Contributor

@Kerry350 Kerry350 commented Jun 20, 2023

Summary

⚠️ This is a rough prototype for the research within #158552.

It's purpose was for R&D and to prove this was possible, the code is not perfect. However, whilst the code is rough, this is the overall approach that we want to validate. Unless there are issues or objections we will move forward with the approach (although cleaner) in #159991.

In a real implementation we would, for example, place the asset generation in to separate functions (possibly a package), and a proper library would serialize to YAML rather than just throwing brittle strings around. We could also place this in a new "custom" fork within the normal install method etc.

@Kerry350 Kerry350 self-assigned this Jun 20, 2023
@apmmachine
Copy link
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@Kerry350
Copy link
Contributor Author

@jlind23 @juliaElastic

Hi folks, would you mind taking a look at this to validate the approach? (the code is just rough).

The prototype issue that pairs with this is here: #158552

Specifically I have written up what I learnt and information on this prototype here: #158552 (comment)

Felix has also written up an issue for the real implementation here: #159991 which contains some open questions from our findings.

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
fleet 1069 1070 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 130.8KB 130.9KB +55.0B
Unknown metric groups

API count

id before after diff
fleet 1185 1186 +1

ESLint disabled line counts

id before after diff
enterpriseSearch 13 15 +2
securitySolution 411 415 +4
total +6

Total ESLint disabled count

id before after diff
enterpriseSearch 14 16 +2
securitySolution 494 498 +4
total +6

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @Kerry350

@jlind23
Copy link
Contributor

jlind23 commented Jun 26, 2023

@nchaulet @juliaElastic would you mind just taking a look at this PR and give Kerry a 👍 if this approach is aligned with our practices.

@juliaElastic
Copy link
Contributor

@Kerry350 Sorry for the late response.
I think the approach looks great.
One thing that I'm wondering, is whether we should add a version field to the API, so that custom integrations can be updated, e.g. in case some fields are missing or has to be fixed.

@Kerry350
Copy link
Contributor Author

Closing this just so we can keep the conversations in #160777.

@Kerry350 Kerry350 closed this Jul 12, 2023
Kerry350 added a commit that referenced this pull request Jul 13, 2023
## Summary

Closes #159991

Fields that have been utilised to fulfil `basic` and `agent` fields can
be easily amended if these are incorrect.

Multiple datasets are supported, and these can contain more than one
type.

## Testing

A curl command similar to the following should allow you to hit the API
(check the credentials etc):

```
curl -XPOST -u 'elastic:changeme' -H 'kbn-xsrf: something' -d '{
    "integrationName": "web_custom_nginx",
    "datasets": [{"name": "access", "type": "logs"}, {"name": "error", "type": "metrics"}, {"name": "warning", "type":"logs"}]
}' 'http://localhost:5601/<BASE_PATH>/api/fleet/epm/custom_integrations'
```

## History / context

- [Prototype
learnings](#158552 (comment))
- [Prototype PR](#160003)

## Results / expectations

API response (with installed assets):

![Screenshot 2023-07-05 at 16 56
33](https://github.com/elastic/kibana/assets/471693/fc4a0bab-7057-430a-8c03-18dd4ee17ab7)

We see the custom integration in "installed integrations" (albeit with a
verification warning):

![Screenshot 2023-07-05 at 16 57
14](https://github.com/elastic/kibana/assets/471693/0c9177d2-2871-490f-9b5c-f338e96484c4)

We see the custom integration in Discover with the logs explorer
profile:

![Screenshot 2023-07-05 at 16 58
20](https://github.com/elastic/kibana/assets/471693/30c556f2-9fcd-416e-8047-5976fc11ffa2)

The assets are installed correctly:

![Screenshot 2023-07-05 at 16 59
06](https://github.com/elastic/kibana/assets/471693/abb82632-f619-4fc3-be93-dc6ce97abedd)

![Screenshot 2023-07-05 at 16 59
20](https://github.com/elastic/kibana/assets/471693/ca1c1da5-1e4b-422c-9edb-0f56e0ed3f98)

![Screenshot 2023-07-05 at 16 59
36](https://github.com/elastic/kibana/assets/471693/8bd60d7e-aebc-4833-b423-eba3336fb42c)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants