Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve Legend Query and Legend Taxonomy performance #1295

Merged
merged 23 commits into from
Jul 20, 2022
Merged

Conversation

akphi
Copy link
Contributor

@akphi akphi commented Jul 13, 2022

Summary

  • Improve performance of Legend Taxonomy and Legend Query - closes Feature request: Improve UX when loading dataspaces #936
    • Optimize when loading query: (from all modes: service, data-space, runtime + mapping)
      • Use new API for dataspace and build dataspace from there
      • BREAKING CHANGE: Only allow showing featured diagrams (other diagrams will be hidden from now on)
      • BREAKING CHANGE: simplify store context hooks' names, e.g. useQuerySetupStore -> useSetupStore
      • BREAKING CHANGE: simplify store naming in LegendQuery
    • Create LegendQueryEditorStore and move graphStateManager into that from LegendQueryStore
      • Check and update Legend Query testss post editor store refactoring
      • Modularize how we render the query editor header label
    • BREAKING CHANGE: update URL patterns for query editor modes to use GAV coordinate ({groupId}:{artifactId}:{versionId}) instead of {groupId}/{artifactId}/{versionId} form
    • Create a query editor mode to build query from dataspace (with specific URL pattern)
    • Make Legend Taxonomy use the new data space analytics API
    • Change in engine - fix problem with dataspace analytics endpoint legend-engine#807
      • Check if runtime is compatible with mapping in data space execution context
      • Enrich data space analysis result
  • bump dependencies
  • minor cleanups:
    • cleanup depot client to support version aliases
    • cleanup usage of depot client methods to make the code more organized
    • renamed variables ending with map to ending with index to better reflect their usage
    • BREAKING CHANGE minor cleanups for typing and config of contextual documentation
    • BREAKING CHANGE reorder parameters of execution-related methods in AbstractPureGraphManager to make them more consistent with the rest of the methods

How did you test this change?

  • Test(s) added
  • Manual testing (please provide screenshots/recordings)
  • No testing (please provide an explanation)

@finos-cla-bot finos-cla-bot bot added the cla-present CLA Signed label Jul 13, 2022
@changeset-bot
Copy link

changeset-bot bot commented Jul 13, 2022

🦋 Changeset detected

Latest commit: dcbb459

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 33 packages
Name Type
@finos/eslint-plugin-legend-studio Patch
@finos/legend-application Major
@finos/legend-dev-utils Patch
@finos/legend-extension-dsl-data-space Major
@finos/legend-extension-mapping-generation Patch
@finos/legend-graph Major
@finos/legend-query Major
@finos/legend-query-deployment Patch
@finos/legend-server-depot Major
@finos/legend-server-sdlc Patch
@finos/legend-shared Patch
@finos/legend-studio Major
@finos/legend-studio-deployment Patch
@finos/legend-studio-extension-management-toolkit Patch
@finos/legend-studio-extension-query-builder Patch
@finos/legend-taxonomy Minor
@finos/legend-taxonomy-deployment Patch
@finos/legend-tracer-extension-zipkin Patch
@finos/legend-extension-dsl-diagram Patch
@finos/babel-preset-legend-studio Patch
@finos/legend-art Patch
@finos/legend-extension-dsl-persistence Patch
@finos/legend-extension-dsl-text Patch
@finos/legend-extension-external-format-json-schema Patch
@finos/legend-extension-external-language-morphir Patch
@finos/legend-extension-external-store-service Patch
@finos/legend-graph-extension-collection Patch
@finos/legend-manual-tests Patch
@finos/legend-model-storage Patch
@finos/legend-query-app Patch
@finos/legend-studio-app Patch
@finos/legend-taxonomy-app Patch
@finos/stylelint-config-legend-studio Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@codecov
Copy link

codecov bot commented Jul 13, 2022

Codecov Report

Merging #1295 (dcbb459) into master (07852bf) will increase coverage by 0.00%.
The diff coverage is 27.55%.

@@           Coverage Diff           @@
##           master    #1295   +/-   ##
=======================================
  Coverage   41.66%   41.66%           
=======================================
  Files        1158     1175   +17     
  Lines       52393    52441   +48     
  Branches    11948    11880   -68     
=======================================
+ Hits        21827    21849   +22     
- Misses      30499    30524   +25     
- Partials       67       68    +1     
Impacted Files Coverage Δ
...plugin/src/rules/no-same-workspace-index-import.js 0.00% <0.00%> (ø)
.../legend-application/src/components/ActionAlert.tsx 9.30% <0.00%> (ø)
...LegendApplicationNavigationContextServiceUtils.tsx 6.66% <ø> (ø)
...nd-application/src/components/VirtualAssistant.tsx 8.44% <0.00%> (+0.05%) ⬆️
...onents/shared/PackageableElementOptionRenderer.tsx 0.00% <ø> (ø)
...rc/stores/LegendApplicationDocumentationService.ts 15.38% <0.00%> (ø)
...-application/src/stores/LegendApplicationPlugin.ts 0.00% <0.00%> (ø)
...tion/src/stores/shared/PackageableElementOption.ts 0.00% <0.00%> (ø)
...kages/legend-art/src/components/ResizablePanel.tsx 13.33% <ø> (ø)
...extension-dsl-data-space/src/DSLDataSpace_Const.ts 0.00% <0.00%> (ø)
... and 149 more

@akphi akphi force-pushed the faster branch 3 times, most recently from 395c7c1 to 6b2a9fd Compare July 14, 2022 21:28
@akphi akphi force-pushed the faster branch 3 times, most recently from 397e49f to 47685cf Compare July 19, 2022 13:01
@akphi akphi marked this pull request as ready for review July 19, 2022 21:20
@akphi akphi marked this pull request as draft July 20, 2022 02:37
@akphi akphi marked this pull request as ready for review July 20, 2022 02:38
Added a new query creation mode from data space, this will mode can be accessed from `Legend Query` using the `url pattern`

```
/query/extensions/{groupId}:{artifactId}:{versionId}/{dataSpacePath}/{executionContext}/{runtimePath}?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is runtimePath an option here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since people are picking the execution context of a data space which already specified with a default runtime. When people pick another option other than the default runtime, we will put that in the url with this param

@akphi akphi merged commit 8b17cfa into finos:master Jul 20, 2022
@akphi akphi deleted the faster branch July 20, 2022 04:12
@akphi
Copy link
Contributor Author

akphi commented Jul 20, 2022

Updated #936 to reflect some of the work done in this PR

@akphi
Copy link
Contributor Author

akphi commented Jul 20, 2022

@MauricioUyaguari This potentially also fixes #1092 so we should check that with some substantial-sized graphs

jinanisha pushed a commit to jinanisha/legend-studio that referenced this pull request Aug 5, 2022
* query-builder: remove the link between query setup and editor

* update custom eslint rules post ESM import upgrades

* query builder: add new simplified graph analytics

* docker: update finos/legend-shared-server to avoid CVEs

* query: optimize service execution query loading flow

* cleanup usage of depot client methods

* query: optimize create query from mapping and runtime flow

* query builder: temporarily hide error indicator for incompatible mapping

* query builder: relax mapping runtime setup checks

* reorder parameters of execution-related methods in AbstractPureGraphManager

* tracer: remove usage of terminology 'span'

* fixes tests and cleanup post rebase

* data space: use engine analytics API for loading up dataspaces

* legend-query: create QueryEditorStore and simplify LegendQueryStore

* legend-query: polomorphize QueryEditorStore

* move getExtraApplicationPageRenderEntries() to @finos/legend-application

* query: new query creation mode from data-space

* add and update changeset

* query: make query editor URL more consistent

* use HOC for store provider wrapper components

* taxonomy: use data space analytics API for loading

* bump dependencies

* minor fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-present CLA Signed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature request: Improve UX when loading dataspaces
2 participants