Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DYN-6295 : multithreaded analytics engine separated from model instance #14595

Merged
merged 8 commits into from
Nov 22, 2023

Conversation

BogdanZavu
Copy link
Contributor

@BogdanZavu BogdanZavu commented Nov 10, 2023

Purpose

Remove Analytics engine dependency on DynamoModel and perform all analytics operations on worker threads.
This will give us a boos in performance of about 5-7% in startup time and avoid any slowness in ADP systems overall in all usage contexts.
For consumers like Dynamo Player and Generative Design is even more valuable because there is no need to instantiate a Dynamo model anymore and worry about the proper execution thread ( along with other changes Dynamo Player and GD will start instantly ).

There are some API braking changes but they are small and can be mediated in a different way if really needed (some are noted to be obsolete in 3.0 ).

DynamoModel.HostAnalyticsInfo
DynamoModel.Version
DynamoModel.AppVersion
The above Dynamo utilities either don't belong there or can be made static. For the moment I chose to make them static but we can do something else.

Some analytics classes and method have become internal because we want to be able to start/have the analytics engine available when DynamoModel is not started yet. Once DynamoModel starts it will pick up the existing analytics instance.
Related to that sometimes we do not want to shutdown the analytics engine when DynamoModel shuts down - in this case it will be the host application responsibility to shut down the analytics service.

No issues observed so far with the new approach.

Declarations

Check these if you believe they are true

  • The codebase is in a better state after this PR
  • Is documented according to the standards
  • The level of testing this PR includes is appropriate
  • User facing strings, if any, are extracted into *.resx files
  • All tests pass using the self-service CI.
  • Snapshot of UI changes, if any.
  • Changes to the API follow Semantic Versioning and are documented in the API Changes document.
  • This PR modifies some build requirements and the readme is updated
  • This PR contains no files larger than 50 MB

Reviewers

@DynamoDS/dynamo

@mjkkirschner
Copy link
Member

@zeusongit @sm6srw can you guys please review this as I think it touches the same apis you are touching.

@QilongTang QilongTang added this to the 3.0 milestone Nov 10, 2023
@QilongTang
Copy link
Contributor

@BogdanZavu Please look at the regressions first

@pinzart90
Copy link
Contributor

Given that all track calls are now on the thread pool, maybe we need to look more closely at the Analytics.Net project to figure out if all the data is thread safe. I know there are bunch of static properties in there...but I think they are only written to during Initialization.
I cannot say the same thing for the AdpDesktopSdkCsharp wrapper code.

@BogdanZavu
Copy link
Contributor Author

BogdanZavu commented Nov 16, 2023

I believe we are good with Analytics.NET - the initialization part remains single threaded and the few statics there are read-only.
As for AdpDesktopSdkCsharp they claim it is thread safe ( for what we need at least ) but apparently there were some problems in the past so to be on the safe side I added a locking mechanism for TrackEvent access.

Given that all track calls are now on the thread pool, maybe we need to look more closely at the Analytics.Net project to figure out if all the data is thread safe. I know there are bunch of static properties in there...but I think they are only written to during Initialization. I cannot say the same thing for the AdpDesktopSdkCsharp wrapper code.

@sm6srw
Copy link
Contributor

sm6srw commented Nov 17, 2023

Yes, some overlap with my obsolete PR but nothing that can't be solved I think.

@BogdanZavu
Copy link
Contributor Author

@QilongTang and team any other concerns ?


lock (trackEventLockObj)
{
if (!ReportingAnalytics) return Disposable;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should try to minimize the amount of code executed within these locks.
Specifically I think we can move the if (!ReportingAnalytics) return Disposable; check above the lock (probably the same for all the locks)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While ReportingAnalytics does only a null check on the service side right now that could change in the future, I think it doesn't really matter , it's safer to leave it as is.

@@ -218,7 +218,7 @@ public string Version
/// <summary>
/// Host analytics info
/// </summary>
public HostAnalyticsInfo HostAnalyticsInfo { get; set; }
public static HostAnalyticsInfo HostAnalyticsInfo { get; set; }
Copy link
Contributor

@QilongTang QilongTang Nov 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea and I have some other change would benefit from this 👍🏻

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are still some places where HostAnalytics is not static (ex. SplashScreen.xaml.cs, PackageManagerExtension.cs, IStartConfiguration and DefaultStartConfiguration, ReadyParams, StartDynamoWithDefaultConfig)
Do they all still make sense to exist and to be non-static ?
Some of these changes will conflict with the "clean obsolete APIs" PR
FYI @sm6srw @mjkkirschner

Copy link
Member

@mjkkirschner mjkkirschner Nov 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, raised this concern as well above -thanks for pointing out how many non static use cases still exist.
In addition
@zeusongit's PR touches this as well dealing with the fact that Application.Current is no longer null in Revit.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Imo the start configurations data structures should remain as is. Those are transient and host info from there ends up being set in DynamoModel on startup.
ReadyParams hostinfo ( perhaps we should just remove it ?! ) delegates towards DynamoModel static now.
Handling host info in SplashScreen.xaml.cs and PackageManagerExtension.cs is a little bit above the scope of this PR so let's handle that separately in the other or a different pr. The model not being always available I believe is the reason why we have separate instances of host info in splash screen and packagemanager.

Copy link
Contributor

@QilongTang QilongTang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One comment then LGTM

@zeusongit zeusongit changed the title DYN-6295 : multithreaded anaytics engine separated from model instance DYN-6295 : multithreaded analytics engine separated from model instance Nov 21, 2023
@QilongTang QilongTang merged commit c00af64 into DynamoDS:master Nov 22, 2023
21 checks passed
@QilongTang
Copy link
Contributor

I feel some future improvements maybe still involved but I would like to merge for now to get things rolling. How and when to set static properties/ singleton is still a challenge in the new Dynamo architecture I can forsee..

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants