Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Back End: Performance tunning(后端数据处理调优) #27

Open
eyehwan opened this issue Mar 6, 2023 · 2 comments · Fixed by #21
Open

Back End: Performance tunning(后端数据处理调优) #27

eyehwan opened this issue Mar 6, 2023 · 2 comments · Fixed by #21
Assignees
@eyehwan @EdmondFrank @lishengbao
Labels
enhancement New feature or request

Comments

@eyehwan
Copy link
Member

eyehwan commented Mar 6, 2023 

- 模型读取数据优化 （@升保）
- es bulk upload 优化（ES内网流量和IO大） （@升保）
- 数据处理：
  - 大字段过多，内存和磁盘存储消耗大（考虑精简enrich字段，或者不收集没有用到的大字段）  (source: @升保)   
    - Issue Body 和 Issue comment body
    - Pull Body  和 Pullcomment body
    - Release Body
    - Commit message
@eyehwan eyehwan self-assigned this Mar 6, 2023
@eyehwan eyehwan converted this from a draft issue Mar 6, 2023
@eyehwan eyehwan added the enhancement New feature or request label Mar 6, 2023
@eyehwan eyehwan moved this from 📋 Backlog to 🏗 In progress in OSS-Compass Dashboard Mar 6, 2023
@eyehwan eyehwan linked a pull request Mar 6, 2023 that will close this issue
Modify the MAX_BULK_UPDATE_SIZE to 5000 #21
Merged
@eyehwan eyehwan changed the title Back End: Performance tunning Back End: Performance tunning(后端数据处理调优) Mar 6, 2023
@eyehwan
Copy link
Member Author

eyehwan commented Mar 7, 2023

  • es bulk upload 优化（ES内网流量和IO大）: testing on going.
  • 大字段过多，内存和磁盘存储消耗大（考虑精简enrich字段，或者不收集没有用到的大字段）: testing on going

@normal-coder normal-coder moved this from 🏗 In progress to ✅ Done in OSS-Compass Dashboard Mar 14, 2023
@lishengbao
Copy link
Contributor

lishengbao commented Dec 26, 2023 •
edited
Loading

社区模型报告优化, 特别是针对openEuler , OpenHarmony这种大社区, 优化完成后社区报告速度提升5-10倍
#96

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@eyehwan eyehwan

@EdmondFrank EdmondFrank

@lishengbao lishengbao

Labels
enhancement New feature or request
Projects
OSS-Compass Dashboard
Status: ✅ Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Modify the MAX_BULK_UPDATE_SIZE to 5000 lishengbao/compass-metrics-model-1
3 participants
@eyehwan @EdmondFrank @lishengbao