feat: app data conversion reports

[chrismclarke]

PR Checklist

• PR title descriptive (can be used in release notes)

Description

This PR lays groundwork for a more comprehensive QA and Optimisation system to make per-deployment recommendations (or future automation). Specifically it adds a Reports script that generates reports based on data processed during content sync.

An initial TemplateSummary report has been included which iterates through templates and enumerates the number of times each component and action type is used. In the future this will be used to help identify components that can be removed from deployment bundles.

Report summaries are stored within the deployment folder, in both json and markdown formats for easier interpretation

This is part of Hosting RFC milestone

4.1.1 Generate manifest and author syntax to allow changes on top manifest

Review Notes

The fastest way to run the subset of the sync process that generates reports is via:

yarn workflow sync_sheets --skip-download

See output in deployment reports folder. Can also see example output below

Author Notes

It would be useful to discuss and consider what other reports might be most useful to authors, and the best way to format output data. E.g. we could also include large assets, asset usage, unused templates, invalid actions etc. Some of these are already captured during sync (errors/warning thrown), but more could be added and moved to reports.

For the current report there doesn't appear to be any use of dynamic data references, however should be noted that the reports generate from static data only. If dynamic data is used, e.g. a row component type defined as @local.use_this_component_type, then only the dynamic text will show in the report and not the intended value. This may mean that some reports aren't as accurate if referring to dynamic data. It might be possible to work around this for cases where naming conventions are used, e.g. assets referenced in data_lists but with a name _asset

The report outputs are populated to the deployment content repo folder. If a deployment doesn't wish to include these reports then they can add to gitignore, e.g.

.gitignore

reports # ignore all reports
reports/*.json # ignore all json reports (keep markdown)

Previously a specific Sheets Summary was included in console outputs. This is now included in reports with the hopes that it might be more obvious/actionable if appearing in git diff when making content PRs. If still useful to keep in the console then it can still be logged (as could any other report) - just not currently to reduce the amount displayed

Dev Notes

Tests can be run by the individual scripts referenced in the spec files

One of the major challenges moving forwards when it comes to optimisation based on reports, is how to understand potential side-effects. E.g. the authors may not use the button component directly, but could reference another component that itself imports it (peer dependency). Similarly for actions, understanding which feature registers which action/action namespace is vital. So before moving ahead with generated optimisations more work will be required to separate out features that register actions and/or component namespaces, and expose a common declaration syntax for those modules that the reporters can understand.

However as this will take some time to do, it might make sense in the short term to at least try to start using the reports to generate outputs that can either be informative or actionable by authors (e.g. identifying poorly optimised/duplicate/hanging assets, templates or data lists)

Future TODOs

(should organise post-merge)

Template Summary Report

• Handle implicit components (will require component manifests or checking imports)
• QA components and actions to flag ones that don't exist
• Identify dynamic references and track separately (initially just text @data.some_ref, later could be populated)
• Authoring syntax to manage optimisations applied from reports

Potential Reports

• Referenced assets (incl. param list, template value and data _asset columns). Both assets referenced that don't exist as well as assets not referenced. Match on filepath/filename? making clear what is/isn't (e.g. data_list column names, local variable)
• Unused templates (go_to action, or from app config e.g. home_screen_template, splash screen, action standalone templates etc.). Direct access still might be useful. Components in unused templates. Likely to happen before asset/feature optimisation.
• Recommendations/optimisations (broad, to be considered more carefully, possible subreport type)

Git Issues

Closes #

Screenshots/Videos

CLI outputs report location during sheet processing

---

Example output from debug generated report

Summary

Actions

type	count
app_update	3
asset_pack	2
auth	2
download_assets	1
emit: @local.child_local_variable	1
emit: completed	36
emit: force_reload	22
emit: force_reprocess	20
emit: force_restart	2
emit: server_sync	6
emit: set_language	3
emit: set_skin	1
emit: set_theme	2
emit: translator_mode_toggle	1
emit: uncompleted	12
feedback	9
go_to	49
go_to_url	4
invalid_action	1
open_external	2
pop_up	32
process_template	2
reset_app	1
save_to_device	2
set_field	98
set_local	17
share	6
start_tour	2
task	2
track_event	2
user	1

Components

type	count
accordion	4
advanced_dashed_box	2
animated_section	5
animated_slides	1
audio	11
button	180
calendar	1
carousel	7
colour_palette	1
combo_box	45
dashed_box	23
data_items	36
debug_toggle	1
declare_field_default	1
declare_global_constant	1
display_grid	3
display_group	106
drawer	1
form	8
html	4
image	25
items	45
latex	1
lottie_animation	11
map	1
nav_group	6
navigation_bar	6
number_selector	9
odk_form	1
parent_point_box	12
parent_point_counter	4
pdf	3
progress_path	2
qr_code	1
radio_button_grid	9
radio_group	73
radio_group_grid	5
round_button	15
select_text	5
set_default	1
set_field	26
set_variable	557
simple_checkbox	26
slider	32
square_button	8
subtitle	45
task_card	15
task_progress_bar	2
template	116
text	654
text_area	5
text_box	32
text_bubble	6
tile_component	22
timer	21
title	358
toggle_bar	35
update_action_list	2
video	3
workshops_accordion	2

Flows By Type

type	subtype	total
data_list		21
data_list	campaign_rows_debug	6
data_list	campaign_schedule	1
data_list	component_demo	6
data_list	debug	20
data_list	debug_lifecycle_actions	1
data_list	example_generator	3
data_list	example_list_override	2
data_list	example_pipe	5
data_list	example_sheet_defaults	1
data_list	generated	11
data_pipe	debug	1
data_pipe	example_pipe	6
generator	example_generator	2
global		2
global	debug	9
template		47
template	component_demo	48
template	debug	307
template	example_generator	2
template	example_hardcoded	1
template	example_list_override	1
template	example_sheet_defaults	1
tour	debug	4

[jfmcquade]

Very nice. I can see how there's quite a lot to be done in order to exhaustively capture some of the useful metrics in these reports, including some refactoring of the app code. But it's pleasing to see the reports generated in this current version.

Previously a specific Sheets Summary was included in console outputs. This is now included in reports with the hopes that it might be more obvious/actionable if appearing in git diff when making content PRs. If still useful to keep in the console then it can still be logged (as could any other report) - just not currently to reduce the amount displayed

I think nice to have this removed and the output tidier.

[jfmcquade]

#2423 should be ready to merge now

[chrismclarke]

Thanks, refactored with 8e4b098
Unsurprisingly I did find the method already existed in the codebase (but within file-utils which wasn't very obvious), so have merged the code to store in shared object-utils

[jfmcquade]

A relevant issue for follow-ups: #1979