Proposal: Theme Review Project Structure #12
Comments
|
I believe we have agreed on the parts and can consider that step complete. We must not forget about documentation, and choosing a format for the documentation. For step 2, Associate the correct tools to the correct parts Here are some tools or references:
|
|
I have made the following changes: Updated the text in Structure 4 Removed Structure 20, (child themes) Added quality 19-21 (custom roles, mime types, user contact methods) |
|
system3 -A user can only submit a complete theme
|
|
System2 -A user can submit unlimited updates for their existing themes that are in the theme directory. |
|
I updated the text in distribute 11 to clarify that this requirement is for copyright displayed on the front. |
|
For distribute9, the themes team goes further than what public domain requires, because, we need to be able to confirm that the image is in the public domain. I updated the text to: |
|
I updated structure2 to say: |
|
I updated quality 8 from should to must. |
|
distribute 5 changed to: |
Let's remove any items that aren't a test (or tests) itself. The structure checks will cover it. [Edit] |
|
@carolinan Apart from my last comment, is this the final list of checks? Have they all been validated and agreed upon? |
|
I've moved Structure11, Structure12 to the distribution list. I left the ids as is. [Edit] Moved Structure1, Structure10 to "safe". We'll need to move to numberless & category-less ids in the next iteration to allow us flexibility as we storm this out. |
|
Here are my thoughts on: System ChecksTool: class-wporg-themes-upload.php This should continue to run its checks, which I documented in #13. Its checks will have some overlap with other tools but it does so with respect to its own purposes which are specific to creating the entry on WordPress.org and filling in the trac ticket. We should not rely on any of these outcomes for the theme review. Instead of returning each error 1 at a time, we should combine all the errors and output them so users can work on multiple problems at once. If the theme passes these checks, the following checks should be triggered:
Any errors found by the tests above should trigger the same error messages and prevent the theme from being uploaded. Structure ChecksTool: Static Code Analysis( Theme-Check/Theme Sniffer/Other ) I'm not sure which tools make the most sense here, but I think the key is that we want to run a subset, priority-1 tests, that look specifically for missing files (and core content). We can circle back and validate more of the details of files with more specialized tools and tests in the Static Code and User Experience Review. Basic Distribution ChecksTool: Static Code Analysis( Theme-Check/Theme Sniffer/Other ) Here too, we should focus on priority-1 issues that look specifically for missing licensing directly related to the theme. We can cycle back and validate other licensing and imagery in the Static Code and User Experience Review. Static Code and User Experience Review rulesTool:
Provided we made it through the priority-1 tests above, the rest of the suite can be divided between the User Experience and the Static Code toolset and triggered asynchronously. NotesI've include Theme-Check/Theme Sniffer/Other because I'm not certain which would be the best static code tool and I'm probably not the best resource to lead that discussion. My gut tells me to keep things simple and use the theme-check for the priority-1 structure and distribution tests and then get into a bigger debate on what to use for the more complicated Static Code review. What are others thinking? Any other ideas? |
|
This match what I had in mind, thank you for putting words to it. |
There is a request for feedback out this week, where we are requesting comments until July 26. I am still hoping that we can find more ways to reduce the requirements. |
What about global14... can use any text, must be the same. I don't know the history but... to me, it applies to a small subset of themes and I would think that other factors like the quality of designs and quality of the copy have a bigger impact on a theme's quality. |
Like WordPress, all themes must be translatable. This is a pretty common problem, and it prevents translation. |
Structure tests, Priority 1? Valid theme name (does not contain WordPress, Theme, Twenty Twenty-series) (system6) Include style.css, index.php, readme.txt, screenshot (Structure2, Structure7) Basic Distribution Checks Priority 1Confirm that the style.css and readme.txt headers contains a valid license and license URL. (distribute6) |
|
Removed: |Structure6|All Themes|must|have tags in style.css that match what the theme actually does in respect to functionality and design|and doesn't use more than 3 subject tags (See: Theme Tag List).|| |
|
Removed |
|
Therefor, this requirement is the same as |
|
distribute8 | All Themes | must | declare copyright for the theme itself. This is a static file content check (that does not previously exist in the theme standard for PHPCS) |
|
Distribute13 | All Themes | must | spell “WordPress” correctly in all public-facing text: all one word, with both an uppercase W and P There is a check for this in WPCS: https://github.com/WordPress/WordPress-Coding-Standards/blob/develop/WordPress/Sniffs/WP/CapitalPDangitSniff.php |
|
quality19 | All Themes | can't | include custom roles |
|
Of the remining requirements that do not already have automated checks, I believe that these are the ones we can add checks for: Doable:distribute8 | All Themes | must | declare copyright for the theme itself.| | Theme check -issue opened | quality14 | All Themes | can't | redirect on theme activation or modify activation| Theme check -issue opened | quality5 | Classic Themes | should | use edit_theme_options capability for determining user permission to edit options, rather than relying on a role (e.g. administrator), or a different capability (e.g. edit_themes, manage_options). Doubtful:Structure9 | All Themes | can't | have minification of scripts or files | | unless the original files are also in the theme folder.
What we can't do:
quality2 | Classic Themes | must | call custom template files using get_template_part() or locate_template(). |
Purpose
This issue is a proposal on how we could structure the technical aspects of the theme review process to converge on a collection of tools, new & existing.
The theme review process and it's associated infrastructure spreads out across numerous domains and includes different projects created in different languages. Rule sets are enforced linearly across the system making it difficult to identify gaps, add/modify rules or incorporate new tools.
The temptation when trying to automate an existing process is to rush into adding more rules to existing tools, but at this point, I think it would be more helpful to separate out the high-level concerns, solidify their rules sets and then assign the right tools. Some of the challenges are new and choosing the right tool for the specific job will be critical in this project's success.
For context, please read:
Table Of Contents
Technical Flow Chart #
Parts #
The parts following represent a general responsibility.
Upload Interface #
Users currently upload their themes via the interface on WordPress.org. The interface provides some basic error messaging when themes have "upload critical" errors.
System Checks #
This is a collection of checks to make sure that the theme can be added to the directory. At a high level, the rules will make sure the theme is unique to the directory and follows the correct naming conventions. It will also make sure that the user is authorized to upload a theme.
See rule list
Structure Checks #
This is a collection of checks to make sure that the theme has the necessary files in the correct format for the theme type.
See rule list
Basic Distribution Checks #
This is a collection of checks that make sure the theme can be distributed based on proper licensing and the absence of overtly abusive structure/content.
See rule list
Static Code Review #
This is a diverse collection of checks that work on static code.
See rule list
User Experience Review #
The is a collection of checks that utilize a browser session to review a theme's user experience.
See rule list
Review Results Interface #
The theme review interface is currently a ticket on themes.trac.wordpress.org. Each theme gets a ticket and all output from the tests are presented to the uploadee.
Example Results
Next Steps #
Appendix #
These were gathered from this theme requirements issue and are subject to change. Additionally, this issue is not the source of truth and any questions or comments about specific rules should be added to the aforementioned issue.
System Check List #
Expand to see list
Basic Distribution Check List #
Expand to see list
Structure Check List #
Expand to see list
Static Code and User Experience Review rules #
There rules are categorized to into
Safe,GlobalandQualityin order to make them more digestible. For this moment in time, they are internal.Safe
These rules indicate whether the theme is safe to install.
Expand to see list
Global
These rules indicate whether the theme is ready for a global user base.
Expand to see list
Quality
These rules indicate whether the theme is of minimum quality.
Expand to see list
The text was updated successfully, but these errors were encountered: