[DEPR]: Asset processing in Paver #31895

kdmccormick · 2023-03-08T15:50:19Z

This Deprecation was accepted on May 3, 2023. It concerned only the Paver asset commands.

At this point, in May 2024, it is now simpler to deprecate Paver in its entirety. Please see:

[DEPR]: Paver #34467.

kdmccormick · 2023-03-10T15:29:36Z

Announcement: https://discuss.openedx.org/t/deprecation-removal-edx-platform-asset-processing-in-paver/9552

Follow-up from: #31790 (comment) Part of DEPR: #31895

kdmccormick · 2023-04-12T19:02:53Z

The new implementation won't be landing in Palm, most likely. I pushed the acceptance date out to May 3rd, and pushed the final removal out to Redwood.

kdmccormick · 2023-05-30T14:54:21Z

This deprecation has been Accepted. Development of the replacement asset script is ongoing (see issues linked in description). When the replacement is ready, I'll put a warning in the output of the old asset processing script and move this issue to Deprecated.

The Webpack configuration file for built-in XBlock JS used to be generated at build time and git-ignored. It lived at common/static/xmodule/webpack.xmodule.config.js. It was generated because the JS that it referred to was also generated at build-time, and the filenames of those JS modules were not static. Now that its contents have been made entirely static [1], there is no reason we need to continue generating this Webpack configuration file. So, we check it into edx-platform under the name ./webpack.builtinblocks.config.js. We choose to put it in the repo's root directory because the paths contained in the config file are relative to the repo's root. This allows us to behead both the xmodule/static_content.py (`xmodule_assets`) script andthe `process_xmodule_assets` paver task, a major step in removing the need for Python in the edx-platform asset build [2]. It also allows us to delete the `HTMLSnippet` class and all associated attributes, which were exclusively used by xmodule/static_content.py.. We leave `xmodule_assets` and `process_xmodule_assets` in as stubs for now in order to avoid breaking external code (like Tutor) which calls Paver; the entire pavelib/assets.py function will be eventually removed soon anyway [3]. Further, to avoid extraneous refactoring, we keep one method of `HTMLSnippet` around on a few of its former subclasses: `get_html`. This method was originally part of the XModule framework; now, it is left over on a few classes as a simple internal helper method. References: 1. openedx#32480 2. openedx#31800 3. openedx#31895 Part of: openedx#32481

…ts` (#32685) The Webpack configuration file for built-in XBlock JS used to be generated at build time and git-ignored. It lived at common/static/xmodule/webpack.xmodule.config.js. It was generated because the JS that it referred to was also generated at build-time, and the filenames of those JS modules were not static. Now that its contents have been made entirely static [1], there is no reason we need to continue generating this Webpack configuration file. So, we check it into edx-platform under the name ./webpack.builtinblocks.config.js. We choose to put it in the repo's root directory because the paths contained in the config file are relative to the repo's root. This allows us to behead both the xmodule/static_content.py (`xmodule_assets`) script andthe `process_xmodule_assets` paver task, a major step in removing the need for Python in the edx-platform asset build [2]. It also allows us to delete the `HTMLSnippet` class and all associated attributes, which were exclusively used by xmodule/static_content.py.. We leave `xmodule_assets` and `process_xmodule_assets` in as stubs for now in order to avoid breaking external code (like Tutor) which calls Paver; the entire pavelib/assets.py function will be eventually removed soon anyway [3]. Further, to avoid extraneous refactoring, we keep one method of `HTMLSnippet` around on a few of its former subclasses: `get_html`. This method was originally part of the XModule framework; now, it is left over on a few classes as a simple internal helper method. References: 1. #32480 2. #31800 3. #31895 Part of: #32481

kdmccormick · 2023-08-18T16:28:56Z

The new asset processing suite has been merged as "experimental". I will continue to test it, and will communicate when it's time to mark the Paver assets suite as deprecated and encourage folks to switch to the new suite.

TODO: * Test dev & prod builds one last time * Test watching again * Print warning in openedx-assets * Add REMOVE-AFTER-VXX comments * Update docs Part of: openedx/edx-platform#31895

kdmccormick · 2023-10-17T17:51:59Z

I am hoping to mark the old asset build as deprecated in Quince so we can remove it in Redwood, although the content libraries V2 project is competing for my time. Here's my WIP PR: kdmccormick/tutor#37

kdmccormick · 2024-02-28T20:51:47Z

@feanil As I alluded to earlier, carrying out this DEPR will allow us to circumvent libsass-python's Python API (which isn't compatible with Python>3.9) and instead go straight to the underlying libsass C API. I'll take this on.

TODO: * Test dev & prod builds one last time * Test watching again Part of: openedx/edx-platform#31895

kdmccormick · 2024-03-20T20:00:17Z

This has renewed urgency, since the Python 3.8->3.12 upgrade is blocked by libsass-python. This must be resolved well ahead of the Redwood cutoff (late April).

Here is my updated plan:

Before the Redwood cutoff, to unblock the Python upgrade:
- I will make paver compile_sass a thin wrapper around npm run compile-sass. Not a breaking change.
- I will update compile_sass.py so that it does not depend on libsass-python. It will still depend on libsass and we will still install libsass-python, but we will not need to execute any Python code from libsass-python. Not a breaking change.
- I will update Tutor to use npm scripts instead of paver. This could be breaking or non-breaking for Tutor users, depending on whether we want to keep openedx-assets around for one more release.
Before the Sumac cutoff:
- I will remove all of pavelib/assets.py. This is the breaking change. Users of the paver assets CLI, such as the openedx/configuration and openedx-unsupported/devstack, will need to update to the new npm CLI.

I've updated the ticket description with the latest dates and migration steps.

FYI @feanil , @awais786 , @robrap , @regisb

robrap · 2024-03-20T20:24:11Z

FYI: @jristau1984 @spencertiberi

TODO: * Test dev & prod builds one last time * Test watching again Part of: openedx/edx-platform#31895

kdmccormick · 2024-03-26T22:05:18Z

UPDATE: This PR switches the old paver static asset build commands to be simple wrappers around npm run commands, with instructive deprecation warnings:

build: compile/watch sass with new npm scripts #34318.

This prepares us to drop the paver asset build commands after Redwood. After this PR merges, I will move the ticket status to Deprecated.

`paver` commands are deprecated for managing static assets. Starting in Sumac, only `npm run` commands will be supported for managing static assets. To ease the transition, both `paver` and `npm run` commands will work in Redwood. However, we want to stop using the *implementations* of the `paver` asset commands right now, as they are blocking the Python 3.11 upgrade. This will also make the removal of `paver` commands more straightforward come Sumac. So, this commit turns these commands/functions: * paver compile_sass (used by configuration) * paver watch_sass (used by configuration and devstack) * pavelib/assets.py:_compile_sass (used by Tutor) into very thin wrappers around the new `npm run` commands. Each of these paver routines now raise a loud deprecation warning, including a message of the `npm run` command that the operator can switch to. We expect no impact to site operators or end users. #31895

BREAKING CHANGE: `openedx-assets` is replaed with `npm run` subcommands. For details, see the changelog entry, or the Open edX DEPR ticket: openedx/edx-platform#31895

BREAKING CHANGE: `openedx-assets` is replaed with `npm run` subcommands. For details, see the changelog entry. For further details and rationale, see the upstream DEPR ticket: openedx/edx-platform#31895

`paver` commands are deprecated for managing static assets. Starting in Sumac, only `npm run` commands will be supported for managing static assets. To ease the transition, both `paver` and `npm run` commands will work in Redwood. However, we want to stop using the *implementations* of the `paver` asset commands right now, as they are blocking the Python 3.11 upgrade. This will also make the removal of `paver` commands more straightforward come Sumac. So, this commit turns these commands/functions: * paver compile_sass (used by configuration) * paver watch_sass (used by configuration and devstack) * pavelib/assets.py:_compile_sass (used by Tutor) into very thin wrappers around the new `npm run` commands. Each of these paver routines now raise a loud deprecation warning, including a message of the `npm run` command that the operator can switch to. We expect no impact to site operators or end users. openedx#31895

kdmccormick · 2024-04-30T22:00:47Z

I am rolling this DEPR into a larger proposed deprecation of all edx-platform Paver usage: #34467

BREAKING CHANGE: `openedx-assets` is replaed with `npm run` subcommands. For details, see the changelog entry. For further details and rationale, see the upstream DEPR ticket: openedx/edx-platform#31895

kdmccormick added the depr Proposal for deprecation & removal per OEP-21 label Mar 8, 2023

kdmccormick self-assigned this Mar 8, 2023

This was referenced Mar 10, 2023

Process edx-platform assets without Paver #31798

Closed

Clean up Paver asset processing code #31908

Closed

This was referenced Mar 10, 2023

Deprecate pavelib/assets.py #31613

Closed

docs: tweak webpack config design in assets ADR #31949

Merged

kdmccormick added a commit that referenced this issue Mar 16, 2023

docs: tweak webpack config design in assets ADR (#31949)

463b64d

Follow-up from: #31790 (comment) Part of DEPR: #31895

kdmccormick mentioned this issue May 22, 2023

Paver update_db is busted #32252

Closed

kdmccormick mentioned this issue Jul 10, 2023

[3/3] build: commit builtinblocks Webpack config and stub out xmodule_assets #32685

Merged

kdmccormick mentioned this issue Jul 20, 2023

Ensure that asset-related env vars and Django settings line up #32805

Closed

kdmccormick mentioned this issue Sep 14, 2023

fix: support for newer npm and macOS #33232

Closed

feanil mentioned this issue Feb 29, 2024

Update all released services to Test on Python 3.11 or 3.12 openedx/public-engineering#234

Closed

kdmccormick mentioned this issue Mar 1, 2024

build: compile/watch sass with new npm scripts #34318

Merged

kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 18, 2024

build: switch to new edx-platform asset build

2b08425

TODO: * Test dev & prod builds one last time * Test watching again Part of: openedx/edx-platform#31895

kdmccormick added a commit to kdmccormick/tutor that referenced this issue Mar 20, 2024

build: switch to new edx-platform asset build

f497ef0

TODO: * Test dev & prod builds one last time * Test watching again Part of: openedx/edx-platform#31895

kdmccormick mentioned this issue Mar 25, 2024

Remove Paver dependencies from Tutor openedx-unsupported/wg-developer-experience#160

Closed

kdmccormick mentioned this issue Apr 18, 2024

feat!: remove dependency on Paver scripts overhangio/tutor#1042

Merged

robrap mentioned this issue Apr 29, 2024

Create arch-bom tickets for 2U specific DEPR follow-up edx/edx-arch-experiments#612

Closed

7 tasks

kdmccormick closed this as not planned Won't fix, can't repro, duplicate, stale Apr 30, 2024

robrap mentioned this issue Jun 11, 2024

[DEPR-2U] Paver edx/edx-arch-experiments#679

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[DEPR]: Asset processing in Paver #31895

[DEPR]: Asset processing in Paver #31895

kdmccormick commented Mar 8, 2023 •

edited

Loading

kdmccormick commented Mar 10, 2023

kdmccormick commented Apr 12, 2023

kdmccormick commented May 30, 2023

kdmccormick commented Aug 18, 2023

kdmccormick commented Oct 17, 2023

kdmccormick commented Feb 28, 2024

kdmccormick commented Mar 20, 2024

robrap commented Mar 20, 2024

kdmccormick commented Mar 26, 2024 •

edited

Loading

kdmccormick commented Apr 30, 2024

[DEPR]: Asset processing in Paver #31895

[DEPR]: Asset processing in Paver #31895

Comments

kdmccormick commented Mar 8, 2023 • edited Loading

kdmccormick commented Mar 10, 2023

kdmccormick commented Apr 12, 2023

kdmccormick commented May 30, 2023

kdmccormick commented Aug 18, 2023

kdmccormick commented Oct 17, 2023

kdmccormick commented Feb 28, 2024

kdmccormick commented Mar 20, 2024

robrap commented Mar 20, 2024

kdmccormick commented Mar 26, 2024 • edited Loading

kdmccormick commented Apr 30, 2024

kdmccormick commented Mar 8, 2023 •

edited

Loading

kdmccormick commented Mar 26, 2024 •

edited

Loading