Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move artifacts outside of edx-platform in Tutor's openedx image #22

Open
2 of 5 tasks
Tracked by #146
kdmccormick opened this issue Jan 5, 2023 · 1 comment
Open
2 of 5 tasks
Tracked by #146

Move artifacts outside of edx-platform in Tutor's openedx image #22

kdmccormick opened this issue Jan 5, 2023 · 1 comment
Assignees
@kdmccormick
Labels
tutor Requires a change to Tutor

Comments

@kdmccormick
Copy link
Member

kdmccormick commented Jan 5, 2023 •
edited
Loading

Background

This is a sub-task of openedx-unsupported/wg-developer-experience#146.

The problem: In the openedx image, static assets are collected & compiled into several sub-directories of /openedx/edx-platform/. So, when you mount your own edx-platform, you override all of those static asset folders. Now, tutor dev launch must re-build assets again, which takes a long time and is completely redundant with the assets that you had to download when you downloaded the openedx image. If you forget to do this, then your edx-platform frontend will be broken.

What we want: To generate static assets outside of the /openedx/edx-platform directory so that mounting edx-platform does not overwrite them.

Tasks

  • Figure out how to change where static assets go. This is very hard.
  • Figure out how this interacts with a developer's desire to rebuild assets that they have changed. In other words, as a developer, would tutor dev run --mount=edx-platform openedx-assets build --env=dev work if I wanted to re-run it? They should just rebuild
  • In the Docker image build, move the artifacts out of edx-platform
  • In the Docker image build and in mounted-directories.sh, convert edx-platform artifact dirs into symlnks
  • Triumphantly remove the rest of Tutor's mounted-directories.sh task (the parts that regenerate assets and node_modules and egg-infos)

Notes

No response
@kdmccormick kdmccormick self-assigned this Jan 5, 2023
@kdmccormick kdmccormick mentioned this issue Jan 5, 2023
Closed
@kdmccormick kdmccormick removed their assignment Jan 5, 2023
@kdmccormick kdmccormick mentioned this issue Jan 5, 2023
Closed
@kdmccormick
Copy link
Member Author

Blocked by openedx/edx-platform#31798

kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
505e9c4 
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
9be70cc 
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
10f8233 
Before this commit, setting up an edx-platform development environment
took multiple steps:

  tutor dev launch
  tutor dev run --mount=/path/to/edx-platform lms bash
  >> pip install -e .
  >> npm clean-install
  >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

  tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
5876a05 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
@kdmccormick kdmccormick mentioned this issue Mar 13, 2023
Closed
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
7222ab1 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
ebab66a 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
kdmccormick referenced this issue in kdmccormick/tutor Mar 13, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
cac093b 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
@kdmccormick kdmccormick mentioned this issue Mar 13, 2023
Merged
kdmccormick referenced this issue in kdmccormick/tutor Mar 14, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
4fbe896 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
regisb referenced this issue in overhangio/tutor Mar 15, 2023
@kdmccormick
feat: full edx-platform setup with tutor dev launch -m ...
f984361 
Before this commit, setting up an edx-platform development environment
took multiple steps:

   tutor dev launch
   tutor dev run --mount=/path/to/edx-platform lms bash
   >> pip install -e .
   >> npm clean-install
   >> openedx-assets build --env=dev

This commit moves the steps under ``run`` into an init task, which
is automatically run by ``launch``. Thus, setup is now one command:

   tutor dev launch --mount=edx-platform

These extra init steps are only applicable when bind-mounting
edx-platform (because bind-mounting the repository overrides
some important artifacts that exist on the image, which must be
re-generated). Thus, the new init tasks exists early if it detects
that it is *not* operating on a bind-mounted repository.

Finally, we try to simplify the Open edX development docs so that
it is clearer how bind-mounting fits into the development process.

These bind-mounts:

* ../build/openedx/themes:/openedx/themes
* ../build/openedx/requirements:/openedx/requirements

existed in the dev lms and cms containers, but they did
not exist in the lms-job and cms-job containers.

This means that themes and requirements that were *built into the
image* would exist in the job containers, but live updates to the
themes and requirements would not apply.

To resolve this, we set ``volumes:`` on the lms-job and cms-job
services so that they match the volumes for the normal lms and
cms services.

Part of: openedx-unsupported/wg-developer-experience#146
Closes: openedx-unsupported/wg-developer-experience#152

This works around (but does not close) these related issues:
* openedx-unsupported/wg-developer-experience#150
* https://github.com/openedx/wg-developer-experience/issues/151
@kdmccormick kdmccormick mentioned this issue Mar 15, 2023
Closed
@kdmccormick kdmccormick closed this as not planned Won't fix, can't repro, duplicate, stale Aug 18, 2023
@kdmccormick kdmccormick self-assigned this Mar 28, 2024
@kdmccormick kdmccormick reopened this Mar 28, 2024
@kdmccormick kdmccormick changed the title Move static assets outside of edx-platform in Tutor's openedx image Move artifacts outside of edx-platform in Tutor's openedx image Mar 28, 2024
@kdmccormick kdmccormick transferred this issue from openedx-unsupported/wg-developer-experience Mar 28, 2024
@kdmccormick kdmccormick added the tutor Requires a change to Tutor label Mar 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kdmccormick kdmccormick

Labels
tutor Requires a change to Tutor
Projects
No open projects
Developer Experience Working Group
Status: Closed
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kdmccormick