add mitxonline networking directions #2594

ChristopherChudzicki · 2025-10-15T19:32:05Z

What are the relevant tickets?

All the PRs we make where we copy-and-paste the setup directions for mitxonline <> Learn.

Description (What does it do?)

This PR adds directions for running MITxOnline and MIT Learn locally in a way that they can communicate.

How can this be tested?

Follow the directions in the README
Sanity checks:
- http://api.open.odl.local:8065/mitxonline/api/v2/courses/ should return same as http://mitxonline.open.odl.local:8013/mitxonline/api/v2/courses/
- After authenticating on Learn via frontend login http://api.open.odl.local:8065/mitxonline/ap/v0/users/me should show your user info (note: no trailing slash!).
Create a program with some required and elective courses. Useful things to know:
- Create courses and programs with ./manage.py create_courseware, ./manage.py create_courseware_page; see --help for details. Note: program readable ids should start with program-v1:.
- Add instructors/relevant data for your program in Wagtail at http://mitxonline.odl.local:8013/cms
Run MITxOnline ETL against your local MITxOnline; ./manage.py backpopulate_mitxonline_data --delete, and ./manage.py backpopulate_mitxonline_data
Ensure product-page-course feature flag is on in your personal posthog
Try backpopulating mitxonline data against local, ./manage.py backpopulate_mitxonline_data
Try Viewing a program page or course page, e.g., http://open.odl.local:8062/courses/COURSE_ID_HERE or http://open.odl.local:8062/programs/PROGRAM_ID_HERE

docker-compose.services.yml

dsubak

You may want to hold off for another, more experienced review, but these instructions are pretty much bang on what I ended up figuring out when I set it up on my own. Thanks for the writeup!

jkachel · 2025-10-16T16:54:45Z

The goal of this is to allow MITx Online and Learn to be set up so that MITx Online can have some data in it, and Learn can pull that data out for its needs, right? This seems OK for that purpose.

mbertrand · 2025-10-16T17:32:36Z

README.md


+### Connecting with MITxOnline
+
+Set up the [mitxonline](https://github.com/mitodl/mitxonline) as indicated there. _Note: A working OpenEdx installation is not necessary for integration with Learn._


Maybe place this sentence after setting the env variables below in MIT-Learn and MITx Online, and starting up both.

Also, OPENEDX_API_CLIENT_ID and OPENEDX_API_CLIENT_SECRET require values for the mitxonline app to start successfully, but they are not mentioned in that repo's README or included in the example .env file. They are mentioned in the "Using Tutor" instructions but I skipped that step.

Not sure if this PR should also enable mitxonline users to log in via mit-learn's apisix/keycloak instance, but if so I could not validate that because something seems off with my local setup and I was unable to log in that way.

Not sure if this PR should also enable mitxonline users to log in via mit-learn's apisix/keycloak instance, but if so I could not validate that because something seems off with my local setup and I was unable to log in that way.

Do you mean click login via MITxOnline login button? I had forgotten that does a surprising infinite loop. I'm not sure what causes that, but logging in via MIT Learn works.

@jkachel Do you have any idea? IMO not critical though.

Yes, I was getting that endless loop initially, then started adjusting other env values with best guesses and got apisix or keycloak errors instead.

mbertrand · 2025-10-16T18:47:52Z

README.md

+KEYCLOAK_CLIENT_ID=apisix
+KEYCLOAK_CLIENT_SECRET=HckCZXToXfaetbBx0Fo3xbjnC468oMi4 # pragma: allowlist-secret
+KEYCLOAK_DISCOVERY_URL=http://kc.ol.local:8066/realms/ol-local/.well-known/openid-configuration
+KEYCLOAK_REALM_NAME=ol-local


I could not log in with these settings. But the backpopulate command works regardless.

If the goal of this PR is just to backpopulate from a local mitxonline instance, then these keycloak settings can be omitted.

If the goal is also to allow login via MIT-Learn's apisix/keycloak instance, then something is off, maybe I have a bad or missing environment value somewhere.

@mbertrand If:

you could log into Learn via Learn frontend

but once logged into Learn, you weren't authenticated with MITxOnline

I believe the issue was an incorrect value of MITOL_API_DOMAIN. I missed it because I had been overwriting it locally so i could use learn.odl.local, but I've removed all those overrides and I'm now back to using open.odl.local

gumaerc

I've been testing this on Ubuntu Linux 24.04. I was able to eventually get the dashboard connected and see my programs / enrollments on the frontend, after changing some of the settings I noted below. However, I was unable to get the program page in Learn working. When I try and load the page for my test program, I get "Error fetching course page metadata for programv1%3Aprogam-v1-test-program" (Yes, that is the readable ID of my test program). The message says "connect ECONNREFUSED 127.0.0.1:8065." This indicates that the API in the watch server (since the metadata seems to be queried by the server and not the client) has mitxonline.odl.local as the MITx Online base url, which is going to evaluate to 127.0.0.1. I think in the docker configuration, if we somehow get the watch container itself to use host.docker.internal:8013 for that, then it should work?

README.md

gumaerc · 2025-10-16T17:38:58Z

docker-compose.services.yml

-      - MITX_ONLINE_UPSTREAM=${MITX_ONLINE_UPSTREAM:-mitxonline.odl.local:8013}
-      - MITX_ONLINE_DOMAIN=${MITX_ONLINE_DOMAIN:-mitxonline.odl.local}
+      - MITX_ONLINE_UPSTREAM=${MITX_ONLINE_UPSTREAM:-host.docker.internal:8013}
+      - MITX_ONLINE_DOMAIN=${MITX_ONLINE_DOMAIN:-host.docker.internal}


I had to set MITX_ONLINE_DOMAIN to mitxonline.odl.local. This allows you to access http://mitxonline.odl.local:8065/ in a browser and then, using your already logged in session in APISIX, you are automatically authenticated to use Django admin in MITx Online.

I'm not entirely sure what the issue was here, but I've changed so that docker.host.internal is not longer used and we just use mitxonline.odl.local everywhere.

ChristopherChudzicki · 2025-10-22T22:47:22Z

env/backend.env

+MAILGUN_KEY=fake
+MAILGUN_SENDER_DOMAIN=open.odl.local


These settings are new. They were in backend.env.example, are not secret, and some value is required for the app to work, so I think they should be in the defaults.

(I removed most of my local env vars to isolate what was actually necessary for mitxonline connection)

ChristopherChudzicki · 2025-10-22T22:47:38Z

env/backend.env

 CORS_ALLOWED_ORIGINS='["http://open.odl.local:8062"]'
-CSRF_TRUSTED_ORIGINS='["http://open.odl.local:8062", "http://api.open.odl.local:8063"]'
-CSRF_COOKIE_DOMAIN=open.odl.local
+CSRF_COOKIE_DOMAIN=.odl.local


I changed the default value of this env var.

ChristopherChudzicki · 2025-10-22T23:08:55Z

@mbertrand @gumaerc This is ready for another look.

@mbertrand I tried to clarify the readme a bit, both in order and expected behavior of mitxonline.

I also dropped host.docker.internal for a scheme more similar https://github.com/mitodl/mitxonline/blob/29e6356817c1b88d34a08b442df293787873712b/docker-compose.yml#L21

docker-compose.services.yml

docker-compose.yml

ChristopherChudzicki · 2025-10-23T00:47:09Z

docker-compose.yml

+    extra_hosts:
+      # Both the browser and NextJS server (containerized) make requests to
+      # api.open.odl.local, forward those to host
+      - "api.open.odl.local:host-gateway"


@gumaerc re

When I try and load the page for my test program, I get "Error fetching course page metadata for programv1%3Aprogam-v1-test-program" (Yes, that is the readable ID of my test program).

I suspect that was happening for all api-dependent metadata generation (though some of them may have fallbacks to static metadata w/o an api call), not just the mitxonline-specific ones.

This should fix it.

dsubak · 2025-10-23T17:20:13Z

FWIW, I took another look at this and ran through the tests. Still looks good (I like the new extra_hosts approach)!

I'll note that I did hit 404s when attempting to hit http://learn.odl.local:8062/courses/COURSE_ID_HERE in step 10, but everything got populated at the database level afaict. Given my lack of familiarity, I wanted to note it, but unless anyone else hits the same, I'm operating on the assumption that I've messed something up and it's not an issue w/ these changes.

mbertrand

I can log in to mit-learn, then when I click the link to the mitxonline course, I am logged in to my local mitxonline as well - 👍

Login still doesn't work in the opposite direction - endless redirects when trying to log in to mitxonline first.

http://mitxonline.odl.local:8013/courses/course-v1:mycourse : works fine

http://open.odl.local:8062/courses/course-v1:mycourse : 404

ChristopherChudzicki · 2025-10-23T19:15:12Z

README.md

+
+```env
+# MIT Learn, shared.local.env (ensure not overwritten in backend/frontend env)
+MITX_ONLINE_BASE_URL=http://api.open.odl.local:8065/mitxonline


@mbertrand I had this wrong previously, MITX_ONLINE_BASE_URL should be in shared (not backend) since it's used by the frontend, too.

mbertrand

Works with the latest changes (and turning off the browser's adblocker to let posthog through).

ChristopherChudzicki mentioned this pull request Oct 15, 2025

Program Summary #2590

Merged

ChristopherChudzicki added the Needs Review An open Pull Request that is ready for review label Oct 15, 2025

ChristopherChudzicki commented Oct 15, 2025

View reviewed changes

docker-compose.services.yml Outdated Show resolved Hide resolved

dsubak approved these changes Oct 16, 2025

View reviewed changes

mbertrand reviewed Oct 16, 2025

View reviewed changes

gumaerc requested changes Oct 16, 2025

View reviewed changes

ChristopherChudzicki force-pushed the cc/mitxonline-networking branch from f2b0fec to c72bdeb Compare October 22, 2025 22:41

ChristopherChudzicki commented Oct 22, 2025

View reviewed changes

ChristopherChudzicki force-pushed the cc/mitxonline-networking branch 2 times, most recently from c4ff3c8 to 81b26ec Compare October 22, 2025 23:05

ChristopherChudzicki commented Oct 23, 2025

View reviewed changes

docker-compose.services.yml Show resolved Hide resolved

ChristopherChudzicki commented Oct 23, 2025

View reviewed changes

docker-compose.yml Show resolved Hide resolved

ChristopherChudzicki commented Oct 23, 2025

View reviewed changes

mbertrand reviewed Oct 23, 2025

View reviewed changes

ChristopherChudzicki force-pushed the cc/mitxonline-networking branch 2 times, most recently from 327c70f to 6788e23 Compare October 23, 2025 19:14

ChristopherChudzicki commented Oct 23, 2025

View reviewed changes

mbertrand approved these changes Oct 23, 2025

View reviewed changes

ChristopherChudzicki added 8 commits October 23, 2025 16:11

add mitxonline networking directions

80f7014

add mailgun defaults

8c6478e

fix MITOL_API_DOMAIN env var

d856c98

clarify expected behavior

4f8ddb5

allow watch to query api.odl.learn

c29e013

fix two remaining learn.odl.local

97ff2d5

add comment

ecfe0c1

put MITX_ONLINE_BASE_URL in shared.env

0b19581

ChristopherChudzicki force-pushed the cc/mitxonline-networking branch from 6788e23 to 0b19581 Compare October 23, 2025 20:12

ChristopherChudzicki merged commit 4788af3 into main Oct 24, 2025
18 of 19 checks passed

ChristopherChudzicki deleted the cc/mitxonline-networking branch October 24, 2025 00:12

This was referenced Oct 27, 2025

Release 0.47.0 #2635

Closed

Release 0.47.1 #2638

Merged


		### Connecting with MITxOnline

		Set up the [mitxonline](https://github.com/mitodl/mitxonline) as indicated there. _Note: A working OpenEdx installation is not necessary for integration with Learn._

		MAILGUN_KEY=fake
		MAILGUN_SENDER_DOMAIN=open.odl.local

add mitxonline networking directions #2594

add mitxonline networking directions #2594

Uh oh!

Conversation

ChristopherChudzicki commented Oct 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What are the relevant tickets?

Description (What does it do?)

How can this be tested?

Uh oh!

Uh oh!

dsubak left a comment

Choose a reason for hiding this comment

Uh oh!

jkachel commented Oct 16, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gumaerc left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChristopherChudzicki Oct 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ChristopherChudzicki commented Oct 22, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dsubak commented Oct 23, 2025

Uh oh!

mbertrand left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbertrand left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

ChristopherChudzicki commented Oct 15, 2025 •

edited

Loading

ChristopherChudzicki Oct 22, 2025 •

edited

Loading