Skip to content

edX Developer Stack

Jump to bottom Edit New page
hkim823 edited this page Sep 15, 2014 · 153 revisions

The edX devstack is a Vagrant instance designed for local development. The instance:

  • Uses the same system requirements as production. This allows developers to discover and fix system configuration issues early in development.
  • Simplifies certain production settings to make development more convenient. For example, it disables nginx and gunicorn in favor of runserver for Django development.

The devstack instance is designed to run code and tests, but you can do most development in the host environment:

  • Git repositories are shared with the host system, so you can use your preferred text editor/IDE.
  • You can load pages served by the running Vagrant instance.

The devstack configuration has the following components:

  • LMS (student facing website)
  • Studio (course authoring)
  • Forums / elasticsearch / ruby (discussion forums)
  • ORA (Open Response Assessor)

For comparison, look at the Production Stack page.

Pre-Requisites

Before installing and using the edX Developer Stack, you should:

  • Understand basic terminal usage: see this introduction to the Mac OS X terminal.
  • Understand Vagrant commands: see the Vagrant "Getting Started" guide for a quick introduction.
  • Devstack uses VirtualBox Guest Additions to share folders via NFS or VBOXFS to allow for easier development. An NFS client should be installed prior to use if your OS doesn't come with it pre-installed. For Windows users, NFS is not supported but VBOXFS is and can be called via an environment variable.

Installing the edX Developer Stack

  • Install the latest VirtualBox >= 4.3.12

  • Install the latest Vagrant >= 1.5.3 (for vagrant >= 1.6, you may need to apply these workarounds - it may be simpler to just use Vagrant 1.5.4, if that is an option for you)

  • Download the Vagrantfile and create the Vagrant instance:

      mkdir devstack
  cd devstack
  curl -L https://raw.githubusercontent.com/edx/configuration/master/vagrant/release/devstack/Vagrantfile > Vagrantfile
  vagrant plugin install vagrant-vbguest
  vagrant up

    • Note: The first time you create the virtual machine (VM) instance, Vagrant will download the base box, which is about 4GB. If you destroy and recreate the VM, Vagrant will re-use the box it downloaded.

    • The base box can also be downloaded using Torrent. Once downloaded, you can add the box using 'vagrant box add (box-name) (path-to-box-file)'.

    • After the VM is booted up, it mounts the shared folders so they can be accessed by both your host system and the virtual system. When it gets to this point it may ask for your password on the host system.

        [default] Exporting NFS shared folders...
  Preparing to edit /etc/exports. Administrator privileges will be required...
  Password:

  • Once the VM is created, log in to the VM via the command: vagrant ssh

  • OPTIONAL [pertains to Windows and others having issues with NFS]: To use VBOXFS, set in your shell the environment variable VAGRANT_USE_VBOXFS = true

  • OPTIONAL [pertains to unix/linux and mac hosts]: To use "preview" from within Studio, you will need to add a line, 192.168.33.10 preview.localhost to your development machine's /etc/hosts file. For example:

      $ cat /etc/hosts
  # Host Database
  127.0.0.1   localhost
  255.255.255.255 broadcasthost
  ...
  $ # if preview.localhost doesn't already exist:
  $ sudo bash -c "echo '192.168.33.10  preview.localhost' >> /etc/hosts"
  $ cat /etc/hosts
  # Host Database
  127.0.0.1   localhost
  255.255.255.255 broadcasthost
  ...
  192.168.33.10  preview.localhost
  ```

  • OPTIONAL - if you wish to customize the location of the source code that gets cloned during devstack provisioning, or if you prefer to work on source trees that already exist on your system, you can set the VAGRANT_MOUNT_BASE env var, which sets the base dir for the edx-platform and cs_comments_service. By default it will be wherever you run vagrant up. Example:

      $ cd $HOME/my-workspace/my-edx-workspace
  $ # This is where you either want vagrant provisioning to clone,
  $ #  or you have existing repositories (although note that vagrant provision will overwrite them):
  $ ls -F
  $ #  You might optionally need to rename edx-ora to ora
  cs_comments_service/   edx-platform/   ora/
  $ # Now, set the environment for vagrant
  $ # (you may want to put this in a file, and 'source' it instead)
  $ export VAGRANT_MOUNT_BASE=$PWD

Using the edX devstack

LMS Workflow

Within the VM instance, switch to the edxapp account: sudo su edxapp

  • This will source the edxapp environment (/edx/app/edxapp/edxapp_env) so that the venv python and rbenv ruby are in your search path.
  • It will also set the current working directory to the edx-platform repository (/edx/app/edxapp/edx-platform).

If you are running the developer stack with the standard environment file, lms/envs/devstack.py, then you can use this command:

paver devstack lms

There is a faster version of this command that starts the server without updating requirements or compiling assets. To ONLY start the server:

paver devstack --fast lms

Note: you can type paver --help to list commands, or paver [command] --help for command options.

Open a browser on your host machine and navigate to http://localhost:8000/ to load the LMS. (Vagrant will forward port 8000 to the LMS server running in the VM.)

The latest version of devstack has the demo course pre-loaded and dummy accounts, you can log in to the website as:

 [email protected] / edx
 [email protected] / edx
 [email protected] / edx
 [email protected] / edx

If you are bringing up LMS with a custom configuration (For example devstack as for developer stack and devstack.py, but you name yours as you require):

  • Update requirements, and compile Sass and CoffeeScript:
paver update_assets --settings devstack lms

Note: paver -h will show other available commands

  • Start your custom configured server:
./manage.py lms runserver --settings=devstack 0.0.0.0:8000

Note: ./manage.py lms -h --settings=devstack will show other available commands on the server.

Studio Workflow

Within the VM instance, switch to the edxapp account:

sudo su edxapp

This will source the edxapp environment (/edx/app/edxapp/edxapp_env) so that the venv python and rbenv ruby are in your search path. It will also set the current working directory to the edx-platform repository (/edx/app/edxapp/edx-platform).

Note: the following steps are for running with the standard configuration file. To run Studio with a custom configuration, follow the instructions as for LMS, above, replacing lms with cms in the commands.

If you are running the developer stack with the standard environment file, cms/envs/devstack.py, then you can use this command:

paver devstack studio

There is a faster version of this command that starts the server without updating requirements or compiling assets. To ONLY start the server:

paver devstack --fast studio

Open a browser on your host machine and navigate to http://localhost:8001/ to load Studio. (Vagrant will forward port 8001 to the Studio server running in the VM.)

  • OPTIONAL: you can see what other commands are available on the server (such as creating users, collecting static files, etc) with:
./manage.py cms -h --settings=devstack

Forum Workflow

  • Within the VM instance, switch to the forum account
sudo su forum
  • Update Ruby requirements (Note that if you get a message for entering a password to install the bundled RubyGems to the system, you can safely ctrl-c out of that. The gems will still be installed correctly for the forum user.)
bundle install
  • Start the server
ruby app.rb -p 18080
  • Access the API at http://localhost:18080 (Vagrant will forward port 18080 to the Forum server running in the VM.)

Developing on devstack

See Developing on the edX Developer Stack once you are ready to start developing on devstack.

Running LMS/Studio Tests

  • Within the VM instance, switch to the edxapp account
sudo su edxapp
  • Run the Python unit tests
paver test_python
  • Run the JavaScript unit tests
paver test_js
  • Run the LMS and Studio acceptance tests
paver test_acceptance

See the docs on testing edx-platform for detailed information about writing and running tests.

Updating Application Versions

Use vagrant provision to update the configuration and all repositories to the master branch. NOTE: this will perform a git clean on your repositories, so make sure you have committed any changes you want to save.

Logs

Logs for the running applications can be found in various application specific folders in /edx/var/log.

Troubleshooting / Issues / Workarounds

Provisioning Issues

edx-platform checkout failure

Sometimes there's a failure to checkout the edx-platform repo (as shown below). This can occur if the edx-platform repository is already checked out on your machine and the git URL for the origin remote is set to an SSH URL (e.g. [email protected]:edx/edx-platform.git) instead of HTTPS e.g. https://github.com/edx/edx-platform.git).

SSH keys are not installed on the Vagrant box so SSH remotes will not work. Use an HTTPS remote instead to resolve this issue. This can be set by using git remote set-url origin <https url> as the edxapp user.

TASK: [edxapp | checkout edx-platform repo into {{edxapp_code_dir}}] **********
failed: [localhost] => {"cmd": ["/usr/bin/git", "ls-remote", "origin", "-h", "refs/heads/master"], "failed": true, "item": "", "rc": 128}
stderr: Warning: Permanently added 'github.com,192.30.252.130' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

msg: Warning: Permanently added 'github.com,192.30.252.130' (RSA) to the list of known hosts.
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/root/vagrant-devstack.retry

localhost                  : ok=125  changed=20   unreachable=0    failed=1



Stderr from the command:

stdin: is not a tty

VirtualBox 4.3.10 NFS Mounts

  • VirtualBox has been updated to 4.3.12 to deal with this bug. VirtualBox can have issues when Guest Additions don't match the version installed when it comes to NFS mounts. In particular, VirtualBox 4.3.10 Guest Additions has an NFS mounting bug and can be fixed with
vagrant up
vagrant ssh -c 'sudo ln -s /opt/VBoxGuestAdditions-4.3.10/lib/VBoxGuestAdditions /usr/lib/VBoxGuestAdditions'
vagrant reload

General tips

  • If you want to build an edX devstack environment from a base precise64 image, you can do the following
git clone https://github.com/edx/configuration.git
cd configuration/vagrant/base/devstack
vagrant up
  • If the /edx/app/edx/edx-platform directory in the Vagrant image is empty, it means the provisioning scripts did not complete successfully. Try re-running the provisioning scripts using vagrant provision.

Signing up a new user (activation message problems):

  • When signing up as a new user, this message is shown in the browser:
We need to verify your email address

Almost there! In order to complete your sign up we need you to verify your email address. 
An activation message and next steps should be waiting for you there.

However, when running the stack locally, there is no email message sent. Rather the message comes out on the console, giving the activation key to use:

Thank you for signing up for edX Studio! To activate your account, please copy and paste this address into 
your web browser's address bar:

            http://localhost/activate/0699e17df6024fb382fadfabed100691

If you didn't request this, you don't need to do anything; you won't receive any more email from us. Please do
not reply to this e-mail; if you require assistance, check the help section of the edX web site.

Using the domain address ‘http://localhost/activate/… may not work when you copy this address into the web browser’s address field as instructed. if a ’Server not found’ message comes up in the browser, substitute

http://127.0.0.1:8001

for localhost. For example:

http://localhost/activate/a19de8c6330f4213ac3313c147297990

becomes

http://127.0.0.1:8001/activate/a19de8c6330f4213ac3313c147297990
  • Database migrations: If you update the platform code, then when using the application you get an error message in your browser like this:
Exception Type:   DatabaseError
Exception Value:  

(1146, "Table 'edxapp.new_foo' doesn't exist")

Exception Location:   /edx/app/edxapp/venvs/edxapp/local/lib/python2.7/site-packages/MySQLdb/connections.py in defaulterrorhandler, line 36

there have been database migrations that you need to perform. Note this will not destroy your data. You can use the following command:

paver update_db -s devstack
  • If you see this error when attempting to run the server:
pymongo.errors.ConnectionFailure: could not connect to localhost:27017: [Errno 111] Connection refused

or this one, when attempting to run the forums:

MOPED: Could not connect to any node in replica set <Moped::Cluster nodes=[<Moped::Node resolved_address="127.0.0.1:27017">]>, refreshing list.

The Vagrant instance not shut down properly. Use vagrant halt to safely stop the Vagrant instance. To fix this, try:

vagrant up
vagrant ssh
sudo rm /edx/var/mongo/mongodb/mongod.lock
sudo start mongodb
  • If you see a database error something like this when running the server:
DatabaseError: (1146, "Table 'edxapp.dark_lang_darklangconfig' doesn't exist")

Then this means that your database is outdated and needs to be migrated.

sudo su edxapp
cd /edx/app/edx-platform
./manage.py lms syncdb --migrate --settings=devstack

and then do the same thing for Studio with

sudo su edxapp
cd /edx/app/edx-platform
./manage.py cms syncdb --migrate --settings=devstack

Forums / Ruby Issues:

  • Bundler::GemNotFound on ruby startup

Your gems need updating (might happen when switching versions of cs_comment_service). Run bundle install to resolve this, and then retry.

If you are getting a password prompt after running bundle install, you will need to fix some directory permissions:

# NOTE do this as user vagrant, not user forum
sudo chmod -R 0770 /edx/app/forum/.rbenv
sudo chmod -R 0770 /edx/app/forum/.gem

Then sudo back to user forum, do bundle install again, and you should no longer be prompted.

Ubuntu:

  • if you see this message:
It appears your machine doesn't support NFS, or there is not an
adapter to enable NFS on this machine for Vagrant. Please verify
that `nfsd` is installed on your machine, and try again. If you're
on Windows, NFS isn't supported. If the problem persists, please
contact Vagrant support.

you need to install nfs using this command:

sudo apt-get install nfs-common nfs-kernel-server

  • If you get an error such as NS_ERROR_FAILURE, try upgrading VirtualBox

  • If you are working on an Ubuntu workstation and your home directory is encrypted you will likely run into NFS problems. If you see this message:

exportfs: Warning: ... does not support NFS export.

To work around the issue set VAGRANT_MOUNT_BASE to a dir outside the encrypted volume.

Windows

  • NPM might fail because of symlinks. If you see this message:
TASK: [edxapp | Install edx-platform npm dependencies] ************************

failed: [localhost] => {"changed": true, "cmd": "npm install ", "delta": "0:00:0
3.358306", "end": "2014-03-24 16:51:37.509494", "item": "", "rc": 1, "start": "2
014-03-24 16:51:34.151188"}
stderr: npm http GET http://registry.npmjs.org/coffee-script
npm http 200 http://registry.npmjs.org/coffee-script
npm ERR! error installing [email protected]

npm ERR! Error: UNKNOWN, unknown error '../coffee-script/bin/coffee'
npm ERR! You may report this log at:
npm ERR!     <http://bugs.debian.org/npm>
npm ERR! or use
npm ERR!     reportbug --attach /edx/app/edxapp/edx-platform/npm-debug.log npm
npm ERR!
npm ERR! System Linux 3.2.0-23-generic
npm ERR! command "node" "/usr/bin/npm" "install"
npm ERR! cwd /edx/app/edxapp/edx-platform
npm ERR! node -v v0.6.12
npm ERR! npm -v 1.1.4
npm ERR! path ../coffee-script/bin/coffee
npm ERR! code UNKNOWN
npm ERR! message UNKNOWN, unknown error '../coffee-script/bin/coffee'
npm ERR! errno {}
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR!     /edx/app/edxapp/edx-platform/npm-debug.log
npm not ok

FATAL: all hosts have already failed -- aborting

Go to https://github.com/edx/configuration/wiki/Vagrant-troubleshooting and follow the instructions towards the bottom of the page about this and other symlink errors in Windows.

Mac OS X

Sometimes, NFS fails the first time you start Vagrant on Mac OS X. If you see this message:

[default] Mounting NFS shared folders...
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
mount -o 'vers=3,udp' 192.168.33.1:'/path/to/devstack/edx-platform' /edx/app/edxapp/edx-platform
Stdout from the command:
Stderr from the command:
stdin: is not a tty
mount.nfs: requested NFS version or transport protocol is not supported

then reload and re-provision the instance using these commands:

vagrant reload
vagrant provision
Add a custom footer
Add a custom sidebar
Clone this wiki locally