Lesson: install hydra jetty

Goals

• Install a copy of hydra-jetty, which includes pre-configured copies of Fedora and Solr
• Learn to start and stop hydra-jetty (which contains Fedora and Solr)

Explanation

In order to use blacklight and hydra-head, you need an installation of Solr. In addition, hydra-head requires a copy of Fedora. Fedora and Solr are both Java web applications that need to run in a servlet container like Tomcat or Jetty.

For developer environments, we have created a package called hydra-jetty which provides both services pre-installed within a Jetty Java application server. Whenever you need Fedora and Solr running in your development environment, just start or stop that copy of hydra-jetty.

TIP DO NOT use hydra-jetty for production installations. The hydra-jetty passwords are well-known and the installation has not been secured for non-local use.

Steps

Step 1: Install the hydra-jetty package

Use the hydra:jetty generator to install the hydra-jetty package by running:

TIP hydra-jetty is a very large download. If you are completing this lesson as part of a workshop, the facilitator may have a copy of hydra-jetty that you can load from a flash-drive rather than downloading over the internet. The workshop facilitator will provide alternate instructions for this step; probably something like cp ../master.zip tmp/

rails g hydra:jetty

This generator is provided by the hydra-core gem.

WINDOWS TIP: Currently this rake task doesn't work on Windows (see jettywrapper issue #14 for status). Workaround: Download https://github.com/projecthydra/hydra-jetty/archive/master.zip, unpack it, and move the unpacked 'jetty' directory to the root of your application.

 

NOTE: the generator requires that your system have curl and unzip installed. If it does not, you may see either of the following unhelpful errors:

Unable to download jetty from https://github.com/projecthydra/hydra-jetty/archive/v8.1.1.zip

To resolve this issue, install a version of curl that works on your system.

Unable to unzip tmp/v8.1.1.zip into tmp_save_dir/

To resolve this issue, install a version of unzip that works from the command line on your system.

This generator can trigger a very long download (over 100Mb of download). When it's done you'll see the directory named jetty has been created. If you run git status you will see output like this

# On branch master
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)
#
#	jetty/

VM Tip: Only necessary if you are running jetty from a shared folder within a Virtual Machine. File sharing in some VMs is not compatible with the file locking mechanism Fedora 4 uses, so you'll need to point your Fedora data directory to a directory that isn't shared with the host machine:

mkdir /tmp/fcrepo4-data
ln -s /tmp/fcrepo4-data jetty/

NOTE you'll have to repeat these steps any time you run rake jetty:clean

Step 2: Make git ignore the jetty directory

We want git to ignore the jetty directory for the same reasons that we don't check our development databases into git -- because it's big and bulky and you don't actually need other developers to have exact copies of your jetty as long as they have all the other code.

We do that by editing .gitignore and adding the something like this:

# Ignore jetty directory (from hydra-jetty)
/jetty

Now commit this change

git add .gitignore
git commit -m "Adds /jetty to .gitignore"

Step 3: Configure and Start Jetty

To configure jetty for use with Hydra-Works, we need to modify the demo app's Rakefile. Open Rakefile at the root directory of the hydra-works-demo rails app and add the import statement such that the file looks like...

require File.expand_path('../config/application', __FILE__)

import File.join(Gem::Specification.find_by_name('hydra-works').gem_dir,
                 'lib/tasks/hydra-works_tasks.rake')

Rails.application.load_tasks

NOTE: Long term, the import statement above will likely be placed in the main hydra generation process and you will not need to add it to the demo app's Rakefile.

Run the configuration specific to Hydra-Works. At the project root, type...

rake jetty:clean
rake hydra_works:jetty:config
rake jetty:start

WINDOWS TIP: This rake task is not currently working on Windows (see jettywrapper issue #13 for status). In the meantime, start jetty manually

cd jetty
java -Djetty.port=8983 -Dsolr.solr.home=/Users/justin/hydra-demo/jetty/solr -Xmx512m -XX:MaxPermSize=128m -jar start.jar

You should see output like this:

Starting jetty with these values: 
jetty_home: /Users/justin/hydra-demo/jetty
jetty_command: java -Djetty.port=8983 -Dsolr.solr.home=/Users/justin/hydra-demo/jetty/solr -Xmx512m -XX:MaxPermSize=128m -jar start.jar
Logging jettywrapper stdout to /Users/justin/hydra-demo/jetty/jettywrapper.log
Wrote pid file to /Users/justin/hydra-demo/tmp/pids/_Users_justin_hydra-demo_jetty.pid with value 8315
Started jetty (40308.7ms)

hydra-jetty has a fair amount of stuff in it, so it may take up to a minute to start. You can check to see if it's started by going to http://localhost:8983/

If Fedora, Solr, or jetty itself are not starting, you'll want to look at the jettywrapper log to diagnose.

Step 4: Review and commit your changes

See the changes from configuring jetty

git status

NOTE: You won't see anything in the /jetty directory since we added it to .gitignore.

After you've viewed which files have been modified, commit the changes:

git add .
git commit -m "Configured jetty"

Step 5: Look at the jettywrapper log

The jetty:start rake task runs jetty as a background job, so jetty's logs won't appear in your terminal. Instead they're written to the file jetty/jettywrapper.log. If you look at the output from running the jetty:start task, you'll see that one line gives you the full path to the file, for example:

Logging jettywrapper stdout to /Users/justin/hydra-demo/jetty/jettywrapper.log

You can open this log file with any text editor or log reader.

vi jetty/jettywrapper.log

Step 6: Monitor the jettywrapper log

Tip: if jetty is taking a long time to start, you can watch its output using the tail command with the path to your jettywrapper.log. For example:

tail -f jetty/jettywrapper.log

As Jetty, Fedora, and Solr start you will see more info being written to the log file. After a few moments you will be able to open jetty at http://localhost:8983/ or http://0.0.0.0:8983/

Step 7: Stop Jetty

You might have guessed this one. In order to stop jetty, at the project root, type

rake jetty:stop

Step 8: Start Jetty again

Before proceeding to the next lesson, make sure jetty is running. If you're not sure whether it's running, go to http://localhost:8983. If jetty is running, a page with links to Solr and Fedora will load. If jetty is not running no page will load.

If it's not running, just use the jetty:start rake task again.

rake jetty:start

Tip: Sometimes people are confused about whether they need to restart jetty when they restart their Rails application. In most cases it is fine to leave jetty running when you start, stop, and restart the Rails application. The only exception is when you make changes to Solr's configuration or Fedora's configuration -- these would be changes to files inside of your copy of hydra-jetty (i.e. jetty/solr/config), not changes to files in your Rails application's Ruby code. In those cases, where you have made changes to Solr or Fedora's configuration, you need to restart Jetty in order for those changes to take effect. The most common change that requires restarting jetty is when you modify the solrconfig.xml or schema.xml in your solr config directory. Normally, changes to your data steams or models do not require restarts to Solr and Fedora because these changes are indexed dynamically by Hydra.

Notes for users of fcrepo4-vagrant

If you are using a virtual machine created with fcrepo4-vagrant notice that you don't need Jetty Wrapper as the VM already comes with Tomcat, Solr, and Fedora.

Next Step

Go on to Lesson: Start the Application & Search for Results or return to the Dive into Hydra-Works page.