QUICK-START.md

Overview

For this example, we'll use a simple freeware chat program to demonstrate the features of gumshoe.

Step 1: Get gumshoe

1. Make sure your system has dependencies.

   To use prebuilt jar, you'll need JRE7:

    java -version
   

   To build from source you'll need maven and JDK7.

2. Get gumshoe.

   Download and unpack gumshoe agent and viewer.

    wget https://github.com/dcm-oss/gumshoe/releases/download/v0.9.0/gumshoe-agent.jar
    wget https://github.com/dcm-oss/gumshoe/releases/download/v0.9.0/inspector.jar
    export GUMSHOE_HOME=`pwd` # maybe put this into .bashrc too?
   

Step 2: Try the target app

Thanks to Chris Hallson for writing this example (and probably forgetting all about it years ago). We'll use my fork of it to make it easier to add options to our java commandline.

1. Get the chat program.

   Download and unpack JavaChat source.

   wget https://github.com/newbrough/JavaChat/archive/master.zip
   unzip JavaChat-master.zip
   cd JavaChat-master
   export CHAT_HOME=`pwd` # maybe put this in .bashrc too?
   

2. Try it out.

   Before looking at the I/O with gumshoe, lets just give this app a try. Open two shell windows and in each one, run:

   cd $CHAT_HOME
   ./start-chat.sh
   

   You should have two identical chat windows. One of them select "server" and click "Connect". The other leave as "client" and click "Connect". They should both show messages that they are connected. Change the name of both to something other than "Unknown". Try sending a message or two.

   Everything working ok? Great!

Step 3: Lets see it already

1. Run with gumshoe

   Again we will use two terminal windows and in one run the chat program as before. Make this one the server and change the name to "server".

   And in the second terminal, start the chat program with:

   ./start-chat.sh --gumshoe
   

   This will open another chat program, and the gumshoe viewer

2. Make something to look at.

   Connect one chat window as server, the other as client. Do something so there is some I/O to examine. Change the name and few times, send some messages, whatever.

3. Look at your I/O

   In the gumshoe window, click the first tool in the toolbar (tooltip should say "Select reports from this JVM"). Reports generated by the gumshoe agent are listed -- by default out chat script will only monitor socket I/O, and report it every 30seconds. You might have to wait for it...

   Select one of the reports and you should see some blocks appear in the top main portion of the window. It may not look very interesting -- JavaChat is a pretty simple program. But make sure you tried different I/O operations to gumshoe has a few different stacks to show you.

4. What is this thing?

   The default graph is a root graph -- the top of the stack immediately causing the I/O is shown on top, the callers that invoked those methods next, and so on down the stack.

   For example, along the top you may see boxes for Socket.read() and Socket.write(). Below Socket.read() may be 3 different boxes that are each a method that called Socket.read(). The width of each box may represent the proportion of I/O. Boxes colored red represent a frame responsible for 50% or more of displayed I/O, orange for 33% or more, yellow for 25%.

   The box width currently represents the number of read operations, although bytes, operations and elapsed time are tracked for reads and writes. Hover over a box to view all values.

5. Just like that, but different.

   Near the middle of the toolbar click on "Configure graph".

   The Direction setting lets you choose either a flame graph or root graph.
   A flame graph starts at code entry points at bottom of stack, and can help identify some upstream triggers of I/O like which of my REST services result in the most I/O? A root graph starts at proximal cause of I/O at top of stack, and can help identify lower-level bottlenecks like is my database or REST client responsible for more I/O?

   The default view uses the raw value for cell width, so if a box is twice as wide as another then that stack frame is involved in twice as much I/O (read operations, write milliseconds or whichever type happens to be selected at the moment). To see frames that may be too narrow to appear otherwise, switch to log(value) or equal width sizing.

   Arrange by value sorts cells so those with the most I/O appear on the left.
   Note this may be confusing when changing other display options as the relative positions will move around more.

   Close this dialog and try the next tool to its left "Choose statistic".
   The first graph was based on number of read operations, but we can choose write operations or combined reads and writes. We can scale based on number of operations, number of bytes, elapsed time, average operation size, etc.

   This demo enabled only the socket I/O probe, but for other probes, different measurements are available to display.

6. Keep it real.

   All the stacks seen so far make great examples of how we can navigate the display, but much of what is visible by default is irrelevant.
   That was intentional (by the defaults in start-chat.sh) so we could use a small, simple program for this demo.

   When using gumshoe with a real project, however, you are probably only interested in I/O related to parts of the program under your control. This is what filters are all about.

   Click the icon for "Stack filters" and check the "drop JDK and gumshoe frames" then click "OK".
   Immediately all the ObjectInputStream and Socket stack frames are gone and you are left with just the things in the javachat application and its libraries. This doesn't look nearly as cool, because javachat is a very simple application
   (which is why we started with display options showing the full stack instead.)

   Here you can add (fully-qualified) packages or classes to look for or exclude from analysis. Click "OK" to filter the stacks just for your view. Command-line options can also apply to these filters directly in the probe to limit initial collection, which reduces the resource usage and overhead of the gumshoe probe.

   Sometimes analysis can also benefit from reducing stacks down to just the top and bottom few frames. For example, the original target application for gumshoe had threads that began with a REST, SOAP, or timer kicking off some action, then filter down through various layers of business logic, finally resulting in a SQL call, a direct TCP socket to another system, or making a REST call to an external system.
   Limiting the view alternately to the top or bottom few frames showed the relative cost of services we provided and services we called, and gave good targets for later filters to probe the full stack of those specific bottlenecks.

Step 4: Now what?

The JavaChat startup script start-chat.sh has additional options commented out. You can uncomment to try the file-io or CPU probes.

Or try gumshoe with your own application. In this example, the viewer was in the same JVM as the target app, but to simplify things you might want to collect data first in a file, then view the contents of the file.

1. Instrument your application

   To enable gumshoe probes, your application must start with a few command-line options set:

   • Gumshoe agent with hooks for all probes:

     -javaagent:gumshoe-agent.jar

   • Configigure the probes with system properties

     -Dgumshoe.socket-io.period=30000

     or replace socket-io with file-io, cpu-usage, datagram-io. See documentation for details.

     You can also configure stack filters with properties. For example, to select your project com.mycompany.proj and a library org.thirdparty.mylib use the property:

     -Dgumshoe.socket-io.filter.include=com.mycompany.proj,org.thirdparty.mylib

2. Collect output from the JVM. By default the output will be put in STDOUT, so this should be redirected to a file. Unrelated lines in STDOUT are not a problem, but if you application adds a timestamp or other text to the gumshoe lines, this should be removed. You can alternately specify another file for gumshoe output using properties like:

    -Dgumshoe.socket-io.output=file:/tmp/socket-io-data.txt
   

3. Run your application and verify data is being collected. The text file should contain matching tags <gumshoe-report ...> with reported data between them including stack traces. These files can get big! Use fewer probes and more stack filters to manage the size. You can also enable/disable monitoring on the fly with JMX, which can limit how much data is collected. If there is nothing between the start and end tags, it could mean the application is not performing any socket or datagram I/O (whichever is being monitored).

4. Start the viewer and open the file:

    java -jar inspector.jar