More preparation for version 2.0.0.rc1.

- Added generated files in docs/code to repository
- Various updates on the manual

See issue #40.

code/deprecated/gui.html

@@ -0,0 +1,63 @@
+<a name=gui><h2>5. Using the Toolkit GUI</h2></a>
+
+The Toolkit GUI can be used by typing one of
+
+<blockquote>
+<code>
+% pythonw gui.py<br/>
+% python gui.py
+</code>
+</blockquote>
+
+Use <code>pythonw</code> when available, <code>python</code>
+otherwise. In both cases the wxPython package needs to be
+installed. The GUI has three advantages over using the command line
+version:
+
+<ol>
+<li>It is faster when using on a file-by-file basis because all
+libraries are loaded either at start-up or when the first file is processed.
+<li>It is easier to use.
+<li>It allows the user to type some input and see what happens.
+</ol>
+
+The main disadvantage is that it is not possible to process all files
+in a directory. Here's a screen shot:
+
+<blockquote>
+<img src="gui-1.png" width="800" height="480"/>
+</blockquote>
+
+There is no separate manual for the GUI, but usage should be pretty
+straightforward. Functionality can be summed up as follows:
+
+<ul>
+
+<li>Use "Load File" to select a file to be processed. 
+
+<li>Use "Load Text" to type in text. This will create a file in
+the <code>data/in/User</code> directory, which is then selected as the
+input file.
+
+<li>Use the"Document Type", "Trap Errors" and "Content Tag" pull-downs
+and text field to enter processing options. Setting the "Document
+Type" option will set the "Content Tag" text field to the default for
+the document type. It will also select the default components for the
+document type. The default components can be overridden by checking
+and unchecking the component check boxes. Note that the three
+pre-processing components are all either on or off.
+
+<li>Use "Process File" to process the input file conform to the selected settings.
+
+<li>Finally, "View Results" will pop up a new window that displays the
+processing results in a more informative way (the large text field in
+the control panel just prints the result file as is). The buttons in
+the results view control what is shown in it. There are two
+exceptions: (i) the "Graph" button will start the Tango annotation
+tool on the results file, showing a graph of the annotation and
+including a "TBox" button which displays an alternative format, and
+(ii) the "TBox" button doesn't do anything yet, in the future it will
+provide a more direct way to the TBox representation.
+
+</ul>
+

code/utilities/make_documentation.py

@@ -3,9 +3,12 @@
 Create manual pages with the docstrings for modules, classes and functions,
 using the modules listed in modules.py.
 
+To update the documentation run this script from the directory it is in.
+
 This code was written because pydoc breaks on some of the tarsqi modules, most
 notably those that import the treetagger.
 
+
 TODO:
 
 - the code does not deal with class methods, see for example TagId in
@@ -14,8 +17,7 @@
 - At some point the code created potentially undesired documentation for the
   module components.preprocessing.wrapper, adding a couple of module-level
   functions that are not in the code, but are imported functions. This has
-  seized to be the case for that module, but this did not happen through an
-  intentional change on my end.
+  seized to be the case for that module, but it is not clear why.
 
 """
 
@@ -28,15 +30,15 @@
 from modules import MODULES
 
 # Set this to True if you want the sources for all functions written to files
-# that are linked to from the module page, this slows down the code a bit and
-# requires 6MB more space.
+# that are linked to from the module page, this slows down the code quite a bit
+# and requires 6MB more space.
 WRITE_FUNCTION_SOURCES = False
 
 # With this one set to True, private functions (those that start with a single
 # underscore) are included.
 PRINT_PRIVATE_FUNCTIONS = True
 
-# directory where the document ation is written
+# directory where the documentation is written
 DOCUMENTATION_DIR = os.path.join('..', '..', 'docs', 'code')
 
 

docs/code/index.html

@@ -0,0 +1,54 @@
+<html>
+<head>
+<link rel="stylesheet" href="css/list.css"> 
+</head>
+<body>
+<h3>Tarsqi Toolkit Module Documentation</h3>
+<ul>
+<li><a href=modules/tarsqi.html>tarsqi</a>
+<li><a href=modules/components.blinker.compare.html>components.blinker.compare</a>
+<li><a href=modules/components.blinker.main.html>components.blinker.main</a>
+<li><a href=modules/components.blinker.wrapper.html>components.blinker.wrapper</a>
+<li><a href=modules/components.classifier.vectors.html>components.classifier.vectors</a>
+<li><a href=modules/components.classifier.wrapper.html>components.classifier.wrapper</a>
+<li><a href=modules/components.common_modules.chunks.html>components.common_modules.chunks</a>
+<li><a href=modules/components.common_modules.component.html>components.common_modules.component</a>
+<li><a href=modules/components.common_modules.constituent.html>components.common_modules.constituent</a>
+<li><a href=modules/components.common_modules.tree.html>components.common_modules.tree</a>
+<li><a href=modules/components.common_modules.sentence.html>components.common_modules.sentence</a>
+<li><a href=modules/components.common_modules.tags.html>components.common_modules.tags</a>
+<li><a href=modules/components.common_modules.tokens.html>components.common_modules.tokens</a>
+<li><a href=modules/components.evita.bayes.html>components.evita.bayes</a>
+<li><a href=modules/components.evita.event.html>components.evita.event</a>
+<li><a href=modules/components.evita.features.html>components.evita.features</a>
+<li><a href=modules/components.evita.main.html>components.evita.main</a>
+<li><a href=modules/components.evita.rule.html>components.evita.rule</a>
+<li><a href=modules/components.evita.wrapper.html>components.evita.wrapper</a>
+<li><a href=modules/components.gutime.wrapper.html>components.gutime.wrapper</a>
+<li><a href=modules/components.merging.sputlink.main.html>components.merging.sputlink.main</a>
+<li><a href=modules/components.merging.sputlink.graph.html>components.merging.sputlink.graph</a>
+<li><a href=modules/components.merging.sputlink.objects.html>components.merging.sputlink.objects</a>
+<li><a href=modules/components.merging.sputlink.utils.html>components.merging.sputlink.utils</a>
+<li><a href=modules/components.merging.wrapper.html>components.merging.wrapper</a>
+<li><a href=modules/components.preprocessing.abbreviation.html>components.preprocessing.abbreviation</a>
+<li><a href=modules/components.preprocessing.chunker.html>components.preprocessing.chunker</a>
+<li><a href=modules/components.preprocessing.tokenizer.html>components.preprocessing.tokenizer</a>
+<li><a href=modules/components.preprocessing.wrapper.html>components.preprocessing.wrapper</a>
+<li><a href=modules/components.s2t.main.html>components.s2t.main</a>
+<li><a href=modules/components.s2t.wrapper.html>components.s2t.wrapper</a>
+<li><a href=modules/components.slinket.main.html>components.slinket.main</a>
+<li><a href=modules/components.slinket.wrapper.html>components.slinket.wrapper</a>
+<li><a href=modules/docmodel.document.html>docmodel.document</a>
+<li><a href=modules/docmodel.docstructure_parser.html>docmodel.docstructure_parser</a>
+<li><a href=modules/docmodel.main.html>docmodel.main</a>
+<li><a href=modules/docmodel.metadata_parser.html>docmodel.metadata_parser</a>
+<li><a href=modules/docmodel.source_parser.html>docmodel.source_parser</a>
+<li><a href=modules/library.classifier.create_model.html>library.classifier.create_model</a>
+<li><a href=modules/library.classifier.create_vectors.html>library.classifier.create_vectors</a>
+<li><a href=modules/utilities.binsearch.html>utilities.binsearch</a>
+<li><a href=modules/utilities.logger.html>utilities.logger</a>
+<li><a href=modules/utilities.mallet.html>utilities.mallet</a>
+<li><a href=modules/utilities.porterstemmer.html>utilities.porterstemmer</a>
+</ul>
+</body>
+</html>

docs/code/modules/components.arglinker.main.html

@@ -0,0 +1,80 @@
+<html>
+<head>
+<link href="../css/module.css" rel="stylesheet" type="text/css">
+<script language="JavaScript" type="text/JavaScript">
+<!--
+function view_code(id) {
+  var newurl = "../functions/" + id + ".html";
+  var w = window.open(newurl,"source code","width=770,height=600,scrollbars=yes,resizable=yes");
+  w.xopener = window;
+}
+//-->
+</script>
+</head>
+<body>
+<a href=../index.html>index</a>
+
+<div class="title">module components.arglinker.main</div>
+
+<pre>
+<a href=#ArgLinker>ArgLinker</a>
+</pre>
+
+<pre>
+Main module for the ArgLinker component.
+
+ArgLinker is a simple Python placeholder for the later to be added
+MaxEnt argument linker.</pre>
+
+
+<a name="ArgLinker"/><div class="section">class ArgLinker</div>
+<pre>
+<strong>Inherits from: <a href=components.common_modules.component.html#TarsqiComponent>components.common_modules.component.TarsqiComponent</a></strong>
+
+Simple Python place holder for a more sophisticated argument linker
+using a MaxEnt classifier.
+
+Instance variables:
+   NAME - a string
+   doctree - a Document</pre>
+
+<blockquote>
+<h3>Public Functions</h3>
+<pre>
+<div class=function>__init__(self)</div>
+Load the Slinket dictionaries if they have not been loaded yet.</pre>
+<pre>
+<div class=function>getLinks(self, sentence, args)</div>
+</pre>
+<pre>
+<div class=function>process(self, infile, outfile, depfile)</div>
+Run the ArgLinker on the input file and write the results to the
+output file. Both input an doutput file are fragments. Uses
+the xml parser as well as the fragment converter to prepare
+the input and create the shallow tree that ArgLinker requires.
+Arguments:
+   infile - an absolute path
+   outfile - an absolute path</pre>
+</blockquote>
+<blockquote>
+<h3>Private Functions</h3>
+<pre>
+<div class=function>_check_pair(self, noun, verb, sentence)</div>
+</pre>
+<pre>
+<div class=function>_crawl(self, sent, objs=None)</div>
+</pre>
+<pre>
+<div class=function>_create_arglink(self, event_token, argument)</div>
+</pre>
+<pre>
+<div class=function>_create_arglinks(self, sentence, alinks)</div>
+For each noun group in the sentence, get the head and find a event
+that it is an argument of.</pre>
+<pre>
+<div class=function>_extractLinks(self)</div>
+</pre>
+<pre>
+<div class=function>_find_by_lid(self, sent, lid)</div>
+</pre>
+</blockquote>

docs/code/modules/components.arglinker.wrapper.html

@@ -0,0 +1,48 @@
+<html>
+<head>
+<link href="../css/module.css" rel="stylesheet" type="text/css">
+<script language="JavaScript" type="text/JavaScript">
+<!--
+function view_code(id) {
+  var newurl = "../functions/" + id + ".html";
+  var w = window.open(newurl,"source code","width=770,height=600,scrollbars=yes,resizable=yes");
+  w.xopener = window;
+}
+//-->
+</script>
+</head>
+<body>
+<a href=../index.html>index</a>
+
+<div class="title">module components.arglinker.wrapper</div>
+
+<pre>
+<a href=#ArgLinkerWrapper>ArgLinkerWrapper</a>
+</pre>
+
+<pre>
+Contains the ArgLinker wrapper.</pre>
+
+
+<a name="ArgLinkerWrapper"/><div class="section">class ArgLinkerWrapper</div>
+<pre>
+
+Wrapper for ArgLinker.</pre>
+
+<blockquote>
+<h3>Public Functions</h3>
+<pre>
+<div class=function>__init__(self, tag, xmldoc, tarsqi_instance)</div>
+Calls __init__ of the base class and sets component_name, parser, CREATION_EXTENSION
+and RETRIEVAL_EXTENSION.</pre>
+<pre>
+<div class=function>create_arglinker_input(self, tmp1file)</div>
+Take the xmldoc variable and create the input needed for the arglinker.</pre>
+<pre>
+<div class=function>import_arglinker_output(self, tmp2file, outfile)</div>
+Takes a file with the results of the dependency parse (tmp2file), finds the arguments
+in there, adds them to the xmldoc, and saves the xmldoc (to outfile).</pre>
+<pre>
+<div class=function>process_fragments(self)</div>
+Apply the Slinket parser to each fragment. No arguments and no return value.</pre>
+</blockquote>

docs/code/modules/components.blinker.compare.html

@@ -0,0 +1,87 @@
+<html>
+<head>
+<link href="../css/module.css" rel="stylesheet" type="text/css">
+<script language="JavaScript" type="text/JavaScript">
+<!--
+function view_code(id) {
+  var newurl = "../functions/" + id + ".html";
+  var w = window.open(newurl,"source code","width=770,height=600,
+                      scrollbars=yes,resizable=yes");
+  w.xopener = window;
+}
+//-->
+</script>
+</head>
+<body>
+<a href=../index.html>index</a>
+
+<div class="title">module components.blinker.compare</div>
+
+
+<div class="section">module functions</div>
+<pre>
+<div class=function>compare_date(date1, date2, creation_year_int)</div>
+</pre>
+<pre>
+<div class=function>compare_day(day1, day2)</div>
+day1 and day2 are strings indicating day</pre>
+<pre>
+<div class=function>compare_in_year()</div>
+</pre>
+<pre>
+<div class=function>compare_month(month1, month2)</div>
+</pre>
+<pre>
+<div class=function>compare_month_week_at_year_boundary(month_year, week_year, month, week, month_day, week_day)</div>
+compare month to week, parameters are all integers</pre>
+<pre>
+<div class=function>compare_short_year(year1, year2)</div>
+year1 and year2 both are of string type</pre>
+<pre>
+<div class=function>compare_week(week1, week2)</div>
+week1 and week2 are strings containing 'W' indicating the type is week.
+Also suppose weeks do not contain 'X'</pre>
+<pre>
+<div class=function>compare_week_month(month1, month2)</div>
+</pre>
+<pre>
+<div class=function>compare_week_month_aux(week, month, week_day, month_day)</div>
+</pre>
+<pre>
+<div class=function>compare_week_month_aux_aux(week_int, month_int, week_day_int, month_day_int)</div>
+</pre>
+<pre>
+<div class=function>compare_year(year1, year2, creation_year_int)</div>
+year1 and year2 both are of string type.
+return: the link between two years</pre>
+<pre>
+<div class=function>get_current_year()</div>
+input: none
+output: integer</pre>
+<pre>
+<div class=function>need_compare_at_year_boundary(year1, year2, month1, month2, day1, day2)</div>
+</pre>
+<pre>
+<div class=function>reverse_link(link)</div>
+</pre>
+<pre>
+<div class=function>simple_compare(time1, time2)</div>
+used only in month section</pre>
+<pre>
+<div class=function>strip_week(week)</div>
+</pre>
+<pre>
+<div class=function>test_compare_date(test_string)</div>
+</pre>
+<pre>
+<div class=function>type_of_day(day)</div>
+</pre>
+<pre>
+<div class=function>type_of_month(month)</div>
+types include: "MONTH", "WEEK", "SEASON", "VAGUE"</pre>
+<pre>
+<div class=function>type_of_year(year)</div>
+</pre>
+<pre>
+<div class=function>year_alpha_to_digit(year, creation_year_int)</div>
+</pre>