CHANGES

Ganymede Release 2.0
29 October 2013
CHANGES

---------------------------------------------

-------------------- Changes from 1.0.12 to 2.0.0 -------------------

RELEASE DATE: 29 October 2013

1. [EVERYTHING] Ganymede Rebooted.. Subversion, Ant

Went through everything in the old Ganymede tree and did lots of
surgery, completely restructuring the source tree, moving files into
new packages.  Created sub packages arlut.csd.ganymede.server,
arlut.csd.ganymede.common, arlut.csd.ganymede.rmi,
arlut.csd.ganymede.admin, to make clear the different roles various
classes in the old arlut.csd.ganymede package play.

Deepak created a very nice Ant-based build system for Ganymede, and
restructured the source code into a package structure to make builds
with Ant convenient.

We imported the old Ganymede CVS repository into Subversion.  I went
through all of the source files and replaced the old CVS keywords with
Subversion keywords.

Subversion is wonderful.  Ganymede 2.0 would never have happened if we
were still stuck with CVS.

2. [SERVER] Localization

The Ganymede clients and server now incorporates a method for
localizing message strings.  A new class,
arlut.csd.Util.TranslationService, has been added which handles string
lookup and templatization from string resource files held in the
src/resources tree.

A considerable portion of the Ganymede server has been modified to use
this localization mechanism, as has the admin console, and the
entirety of the Ganymede graphical client.

The default, American English version of these localization files held
in the src/resources tree are simply called <classname>.properties.

When the Ganymede server is run under a JVM that is configured to use
a different locale, the TranslationService class will automatically
read any translated resources from the appropriate file, if it is
present in the appropriate directory.

For instance, if an adopter wanted to translate the messages used by
the arlut.csd.ganymede.server.GanymedeSession class into French, he
would create the file

src/resources/arlut/csd/ganymede/server/GanymedeSession_fr.properties

Ganymede uses the standard Java Internationalization method for
handling message strings, with interpolated parameters injected into
the message strings with tokens like {0}, {1}, and etc.

The new Ant build system provides a task to cross check the default
localization resources against the Ganymede source code, to ensure
that the localization message strings match the usage in the Ganymede
source code.  If you run 'ant validate' in the src directory, a Perl
script is run to do this analysis and verification.

3. [SERVER] Added support for SSHA password hashing in DBPasswordField

Here at ARL, we're working to move from NIS to LDAP for our Unix and
Mac OS X systems.  Since the OpenLDAP server doesn't support md5Crypt
and since traditional Unix crypt is just too inadequate in the face of
John the Ripper and other brute force dictionary attacks, I've
implemented support for encoding passwords entered into Ganymede using
the Netscape Salted SHA-1 hash algorithm.

See http://www.openldap.org/faq/data/cache/347.html for more details
on the SSHA hash format.

Nota bene: From what I've read, SSHA is actually a relatively cheap
hash algorithm.. it doesn't contain the arbitrary delay features that
something like md5Crypt uses to slow down the hashing algorithm.  As a
consequence of this, attempts to use brute force dictionary attacks
against SSHA hashes can be pretty successful.  The primary benefit of
SSHA over Crypt in the LDAP context is that longer passwords can be
supported (vs. the 8 character limit of traditional Unix Crypt).

4. [SERVER] Integrated Jython interpreter support in the server

XXX - to be written.  Discuss DBEditObject subclasses, tasks.

5. [SERVER, CLIENT] Java 5.0 fixes

Sun, long may their name be praised, decided in Java 5.0 to
arbitrarily make 'enum' a language keyword, despite never having put
it on the reserved keyword list.  We had to go through the entire
source code and replace 'enum' with 'en'.

In a number of other places, we had to explicitly cast some parameters
when making reflection calls to avoid confusion between an older
method call that has null as the final parameter and the newer Java
5.0 varargs versions of those methods.

We've made the ant build files for everything specify a source code
and class file target level of Java 1.4, so nothing in Ganymede 2.0
should require a Java 5.0 system.

6. [CLIENT] Cleaned up some threading

We're now using FoxTrot (http://foxtrot.sourceforge.net/) to allow
some activity in the client to be synchronous in ordering but
asynchronous in that we allow the GUI thread to continue redraw and
processing further GUI events until certain server calls complete.

This fire-and-wait support makes the GUI client feel much snappier in
a few places.

7. [SERVER] The return of passive password capture

Thanks to Deepak, we have now re-enabled the passive password capture
logic in the PasswordDBField class that was removed in 1.0.10.
Previously, the big showstopper with passive password capture was that
it was possibly to capture erroneous plaintext that nonetheless
matched a stored hash format, particularly the 8-character-significant
traditional UNIX 'crypt' hash function..

Now, passive password capture is back, but it is smart enough not to
capture plaintext on login if the password is not known from a hash
with adequate specifity.  The whole point of having passive password
capture is that it enables Ganymede to learn about plaintext from
interacting with a user, while still allowing us to forego saving
plaintext passwords persistently to our ganymede.db file.

8. [SERVER] Made transaction commit more resilient to exceptions in commitPhase1

The DBEditSet's transaction commit logic was vulnerable to getting
into a confused and broken state if an object's commitPhase1() method
threw an uncaught exception.  It's fixed now so that the client gets
an exception report and the transaction unwinds itself back to a
consistent state, ready to try committing again (though it will
probably fail if the error condition in the plugin classes is not
fixed).

9. [SERVER, CLIENT] Added SSL support, 1.4 requirement, Firewall friendliness

Added the arlut.csd.ganymede.server.GanymedeRMIManager class to handle
all RMI exports, with support for SSL encryption of client/server
communications through the
arlut.csd.ganymede.common.RMISSLClientSocketFactory and
arlut.csd.ganymede.common.RMISSLServerSocketFactory classes.

The Ant build process now incorporates automatic generation and
distribution of key and certificate material into the appropriate jar
files when the jar files are built.

The SSL support now present in Ganymede requires a minimum of Java 1.4
to work.  We are therefore intending to make JDK 1.4 a minimum
requirement of Ganymede 2.0.

It is now possible to specify the port number that the Ganymede server
should use for publishing its RMI object targets.  This means that the
Ganymede server will now use only two ports, one for the rmiregistry
and one for client access, and that these can be specified for
compatibility with a fixed firewall configuration.

10. [SERVER, CLIENT] Changes to Query system (Deepak)

Deepak implemented support for a text based (ANTLR-based) querying
language that can be used by Jython code to do queries on the Ganymede
server.  Deepak also added the new
arlut.csd.ganymede.common.QueryResultContainer class, which acts like
the old arlut.csd.ganymede.common.QueryResult class, but which
implements the java.util.List interface for convenient use from
Jython.  At this writing, this new QueryResultContainer isn't actually
being used, yet.

In specifying the ANTLR grammar, Deepak defined a new dereference
operator, which allows for a query to express a constraint based on
the value pointed to be an invid field.  By using this dereference
operation, I was able to drastically simplify and speed up the
GanymedeSession query logic by pulling some of it inside-out.

11. [SERVER, CLIENT] Removed the OwnerObjectsOwned field from OwnerBase

With this change, Owner Group objects no longer include an
InvidDBField tracking objects owned by that Owner Group.  Instead, it
is the responsibility of the owned object to point to its owner.

By making this relationship non-symmetrical, we gain the ability to
add or remove objects from an owner group during a transaction without
having to check out the owner group for exclusive editing.  As such,
this change will significantly improve concurrency in the Ganymede
system when many objects are owned by a single owner group.

This change was made possible by the (now quite old) change described
in 0.99.8, #17.

The 'Objects Owned' pane is still present in the client when viewing
or editing an Owner Group object, but this is now done using discrete
query logic rather than the old InvidDBField lookup, again for the
benefit of avoiding Owner Group locking.

12. [SERVER] Added a per-DBObjectBaseField lastChange variable

For the entire history of Ganymede, the GanymedeBuilderTask has
consulted a lastChange variable in each DBObjectBase to decide whether
any objects of a given type (user, group, system, etc.) has been
changed since that builder task last ran.  Our custom builder tasks
use this to avoid doing builds when the builder task is not interested
in what has changed since it last ran.

One problem with this has always been that the object-level timestamp
didn't really have as much granularity as we would desire.  Now, each
DBObjectBaseField has its own timestamp, so we can write builder tasks
that will, say, check to see if any user has had their username or
password changed since the builder task last ran.  Previously, all the
builder task could do was check to see if any user had been edited in
any way since it last ran.  This meant that editing the Notes field on
a user would trigger unnecessary activity in all builder tasks that
cared at all about user changes.

13. [CLIENT] Numerous fixes in the client's query table logic

Fixed a number of bugs in the client, including the bug that allowed
an object to be deleted in the client then viewed in an empty state
through the use of the query result table pop-up menu.  Now, all
checks against an object being already marked as deleted are done at
the proper place, rather than being done in the tree response code.

Also, I fixed bugzilla bug #14, which caused the title of query
windows not to be properly updated when a transaction is committed and
the query windows are refreshed, possibly with a different number of
entries (see 1.0.12, item 14, below).  While I was working on that, I
shuffled some code around in the client for better thread safety when
accessing some windowPanel data structures, as well as the use of a
common windowPanel.setWindowTitle() method for picking and maintaining
unique window names.

14. [ADMIN CONSOLE] Significant fixes to bugs in console login process

Improved the admin console login procedure.  The server now returns
an explicit dialog refusing admin console login if the user provides
an unprivileged or incorrect username and password.  The admin
console now responds properly to a failure to login or to the
Ganymede server being down.  The admin console will now also
repeatedly test the server to see whether it has come up, and can be
told to attempt a new connection if the server is believed to come
back up.

Much of the connection establishment threading has been improved in
design.  The use of the arlut.csd.Util.booleanSemaphore also helps
some, by allowing for proper exclusion of overlapping connection
threads.  The connection thread now properly uses the SwingUtilities
to handle proper issuance of gui updates in response to the
connection thread.

Good stuff, and long overdue.

15. [NIS PASSWORD CLIENT] Removed the rpcpass program

I've taken out the old rpcpass program which was used in conjunction
with the -x option to Thorsten Kukuk's NIS yppasswdd daemon for Linux.
The vaguely named rpcpass was used to propagate password, shell, and
finger info changes from the Linux yppasswdd daemon into Ganymede.

Unfortunately, the passwords that the yppasswdd is capable of
transmitting to Ganymede are in the old Unix crypt() format, which
means that passwords submitted in this way are useless if Ganymede
needs to sync to anything but NIS.

Therefore, and given that we at ARL have not used this code in nearly
five years, I removed it.  If you are reading this out there in
Internet land and decide that you've got to have it, let me know and
I'll see about resurrecting it from Subversion.

16. [XMLCLIENT] Massive speed-ups in the XML client, reversal of method calls

Previously, when the xmlclient was used to extract data from the
Ganymede server, the xmlclient would publish an RMI target and allow a
thread on the server to repeatedly call the client, pushing XML data
from the server.

This had the disadvantage that the xmlclient could not be run on a
system with a local firewall blocking incoming TCP connections.  With
SSL support now added to the Ganymede server, this reverse flow of RMI
method calls also meant that the xmlclient needed its own SSL
certificates and key material, or else XML sent from the server to the
client would be unprotected on the wire.

Now that the xmlclient pulls data down from the server with iterative
method calls, the XML is naturally protected with the existing SSL
connection.

In addition to this change of direction for the method calls, the
arlut.csd.Util.BigPipedInputStream class has been fixed so that it
actually will properly increase the size of the buffer used for
internal pipes within the Ganymede server.  In the case of the pipes
used to feed the xmlclient, this means that the server can now provide
64kbyte chunks from the XMLWriter, rather than the default 1k chunks
normally provided by PipedInputStream.  This has the effect of
reducing the number of RMI calls required to transmit XML to the
client by a factor of 64, resulting in a massive speed-up.

In testing here, this change made the new code 5 times faster at
wall-clock time in doing a system-local xmlclient dump to /dev/null,
even with the overhead of SSL encryption included, relative to
previous releases.

17. [SERVER] Use of external rmiregistry eliminated

The Ganymede server now implements an rmiregistry in-process, rather
than depending on an external rmiregistry support process.  This
simplifies the startup and shutdown of the Ganymede server, as well as
operational usage generally.

18. [XML] Improvements to namespace handling in <ganyschema>

In some testing we were doing, we got surprised at the case
sensitivity of namespaces that we were creating with the xmlclient.
The surprise was that by default, namespaces created are
case-insensitive.  To be fair, our documentation said this, but it was
just too easy to forget this when crafting xml test data.  To address
this, we have made the system warn if the case-sensitive attribute is
not present in <namespace>.

Also, the Ganymede server now properly handles switching a namespace
from case-sensitive to insensitive, or vice-vera, as needed, and
checks to make sure that this redefinition operation can be done
safely on the namespace-constrained data loaded in the server.  The
admin console's schema editor still does not allow changing the case
sensitivity of a namespace, but the xmlclient may be used to do this.

19. [SERVER] Fixed a bug that made newly defined object types inaccessible

Oops!  We weren't properly updating the baseTransport cache after
schema editing.  This resulted in a failure whenever we tried to use
the Ganymede client to access an object of a type that did not exist
before the schema edit immediately after the schema edit took place.

The only reason this didn't bite us before was that we haven't added a
lot of object types through the schema editor without subsequently
shutting down and restarting the server.

Gotta love really exercising old code paths in relatively new ways.

20. [CLIENT,SERVER,userKit] Scalability work

Lots of work on increasing the scalability of the Ganymede server and
client.  At the time of this writing, we've tested the userKit to a
loading of 250,000 users and 250,000 groups, a total of some 7.5
million data fields, with a nominal (fully garbage-collected rest
state) memory usage of around 600 megabytes on the server.  In order
to get there, we had to fix some poor algorithmic choices in certain
points in the Ganymede server.  This included a complete cut over from
using the old cheapo QuickSort implementation I slapped together in a
day from someone's online course notes to using the java.util.Arrays
sort algorithm that came in Java 1.2.

I also fixed a bug in the userKit userCustom.java class that made it
do about a bazillion times more work than it should have when doing
processing, thanks to a failure to set and retain the schemaConfStamp
time stamp variable properly.

The Ganymede transaction logging system is now very much more
efficient as well.  Previously, all events that needed to be logged in
conjunction with a transaction would be assembled into a Vector of
DBLogEvent objects before any events were logged to disk.  Now, the
transaction commit logic streams these log events to the logging
system as they are generated, so that garbage collection can occur
during the logging process.  This should help reduce memory loading on
very large transactions.

Made some improvements in the XML import logic to intern commonly
occuring strings, again to reduce memory usage.

21. [CLIENT,SERVER] Invid redefined, private constructors, InvidPool

The Ganymede object id pointer class, arlut.csd.ganymede.common.Invid,
has been modified slightly to make its constructors private, in order
to force code wanting to create Invids to go through the Invid class'
static factory method, createInvid().  The purpose of this is to allow
for intelligent re-use of Invid objects on the server, so that the
server will only need to keep one copy of any given Invid.  This has
an effect similar to the interning that the Ganymede server has always
done when loading strings from its database, and should help lower the
server's memory usage a bit on large datasets.

In addition, an InvidPool has been implemented on the server, which
uses a hashing SoftReference data structure to efficiently implement
Invid interning.  This seems to cut memory loading on the server with
large datasets by 15% or more.

22. [CLIENT] Reworked the client layout/rendering algorithm

The layout/rendering algorithm for the client object windows and the
admin console schema editor has been rewritten to behave better in a
greater variety of circumstances.  In particular, bugs that were seen
when using the multi-line label field have been addressed.

23. [SERVER,ADMIN CONSOLE,CLIENT] New data field type added for Sync Channel

Introducing the first new DBField type since the introduction of Float
back in 1999.. the FieldOptionDBField!

The purpose of the FieldOptionDBField is to allow the client and/or
xmlclient to configure the list of objects and fields that the new
Sync Channel synchronization channels (to be introduced hereafter)
will use when writing out an XML description of transactions to disk
as they happen.

The intention is that the Sync Channel will write out an XML
description in a known format for each transaction that is committed,
and a dedicated queue runner will be launched on an asynchronous basis
to sweep up the XML records for the queue, up to a fixed transaction
number, for integration into directory services oriented around change
requests rather than complete dump and reloads.

24. [SERVER] Made DBEditObject.deleting variable private

I've made the deleting boolean in DBEditObject private rather than
protected, so that use of it can be controlled better.  If your custom
plugins consult deleting directly, they will need to be tweaked to use
"isDeleting()" rather than "deleting".

25. [SERVER, XML] Corrected xmlclient behavior for namespace swaps

The xmlclient is intended to allow just about any legal change to the
Ganymede server's data to be made by submitting a declarative xml
file.  Unfortunately, it turned out that one thing the Ganymede server
could not do when handed an xml file was to swap two namespace values
around.

That is, suppose you wanted to rename two user accounts
simultaneously, giving account A the name previously associated with
account B, and account B the name previously associated with account
A.  If you try to do this in the GUI client, you'll find that you need
to make up some transient name to act as a placeholder for account A,
then give A's name to B, then go back and give B's name to A, erasing
the placeholder in the A account.

This kind of fine grained control over the order of operations isn't
possible when using the xmlclient, so I have tweaked the server's
namespace management logic so that it will allow non-interactive
sessions to have an arbitrary number of namespace linkages 'in flight'
at a time.  Giving name 'A' to account B will be allowed, even though
it means that for a brief window, both account A and the old account B
will be associated with the 'A' name.  If the name 'A' is not unlinked
from the old account A by the time the xmlclient's transaction
commits, however, an exception will be reported and the transaction
will be canceled.

This logic is even capable of resolving object renaming cycles, such
as would occur in an XML file that required

  System A to be renamed as System B
  System B to be renamed as System C
  System C to be renamed as System A

26. [SERVER, XML, EVERYTHING] Object labels no longer virtualizable, must be namespace constrained

The Ganymede server has never previously actually enforced a
requirement that all objects must have unique labels.  I thought this
very clever in my initial design, allowing DBEditObject plugin classes
to completely fabricate labels for objects on demand.

In fact, this was a very stupid idea, as it makes all sorts of things
much more complicated, including the XML dump and load logic.  Not
being able to know for sure that an object's name is unique in an xml
dump means that the xml file format is basically kind-of-close to
useful, but not actually trustworthy.

In Ganymede 2.0, all object types registed in the Ganymede server must
have honest to goodness label fields, guarded with a namespace
constraint to ensure unique labels within the object type.  It is
still permissible to allow multiple kinds of objects to have their
label fields share a namespace if you so desire.

Object types in the base Ganymede system that did not meet this
criterion, such as the Object Event type, have been modified to
include a hidden label field that is automatically maintained by the
new preCommitHook() method added to the DBEditObject class.

This change may cause some difficulty for people trying to transition
a Ganymede 1.0 environment to 2.0, but I've put in some logic to try
and help you out.  When you load an old 1.0 database that has any
object types without namespace constrained label fields, the Ganymede
server will print a warning on startup.  When you go to edit the
schema, either with the GUI schema editor or the xmlclient, your
schema change will refuse to commit unless you make sure that all
objects meet this constraint.

As well, any built-in Ganymede types that are affected by this change
will automatically be tweaked for you by the system the first time you
start the server on top of an old ganymede.db file.

This change bumped the DBStore version number to 2.11.

27. [SERVER] Clarified that Vector fields may not hold duplicates

Previously, certain parts of Ganymede implicitly assumed that
duplicate values were not allowed in vector fields, but no code made
sure that duplicates were not introduced into vector fields.  I have
now introduced code to test and enforce that constraint.

This change may slow down bulk loading, but it will assure that we
have a consistent semantic for vector fields, and other code in the
Ganymede server can safely rely on the uniqueness of vector field
elements.

28. [SERVER] Added GanyPermissionException to server

Several classes in the Ganymede server now throw
arlut.csd.ganymede.common.GanyPermissionException when permission
problems are encountered.  This change was to help insure that
Ganymede server internal code was properly tracking permissions
failures.

As a result of this change, custom schema kit code written to the
Ganymede 1.0 series may need to be altered to catch the
GanyPermissionException, as it is a declared exception that derives
from RemoteException for compatibility with the client.

In some cases, methods that previously returned ReturnVal to indicate
permissions failures may now throw GanyPermissionException.

29. [SERVER] Added entirely new Sync Channel build systems

With Ganymede 2.0, the Ganymede server now has an entirely new family
of synchronization services.

Previously, all builds were done through the use of the
GanymedeBuilderTask and its subclasses, using a two phase process.  In
phase one, portions of Ganymede's internal datastore were locked
against changes while the builder task scanned the database and wrote
data (usually text files) out to a specified directory.  In phase two,
the database was unlocked, and the builder task ran an external
synchronization process that used the data files written out by phase
1.  When the external build system finished, the task's execution was
complete, and a new build could be scheduled and issued for that same
task.

All of that is still there, and it all works the same way that it ever
did.  Now, though, there is a second system, based on the SyncRunner
and syncChannelCustom classes and the use of an XML file format.  I
call it the 'Sync Channel' system.

The Sync Channel system comes in three flavors.. "Automatic Full
State", "Automatic Incremental", and "Manual".

The Automatic Incremental Sync Channel system is the most different
from GanymedeBuilderTask.

It is based on tracking the changes made to the Ganymede data store.
When a change is made, the Ganymede transaction commit logic examines
all Incremental Sync Channels registered in the server, looking to see
if any of them are interested in the objects and fields that have
changed.  If any Incremental Sync Channel is so interested, that Sync
Channel will write out an XML description of the transaction with
respect to the objects and fields that the Sync Channel cares about.

Each standardized transaction description is written out to a
directory devoted to the Incremental Sync Channel, in a file whose
name is the transaction's id number.  The Ganymede server then
triggers the external service program specified for each Incremental
Sync Channel, passing it the number of the highest transaction known
to have been successfully written out at the time that the service
program is run.

The service program is intended to read all of the transactions up to
and including the number given it, and to integrate those changes into
whatever directory service the Sync Channel is intended to support.

There's much more to read about this major new feature, including
details about the "Automatic Full State" and "Manual" Sync Channels,
in doc/synchronization/index.html, or you can go on the web to
http://tools.arlut.utexas.edu/svn/ganymede/trunk/ganymede/doc/synchronization/index.html
for the details.

30. [SERVER] Forced java.rmi.server.randomIDs on by default

In conjunction with the adoption of SSL in Ganymede 2.0, the
definition of the property statement 'java.rmi.server.randomIDs=true'
is necessary to avoid direct JRMP wire-level access to RMI objects
intended to be published by Ganymede for the benefit of a single
client.

With this property defined in the runServer script, Ganymede RMI
objects will be given cryptographically secure 64 bit random object id
numbers.  Anyone wishing to hijack an RMI-published object would have
to have some way of guessing a 64 bit random number in order to send a
JRMP message to the Ganymede server forcing an unauthenticated
operation.

In theory turning off java.rmi.server.randomIDs should make the
Ganymede server startup marginally faster, but any modern system that
provides high entropy random numbers through a /dev/random device
shouldn't see any delay to speak of anyway, and the Ganymede server is
much, much, much more secure with both SSL and randomIDs turned on
than it is with just one or the other of them turned on.

31. [XMLCLIENT] Added sync argument to allow for constrained dumps

Now that Ganymede 2.0 allows you to define object and field
constraints on sync channels, the xmlclient has been elaborated to be
able to apply a sync channel constraint to a dump.  This makes it
possible to dump out a tailored XML report that only contains the
object and field types that a given synchronization target might be
interested in.

The ultimate purpose of this is to allow an administrator to pull the
full state of Ganymede with respect to certain objects and fields, for
the purpose of doing initial setup of an external directory service
target, before setting up a sync channel to transmits deltas as they
are committed to Ganymede.

Use is simple, just type

        xmlclient -dumpdata sync=Systems > systems.xml

and Ganymede will dump out all data that fits the Systems Sync Channel
constraint.

32. [SERVER] Ripped out the old schema printing logic

The Ganymede server, from way back when, included logic for printing
its schema definitions in a readable form to HTML or plain text.  All
of that code really became redundant when the xmlclient was
introduced, so I have gone through and removed it.  The Ganymede
server will no longer support the ganymede.htmldump property, and will
no longer support dumping the schema to an HTML page on server
startup.

If anyone is interested in generating a schema web page from the
xmlclient -dumpschema output, that should just be an XLST script away.

Getting rid of that old code helps reduce the maintenance burden on
Ganymede, as I don't have to change schema printing logic in three
places anymore.

33. [SERVER, XML] Miscellaneous fixes to XML input processing

The Ganymede server's XML input processing is now far more robust in
terms of dealing with invid label resolution, regardless of the order
in which objects are presented in the XML input, and especially in the
case where objects are renamed as the XML file is processed.

The status report at the end of an xmlclient data batch submission has
been fixed.

Fixed the bug that led to sshatext being ignored on XML input.

Fixed the bug that caused leading or trailing white space to be
ignored in scalar string elements.

34. [SERVER, CLIENT]  The client now catches and displays exceptions reliably

The Ganymede client now catches and displays exception conditions to
the user when they happen, rather than just dumping them out the back
door to STDERR.  When an unhandled exception occurs, the Ganymede
client will display a dialog showing the exception.  The dialog
includes a button to allow the user to report the exception to the
Ganymede server for logging.

35. [SERVER] Improved various error messages from Invid fields

Lots of clean ups to make the error messages that occur while linking
and unlinking objects more readable.

36. [SERVER, CLIENT] The server reports bad regular expression queries to the client

The server will now send a dialog to the client explaining about a
malformed regular expression query, rather than silently failing to
return any data.

37. [SERVER] Honor DBEditObject.canClone()

The canClone() method in DBEditObject wasn't being checked by
GanymedeSession, which meant that users could clone objects that the
schema kit author intended not to be cloneable.

This bug was reported by Fredrik.A.Bergman@ericsson.com.

38. [SERVER] Fixed non-namespace constrained IPDBField fields.

IP fields that are not namespace constrained no longer throw a rod
when you try to change them.

This bug was reported by Fredrik.A.Bergman@ericsson.com.

39. [SERVER] Properly handle regexp description in XML input

Made sure to read/set the regular expression description attribute
when processing xml input.

This bug was reported by Fredrik.A.Bergman@ericsson.com.

40. [SERVER] Properly clear qr cache in InvidDBField

Made sure to clear the qr cache in a few more places so as to not
allow lingering choice lists to be retained.

This bug was reported by Fredrik.A.Bergman@ericsson.com.

41. [SERVER] String field schema consistency checks now in GanymedeValidationTask

I've added support in the DBField class to allow a consistency check
to be made on values held in a field against the static constraints
specified in the Ganymede schema.  This check is integrated into the
old verifyNewValue() call for StringDBField, and the
GanymedeValidationTask will run the field contents integrity test when
you do the database consistency test from the admin console.

Most constraints on fields are still implemented by dynamic method
hooks in the DBEditObject class, and those constraints can't really be
tested out except at commit time, when the editing context is
available.

This change came about because we found that a system record in the
laboratory's Ganymede server had a colon character in its Manufacturer
field, despite our having modified the schema to prevent that some
time ago.

Note that if you apply a new constraint to a string field, there is no
guarantee that previously existing data will match the new constraint
until you run the Database Consistency Test / GanymedeValidationTask
to be sure.

42. [ADMIN CONSOLE] Split sync tasks into separate task table

The admin console now includes a second task monitor table, devoted to
sync tasks (i.e., schedulable tasks derived from
arlut.csd.ganymede.server.GanymedeBuilderTask or
arlut.csd.ganymede.server.SyncRunner).

Adding a third tab for this in the admin console makes it easier to
track builds and what-not.

43. [SERVER, XMLCLIENT] Added logic for handling space characters on the command line

Previously, the Ganymede xmlclient and server, both of which require
the ability to process command line arguments, did not properly handle
command line arguments containing whitespace.

This was due to a major design flaw in how the Java runtime
environment handles command line arguments.. in Java, rather than
letting the shell break the command line into an argument list, the
runtime is responsible for doing this, and it always breaks on
whitespace, regardless of attempts at quoting or escaping arguments.

This made it impossible to use xmlclient to do sync channel dumps with
sync channels whose names contained whitespace.

Now, the Ganymede command line argument parser recognizes HTTP-style
space character encoding.. any "%20" sequence encountered on the
command line will be treated as a space character instead.

The xmlclient wrapper script is now written in Perl, and converts
space characters in arguments to %20 for passing into the xmlclient
Java executable.

44. [SERVER, XMLCLIENT] Added support for full-fidelity dumping and loading

The xmlclient now has support for an -includeHistory command line
argument, which causes the Ganymede server to include the historical
fields (creation time, creator info, modification time, modification
info) when dumping out Ganymede objects.

The xmlclient now has support for an -includeOid command line
argument, which causes the Ganymede server to include the precise
invid for dumped objects in the <object> element's oid attribute.

The Ganymede server now has support for a -magic_import command line
argument, which causes the Ganymede server to support loading the
historical fields from an xml file, as well as using the oid attribute
to create objects with a predetermined invid number.

Between -includeHistory, -includeOid, and -magic_import, it's now
possible to dump a server's complete state to an XML file, modify the
XML dump by hand to alter the schema (for instance) and then load it
into a new server, with all invids and historical fields matching the
original server's state.

45. [EVERYTHING] Added support for server-defined custom tabs in the client

This change introduces tabs to the Ganymede schema model.  Adopters
can now place user-defined fields into user-defined tabs, and the
client will display the appropriate fields in separate fields,
accordingly.

The goal here is to make it possible to add more data fields to an
object type without overwhelming the user.

46. [CLIENT, ADMIN CONSOLE] No More Redundant Login Box When Running As Application

The Ganymede client and admin console are both designed to be able to
function either as an application or as an applet, depending on the
manner in which they are launched.

When they are run as applets within a web browser's window, the applet
must remain open in order to avoid terminating the applet.

Things are different when the client and console are started as
applications, however.  In this case, the login box that is displayed
when the program is started is fully under control of the application.

With this change, the Ganymede client and admin console, when run as
applications, will now close their initial login windows after the
user has logged into the client or console.

This eliminates the redundant/vestigial login window that used to hang
around uselessly after logging in to the client and console.

When running as an applet (using the 'native' or 'plugin' mode from
the Ganymede web launcher), the applet window stays up still, of
course.

47. [CLIENT, ADMIN CONSOLE] Application Mode with Java Web Launch

In order to take advantage of the previous change, I've changed the
JNLP launch files for the Ganymede client and admin console so that
they are run as applications rather than as applets.

I had already moved in this direction by signing the client and
console jar files with the SSL key material.  Going ahead and making
the client and console run as applications gives me control over
disposing the login window, allows me to manage threads a bit better,
and lets the client/console connect to whatever machine hosts the
Ganymede server, even if different from the web server from which the
client or console is downloaded.

This will help when it comes time to support fast failover of the
Ganymede server.

48. [XMLCLIENT, SERVER] Added support for running queries with xmlclient

The xmlclient now supports queries using the GanyQL query language
that Deepak developed.

See doc/xml/xmlclient.html and doc/querylanguage/index.html for full
details.

49. [SERVER] Fixed invid vector cloning

Previously, if you cloned an object that contained an Invid vector,
any failure in the vector cloning would cause no items at all to be
cloned, and the cloned object would have that field empty.

Now, the InvidDBField.addElements() method is cognizant of the
partialSuccess intention for object cloning, and will clone partial
objects.

50. [SERVER, CLIENT] Miscellaneous fixes to client operations

I fixed up the logic for the 'default owner' dialog in the client, so
that you can close it without making a selection to no ill effect.  It
will also remember your previous owner group settings, so that you can
revisit the dialog and see what defaults you have in effect.

I tweaked the server and client so that the client is informed when an
object being edited has its label field changed.  All objects that
point to the edited object through an invid link will be refreshed in
the client to reflect the object's new name.

This is even true for view-only object windows.  The logic here is
that changing the label for a linked object doesn't truly change the
object being viewed, so having the link label reflect the new name is
actually no change at all.  It is a bit at odds with the fact that an
object view window is a flash-frozen view of an object, though.

Newly created objects are now given default object labels on the
server until such time as their label field is set.  The confusion
about these temporary labels is what led to making the client react
visibly to object renaming.

51. [CLIENT, JDATACOMPONENT] Fixed listHandle

One of the primary data structures used in the client, the
arlut.csd.JDataComponent.listHandle, turned out to be not fully
defined.  It was lacking a hashCode() method, and certain revisions I
made in #50, above, needed it to be reliably hashable based on content
rather than object identity.

By fixing this, it ensures that the optimized VectorUtils class
methods can properly manipulate listHandle vectors.

52. [SERVER, CLIENT, JDATACOMPONENT] Added support for server-side reformatting of input

The Ganymede system now has the ability to respond to string input
from the client by approving a canonicalized version of the input.

Previously, the server could accept a given string, reject it
outright, or engage the user in a wizard sequence.

Now, the verifyNewValue() method in the DBEditObject subclass can use
the new setTranformedValueObject() method in the ReturnVal to signify
to the Ganymede server and client that the submitted value was
accepted but that it was transformed in some fashion and that the
client should refresh the GUI widget.

We use this with the GASHARL schema kit to auto-format MAC address
entry in the Embedded System Interface object's "Ethernet Info"
field.. if the user enters "AB-CD-EF-11-22-3", the server will
automatically reformat this string to "ab:cd:ef:11:22:03", in keeping
with our policy, and the user will get immediate visual feedback that
the data was massaged by the server.

Good stuff, although it doesn't make the code on the client and server
any simpler. ;-/

53. [CLIENT, ADMIN] Use of Java 1.4 Preferences API to remember window position, etc.

The Ganymede client and admin console now use the Java 1.4 Preferences
API to remember window positions and various default selections from
invocation to invocation on a given client system.

54. [CLIENT, ADMIN] Added support for the scroll wheel

The Ganymede client and admin console now properly support the scroll
wheel in the custom tree and table components.

55. [SERVER] Fixed a bug relating to limited privilege users in the incremental sync channel logic

Came across an interesting bug when dealing with a user who was set up
with extremely limited privileges in Ganymede.  The emitXMLDelta()
method was assuming that all fields defined on an object would be
present (not null), and would fail if an administrator with limited
privileges edited an object (explicitly or implicitly) in which he did
not have permission to edit or see all fields.

The bug caused a NullPointerException to be thrown, and a transaction
involving this scenario could not successfully proceed to commit.

56. [SERVER] Liberalized anonymous linking behavior for non-privileged admins

The DBEditObject class is meant to allow normal permissions to be
overridden to allow linking to a symmetric field, even if the admin or
user in question did not have permission to edit the target object
and/or field.

Unfortunately, this logic had a loop hole that caused anonymous object
linking to break if the admin or user did not have permission to
create the target field and the object, as persisted in the object
base, did not contain an defined instance of the target field.

This has been fixed by a change to the InvidDBField.bind() method.

57. [SERVER] Various cleanups to avoid proliferation of switch statements on field type

I created a handful of static factory methods on the DBField class,
and modified a lot of classes to use these factory methods, rather
than to repetitively have their own switch statements on the set of
field types defined.  Now, adding a new DBField subtype is (almost!)
as easy as editing a few static factory methods at the top of DBField,
rather than having to edit a scattered collection of internal Ganymede
code modules.

In fact, in doing this work, I fixed a bug that made the DBJournal
improperly fail to process FieldOptionDBField data in the journal if
the server was abnormally terminated without consolidating its
database.  This is precisely the sort of bug that this change is
intended to help prevent in the future.

58. [CLIENT] Misc fixes.. case insensitive sort, window size, motd window

The client now uses a case insensitive sort throughout.  The default
window size for object view windows is now big enough not to require
a horizontal scrollbar.

The 'Message of the day' window now appears with the top of the
message visible in the dialog, rather than the bottom.

59. [SERVER, CLIENT] Added support for user photos

By redefining the getImageURLForObject() method in a custom user
DBEditObject subclass, it is now possible to display user photographs
in the client.

The getImageURLForObject() method is designed to return a URL to a
photograph for the user located on a web server accessible by the
Ganymede client.

There is no support for user photographs in the Ganymede server,
itself.

60. [SERVER] Fixed nested monitor deadlock in GanymedeScheduler, scheduleHandle

The GanymedeScheduler and scheduleHandle classes had a big opportunity
for a nested monitor deadlock.  We haven't previously hit the deadlock
in the many years we've been running Ganymede as far as I know, but
there it is regardless.

scheduleHandle now drops all synchronization before calling
notifyCompletion() on the GanymedeScheduler.

This fixes a bug that caused things to seize up a bit if a user sent a
disableTask command to a task while the task was running in the
scheduler.

61. [SERVER] Added Event Reporting When External Build Processes Fail

I've added an 'externalerror' event to the System Events predefined in
the Ganymede Server.  An 'externalerror' event is generated (and,
optionally, email is sent, according to the definition of the
externalerror System Event object in the Ganymede datastore) when the
Ganymede server receives an non-zero result code upon executing an
external process to service a Sync Channel.

I've added similar code to the GASHBuilderTask and the IRISBuilderTask
classes defined in the gasharl schema kit, but the launching of
external builder scripts in response to builder task build phase 2
execution is not handled in the GanymedeBuilderTask base class, so
I've not added 'externalerror' support for all conceivable Builder
Tasks.

See the runFullStateService() and runIncremental() methods in the
src/ganymede/arlut/csd/ganymede/server/SyncRunner to see how to add
event logging for errors encountered when running external build
scripts.

62. [XMLCLIENT] The xmlclient will no longer leave itself logged in on query exceptions

The xmlclient had an error, in which any exception thrown on the
server during query processing would cause the xmlclient to terminate
without properly logging out.

Now, if a GanyQL query submitted to xmlclient causes an exception on
the server (a parse exception, say), the xmlclient will report the
exception (once!) before doing a clean logout from the server.

63. [XMLCLIENT] The xmlclient no longer fails on object deletion

Brian Worden and his team at fg-networking.de provided a patch to fix
a NullPointerException bug that occurred when deleting objects using
the xmlclient.

64. [XMLCLIENT] The xmlclient can now import files dumped by Ganymede

Another bug reported and fixed by Brian Worden and the guys at
fg-networking.de.  Ganymede was using different minor version numbers
for its dump format between the code that generated XML dumps and the
code that handled reading those dumps.

65. [EVERYTHING] German translation added

Stefan Bier (bier@fg-networking.de) translated all of the Ganymede
message strings into German.  Users running the Ganymede client on a
system with the default locale set to German will see translated
messages in the GUI.  Messages generated by the Ganymede server will
only be translated if the system running the server is also using a
German language locale.

If you like, you can force any of the Ganymede programs to use German
by including -Duser.language=de in the Java invocation line in the
startup script, as follows:

$JAVA $DEBUG -Xms$MINHEAP -Xmx$MAXHEAP -Duser.language=de arlut.csd.ganymede.client.glogin properties=$PROPERTIES

Thanks Stefan!

66. [SERVER] Added Support for Custom Inactivation and Removal Notifications

I've modified the DBEditObject class to define a pair of new hooks,
reactToExpirationWarning() and reactToRemovalWarning(), and modified
the GanymedeWarningTask to take advantage of them.

These new DBEditObject hooks are designed to allow custom DBEditObject
subclasses to do custom notification when a given object needs to have
inactivation or removal warnings sent out, as determined by the basic
logic in the GanymedeWarningTask.

67. [SERVER] Fixed dumpAndArchiveTask so it won't throw an exception if it is scheduled while the schema is being edited

The dumpAndArchiveTask consolidates the Ganymede journal into the
ganymede.db file on a periodic basis.  It had a bug that would result
in a semaphore exception if it was scheduled during schema editing.

68. [SCHEMA EDITOR] Fixed the right-click menu on embedded object types.

The schema editor was popping up the wrong right-click menu on
embedded object types in the tree control.  This bug made it
impossible to create new fields in embedded objects other than during
the same schema editing session in which the embedded object type was
created.

69. [SERVER] Improved error handling reporting for XML Sync Channels

Improved the transactional Sync Channel processing so that the
non-existence (or non-writability) of the registered queue directory
results in a specific error message explaining that, rather than
speculation that the disk might be full.

70. [XMLCLIENT] Improved error handling reporting for xml data uploads.

The xmlclient (actually the code in the server which interacts with
the xmlclient) has had its error handling improved.  Certain classes
of document errors are now handled more gracefully, with more explicit
error messages.  Also fixed the issue that led to the xmlclient
stalling if the XML document was not properly terminated.

71. [SERVER] Added lookupInvid and lookupInvidLabel methods to DBObject

These new methods are designed to provide a fast, convenient way for
creators of DBEditObject subclasses to dereference Invids to the
objects and labels which those Invids resolve to in the proper
transactional context.

Also added getParentInvid() to DBObject, which is applicable to
embedded objects.  It returns the Invid of the object which contains
the embedded object.

Also added isDefined() methods to DBObject. which can be used to test
to see if a named or numbered field is present and non-empty.

These new methods should make a lot of typical DBEditObject subclass
logic a bit easier to implement.

72. [CLIENT] Added XML support capability to the GUI client

The Ganymede GUI client now has the ability to transmit and receive
Ganymede XML data files to and from the server, in addition to the
command line xmlclient program.

There is now a 'Submit XML Data' menu item under the File menu that
supports transmitting XML to the server, and it is now possible to
save or email query results from the query table in 'Ganymede XML
Format'.

With these two features, it is now possible to compose a query with
the GUI query builder, get the results, save them in XML format, do
some quick editing with the text editor of your choice, and then
transmit the changes to the server, all without leaving the Ganymede
GUI client.

Note that you can't change the schema with the Ganymede GUI client,
because the schema can't be changed when a user is logged in.  If you
want to change the schema, you'll have to be sure all users are logged
out and use the standalone xmlclient.

73. [CLIENT] Fixed some threading issues that appeared in Iced Tea OpenJDK 7

The Ganymede client had some threading issues that appeared in the
OpenJDK 7 packages that Red Hat ships with Fedora 8.  I've slightly
reworked a bit of the threading to try and avoid the issues.

74. [CLIENT] No longer prompt for persona if the user selected one in the login box

The Ganymede client will no longer pop up the persona select dialog if
the user specified a persona at login time, by logging in as
'username:persona name'.

This is a nice little optimization that streamlines the login process
a bit.

75. [SERVER] Added support for Sha Unix Crypt password hashing in DBPasswordField

We have added support for the new SHA256 and SHA512-based 'Unix Crypt'
hash algorithms introduced by Red Hat, Sun, IBM, and HP.  A discussion
and link to the specification for this algorithm can be found at

http://people.redhat.com/drepper/sha-crypt.html

These new hash algorithms are intended for widespread adoption in a
way that OpenBSD's bcrypt hash algorithm never achieved, while
supporting scalable computational requirements to 'future-proof' the
hash algorithm.

76. [SERVER] Removed user login and logout events from user history panel

Previously, when you viewed the history of changes to a user object,
you'd see a ton of login/logout events if that user happened to be a
Ganymede administrator, or even an end-user change his password
through Ganymede.

We're now filtering out this event spam from the retrieveHistory
method on the DBLog, so that it will be easier to see actual changes
committed against a user object.

77. [SERVER] Rewrote logscan.pl external log scan accelerator

The external logscan.pl log scanning accelerator has been re-written
to perform efficient date-bounded searches through the log file.  Our
own log file is now up over 80 megabytes, which is starting to get a
bit long to grep through, which is all that logscan.pl really did
before.

The logscan.pl script is now much smarter, and takes lower and upper
time bounds on the command line.  If a lower time bound is provided,
the logscan.pl script will actually perform a binary search through
the Ganymede log file, seeking to find the first record that occurs
after that point in time.

The Ganymede DBLogFileController class has been amended slightly to
interact properly with the new, smarter logscan.pl.  If you're
upgrading to a version of the Ganymede server more recent than
Subversion revision number 7820, you'll want to be sure to update the
logscan.pl script.

The Ganymede server and the logscan.pl script are designed such that a
newer Ganymede server will continue to work properly with an older
logscan.pl, but moving to the newer logscan.pl will improve log
scanning efficiencies significantly.

78. [CLIENT] Cloning viewable objects now allowed in the graphical client

The Ganymede graphical client previously did not allow the user to
clone objects unless they had the right to edit those objects.  This
was an unnecessary restriction that the client was imposing, and has
been removed.

Now, Ganymede will allow cloning any object so long as the user has
the right to view the object to be cloned, and the right to create new
objects of that type.

79. [EVERYTHING] ReturnVal amended for ease of use

The ReturnVal class has been altered to provide greater programmatic
ease of use, and to simplify internal logic in the Ganymede server.

Most adopters of Ganymede will not need to concern themselves about
the changes made to ReturnVal, save for the following:

Most setter methods on ReturnVal now return 'this'
rather than void, allowing Smalltalk-style method chaining..

e.g., you can now do things like

  ReturnVal retVal = new ReturnVal(false);
  return retVal.setErrorText("Oops, we failed").addRescanField(objId, field.getID());

when you want to modify and return a ReturnVal object, rather than

  ReturnVal retVal = new ReturnVal(false);
  retVal.setErrorText("Oops, we failed");
  retVal.addRescanField(objId, field.getID());
  return retVal;

Because the setter methods on ReturnVal now return a ReturnVal object,
rather than void, plug-in code that was previously compiled against
the Ganymede libraries will need to be recompiled in order to be
compatible with this change.  Otherwise, a 'NoSuchMethodException' may
be thrown at run time when plug-in code is called which was compiled
with versions of Ganymede prioer to Subversion repository version
7845.

For convenience's sake, the ReturnVal class has four new static
methods which perform the proper null check as part of their
operation, reducing the necessity for the constant

  if (retVal == null || retVal.didSucceed())

idiom.  Now, one can say

  if (ReturnVal.didSucceed(retVal))

to achieve the same effect.

The four new static methods on ReturnVal are

  static public boolean didSucceed(ReturnVal retVal);
  static public boolean hasTransformedValue(ReturnVal retVal);
  static public boolean isDoNormalProcessing(ReturnVal retVal);
  static public boolean wizardHandled(ReturnVal retVal);

The biggest change to ReturnVal which assists in the internal logic of
the Ganymede server is the addition of the static ReturnVal.merge()
method, which can be used to intelligently combine the results from
multiple ReturnVal-returning operations, preserving and/or combining
the important bits of each as processing continues.  The Ganymede
server has been converted to use this throughout, and benefits from
simplified and more regular processing logic in the DBField subclasses.

Note that in conjunction with this new merge logic, the single
parameter ReturnVal constructor behaves a bit differently than it did
previously.

Before, a ReturnVal object created by calling

  result = new ReturnVal(true);

would create a ReturnVal for which result.success is true, and for
which the doNormalProcessing propery is _false_.

Now, either of the following

 result = new ReturnVal(true);

-

 result = ReturnVal.success();

result in a ReturnVal that has both success and doNormalProcessing set
to true, and

 result = new ReturnVal(false);

results in a ReturnVal that has success and doNormalProcessing set to
false.

This change will generally only affect you if you have written custom
wizard code, but as the semantics of the single parameter ReturnVal
constructor are different now, I recommend taking a look at your code
to see whether something was previously depending on the
doNormalProcessing flag being set to false on a 'new ReturnVal(true)'.

80. [CLIENT] GUI improvements in the StringSelector object

Gil Kloepfer asked if the StringSelector object (the GUI component
used in the client for dealing with lists of Strings or Invids) could
be made to take up less space when the field connected to it had no
data in it.

That is, rather than taking up the equivalent of 12 lines of text in
height, Gil wanted an empty String or Invid vector field to only take
up a few lines of text worth of height unless and until such time as
more data was put into such fields.

I have modified the StringSelector class to do this properly.  Now,
the Ganymede client will only allocate a few lines of vertical space
for empty StringSelector objects (enough to maintain the visual
familiarity of the component), and will automatically grow as needed
until it reaches the size we were previously using.

81. [SERVER] Reduced automatic field rescanning

Previously, the Ganymede server was ordering the Ganymede GUI client
to rescan fields every time they were edited by the client.  This
involved an unnecessary extra round-trip call to the server after
every field modification performed by the user.

Now, the Ganymede server is a bit more picky, and only orders a rescan
if the server code performed a value transformation on the submitted
value (see change 2.0 #52, above).

This improves performance, and will allow the StringSelector to
properly highlight newly entered strings when a choice list is not
forced on the user.

82. [SCHEMA] Gasharl schema DHCP support tweaked

The Gasharl schema code shipped with Ganymede has been elaborated a
good bit to support per-host and per-group custom DHCP boot options.

The arlut.csd.ganymede.gasharl.systemCustom and
arlut.csd.ganymede.gasharl.dhcpGroupCustom classes include useful code
that you might want to look at if you find yourself wanting to support
cloning of objects that themselves contain embedded objects.

83. [SERVER] Improvements to mail thread reliability

We had an incident in which the mailer thread in the Ganymede server
got stuck trying to read from a socket from an SMTP server that was
having some issues.  I've set SO_TIMEOUT on the mailer socket to help
out the mailer thread, which will now interrupt socket reads if an
individual socket read takes longer than 15 seconds to complete.

In addition, various small tweaks to the SMTP handling logic in the
Qsmtp class have been made to make Ganymede more compliant with SMTP.

84. [CLIENT] Ganymede client displays basic information about concurrent use

The Ganymede client will now display a little message showing how many
users are logged into the Ganymede server.  Whenever any user commits
a transaction (causing the build icon to start animating), a message
will appear for a short time identifying the user who made the commit.

This change is intended to provide a bit more context for Ganymede
users, so they can get a better idea if other administrators are
making changes at the same time they are.

(This actually went into the Ganymede repository a little while ago,
but I had neglected to mention it in CHANGES previously.)

85. [SERVER] Added a ctrl-C handler to allow for clean shutdown upon kill

It turns out that Java 1.3 added support for the
java.lang.Runtime.addShutdownHook() method, which allows a Java
process to register a thread to be invoked in response to ctrl-C/kill.

I've taken advantage of this mechanism to allow Ganymede to do a clean
shutdown on ctrl-C/kill.  This will help us make sure our logs are
flushed and any email is drained.

kill -9 on the Ganymede server process will still terminate
unconditionally of course, if that is required.

I hadn't known that Java had this feature, until we started working
with JBoss and saw that it was doing this somehow.

86. [CLIENT, ADMIN CONSOLE] Persisted the look and feel selection

The admin console and graphical client now remember what look and feel
a user last chose on any given system, and automatically re-instates
the desired look and feel at startup.

87. [CLIENT] Added a reset button to the calendar widget

The popup calendar widget now features a reset date button, which can
be used to revert the calendar to the date that was originally set in
the calendar.  Useful for editing dates in object data as well as for
setting the start date for object and admin persona history in the
history panels.

88. [EVERYTHING] Reworked the build process for Ganymede to make it require Java 5

As of 7 July 2008, we're going to go ahead and make Ganymede dependent
on Java 5, so we can take advantage of some new features in Java 5,
most especially the defaultUncaughtExceptionHandler mechanism.

Going forward, we'll bring Java 5 features into Ganymede as seems best
appropriate.

89. [SERVER] Refactored DBNameSpace classes

Gil reported a bug with concurrent xmlclient activity that was capable
of confusing the Ganymede object store's namespace system.  We have
now rewritten the DBNameSpace classes top to bottom.  We fixed some
unsynchronized operations so as to prevent race conditions, and
improved collision detection logic in several respects.

Users may notice that the memory loading for DBNameSpaceHandle class
has been reduced somewhat, as well.

90. [CLIENT] Converted the client's report window to use JTable, support printing

James replaced the old ARL-custom table component in the Ganymede
client with a Swing JTable-based keyed table.  This enabled fancier
sort control, and most especially made it possible to print query
reports.

91. [CLIENT] Added extended keyboard navigation to the tree component

The tree component used in the Ganymede client and admin console
schema editor now support full content search through prefix match
with keyboard text entry.

92. [CLIENT] Fixed numerous bugs when using the GTK look and feel on Linux

The Ganymede client now renders properly when using the GTK look and
feel on Linux.

93. [CLIENT, ADMIN CONSOLE] Client, console work properly as an applet once again

The Ganymede client and admin console have been broken for some time
when run as an applet.  The client and admin console now behave
properly whichever method (applet, plugin, webstart) you choose.

94. [CLIENT, ADMIN CONSOLE] Expanded the 'About Ganymede' dialog.

The 'Credits' and 'About Ganymede' dialogs have been folded into one
expanded 'About Ganymede' dialog.

95. [SERVER] Reworked handling for embedded objects

Previously, the schema editor allowed edit-in-place invid fields to be
configured to point to invid fields in the embedded object other than
the container field, which is never the right thing to do.

In addition, we discovered some general issues with using symmetric
links for the container<->embedded object relationship.  We had a case
in which we wanted to allow a specific embedded object to be embedded
within multiple container fields in a single container object.  In
this case, using a symmetric link with a specified target field was
impossible.

In response to this, we made modifications to the Ganymede server to
ensure that all edit-in-place fields were bidirectionally
asymmetrically linked.  The container's edit-in-place field no longer
supports a designated target field, and neither does the embedded
object's container field.

Instead, the back-pointer tracking for both ends of the
container->embedded object relationship is now done through the global
back pointers structure in DBStore.

When we implemented this, we discovered that the back pointers
structure had its own problems..

96. [SERVER] Improved asymmetric backlinks tracking

The hashing structure used to track objects pointing to a target
object through an asymmetric invid link had a number of bugs,
including a complete failure to track checkpoints and rollbacks, as
well as a complete lack of proper encapsulation.

I rewrote the backpointers tracking system, placing it in the form of
the new arlut.csd.ganymede.server.DBLinkTracker class.  The
DBLinkTracker class features efficient checkpoint/rollback support,
and performs less unnecessary synchronization than the old manual
hashtable structure did.

97. [SCHEMA] Additional gasharl DHCP support

The gasharl schema kit's DHCP support is now entirely self contained,
and uses the refined embedded object support logic to enable flexible
configuration of the entire dhcpd.conf file generation without need of
external configuration files.

98. [CLIENT] Cleaned up, consolidated history panels

I've cleaned up and consolidated the history and admin panels in the
Ganymede client.  The GUI controls for choosing a time span to view
history for is now simpler and easier to use.

99. [SERVER] Reworked encapsulation in several server classes

I modified DBObjectBase, DBObjectBaseField, DBField and several other
server classes to mark all (or most) member variables as private, and
added accessor methods to set and get these values, with proper
security controls.

General big clean up to improve encapsulation, honestly.  Things are
much cleaner now, with less worries about accidental encapsulation
violation.

Also reworked the data structures in DBObject and DBObjectBase so that
we now hold our fields and field definitions in a sorted list, rather
than in a custom hashed data structure.  We now use
java.util.Arrays.binarySearch() to efficiently find fields in
DBObjects and DBObjectBases.

100. [SERVER] Converted from using gnu.regexp to java.util.regex.

I took out the use of the old gnu regexp package from the server, to
reduce the size of the Ganymede download.

101. [SERVER, CLIENT] Improved object locating logic

The various open object dialogs (view/edit/delete/inactivate, etc.)
now have automatic alias handling.  Now that all objects are required
to have namespace constrained labels, the open object dialogs are able
to find / display objects that may match the name given in the
appropriate namespace, even if the name is attached to the namespace
through a non-label field.

Even more, if the namespace is attached to fields in a different
object type, requesting an object of a given name might result in a
object of a different type being returned.

I.e., when using Ganymede to manage accounts for Active Directory,
users and groups are required to be in the same namespace.  With this
change, you could ask to view either a user or a group of a given
name, and if you guessed wrong about the type of the object, the
client will notify you that the object you asked for is of a different
type.

102. [SERVER] Added integral cracklib support

The Ganymede server now includes built-in support for cracklib
password quality checking and history maintenance.

103. [CLIENT] Implemented better Mac OS X client rendering

The Ganymede client now looks like more like a proper Mac OS X
program, with support for common Mac keybindings, the Macintosh
Application menu, a Mac-like integrated toolbar, modal dialogs as
property sheets, and so forth.

104. [SERVER, CLIENT, XMLCLIENT] Added support for transaction comments

Ganymede now supports comments on transactions.  If you hold down Alt
or Option when pressing the Commit button, a dialog will pop up and
prompt you for a comment to attach to the transaction.  The comment
will be included in all email generated by Ganymede for that
transaction, and the comment will also be included in logging for the
transaction.

With xmlclient, you can specify a comment by adding a <comment>
element under the <ganydata> element.

I.e.,

<ganydata>
  <comment>The user asked the help desk to reset his password.</comment>
  <object type="User" id="jamesf">
    ..
  </object>
</ganydata>

If the transaction can be successfully committed, the contents of the
<comment> element will be included in the email and logging.

105. [SERVER, XMLCLIENT] Loosened the character restrictions on Ganymede names

Ganymede object type names and field names can now include any
character in Unicode that is legal for representation as an XML 1.1 /
XML 1.0 version 5 element or attribute name.

Unlike XML element names, Ganymede object type and field names *can*
include a space character, which is represented as an underscore in
Ganymede's XML dialect, and *can not* include an underscore character.

106. [SERVER] Incorporated the Java genericized Collections API

The logging system has been revised internally to use List instead of
Vector, and Set rather than my old home-spun
arlut.csd.Util.VectorUnion stuff.

This has had some changes on method definitions that customizers may
run into when compiling their code against the Ganymede server from
this point on.

In particular, DBEditObject.getEmailTargets() must now return a
List<String> instead of a Vector.

107. [SERVER] Improved email sent by logging system

Formerly, the Ganymede logging system sent out a lot of mail with a
generic 'Transaction Log' message.

The logging subsystem has been improved so that the subject line is
much more descriptive.  This change includes logic for both small and
large transactions.. small transactions will have the object types and
names included in the subject line, along with whether they were
created, edited, or deleted.

Large transactions will have a summary report that includes a count of
objects created, edited, or deleted, and their types, if a reasonably
small number of object types were affected.

108. [SERVER, ADMIN CONSOLE] Added monitoring of build / queue status to admin console

The Ganymede server now monitors queue size for incremental sync
channels and provides that information to the admin console.  For
incremental sync channels, the Ganymede server now distinguishes
between an empty sync queue, a non-empty sync queue that is being
properly serviced, and a stuck queue resulting from a sync channel
service program that could not be run successfully.

The server also provides build success information for non-incremental
sync channels and GanymedeBuilderTasks to the admin console, based on
the exit code returned by the service program.

The Sync Monitor tab on the Admin Console now has a 'Status' column
that displays this build status information.  The older 'Status'
column has been renamed 'Scheduling Status' to distinguish it from the
build status that is now being carried in the 'Status' column.

Added a red warning icon that will appear in the tab for 'Sync
Tasks' if one or more builds are currently broken.

109. [SERVER] Removed RMIGC from the bin/runServer script

As of Java 1.6.0_18, the Java garbage collector seems to behave
somewhat badly in the presence of modified values for

 sun.rmi.dgc.client.gcInterval and
 sun.rmi.dgc.server.gcInterval

I have removed the RMIGC variable setting these properties to a very
high value from the server's runServer script, to avoid gross memory
loading behavior in the Ganymede server.

Under Java 1.6.0_18 and later, the old gcInterval property settings
was causing our Ganymede server to balloon up to 340 megabytes during
operation, due to full image garbage collection only being performed
once an hour.

With the RMIGC variable removed, the server now keeps much closer to a
steady state and does not go on wild ballooning trips through memory.

110. [CLIENT, ADMIN CONSOLE] RMI Registry port numbers are now really honored

The Ganymede client and admin console previously did not actually
honor the RMI registry port number provided in the jnlp file /
properties file, defaulting to port 1099 in all cases.

The clients and admin console now honor the specified rmi registry
port number.

111. [CLIENT] Improved the client's date picker with SwingX JXDatePicker class.

The Ganymede GUI client now uses the Swing Labs' JXDatePicker class to
provide a drop-down calendar for date fields, rather than requiring a
pop-up window incorporating our own heavy-weight calendar component.

Wizards and expiration/removal panels still use our panel calendar.

112. [SERVER] Improved security for XML client queries.

The server will no longer provide password hash information to
xmlclient queries unless the xmlclient is given supergash-level
privileges.

113. [SERVER] Simplified and updated query language, updated ANTLR

The Ganymede Query Language no longer requires field names and object
types to be single or double quoted in GanyQL queries.

The GanyQL is now being constructed with ANTLRv3.2, and I've updated
the ANTLR runtime included with the Ganymede distribution to ANTLR3.2.

The Ant files for Ganymede now bodily include the ANTLR3.2 runtime
classes into the produced GanymedeServer.jar, rather than requiring
the runServer script to add the ANTLR runtime to the class path.

114. [CLIENT] Added comprehensive error detection and reporting to the client

The Ganymede client includes a feature to report any uncaught
exceptions that occur on the client and report them to the server to
be logged.

Previously, these exceptions had to be caught one by one throughout
the client code.  This tended to miss exceptions that were thrown from
the GUI thread before calling non-GUI client code.

We have now added an UncaughtExceptionHandler to the client which will
report any exceptions that occur anywhere in the client after login to
the server.

115. [CLIENT, ADMIN CONSOLE] Added the complete CREDITS file to the 'About Ganymede' dialog

The About Ganymede dialog used to refer the viewer to the CREDITS file
in the Ganymede distribution.

We have now bodily included the contents of the CREDITS file into the
About Ganymede dialog's properties file.

Note that we will need to update the properties file whenever we
update the CREDITS file.

Note also that the German localization for Ganymede still has a
truncated version that refers the user to the distribution's CREDITS
file.

116. [EVERYTHING] Moved to require Java 6

Since we are now incorporating SwingX in the Ganymede client, and
SwingX since after release 1.0 has required Java 6, we are going to
officially require Java 6 ourselves in the Ganymede client.

The server still does not (at this time) contain any Java 6
dependencies, but we will consider ourselves free to introduce them
from this point on.

(27 September 2010)

117. [ADMIN CONSOLE] Reflowed server info fields in admin console

I have reworked the top panel in the admin console to reduce the
vertical space taken up by those constant fields.

This will provide more space for the log box.

118. [SERVER] Fixed a security vulnerability in PasswordDBField

The arlut.csd.ganymede.rmi.pass_field remote interface was providing
remote access to a method that allowed an override of Ganymede's
permission system.  This method was created for the purpose of
supporting the xml data loading subsystem, but it should not have been
exposed to remote clients.

This vulnerability would allow a user who had both valid login access
to the Ganymede server and permission to edit an object containing a
password field the ability to set the password in that field, even if
the Ganymede server was configured not to allow the user to edit the
password field specifically.

In order to take advantage of this vulnerability, an attacker would
need to create his own custom client to call the vulnerable published
method.

And, as mentioned above, valid authority to login to the system and
edit the object in question.

This method has been removed from the pass_field remote interface,
removing the ability for a custom client to bypass field permissions.

119. [SERVER] Added SyncMaster interface to augment Sync Channel feeds

We've added a new arlut.csd.ganymede.server.SyncMaster interface that
can be used to programmatically enhance the context passed to Sync
Channels for incremental synchronization.  With a SyncMaster attached
to a Sync Channel, you can pull in information from any part of the
Ganymede object database and include it as needed in a Sync Channel's
incremental XML sync files to provide whatever context might be needed
to allow the XML processor to communicate with its backend services.

120. [ADMIN CONSOLE] Fixed some errors in the schema editor

The schema editor's namespace editing panel has been fixed.
Previously, when you clicked on namespace name, it would only show you
fields managed by it if they were string fields.  Now, any namespace
managed field (i.e., I.P. Address, Numeric) will be properly shown.

121. [SERVER] Improved error handling in DBLog

The DBLog code would throw an exception if the mail host could not be
reached in such a way that the log message would be lost.  This has
been repaired, and all messages will be fully logged even if the SMTP
host cannot be found.

122. [WEB] Improved password change web page

The password change CGI has been significantly cleaned up, and now
properly autoconfigures the URLs it uses when generating forms, rather
than depending on a pre-specified URL when you run installWeb.

123. [WEB] Added GanyQL query web page

The installWeb script now installs a CGI that allows you make GanyQL
queries against the Ganymede server.

124. [CLIENT] Fixed password field focus problems on multi-monitor Linux

The Ganymede client's double entry password field was using a
deprecated method for doing focus handling.  This was causing problems
where you couldn't click on a password edit field and have it take
keyboard focus.

This wasn't revealing itself in my testing, but it came up on a
Linux box with more than one screens attached to its X Server.

Fixed.

125. [SERVER] Added support for the OpenBSD bcrypt hash format.

The PasswordDBField class in the server now understands the OpenBSD
bcrypt hash format, in addition to SHA Unix Crypt and all the rest
already supported.

126. [SERVER] Admins are now allowed to give objects away

Previously, admins in one owner group were not allowed to give an
owner group they did not belong to possession of an object.

Now, ownerCustom is tweaked to allow an admin in one Owner Group to
add another Owner Group to an object's Owner List field, and then take
his Owner Group out of the Owner List, thereby transferring
responsibility of the object away from his group.

The new 'ganymede.allowdonations' property can be used to control this
behavior.  If the property is set to 'true', admins will be able to
give away objects.  Otherwise, not.

127. [CLIENT, SERVER] Object metadata fields can now be sent to Sync Channels

The standard object metadata fields (creator, creation date, last
modifier, last mod date) can now be selected for output to an XML Sync
Channel.

128. [CLIENT] Improved Sync Channel field editor

The client's sync channel field editor now better represents the valid
options for a full state Sync Channel.

129. [SERVER] Added checkpoint / rollback to GanymedeExpirationTask

The GanymedeExpirationTask was not previously protecting its
activities with a checkpoint / rollback pair, so a failure to
inactivate or remove any object scheduled for expiration/removal would
block all other expirations / removals during that run of the task.

The GanymedeExpirationTask now uses checkpoint / rollback for each
object that it attempts to inactivate or remove, so a problem in one
object won't prevent other objects from being properly processed.

130. [SERVER] Added support for emailing uncaught exceptions

The GanymedeUncaughtExceptionHandler will now send email to the
addresses listed in the ganymede.bugsaddress property if an uncaught
exception is handled.

131. [SERVER, CLIENT] JDK 7 Compatibility, JDK 1.4 Incompatibility

I made calls to FoxTrot, etc., conditional on the version of Java
being run so as to make the client work properly under JDK 7.

I took out some logic that was in place to handle running on JDK 1.4,
which is now officially dead and gone.

132. [SERVER, RMI, CLIENT] Removed the 'isCrypted' method from pass_field, PasswordDBField.

The isCrypted() method was increasingly useless as it only checked for
the presence of a traditional Unix triple-DES crypt hash.

There are better ways to query a pass_field's supported hash types
using the BaseField remote interface, anyway.

133 [SERVER, RMI] **security** Removed getValueString() from db_field interface

Removed getValueString, getEncodingString from remote db_field interface.

These methods made it possible for users remotely logged into the Ganymede
server to access information that was meant to be hidden from them.  In
order for the user to access restricted information, the user would have
to have permission to get a read-only or editable copy of a Ganymede
database object via GanymedeSession.view_db_object() or edit_db_object().

Once they had access to such an object, they were able to call getValueString()
or getEncodingString() on any field in the object, even if those fields
were not meant to be readable by the user.

This did not put password data at risk because PasswordDBFields have never
returned plain or hashtext via either of these methods.  It could reveal
Strings from fields that were not meant to be visible to the user.

In order to have taken advantage of this vulnerability, users would have
had to have built a custom client that abused these methods.  Not a trivial
matter, but not impossible either.

134. [CLIENT] The client now only supports saving / mailing  object information in XML

The client now has support for using the Ganymede XML format when
saving or mailing object data.  This is exposed to the user by an item
in the per-window 'Object' menu.

The old ad-hoc ASCII formatting used when saving / mailing data has
been removed.

135. [CGI] The password CGI shows the password characters entered count.

The Perl CGI script that handles password changes for end users now
uses JavaScript to provide a dynamic display of the number of
characters entered / the number of characters required so that users
don't have to count the characters in their passwords themselves.

136. [CLIENT] Improved CSV generation from the query result table

The query result table now emits a more standard version of a CSV file
when you save query results in that format.

137. [XML] Fixed XML parsing of empty fields

The XML parser in the Ganymede server now properly handles empty field
elements (e.g., <Password/> or <Password></Password>) and treats them
as a directive to delete the field.

This only works in scalar fields.  Array fields will still require you
to do something like this: <Groups><set></set></Groups> to clear all
members of the field.

138. [SERVER] Added versioning of Ganymede journal files for reliability

The Ganymede server was not previously able to identify the case where
a ganymede.db file on disk was paired with a journal file from an
incompatible version of Ganymede.

The journal file now contains version information, and the Ganymede
server verifies that the journal file matches the version of thd
ganymede.db file on startup.

139. [SERVER] Internal cleanup in Ganymede startup sequence

The arlut.csd.ganymede.server.Ganymede class has had its main() method
considerably cleaned up, with the server's startup logic broken into
several discrete methods.

This is to improve the readability and maintainability of the server
code.

In addition, the server is now more resilient to errors that can arise
during certain portions of the startup process.

140. [SERVER, CLIENT] Show object windows causing problems at commit.

The Ganymede client will now make visible and select a window editing
an object if that object is missing fields or otherwise failed to
validate at commit time.

Previously, it was possible to have a partially filled out object in
the background (or iconified) in the client, and it wasn't always
clear which object caused the problem if something was broken at
commit time.

This was especially the case if an object had not been filed out at
all, in which case it would not have a recognizable label that could
be presented to the user to identify the problem object.

Now, when the user hits commit in the client and there is a problem,
the object edit or create window that contains the problem will be
brought to his attention.

141. [SERVER] Made server startup more robust

The Ganymede server now properly survives a failure in the DBLog
initialization logic.

142. [CLIENT] Slimmed the GUI client's handling of embedded object panels

Embedded object panels in the client are now slimmed down so that they
fit inline with the rest of the components in a client panel, without
having a title border around it.

143. [SERVER] Added the ability for the server to capture stderr/stdout from build scripts

The arlut.csd.Util.FileOps class now has a runCaptureOuputProcess()
method which will run an external process and capture the output for
logging and/or analysis, as appropriate.

We now log whatever is written to the stderr stream during the
execution of an incremental sync service program.

144. [SERVER] Major refactoring of GanymedeSession class

I have radically cleaned up the GanymedeSession class by breaking a
lot of the functionality into the server's new DBPermissionManager and
DBQueryEngine classes.

During the process of accomplishing this, I've improved
GanymedeSession's encapsulation a great deal, with almost all
variables in GanymedeSession now being private with accessors added.

All of the RMI-accessible Session methods in GanymedeSession are now
grouped together and properly call checklogin() at method entry.

145. [RMI] Removed deprecated methods from Session interface

I have removed the long-deprecated getTypes() and getRootCategory()
methods from the Session interface.

146. [CLIENT] Made the object view/edit/delete object dialog case insensitive

The view/edit/delete object dialog now does a case insensitive match
when looking for exiting objects that prefix match the text in the
label field.

147. [SERVER] Fixed a long-standing memory pinning problem with schema editing

The server no longer holds a reference to the outdated version of the
persona base after a schema edit is performed.

This was causing memory usage to stay inflated until a restart after
editing the schema with the admin console.

148. [CLIENT] Improved window size and text presentation in client.

The Ganymede client now makes sure to show the beginning of text that
is too big to fit in a standard-sized text field, rather than the end.

The standard-sized text fields in the client are a bit bigger now.

The Ganymede client will create slightly larger internal object
windows to compensate, but now has logic which shrinks newly created
object windows if the Ganymede client's main window is too small to
fit its prefered size.

149. [CLIENT] Closing the main client window now terminates the client

Previously, closing the main client window when run as a command line
or Java Web Start application caused the login window to appear again.

The Ganymede client will now simply terminate when the main window is
closed.

150. [SERVER] The server now supports a Ganymede.returnaddrdesc property.

The Ganymede Server now supports a new system property,
'Ganymede.returnaddrdesc'.  This property may contain the extended
human readable rfc822 return address for inclusion in the 'From:'
field of any mail sent out by the Ganymede server on behalf of itself
rather than a logged-in user.

I.e., you might have

  Ganymede.returnaddr=ganymede-server@yourdomain.com
  Ganymede.returnaddrdesc=Ganymede Server <ganymede-server@yourdomain.com>

The returnaddr property is used in the SMTP handshake, and the
returnaddrdesc is used in the headers.

151. [SERVER] Fixed a rare deadlock vulnerability with XML sync channels

The XML Sync Channel subsystem was doing a nested dumplock, but was
not preventing other threads from getting into the writer list queue
after the outer dump lock was established and before the inner dump
luck could be.

I have modified the DBWriteLock so that it will not permit writers to
get on the write lock queue until there are both no dumpers waiting to
be established or established on a competing DBObjectBase.

152. [Admin Console] Display both locks waiting to be established and locks held

The admin console now displays both the number of locks waiting in
their establish routines and the number actually established and held.

153. [Admin Console] Cleaned up monitor tables, date formatting

The admin console now uses a much briefer and more readable date
format, with configurable localization patterns in the
GASHAdminDispatch.properties file.

There are two separate date format strings available, one for
timestamps that occur in the same day as current time, and one for
times that are not.

I also improved the task highlighting for running or disabled tasks in
the Sync Monitor, and changed the column ordering in the Scheduled
Task Monitor to have 'Last Run' be the last column as it is in the
other admin console table views.

154. [CLIENT] Better behavior on login failure

The client now simply clears the password field and prompts for
re-entry if the user provides invalid credentials.  All other causes
for login failure (server going down, schema being edited, etc.) will
still pop up a nuisance dialog to the user.

155. [CLIENT] The client will now indicate when a viewed object is
              inactivated in the object window title bar

When viewing an inactivated object, the client will now include the
inactivated status in the text of the object window's title bar.

156. [SERVER] Added a handy getLabel(Invid) method for DBEditObject subclassers

This is now being used by the updated userKit, and makes things a bit
easier when looking up remote labels in custom object code.

Note that GanymedeBuilderTask has already provided this, so this is
just meant for custom DBEditObject subclasses.

157. [USERKIT] Synchronized userKit up to modern Ganymede, added ShaCrypt password import

We don't use the userKit ourselves at the lab, and the code had
suffered some bitrot.

We went over userKit and made sure that it worked with the latest
Ganymede code, and added the ability to handle the import and
generation of BCrypt and modern ShaCrypt password text.

158. [SERVER] Fixed a Sha256Crypt / Sha512Crypt verifyPassword bug for short salt strings

The verifyPassword() method in the Sha256Crypt and Sha512Crypt classes
had a bug that was causing password validation to fail if the salt
string in pre-generated Sha256Crypt/Sha512Crypt hash text was shorter
than 16 characters.

This wasn't a problem for ShaCrypt passwords generated by Ganymede,
because Ganymede always uses 16 character salt strings when generating
ShaCrypt hashtext, but it came up when using the updated userKit to
import shacrypt passwords from older /etc/shadow passwords on Linux.

159. [SERVER] Improved handling of RMI edge cases in server startup

The Ganymede server was not fully configuring the RMI system to
completely honor the ganymede.serverhost property.  The RMI regsitry
was being bound to the hostname / IP address listed in this property,
but the server-side RMI proxies were using whatever name was returned
from the Java networking libraries for the identity of the localhost.

This made it impossible to successfully run Ganymede on a system whose
public IP address was firewalled.

It is now possible to explicitly set this property to 127.0.0.1, and
all server and client operations will now work properly over the
dedicated loopback address.

160. [SERVER] Improvements in internal encapsulation and multithreaded security

Previouly, developers of plugin classes were required to deal with
making defensive clone() calls on the results from

  DBObject.getFieldValues()
  DBObject.getFieldValuesLocal()
  DBField.getValues()
  DBField.getValuesLocal()

As well as dealing with possible null values from

  DBObject.getFieldValues()
  DBObject.getFieldValuesLocal()

Now, the DBObject getFieldValues methods will always return an empty
Vector where previously they would return null, and the DBField
getValues methods will return a copy of the internal field Vector,
rather than providing direct access.

In addition, the DBField and DateDBField class now cooperate to make
defensive copies of Date objects passed or returned from DateDBField
objects.

161. [SERVER, GASHARL] Made more DBObject methods final, added @Override annotations

I went through and made a bunch of previously unprotected methods
final, and have added the @Override annotation to intentionally
overridden methods in the arlut.csd.ganymede.server and
arlut.csd.ganymede.schemas.gasharl packages.

In addition, a great deal more of DBObject's state is now private to
improve encapsulation and security.

162. [SERVER, CLIENT, SCHEMA CODE] Reworked internal representation of IPDBField

Previously, all IP Addresses in Ganymede were held in IPDBField
objects as Byte[] objects, and getting and setting these values left
the (server-side) caller with a reference to the Byte[] held in the
IPDBField after setting, with the ability to randomly mess with the IP
data at a later time, possibly breaking security checks in the schema
code.

Now, the new arlut.csd.ganymede.common.IPAddress class is used as the
value stored in IPDBField objects.  The on-disk representation of the
IPDBField objects have not changed, so this is a backwards-compatible
change with regards to ganymede.db and journal files, but any custom
schema code that is configured to get callbacks when IPDBField values
are changed will now be called with IPAddress objects instead of
Byte[] objects.

As such, this is an **incompatible** change with previous schema code
that ever deals with the old Byte[] representation, and such code may
throw a ClassCastException if not changed to use the new IPAddress
object type.

The good news it that the IPAddress object is serializable, final and
immutable, so whenever you create or pass around an IPAddress you can
be sure that nothing will ever mess with its contents post-creation.

I've also moved all the standard IPv4 and IPv6 serialization /
deserialization logic to the IPAddress class, and any classes in the
Ganymede server or client that previously had code for creating a
Byte[] out of an IPv4 or IPv6 string representation now depends on
IPAddress to do that work.

163. [BUILD, CLIENTS] Added Permission and Codebase to client/admin console jars

As of Java 7u25, Oracle has added support for a couple of new
attributes in the jar manifest to provide limits on what sort of
environment a given jar was built to run in.

We have modified build.xml so that the Ganymede client and admin
console jar files now declare where they are expecting to be run from.

As a consequence of this change, the Ganymede client and admin
consoles will now refuse to run on Java 7u25 or later if they are
built on a different domain from that which they are downloaded from.

See

  http://docs.oracle.com/javase/7/docs/technotes/guides/jweb/no_redeploy.html

for details.

164. [BUILD] Improved build.xml with instructions on how to sign jars

As of Java 7u40 and later, the JRE will no longer passively accept
self-signed certificates for jar files.  In response to this, I have
modified the build.xml script in src to allow the necessary pieces for
signing the jars with a private / public key pair signed by a
certificate from a Certificate Authority recognized by the Java
Runtime Environment to be held outside of the Git repository.

Now, all of the sensitive pieces for signing are held in an external
src/build/crypto/crypto.properties file which is listed in .gitignore
to prevent that material from being accidentally included in a Git
repository.

-------------------- Changes from 1.0.11 to 1.0.12 -------------------

RELEASE DATE: March 1, 2004

1. [CLIENT] Properly update StringSelector count display on manual item entry

The StringSelector widget used to manipulate vectors of strings and
object references in the client was not properly updating the item
count when an item was manually entered through the text box.

2. [SERVER] Fixed regression in object event label display

I had neglected to edit the objectEventCustom.java class to cohere
with the change described in CHANGES 1.0.11#8.

3. [SERVER] Fixed exception vulnerability in admin console, scheduler code

We had an occasion in which the Ganymede scheduler terminated during
the server's normal operations, as a result of cascading failure in
the admin console notification system.  The changeStatus() method in
serverAdminProxy was not properly checking the done flag, so a
NullPointerException was thrown where the GanymedeAdmin code was only
expecting the possibility of a RemoteException.  The
NullPointerException was being thrown up through the Ganymede.debug()
call frame, which led to the scheduler being killed.

I have fixed the serverAdminProxy.changeStatus() method so that the
NullPointerException will not re-occur, and I have fixed the Ganymede
scheduler so that nothing but an actual interrupt on the scheduler
thread will cause the scheduler to terminate.

4. [SERVER] Turned off extraneous debug output from XML processing logic

I had accidentally left the debug flag enabled in the xmlobject and
xmlfield classes, which caused a lot of extra output to be displayed
when processing XML files through the xmlclient.

5. [SERVER] Removed 1.4-only StringBuffer method usage from DBEditObject.java

It turns out that java.lang.StringBuffer gains a method overload that
executes special code in the event that the parameter passed to append
is itself a StringBuffer.  When I compiled the Ganymede code with a
1.4.1 JDK, code that previously was interpreted to have an implicit
toString() method call now uses the optimized StringBuffer call, which
doesn't exist in JDK 1.3 and earlier.  Putting the toString() in
explicitly forces javac to use the String parameter method, which is
present in all versions of the JDK.

This fix should make it possible to run the Ganymede server on JDK
1.3.1 or earlier again.

Thanks to Maura Mathieu (maura.mathieu@sdis66.fr) and Kent Wick
(kent.wick@mhmr.state.tx.us) for reporting this.

6. [SERVER SCRIPTS] Fixed RMIGC definition in runServer for ksh

Per Kent Wick, the RMIGC definition in the runServer templates has
been surrounded by single quotes.

7. [SERVER] Preparations for logging to PostGreSQL

The Ganymede log classes have been reworked in preparation for
supporting SQL databases for log event storage and retrieval.  1.0.12
is not SQL-ready, quite, but I've done some refactoring to allow for a
variety of DBLogController implementations.  I've implemented a
DBLogPostGreSQLController class for logging to PostGreSQL with an
appropriate schema, but I'm not including the appropriate support
files necessary for initializing the database and what-not.

Next release if all works out well.

8. [SERVER, CLIENT] Reworked client and server to support direct login

In all previous versions of Ganymede, the login process is a two-step
dance.. first the client (GUI or xmlclient) calls up the appropriate
login method on the server, passing a reference to a callback object
hosted by the client, then the server uses that callback object to
query the client or xmlclient for username and password.

I initially did this (6-7 years ago) to throw a (vanishingly small)
roadblock into the path of people trying to packet sniff the login
process, and so that people wanting to do a replay attack would have
to emulate the full RMI protocol.

This was really always complete silliness, of course.  Despite that, I
had no real motivation to rework the login process until David Cummins
reported that he was unable to use the Ganymede client from his
firewalled desktop systems.  Given the increasing prevalence of
defensive firewalling on the local system level, it seemed like it was
time to rejigger the Ganymede login process.

Users of a locally firewalled system will now be able to log in and
use Ganymede, but the experience will still be second rate.. the
server won't be able to send any build status messages to the client,
and if the server needs to timeout or force disconnect the client, the
client won't get the message.

NOTE: One side effect of this change is that certain methods in the
Ganymede server are now marked as throwing a 'NotLoggedInException'
condition.  If you have custom code that calls any of these methods,
you may see build problems when you go to rebuild your code for use
with Ganymede 1.0.12.  Surrounding the offending calls with an
appropriate throw..catch clause should be all you need to do to make
your code compatible with 1.0.12.

9. [CLIENT] Improvements to text messages in dialogs

The common composite dialog class used in the Ganymede client and
admin console has been improved to support adaptive word wrapping and
scrollbars.  This change was motivated by the client's new ability to
report remote exception traces in a dialog.

10. [CLIENT/CONSOLE/SERVER] Reworked RMI API to be system firewall transparent

This is an extension of the 1.0.12#8 work.  It used to be that the
server used RMI calls to the client and admin console to send
notification for build status and force off (on the client) and all of
the data displays on the admin console.  The problem with this is that
it required the client/console to open a socket and wait for an RMI
call from the server to send update information.

I have now reworked the server, client, and admin console so that all
transmission of data from the server to the client is done by having
the client/console run a polling thread which repeatedly calls an
asynchronous message port on the server.  Since the data is flowing
through the result of a forward RMI call, no socket needs to be opened
up for listening on the client/console system, and full functionality
is achieved even in the presence of a client-side system/personal
firewall.

This change also speeds up login and noticeably reduces the memory
loading on the server.  A win/win/win deal.

11. [CLIENT] Fixed gclient.getObjectHandle()

The getObjectHandle() method in the client's gclient class didn't
properly deal with an empty list back from the GanymedeSession's
queryInvids() method.  This could result in an exception being thrown
rather than a dialog being shown in certain circumstances in the
client in which a user tries to edit an object reference that he
doesn't have permission to edit or view.

12. [SERVER] Improved namespace conflict message

The DBField class in several places had code to report namespace
conflicts.  Created a common method to generate a more detailed error
dialog which informs the user what object/field is conflicting.

13. [SERVER] Cleaned up DBObject.getTypeName()/getTypeDesc()

The DBObject class had two methods, getTypeName() and getTypeDesc()
which did exactly the same thing.  Embarassing.  I went through and
changed all instances of DBObject.getTypeDesc() to
DBObject.getTypeName() calls, in keeping with the db_object interface.

14. [CLIENT] Added row count indicator to query window

In response to a so-simple-it's-brilliant suggestion by David Cummins,
I have tweaked the Ganymede client to display a count of the rows
returned in the title bar of the client's query results window.

15. [CLIENT] Various tweaks for better behavior under JDK 1.5 beta

I've made some slight tweaks to the client and console's GUI layout to
improve behavior when running under the JDK 1.5 beta.  This includes
modifications to the JNLP launch files in webforms so that the user
will be given the option of using versions 1.2 through 1.5 of the Java
runtime.

-------------------- Changes from 1.0.10 to 1.0.11 -------------------

RELEASE DATE: December 9, 2002

1. [SERVER] Put safety interlock in DBEditSet.addNewObject()

The DBEditSet class had a vulnerability that could conceivably cause
transaction corruption if an adopter wrote custom code that attempted
to pull new objects in to the transaction in a custom DBEditObject
classes commitPhase2() method.

Now addNewObject() will throw a RuntimeException() if someone attempts
to pull new objects into a transaction during the transaction's
commit() process, causing the transaction to cleanly abort.

2. [CLIENT] Made the Ganymede login applet show the server name

Added information on the login box about the location of the Ganymede
server that the client has bound to.

3. [SERVER] Added new ganymede.subjectprefix property to the server.properties file

The server now supports a new server property, entitled
'ganymede.subjectprefix'.  If this property is set to a string that
begins and ends with single quotation marks, the text included between
the single quote marks will be used as the identifying subject prefix
for email sent by the Ganymede server.  If no such server property is
defined, the server will use 'Ganymede: ' as the subject prefix.

4. [SERVER] Fixed GanymedeSession addResultRow(), internalQuery() to honor editableOnly query filtering

The query logic in GanymedeSession has always been grotesquely over
complicated, and it turns out that there has been a significant error
in the addResultRow() method, which was neglecting to filter out
non-editable objects from queries that specifically request that
non-editable objects be filtered, in the case that
GanymedeSession.internalQuery() was used to execute the query.  In
addition, the addResultRow() logic in the old non-supergash,
non-internal case was confused and confusing, so I cleaned that up a
bit as well.

I have swept through my code looking for instances of internalQuery()
usage, and I have made sure that all queries supplied to
internalQuery() in non-supergash sessions explicitly turn off the
editableOnly restriction.  This involved some changes in the
schemas/gasharl/custom_src code.

5. [SERVER] Modified DBObject.checkRequiredFields() to check all fields

The checkRequiredFields() method in DBObject was only checking the
custom-specified fields, rather than all fields.  This made it
impossible for a DBEditObject plugin's fieldRequired() method to
enforce field requirements for the expiration and removal fields.

6. [SERVER] Optimized XMLReader and serverAdminProxy slightly

Reworked both XMLReader and serverAdminProxy classes to use an
efficient circular buffer rather than continually doing
System.arraycopy operations within a Vector.  Won't make any
noticeable difference, but after seeing Doug Lea's optimized
concurrency structures (at
http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html),
I felt like tinkering.

7. [SERVER/XML] Fixed feedback on transaction failure in the xml subsystem

The Ganymede server's XML transaction logic was faulty, and would
attempt to report a successful commit even in the circumstance where
the commit-time transaction verification logic caused the transaction
to be rejected.

Now, an xmlclient transaction that could not complete due to
consistency problems during transaction commit will no longer report a
confusing mix of success and failure messages.

8. [SERVER] Changed plug-in API for object labels

The Ganymede object model has always had a rather poorly thought out
implementation of object labeling, with two separate means of
expressing object labels.  The first was by designated label field,
the second was by calculated label field using the DBEditObject
plugin's getLabelHook() method.  There were a number of problems
with this.

First, the Ganymede query engine completely ignored
getLabelHook()-derived labels, and if an object type had both a
designated label field and a getLabelHook() method in a DBEditObject
plugin, the getLabelHook() was used for many but not all
things.. querying on an object's label simply ignored the
getLabelHook() method's output and searched on a label field if
defined, or simply failed if no label field was assigned.

Now, the Ganymede query engine will check the new useLabelHook()
method for a plugin, and if that method returns true, the label field
will be ignored in queries, in favor of the results from
getLabelHook().

To prevent things from being too confusing, the DBObject getLabel()
method will now also consult the new useLabelHook() method, and if
useLabelHook() does not return true, the getLabelHook() method will
simply not be used under any circumstances.

9. [SERVER] runServer now specifies less frequent full-heap GC

Since 1.2, Sun's HotSpot JVM (server-mode) has incorporated
generational garbage collection, which allows the JVM to perform
limited-scope garbage collection passes over recently created objects
relatively frequently, while avoiding full heap garbage collection
passes unless and until the full heap as a whole is filled.  This
allows the CPU loading due to garbage collection to be dramatically
reduced, relative to a more naive full-heap garbage collector.

Unfortunately, the Ganymede server has in the past not been
benefitting greatly from generational garbage collection, due to a
peculiarity of the RMI system.  By default, RMI programs that access
remote objects (as the Ganymede server does to asynchronously send
messages to the admin console and to the gui clients) do full-heap
garbage collection on a periodic basis (once a minute) in order to
quickly release any remote RMI objects.  This extra garbage collection
activity is useless for the Ganymede server, as by design it will
never relinquish references to the few remote objects it talks to
until such time as the client or admin console disconnects/quits,
whereupon the issue of client-side local garbage collection becomes
moot.

I've recently become aware of an RMI system tuning parameter for
recent JVM's which allows this RMI full-heap gc interval to be tuned.
I have edited the runServer script in the Ganymede distribution to set
this tuning parameter to do full-heap gc once an hour.

Previously, our Ganymede server was taking around .5 seconds every
minute doing the full-heap GC cycle, now this overhead is only done
once an hour, at the cost of a rather greater proportion of the
server's allocated heap size 'in-use' on average.

All of this is documented in the new runServer script, and this
parameter can be tweaked as you like.  See
http://java.sun.com/docs/hotspot/gc/ for complete details.

10. [CLIENT] Several small improvements to the Ganymede user interface

You can now hit enter after typing a search value in the query box and
the query will be submitted for you.  This should save people from
having to mouse down to the 'submit' button for each query.  You will
still need to manually mouse down to the 'add choices' box if you want
to add an additional search constraint.  In addition, if a query
returns no results, you'll now have the option to pop the query box
right back up, revise your query, and resubmit.

The GUI widget used for changing passwords in Ganymede will now
provide a quick dialog box telling you that you didn't type the
password the same way twice, rather than silently clearing the fields
and having you try again without any guidance as to why.

Thanks to Marcus Walker and Tom Embleton for these suggestions.. I
hope the changes I made help out.

11. [SERVER] Improved logging for newly created objects

I slightly reworked the commit_createLogEvent() method in DBEditSet so
that newly created objects will use DBObject.getPrintString() to
create a full summary of the newly created object, as is done when
such objects are deleted, rather than using the DBEditObject.diff()
method.

This has the advantage that newly created objects which contain
embedded objects will include full information on the embedded objects.

12. [SERVER] Reworked ownership inheritance

The support in GanymedeSession's recursePersonaMatch() method for
recursing up the ownership tree to find matching personas was broken
in the case when an owner group contained no members.  In such cases,
the upwards recursion stopped inappropriately.

This was a pretty major logical error, and one which has left support
for the Ganymede owner group hierarchy broken for quite a long time.

Upon looking at the code, I found a lot of gross stuff in the whole
owner group permission handling logic.  I went in and committed major
surgery on the owner group code.  The code is now simpler and less
prone to security problems due to inappropriate caching.  In addition,
there may be a noticeable decrease in the time taken when a new object
is created, as the owner group list calculation code is far more
efficient now.

13. [SERVER] Tweaked object creation ownership behavior

Previously, the Ganymede server would complain any time someone tried
to create an object with more than one default owner selected.
Ganymede will now allow this, but will complain if no owner list has
been manually selected in the interactive case.  Ganymede will now
provide a warning if it had to pick an owner group at random to put a
newly created object in when run with a non-interactive client.

14. [CLIENT] Added menu options to view object window, cleaned dialogs

The view object window now has several menu options to allow editing,
deleting, cloning, and inactivating of objects from the view object
window.

Cleaned up several dialogs in the code.

-------------------- Changes from 1.0.9 to 1.0.10 -------------------

RELEASE DATE: April 2, 2002

1. [SERVER] Consistency checks should not be forced if 'enableOversight' not set

We were having some problems with the nightly expiration task failing
due to a consistency check failure unrelated to the actual expirations
at hand.  This was due to the default DBEditObject commitPhase1()
method calling consistencyCheck() on each object involved in the
expiration transaction, despite the expiration task's having elected
to turn off enableOversight.

Now, if the GanymedeSession's enableOversight flag is turned off, the
default commitPhase1() method won't bother to run consistencyCheck(),
and will simply return a positive success value.

This comes in handy, as the nightly GanymedeExpirationTask runs with
enableOversight turned off, so that we can allow object
expiration/removal that would break a required field or other
consistency guarantee if need be, rather than having the nightly task
be blocked from committing its changes if anything would lead to a
slight inconsistency.

Note as always that the default DBEditObject commitPhase1() method can
be overridden with custom logic if it is not desireable to have any
consistency checks skipped in response to the enableOversight flag
being reset.  See the next item for more.

2. [SERVER] Made default transaction commit-time consistency check smarter

Previously, the default DBEditObject.commitPhase1() method rejected a
commit if the object's consistencyCheck() method failed.  Now,
commitPhase1() goes a step further, and only returns a failure code if
the object's consistencyCheck() fails, and the consistencyCheck()
succeeds for the original version of the object going into the
transaction.

That is, if a user is processing a transaction and happens to involve
in his working set an object which was previously inconsistent, we
won't hold it against him and force him to correct the previously
existing inconsistency.  Instead, we will depend on the
GanymedeValidationTask to detect such conditions for a knowledgeable
admin to fix, see the next item.

3. [SERVER] Improved GanymedeValidationTask to run consistencyCheck()

The GanymedeValidationTask scans through the database checking objects
for consistency.  Previously, however, the only check this task was
performing was to see if any required fields were missing.  Now, the
GanymedeValidationTask will also call consistencyCheck() on each
object, allowing inconsistencies that have creeped in from
transactions run with enableOversight turned off to be easily detected
on demand.

4. [SERVER] Turned off passive password capture

Turned off passive password capture.  Ganymede stores passwords in a
variety of hashed forms, which are strings that can be used for
validating a password but not which can't reveal the actual text of
the password without the use of a brute-force encryption search.
Plaintext is never saved on disk unless the password field is
specifically configured to do so in the Ganymede schema.  Things like
sync'ing passwords to Windows NT which require plaintext passwords
have to obtain them from temporary storage in the Ganymede server.

Formerly, whenever a user logged into Ganymede, Ganymede would
temporarily acquire and retain the plaintext used to login to Ganymede
in RAM only.  This plaintext could then be passed to Windows NT for
synchronization.  In this way, users would not necessarily have to
change their passwords in order for their account to be sync'ed.

Unfortunately, it turns out that the password identity validation is
not perfect.  A UNIX password hashed using the traditional Unix
crypt() routine is only significant to 8 characters.  If a user logged
into Ganymede for the first time since the Ganymede server was
restarted with a mistyped version of their password that was correct
to 8 characters but then diverged in the ninth, Ganymede would
remember the incorrect plaintext, and insist on seeing it again before
allowing the user to login until such time as the server was
restarted, and the plaintext password information was lost.  Worse,
such carelessly entered passwords could get propagated to our Windows
NT domain controller in their erroneous form, and Windows NT's
password hashing is significant to 14 characters.

This is not a reasonable or tenable situation, so the passive
plaintext password capture logic has been disabled.

5. [SERVER] Better error message on link permission failure

The Ganymede server will now explicitly report a permissions problem
when an attempt is made to link to an uneditable object.

6. [SERVER] Re-enabled optional mailout event logging

I had disabled logging of mailout events, to keep the Ganymede log
from being flooded by non-critical email sent out by the Ganymede
server.  Unfortunately, the inevitable question arose here about
whether Ganymede did or did not send out an email message warning
about an upcoming account inactivation.  Having a log of all advisory
email sent out by Ganymede would have helped clarify the situation, so
this function is being added back to the Ganymede server as an
optional logging facility.

Now, if the Ganymede server's ganymede.properties file defines a
ganymede.maillog property, the file named in that property will have
all mailout events logged to it.  If the ganymede.maillog property is
not set, no mail events will be logged on disk.  By default,
the server's ganymede.properties file is installed with no ganymede.maillog
property defined.

If you are using a schema kit that sends out a lot of mailout events,
you may only want to enable mail event logging for short periods as a
diagnostic.

As with all properties defined in the ganymede.properties file, the
server must be stopped and restarted for any changes in the properties
file to take effect.

7. [SERVER] Improved logging/email for embedded object modifications

All informational logging of changes made to embedded objects now
includes the name of the top-level object which holds the embedded
object.  For our usage, this makes it possible to tell what system
was involved when an interface definition was changed.

8. [SERVER] Clean ups and optimizations to the admin console handling

I found that there was some really ugly code in the GanymedeAdmin
class that handles communications to and from attached admin consoles.

I substantially reworked this class for better encapsulation, and
while doing so wound up removing an extra RMI call that the server was
attempting to make to each admin console for every line of log
information sent to the console.  The serverAdminProxy was eliminating
a lot of these calls if lines of logging information were being sent
rapidly to the admin consoles, but there was no need to have the
server attempt to send so much unnecessary data to the attached admin
consoles.

9. [SERVER] Shutdown sequence refinements

In the process of doing work for #8, above, I introduced a class cast
exception into some of the admin console code that resulted in the
server getting stuck during shutdown.  In fixing this, I revised the
shutdown sequence to be more verbose and to ensure that arbitrary
exceptions thrown late in the server shutdown process won't prevent
the server from exiting.

10. [CLIENT] Cleanups to the GUI calendar widget

I did a cleanup job on the JpanelCalendar class, to make the thing
easier to read and maintain, and to improve its layout in the face of
window sizing.  When viewing, rather than editing, objects' expiration
and removal date fields, the client will now show a full-size,
display-only calendar rather than the cheap text label that it used to
show in that circumstance.  When editing these fields, the old label
showing what the expiration/removal field was set to in the calendar
panel can now be used to jump the calendar back to the calendar page
containing the selected time.

11. [SERVER/XML] Liberalized the server's behavior with xmlclient deletions

The server was previously rejecting any XML transaction that attempted
to delete a non-existent object.  Now, the server will merely transmit
a warning to the xmlclient that an attempt to delete a non-existent
object was attempted, without completely rejecting the valid deletion
commands.

This change was made in response to a comment by Gaurav Bhargava at
Villanova University.

12. [SERVER] Fixed slight inconsistency in object link logging

The Ganymede server's logging and email notification logic had a small
inconsistency in it.  When logging changes to invid (object reference)
fields, the Ganymede server logged old values removed using the label
for those objects that pertained at the end of the transaction in
question.  If an object's name changed during the transaction due to
other editing, something like the following would be logged:

 objectchanged
 -------------

 I.P. Network ATL - Outgoing Inet Access Only, <267:96> was modified.

 Fields changed:

 Records
        Deleted: [10.5.16.132  - 08:00:20:9a:16:f2]

 objectchanged
 -------------

 I.P. Network ADG/OMG - Outgoing Inet Access Only, <267:79> was modified.

 Fields changed:

 Records
        Added: [10.5.16.132  - 08:00:20:9a:16:f2]

 objectchanged
 -------------

 System corona's Embedded System Interface, '[10.5.16.132  -
 08:00:20:9a:16:f2]',  <265:970> was modified.

 Fields changed:

 I.P. Address
        Old: 10.8.16.186
        New: 10.5.16.132
 Ethernet Info
        Old: 8-0-20-9a-16-f2
        New: 08:00:20:9a:16:f2
 I.P. Network
        Old: ATL - Outgoing Inet Access Only
        New: ADG/OMG - Outgoing Inet Access Only

In this case, the 'Records Deleted' part of the 'ATL - Outgoing Inet
Access Only' objectchanged log entry is showing the new name for the
interface that was deleted, even though when that interface was in the
ATL network, it had a different name.

This has been changed so that whenever an object is removed from
an invid field, the removal will be logged using the object's name
as of the start of the transaction.  This would make the above logging
sequence look like this:

 objectchanged
 -------------

 I.P. Network ATL - Outgoing Inet Access Only, <267:96> was modified.

 Fields changed:

 Records
        Deleted: [10.8.16.186  - 8-0-20-9a-16-f2]

 objectchanged
 -------------

 I.P. Network ADG/OMG - Outgoing Inet Access Only, <267:79> was modified.

 Fields changed:

 Records
        Added: [10.5.16.132  - 08:00:20:9a:16:f2]

 objectchanged
 -------------

 System corona's Embedded System Interface, '[10.5.16.132  -
 08:00:20:9a:16:f2]',  <265:970> was modified.

 Fields changed:

 I.P. Address
        Old: 10.8.16.186
        New: 10.5.16.132
 Ethernet Info
        Old: 8-0-20-9a-16-f2
        New: 08:00:20:9a:16:f2
 I.P. Network
        Old: ATL - Outgoing Inet Access Only
        New: ADG/OMG - Outgoing Inet Access Only

This misfeature was reported by Gil Kloepfer, here at ARL.

13. [CLIENT] Put warning dialog up when editing inactivated objects

A good number of admins here at ARL have ran into problems when they
tried to manually edit inactivated users, rather than using the
reactivation function that is available on the object tree right-click
menu for inactivated objects.  The problem is that most admins here
just set the password and commit the transaction, without realizing
that they need to clear the Removal Date field.  Our server is
configured to provide a 'reactivate' wizard when users use the
right-click reactivation function, but users too often don't realize
or remember that that function exists.

In response to this, I have implemented a dialog box which will pop up
if an admin attempts to edit an inactivated object, prompting the user
to attempt to use the server's reactivation wizard rather than
directly editing the object.

This is pretty clean, but this will put more pressure on schema kit
developers to insure that if they want an object type to be able to be
inactivated rather than simply removed, they really will need to
provide a reactivation function.

14. [CLIENT] Synchronized the transaction cancel button to avoid exceptions

The gui client now synchronizes its cancel button, to ensure that all
object windows (which load data from the server on their own threads)
have stopped loading before the client tells the server it wants its
transaction cancelled.

Previously, some of the client's loading threads could briefly race,
continuing to try and call data loading methods on the server after
the client's main GUI thread had already cancelled the transaction,
which could cause the server to throw an exception down to the client.

This change just serializes transaction cancellation on the client
side a bit better.

15. [CLIENT, ADMIN CONSOLE] Miscellaneous GUI cleanups, improvements

Quite a varied collection of small improvements in the GUI client and
the admin console.

In the client, the history panel now uses the OneTouchExpandable
option in its splitter panel, and sets its initial size so all the
historical fields are visible.

The expiration and removal tabs now show the appropriate icon from the
tree when they are present.  This will hopefully make it less likely
that users will miss noticing that these tabs are present when editing
certain user accounts.

The client's vertical splitter supports the OneTouchExpandable option
as well, making it possible to entirely hide the tree if more space is
needed for showing object windows.

The admin console's layout has changed very slightly, with a splitter
bar added between the log panel and the bottom status tables panel,
and the 'clear log' button removed and put in the Control menu, where
it won't take up screen real estate.

16. [SERVER] Better exception handling in checkpointed code

The object linking code was not properly catching exceptions that
might be thrown by plug-in code, and so certain object linking and
unlinking operations could leave checkpoints on the server's
checkpoint stack.  Due to the thread identity checking logic in the
DBEditSet checkpointing routines, this could cause a session on the
server to refuse any further checkpointing, causing the user to have
to cancel the transaction.

Now, if any user-level code throws exceptions during object linking or
unlinking, the checkpoint stack will properly be cleared.

This is in response to a bug report by Gil Kloepfer.

17. [CLIENT] Fixed the clients' failure to terminate on window closure

The gui client and admin console were not properly terminating the JVM
when the clients were run as applications rather than applets, and the
user closed the window using the operating system controls rather than
the 'quit' button.

The login box will now properly terminate the JVM if the close window
button is used.

This is in response to a bug report by Gil Kloepfer.

18. [SERVER] Cleanups and refinements to DBEditSet class

The transaction commit() method code in DBEditSet has been radically
reworked.  The biggest part of this reworking is a simple refactoring
to make the code far simpler to read, but a number of reliability
improvements were made in the process, and a few extremely unlikely
yet nonetheless possible security race conditions were fixed.

In addition, a lot of DBEditSet member fields have been made private,
and special accessors have been added and method calls altered where
needed to allow for interaction with other classes.

19. [SERVER] Slight API change for DBEditObject subclasses

Previously, the DBEditObject class held a 'committing' boolean flag
that needed to be set and cleared in specific methods in order to
coordinate field locking during the transaction commit process.

The problem is that access to this boolean flag was unsynchronized,
making it possible in theory for a system running the Ganymede server
with multiple threads on multiple CPU's to allow changes to be made to
fields after the commit process has started for a given object.

This race condition is extremely unlikely, as it would depend on a
system having a non-coherent memory access model in the absence of
specific object synchronization, but the Java Language Specification
does allow for such implementations.

In any case, the committing boolean has been replaced by an
arlut.csd.Util.booleanSemaphore object, and made private.  Subclasses
of DBEditObject which override commitPhase1() and/or release() should
now use the DBEditObject setCommitting() method to set or clear that
flag.

None of the custom object plugins or schema kits that we have provided
overrided those two methods, so I think it'll be unlikely that this
API change will affect people, but if you are not able to recompile
your schema kit and/or plugins with this version of the server, you
may need to look at this issue.

20. [SERVER] Shutdown Expedited

The GanymedeBuilderTask base class will now inhibit any builder tasks
from running at all once a server shutdown command has been issued.
Previously, the server shutdown process only blocked phase 2 builds.
This change makes it so that the server won't bother trying to write
out data files that it won't do anything with before shutdown.

21. [SERVER, ADMIN CONSOLE] Added support for Apache md5 hash format

The Ganymede PasswordDBFile class now has support for working with
passwords hashed using the variant Md5Crypt() algorithm that the
Apache web server supports for its htpasswd files.  Implementing this
support involved changes to the Ganymede database file format (now up
to file format 2.4), along with changes to the RMI interface
definition for the schema editor and for the password field interface.

Because of these changes, Ganymede 1.0.10 will not work properly with
an admin console or client from Ganymede 1.0.9 or earlier.

-------------------- Changes from 1.0.8 to 1.0.9 -------------------

RELEASE DATE: December 4, 2001

1. [SERVER] Fixed xml parser error introduced in 1.0.8

The server side XML parsing code in Ganymede 1.0.8 had a major bug.
The 'addIfNotPresent' vector field mode was made the default, but
support for handling the addIfNotPresent mode was not implemented for
invid vector fields.  Loading data into invid vector fields silently
failed.

This has been fixed, and the xml file processing now works properly
for all kinds of vector fields.

In addition, the server's xml code now reports whenever it sees an
<invid> element that it is not able to resolve during the data
consolidation phase.

Apologies to all and thanks to Gaurav Bhargava at Villanova University
for reporting this brown paper bag bug.

-------------------- Changes from 1.0.7 to 1.0.8 -------------------

RELEASE DATE: November 16, 2001

1. [SERVER] Fixed Invid, String, IP array fields disk i/o to handle large arrays

The InvidDBField, StringDBField, and IPDBField classes support array
values for object pointeers, strings, and IP addresses, respectively.

Prior to this release, the length of these arrays were limited to 32k
entries, as a 16 bit signed short was used to store the array length
on disk.

This has now been changed to a 32 bit signed int, so that Ganymede's
disk format can now handle 2 billion items per array.

That doesn't mean that the server will be able to behave well with 2
billion items in a String field, but there at least won't be an
artificial cap of 32k items anymore.  This will be especially valuable
in that owner groups will no longer be limited to 32k items per group.

This is an incompatible change with versions 1.0.7 and earlier of
the Ganymede server.  ganymede.db and journal files written
by Ganymede 1.0.8 and later won't be readable by Ganymede 1.0.7 and
earlier.

2. [SERVER] Fixed scheduler to allow enabling/disabling of on demand tasks

The Ganymede scheduler code to handle task enabling/disabling from the
admin console had a bug that prevented it from being able to handle on
demand tasks.  This has now been fixed, and any kind of task should be
able to be disabled.

This can be useful if you want to commit changes to the Ganymede
database without having the server trigger the builder tasks.

3. [CLIENT] Fixed exception in timed out querybox

The querybox code was improperly retaining a reference to a
client-side cache structure, making the re-use of the querybox
impossible after the client was forcibly timed out.

The querybox now takes care to get an updated reference to the caching
structures it needs, so that re-use of the client querybox after
a forced time out will not cause an exception to be thrown.

4. [SERVER/XML] Improved password importing support in the XML system

The server's xmlfield code was missing support for processing the
"lanman" and "ntmd4" XML attributes for the <password> element in the
Ganymede XML file format when loading data.  This support has now been
added, and it should now be possible to write scripts to load user
accounts, passwords included, from a Samba password file.

In addition, the XML client can now simultaneously load passwords in
several hash formats, so if you already have a Samba file with LANMAN
and NTMD4 password hashes along with your UNIX passwd file's crypt and
md5 crypt passwords, you can load all four hashes into accounts in
Ganymede at once.

Ganymede obviously can't verify that the hashes match in this case,
but if you already have been synchronizing passwords to different hash
formats, this will let Ganymede support your accounts without
requiring that your current passwords be available in plaintext.

This entails a slightly incompatible change in the RMI API between the
client and the server, so the Ganymede 1.0.8 and later xmlclients
should not be used with a 1.0.7 or earlier server.

5. [CLIENT/SERVER] Fixed query filtering in Ganymede client and server

The Ganymede client has for a long time had a (nonworking) 'Filter
Query' menu option that was intended to allow users in many owner
groups (including supergash) to set a filter so that only objects
owned by a given list of owner groups would appear in the tree.

It works now, but it's a touch inelegant.  Ultimately, the answer to
overloading in the tree is to force a hierarchical data
representation, but until such time as we can rework the Ganymede data
model that deeply, you can apply and clear owner filters using
the "Set Owner Filter" menu item from the client's File menu, and the
tree and object choice lists in objects being edited will be
dynamically updated.

Voila, 'supergash' level power while still being able to 'drill down'
on a reasonable subset of the server's data.

This entails a slightly incompatible change in the RMI API between the
client and the server, so the Ganymede 1.0.8 and later clients should
not be used with a 1.0.7 or earlier server.

6. [SERVER] Fixed nullPointerException throw on empty regexp string

Fixed a couple of places in DBQueryHandler that could result in a
null or empty string being passed down to the gnu.regexp.re constructor.

7. [XML/SERVER] Improved semantics of XML vector field handling

Previously, the default mode for vector fields in Ganymede's XML
support was "add".  That is, the following XML snippet will
cause "jonabbey" to be added to the Aliases field in the broccol
user object.

<object type="User" id="broccol">
  <Aliases>
     <string val="jonabbey"/>
  </Aliases>
</object>

This is true even if "jonabbey" was already listed in the Aliases
field.

Now, a new vector field mode has been added, "addIfNotPresent".
This mode will now be the default when a vector handling mode
is not specified.  In other words, the above snippet is equivalent
to

<object type="User" id="broccol">
  <Aliases>
     <addIfNotPresent>
       <string val="jonabbey"/>
     </addIfNotPresent>
  </Aliases>
</object>

Note that "add" mode for Invid fields was always handled as if it were
"addIfNotPresent", as the InvidDBField.addElements() method
automatically filters out duplicates.

I have updated the XML docs to reflect this change.

In addition, the <set> mode can now properly be used to remove
redundant copies of strings from a vector string field.. this did
not work properly before.

In other words, if we have an object with the following values

<object type="User" id="broccol">
  <Aliases>
     <string val="jonabbey"/>
     <string val="jonabbey"/>
  </Aliases>
</object>

then

<object type="User" id="broccol">
  <Aliases>
     <set>
       <string val="jonabbey"/>
     </set>
  </Aliases>
</object>

will properly remove one of the "jonabbey" strings.  Before, the
XML field handling logic simply saw that the "jonabbey" string
was already in the vector (at least) once, and so didn't bother
to remove the redundant copy.

8. [ADMIN CONSOLE/SERVER] More fixes to prevent memory pinning with schema editor

The admin console's schema editing code has continued to cause memory
on the server to be pinned after the schema editor window has been
closed.  I've added a great deal of manual object dissolution and
reference clearing to prevent memory pinning in the server.

More importantly, the admin console's schema editor code now properly
handles clearing the server-side schema editor context if the window
is closed through the window system's controls rather than through the
'commit' or 'cancel' buttons.

-------------------- Changes from 1.0.6 to 1.0.7 -------------------

RELEASE DATE: October 11, 2001

1. [SERVER] Deleted objects now have all of their fields logged

The Ganymede server has not been logging the status of objects when
they are deleted, which has made it in some cases impossible to
determine what was in an object when it was deleted.  In theory,
today, all objects created in the server will be logged with their
initial state as well as all changes made, but at ARL we had a great
deal of objects that were imported using the old 'directLoader' hack,
without any logging done.

The real advantage to this for other users will not so much be
for scanning the log file as it will be to make the object deletion
email more informative.

2. [SERVER] Better error message on field creation failure

The server now gives a better error message if a user tries to link an
object to a (previously undefined) field that the user does not have
permission to create.

3. [SERVER] Made sessions smarter about refusing to edit objects during commit

The DBSession class has been modified to disallow editing of new
objects during a transaction's commit or abort process.  This is to
help make sure that any code during the commitPhase1() and
commitPhase2() DBEditObject customization methods that seeks to bring
new objects into the transaction after the commit has already started
will trigger a runtimeException, rather than silently confusing the
commit logic.

This change shouldn't have any effect on any code provided with the
Ganymede distribution or the userKit, but might force an error in
custom code if the customizers were doing some sort of dangerous
activities during the commit process.

4. [SERVER] Amended wizard shutdown process

The GanymediatorWizard class had a loophole that would cause wizards
to not be unregistered from a GanymedeSession if the wizard's state
had been forced to DONE outside of the GanymediatorWizard.unregister()
method.  This has been fixed so that any time a wizard goes through
returns a success() or fail() result it will be unregistered.

This fix took care of a bug in the laboratory's GASHARL schema kit
operations, but is very unlikely to be affecting anyone out there
unless you've done just the right (wrong) kind of custom schema
coding.

5. [SERVER] Put loopback binding check into server startup

The server now checks to make sure that it is not being bound
to a loopback address on startup, which would prevent clients from
being able to communicate with the server from remote systems.

The server will now generate a message of the form

 ** Error **

  Both the system hostname (hostname) and the ganymede.serverhost
  definition (hostname) resolve to the 127.0.0.1 loopback address

  The Ganymede server must have an externally accessible IP address or
  else clients will not be able to communicate with the Ganymede server
  from other than localhost.

  If you really want to be only usable for localhost, edit the runServer
  script to use the -forcelocalhost option.

if the server is not able to figure out a hostname that will resolve
to a network-accessible IP address.

RedHat Linux 7 systems are particularly prone to this, as the default
configuration for such systems includes the system's hostname in
/etc/hosts as mapping to 127.0.0.1, and the system 'hostname' function
returns by default a non-fully-qualified hostname.  The combination
of these two conditions resulted in the Ganymede server registering
itself with the rmiregistry process as being on IP address 127.0.0.1.

This change was in response to a problem report from Steve Lemons
(steve.lemons@arrisi.com).

6. [SERVER] Objects in the process of being deleted may not be linked to

The server's object binding logic had a bug which would allow an
object which was being deleted to be improperly linked into an Invid
field in another object.

This occurred in a bit of custom code in the GASHARL schema kit where
unlinking a remote object in a given context resulted in the object
being deleted by the custom server-side code.  The client, however,
showed the unlinked object as a valid choice to be added back into the
link field.  If the user tried to put the (now deleted) object back
into the field, the link was made, even though the object was marked
for deletion when the transaction committed.

This resulted in an Invid field containing an Invid pointer to an
object which no longer existed, which is a bug.

The server's InvidDBField logic is now smart enough to forbid linking
to an object which has been marked for deletion by the transaction.

This was a very obscure integrity violation vulnerability.  The server
and client have always cooperated in other ways to prevent this
condition from arising.  It was only due to some odd custom coding
in the GASHARL schema kit that this vulnerability was revealed.

7. [CLIENT] Optimized out redundant network calls in the GUI client

The Ganymede GUI client has been optimized to remove a great many
redundant RMI calls when viewing objects on the server.  The client
can now display objects with only about half the RMI calls that the
client used previously.  This will speed up the client noticeably on
slow network links, and will reduce loading on the server slightly.

-------------------- Changes from 1.0.5 to 1.0.6 -------------------

RELEASE DATE: August 15, 2001

1. [SERVER, ADMIN CONSOLE] Fixed delayed memory release for schema editing

The Ganymede server wasn't properly clearing references to a bunch of
temporary data structures when a schema edit was cancelled by the
admin console.  This had the effect of increasing our server's memory
footprint by a couple hundred kilobytes or so until such time as the
admin console was stopped and restarted.

2. [WEBACCESS] Fixed ganypass.pl to allow < and > chars in passwords

The ganypass.pl script, which provides a CGI interface to the
xmlclient for changing users' passwords, was improperly failing to
convert < and > to the XML entity escapes &lt; and &gt;, which could
cause the server's XML parser to reject password changes that involved
those characters.

3. [SERVER] Fixed several journal processing bugs

Yowtch.  Several nasty bugs that were lurking here.  None of these
bugs caused any problems if the server was shut down cleanly and
restarted, but if the server was abnormally terminated, the following
bugs caused problems during transaction recovery on startup.

Firstly, the server had a bug that prevented successful startup if the
server was killed without getting a chance to consolidate the database
into a new ganymede.db file, and if one of the transactions in the
journal file involved an deleted object that had one or more
namespace-constrained database fields.

This was due to an attempt to use Ganymede.internalSession before that
variable was defined.  This has been fixed by making the
DBNameSpaceHandle.getField() method look up objects directly out of
the DBStore tables without using a GanymedeSession, if needed.

Thanks to Gaurav Bhargava at Villanova University for reporting
this first bug.

Secondly, there was a bad bug that caused transaction processing in
the journal handling code to repeatedly process object records in the
journal.  The journal processing code was not clearing the entry
vector at the end of each transaction's processing, which caused the
server to do a lot of repetitive work and to generate misleading
warnings about 'objects already deleted' in the object deletion case.
Ugh.

Thirdly, there was a bug, subtle in causation but HUMONGOUSLY AWFUL in
effect, which caused the server to *eat* changed scalar fields when
processing journal entries.  This bug has been in Ganymede since
1.0pre1, when I moved field storage in DBObject to a packed hash as a
memory optimization.  The replaceField() code that edited the packed
hash to replace fields with the versions noted in the journal
transaction entries silently failed if it could not find the field it
was looking for, causing the changed fields to be simply dropped on
the floor.  That's what I get for recklessly optimizing my data
structures.

Data loss bugs in Ganymede have been rare, thankfully, but this was an
ugly one, and it bit me when I was testing the fixes to the first two
journal bugs.  My apologies if it bit anyone else out there.

The good news is that I've cleaned up the journal loading code a fair
bit, and that I have significantly improved the debug instrumentation
in this section of the code.

-------------------- Changes from 1.04 to 1.0.5 -------------------

RELEASE DATE: July 27, 2001

1. [DISTRIB] Fixed a number of bugs for JDK 1.4 compatibility

Fixed a few places that assumed SimpleTimeZone.getDefault()
would return an object of class SimpleTimeZone.

Fixed the jar packaging to satisfy Java 1.4's slightly pickier
serialization logic.

Because of these changes, the Ganymede 1.0.5 server will not be
compatible with clients from 1.04 or earlier.

2. [DISTRIB] Added support for using Java Web Start to launch clients

Sun's excellent Java Web Start provides the best way yet to launch the
Ganymede clients on Windows.  I have made changes to installWeb to
support the Java Web Start option in the Ganymede launcher.

3. [CLIENT] Fixed java.security.AccessControlException in client

On some versions of the JDK, the client, when running as an applet,
was refusing to allow changes to the animated build status icon in
response to RMI callbacks from the server.  This robbed the user of
valuable feedback and resulted in java.security.AccessControlException
messages being thrown on the server in response to the client
notification call.

I have added a new 'SecurityLaunderThread' to the client, which waits
for build status messages from the server and updates the build status
icon.  Since the SecurityLaunderThread was started by the client and
not by the RMI system, it has proper permissions to play with the
Swing Event Queue, and everything works properly.

4. [SERVER, CLIENT] Reworked permissions handling for built-in fields

The Ganymede server previously did not separately track permission to
edit the "built-in" fields.  That is, the permissions editor in the
client did not allow per-field permissions to be set for the standard
fields that are automatically present in every object, including the
Owner List, Expiration Date, Removal Date, and Notes fields.  Instead
of allowing access controls for these fields to be set in the standard
permission matrices in the Role objects, all of these fields were
accessible with the same permission as the containing object itself.
If a user had permission to edit an object, that user automatically
had permission to edit these four system fields.

This turned out to be a security hole, because there are many cases in
which it is desirable to allow an object to be minimally editable by
non-privileged users (so that end users can change their own
passwords, for instance), where such users may not be trusted to edit
the owner list, notes, expiration date, and removal date fields.

The client's permissions editor and the server now allow explicit
setting of permissions for the Owner List, Notes, Removal, and
Expiration Fields.  The four 'historical' fields used to track object
creation and modification timestamps and admins are never allowed to
be edited by the user, and so are still left out of the permissions
editor.

In the process of doing this work, a few potential vulnerabilities
that might have been exploitable by a rewritten Ganymede client were
closed.  The Ganymede philosophy is that the server should not trust
the client for anything, and a few security assumptions have now been
made explicit.

The server now makes several checks and forbids the granting of
privileges that would be liable to break the Ganymede security
model.  For instance, non-privileged end users will never be allowed
to edit the owner list field for objects, regardless of whatever
permission for the Owner List field might be granted in the server's
Default role.

5. [CLIENT] Improved GUI cleanup

Miklos Muller reported that the client was leaving the permissions
editor up if the server forced a client disconnect due to a timeout.

I looked into things, and the code for handling resource cleanup on
window close in the client was still very messy, despite Bug fix #11
for release 1.0, below.

Things are a good bit cleaner in the client, now, and all popup
permission editor windows should now be put away properly when the
window that the permission editor was launched from is closed.

6. [DISTRIB] Changed numbering scheme

I realized that if I have more than 6 more 1.0x releases, I'll
suddenly be at 1.1, which I don't want to do by accident.  So, we're
going from 1.04 to 1.0.5.

7. [DOCUMENTATION] Added an upgrade guide, improved XML docs

I've written a short upgrade guide that explains how to go about
upgrading the server, schema kit, and clients when a new version of
Ganymede comes out.

I've also done a fair amount of rework of the xmlclient documentation.

8. [SERVER] Made the server use a default address for email if necessary

The Ganymede server typically sends mail out for actions taken by
admins on behalf of the admin's email address.  That is, Admin Persona
objects have an 'email address' field, and when the Ganymede server
sends out mail in response to that admin's actions, mail from Ganymede
is sent with the admin's address used as the 'From:' header.

Previously, if Ganymede could not determine an email address to use
for an admin, it would try to send mail out with a null return
address, which caused the mail attempt to fail.

Martin Vogt <vogt@itwm.uni-kl.de> reported this bug, and the proper
fix for it, back in March of 2000, but I didn't realize how likely
this problem was to affect new users until I did some testing
of a fresh install on a new system.

9. [SERVER] Fixed field/object permission interaction

Previously, the GanymedeSession.getPerm() method filtered per-field
permission bits against the permission bits for the object as a whole.

This makes sense for the view and edit bits, as you can't view or edit
a field if you can't view or edit the object it is contained in, but
it does not make sense for the create bit, as it might well be
appropriate to be able to create a new field in a pre-existing object
that you can edit, even if you don't have the right to create a new
object of the type.

The server is now smart enough to avoid filtering the create bit
for a field against the create bit for its object.

-------------------- Changes from 1.03 to 1.04 -------------------

RELEASE DATE: July 9, 2001

1. [SERVER] Fixed NullPointerException thrown in XML Login path

The Ganymede server's xmlLogin() method was throwing a
NullPointerException if a user submitted an XML file through the XML
client with an invalid username and/or password.  The server now
avoids having that exception thrown by explicitly checking the success
of the username/password login before trying to construct a
GanymedeXMLSession.

2. [CLIENT] Fixed date reversion in the pop-up calendar GUI component

I fixed the pop-up calendar's reversion handling for when the user
attempts to change the date/time in a date field and chooses a date
that is out of range.

3. [CLIENT] Enhanced list GUI boxes in the client

The client's JstringListBox and StringSelector components have
had their behavior enhanced, with a 'select all' button added
at the top of the StringSelector, and with the JstringListBox
keeping all items added selected, to allow easy undo of
item moves.

The JstringListBox is now capable of supporting item dragging
in the list.

4. [CLIENT] Enhanced query box

The query box now uses a pair of StringSelectors to represent
the list of field choices rather than a bunch of checkboxes.

5. [SERVER] Fixed namespace constraint handling during schema editing

Previously, it was possible during schema editing to set a namespace
constraint on a field that contained data in violation of that
constraint.  Now the schema editor will not allow any schema edits to
be made that would allow a namespace constraint to be placed on a
field that contains data in violation of such a uniqueness constraint.

With this change, it should now be impossible to put the server into a
state where fields hold data in contradiction to an expressed
uniqueness constraint.

Thanks to Gaurav Bhargava at Villanova University for reporting
this bug.

-------------------- Changes from 1.02 to 1.03 -------------------

RELEASE DATE: June 22, 2001

1. [SERVER] Fixed non-event related mail sending in DBLog

The DBLog class had an error, effectively, that prevented 'mailout'
class mail from being sent, although it was logged.

Added a 'sendMail' method to DBLog to simplify the sending
of mail by plug-in code and custom tasks.

2. [SERVER] Fixed startup default permissions for non-supergash users

Miklos Muller, mmuller@lbcons.net, reported a bug that caused Ganymede
to not allow non-supergash users to login to a freshly initialized
Ganymede server.  Editing the Default Role object's Default Access
Bits field, works around the problem, but unless this is done, an
exception will be thrown in GanymedeSession.resetDefaultPerms().

I have corrected the GanymedeSession resaultDefaultPerms() method so
that it will not freak out when it doesn't find any definition for the
Default Access Bits field in the Default Role object.

This bug was avoided by users who used the userKit's loader.pl script
to initialize the Ganymede database.

3. [SERVER] Fixed infinite loop bug in XML parser

The XML parsing code had a bug that would leave it looping infinitely
if the XML stream was closed before completion of parsing, either due
to an incomplete XML file being fed to the xmlclient, or through the
xmlclient dying suddenly.  This didn't break the server, but it did
take up a lot of processor time, and kept all of the memory associated
with an XML session pinned in the server's heap.

4. [SERVER] Fixed default label field for persona objects

The persona object schema definition was improperly defaulted to use
the older 'name' field as the label, rather than the composite 'label'
field.  This could cause errors when trying to login with a newly
created persona.

This bug was reported by Miklos Muller as well.

-------------------- Changes from 1.01 to 1.02 -------------------

RELEASE DATE: June 7, 2001

1. [SERVER] Fixed an incompatibility with JDK 1.1 on the server

The server code included a use of a generic 'add()' method
on the java.util.Vector class that didn't get added until Java
1.2.  Replaced it with addElement().

I'm going to have to put up some sort of nightly build directory until
I can get network CVS access going, putting out new releases like this
is silly. ;-)

-------------------- Changes from 1.0 to 1.01 -------------------

RELEASE DATE: June 7, 2001

1. [SERVER] Fixed an IllegalArgumentException with empty namespaces

Michael Jung (mikej@confluenttech.com) reported an exception thrown
during startup.  It turned out to be due to some of the checkpoint
code trying to deal with an empty namespace.  Easily fixed, so
I did.

-------------------- Changes from 1.0pre1 to 1.0 -------------------

RELEASE DATE: May 31, 2001

1.[SERVER] Made server do better server-side cleanup for stuck clients

The RMI system has the ability to get into a state where a TCP/IP
handling thread waits indefinitely for communication from a client.
Prior to this fix, the server would wait forever for a dedicated
thread to try and notify the client that it is being forced off before
handling the server-side client logout housekeeping.

Now, the forceOff() method in GanymedeSession will expunge the client
from the server on the server-side before notifying the client that it
has been disconnected.

Note that this fix may still leave those dedicated background threads
stuck in the JVM trying to finish reading from a socket from the
(presumably dead) client.  I can't do much about this other than wait
for Sun to fix their RMI implementation so that RMI reads on a socket
will eventually time out.  In my experience, threads hanging due to a
partial socket read is reasonably rare, so I'm not overly worried
about hanging disconnect threads clogging up the server.

2. [SERVER, ADMIN CONSOLE] Added support for memory status tracking

The server and admin console now collaborate to display the amount of
free memory and the total amount of memory taken up by the server, to
be updated once a minute by a new Memory Status Updater task.

3. [SERVER] Added the garbage collection task back in

I added back in the on-demand garbage collection task, which is an
interesting thing to play with now that the admin console shows memory
usage on the server.  It looks like System.gc() doesn't do much of
anything under the HotSpot server VM.  Not too surprising, as the
HotSpot server VM is already optimized for continual, incremental
garbage collection, but having the ability to launch a garbage
collection task from the admin console's task monitor may be helpful
on some VM's.

4. [SERVER] Cleaned up setLastEvent() usage in GanymedeSession

I cleaned up usage of the diagnostic setLastEvent() stuff in
GanymedeSession.  Viewing or editing objects that contain embedded
objects will no longer cause notification for the embedded stuff to be
sent to the client.  In addition, all setLastEvent()-style
notification for non-interactive sessions has been turned off, to
reduce the overhead during XML bulk-loading.

5. [CLIENT] Informative dialogs are no longer modal

The Ganymede GUI client was occasionally freezing up on start-up if
the user attempted to bring up a toolbar dialog at the same time the
client was working in the background to bring up the MOTD dialog.

This was a result of the HTML messageDialog class used for the MOTD,
Credits, and 'About Ganymede' dialogs being configured to lock out any
other GUI interactions until those dialogs were released.

I don't think there was ever any compelling reason to make these
dialogs modal, so I have made them not be so.  There's now a small
risk that a user might ignore the MOTD dialog without dismissing it,
but that's a far less serious outcome than having the client lock up.

Thanks to Tom Embleton for bringing this back to my attention.

6. [SERVER] Memory leaks fixed

Once I had the admin console tracking memory usage, I quickly saw that
the Ganymede server was leaking memory over time.  At ARL, our
server's heap was starting out at about 10 megs, and after 8-9 days,
the heap had nearly doubled in size.  I ran the Ganymede server
through a profiler for a couple of weeks and identified several places
where bits of code were holding on to references to things that were
no longer needed.  Some of these were really obscure.. things I would
*never* have found were it not for the assistance of the profiler.

After a month or so of analysis and code work, the Ganymede server's
memory usage has been brought under far better control.  I've not yet
had a chance to run the newly updated server for long enough to judge
whether the heap usage is truly stable over the space of weeks and
months, but I can say that the server's memory growth rate appears to
have dropped to less than a tenth of what it was before, at worst.

7. [SERVER] File descriptor leak fixed

The arlut.csd.ganymede.util.zipIt class was neglecting to close files
that were being read in for adding to a zip file.  This caused the
Ganymede server to run out of file descriptors during nightly
archives.  Apparently this problem was masked on earlier versions of
the JVM by the FileInputStream's finalize() method doing clean-up.
Since finalize() is not guaranteed to ever be called, moving the
Ganymede server to a new JVM made file descriptor starvation start
appearing.

I have now fixed the code so that finalization is no longer required
for proper resource release, and everything is working well.

8. [SERVER] Added cryptographic hash support for Samba

The server now includes classes to support both forms of password
hashing used in Samba's encrypted password file.  This will simplify
support of Samba in userKit 1.0, as the NTSambaBuilderTask will now be
able to emit a file with all of the plaintext passwords pre-hashed,
rather than depending on a hard-to-install Expect script to drive
smbpasswd to insert updates into the Samba password file.

9. [SERVER/ADMIN CONSOLE] Added a 'force rebuild' menu item to admin console

The admin console now has the ability to force a mandatory rebuild of
all external data.  Previously, it was possible to manually run a builder
task from the task monitor table through the right-click popup menu, but
that builder task would typically not actually write out data if the
object bases in Ganymede that contained the data had not been changed
since the last run of that builder task.

Now the admin console has a 'force build' option under its Control
menu which forces the running of all builder tasks registered in the
system, and that forces those builder tasks to treat all object bases
as having been dirtied since the last run of the builder tasks.  This
allows an admin to force a dump of all external data files generated
by the builder tasks, and the consequent execution of all external
update scripts.

10. [ADMIN CONSOLE] Improved network error handling

The admin console would previously force a disconnect from the
server in the event that any network operation returned a RemoteException.

Unfortunately, a RemoteException may be returned even if the network
connection remains good.  Any exception that occurs on the server
in response to an RMI call results in a RemoteException.  Having
the admin console do a force disconnect on any error was a bit harsh.

Now, the admin console will append the stack trace information for
any RMI remote exception to the console's log area, where it may be
seen.

11. [CLIENT] Cleaned up resource retention on the Ganymede GUI client

The Ganymede client was doing very little in the way of cleaning up
allocated structures.  So little, in fact, that it was retaining
references to remotely exported fields from the server, even if the
client had been logged out.

The client has now had some extensive work done to disassemble object
structures when appropriate.  This will very significantly reduce
memory loading on the client.

12. [CLIENT] Made the client's query window non-modal

The client's query window's modality was causing the JpopUpCalendar to
lock up when someone tried to edit the date and time of a time field
search.  Making the query window non-modal takes care of this.

In addition, I reworked the way that threading is handled for the
'waiting for query' animation and table window display to make sure
rendering operations for them are handled on the GUI thread, which
should help avoid some exception traces that were occasionally
being thrown on the client.

13. [GUI, SCHEMA EDITOR] Fixed a race condition in the tree node drag code

The schema editor provides support for re-ordering fields within an
object, or objects within an object category folder.  The tree GUI
component had a race condition, however.  Depending on the rate at
which the Java GUI thread produced mouse motion events and the rate at
which the user moved the mouse, node dragging could result in the
wrong node being moved.

This of course, made the schema editor amazingly frustrating to try
and use if conditions allowed this bug to be expressed.  This bug has
been around for years, transiently appearing and disappearing
according to the speed of GUI thread processing on the JVM on which
the schema editor was run.  I am *very* happy to have it fixed at
last.

14. [SERVER] Fixed transient logic errors in permission editing

There were a few places in the Ganymede server where the Ganymede
logic was invalidly depending on the post-serialization initialization
of transient fields in objects passed over RMI.  This is not in
keeping with the way that Java serialization actually works, and in
one instance this logic flaw resulted in changes made to permission
matrices by editing Role objects not taking effect until the server
was stopped and restarted.

Nasty, nasty bug, and a pain to track down.

I've fixed this, and audited all of the server code to insure that
there are no remaining attempted dependencies on post-serialization
initialization of transient fields.

15. [SERVER XML] Fixed missing attribute error reporting in XML parser

The Ganymede server's XML parsing code is designed to report to the
xmlclient a record of the last 30 XML elements parsed prior to
encountering an XML validity error (such as an invalid character,
say).  Unfortunately, the class that represents XML entities (such as
'<object type="I.P._Network" id="CSD - No Inet Access">') had a bug in
it's toString() method that caused every other attribute to be skipped
over when regenerating a string representation for the node.

This was a small bug, but made it maddeningly difficult to track down
syntax errors in XML files created for use with Ganymede, if the odd
numbered attributes in the nodes were not unique enough to identify
position in the XML input file.

16. [CLIENT] Tweaked the Ganymede client's GUI

I have tweaked a number of aspects of the Ganymede client's GUI to
make using the Ganymede client a bit more convenient, mostly in the
way of reducing the default length of fields in the object
viewing and editing windows.

17. [SERVER] Fixed DBNameSpace rollback()/unmark()/commit() bug

The server had a vulnerability in the DBNameSpace class that would
cause the commit() method to fail in circumstances where a namespace
rollback() operation had been followed by an unmark() operation on a
value previously allocated in that namespace.  The
DBNameSpace.commit() method could in this circumstance throw a
NullPointerException at a point in the transaction commit process that
would leave the transaction in an unfinished state, causing confusion
and mayhem all around.

This bug only manifested one time in the year and a half that Ganymede
has been in production here at ARL:UT, and appears to have manifested
at a time of heavy contention for resources in the namespace, such
that multiple transactions were trying to allocate namespace-guarded
unique values concurrently.

In response to this, the DBNameSpace.java code was largely rewritten.
The rewrite fixed this problem, and also significantly reduced the
memory requirements for unique value namespace handling.

18. [CLIENT, ADMIN CONSOLE] GUI tweaks for 1.0

Added 'About Ganymede' dialog to Admin Console.  Added keyboard
accelerators to Admin Console menus.  Tweaked menu names in Admin
Console to allow unique and intuitive accelerators.

Tweaked the messageDialog class so the client's MOTD can be dismissed
by hitting spacebar.  Tweaked the menus in object windows in the client,
added keyboard accelerators to menus.

Did some more cleanup work on the tree GUI component.  When you drag a
node in the schema editor, the node you are dragging will now be
highlighted in blue during the drag operation, in addition to the icon
drag that the tree has had.  Made right-clicking on nodes select the
node before popping up the pop-up menu, as a visual reference point.

19. [SERVER] Fixed DBObject synchronization

The server ran into a deadlock race condition where logins could not
be processed while a database dump was in progress at a certain point.

This arose out of the conversion to a packed array hash structure
for field storage in the DBObject class within the last year.  There
were a number of places in which I was sloppy with nested monitor
synchronization.

I have thoroughly reworked the DBObject synchronization logic
to avoid all such problems.

-------------------- Changes from 0.99.9 to 1.0pre1 -------------------

RELEASE DATE: January 30, 2001

1. [SERVER] Reworked server synchronization logic

The GanymedeServer.login() method was vulnerable to a nested monitor
deadlock due to a section being synchronized on both the server's
GanymedeServer object and its DBStore object.

Basically, I was getting way too fancy in synchronizing on various
server objects instead of defining a proper semaphore class and using
that.

I added the loginSemaphore class, which takes care of everything
without my having to worry about n^2 different possible code paths.

I modified the admin console to report the login semaphore count for
each log entry.. this makes it easy to see at a glance how many users
are logged on for each log entry, and to insure that the semaphore
count is incremented and decremented properly under all circumstances.

In the process of all this, I realized that I was unnecessarily doing
a notifyAll() every time I did a synchronized operation on the
server's DBStore object.. I had misunderstood when it was necessary to
do a notify when I originally wrote a lot of this code.

The sweepInvids() and checkInvids() methods have been fixed so that
they are no longer synchronized on GanymedeServer but they do now
establish a dump lock on the server's database, to insure that the
object walk is done safely.

2. [SERVER] Added getBackLinks() method to the DBObject class

Martin Schneider <martin@ira.uka.de> reported that he needed to be
able to get the Vector of Invid's pointing to a given object through
asymmetric links, in the same way that was possible by using the old
backlinks field.  To support this, I added a getBackLinks() method to
the DBObject class that directly consults the DBStore's backPointers
hash tree.

3. [ADMIN CONSOLE] Fixed admin console confusion with multiple logins

The admin console had a bug that would cause the console user table to
become confused if more than one user with the same persona name was
logged in at the same time on the Ganymede server.

This problem only happened when an admin logged into the server with
admin privileges more than one at a time.  Logging in without admin
privileges did not cause this problem, because the server assigns
unique usernames to logins, but the server does not guarantee unique
persona names.

The server now reports to the admin console each login's unique user
name, which the admin console uses for login tracking, but the admin
console will continue to display the user's (possibly non-unique)
persona name.

This one has bugged me for awhile, it's good to have this fixed.

This change (and change 5 below) change some of the semantics of the
admin console's operations, so you should not use an older version of
the Ganymede admin console with a 1.0pre1 server.

4. [CLIENT] Right clicking on a selected JTable row no longer de-selects the row

This one has bothered me for a long time, too.  I changed the
baseTable, rowTable, and tableCanvas classes in src/JTable so that
right-clicks on a table row won't ever deselect that row if already
selected.  Left-clicking on a selected row in the rowTable will still
deselect the row.

5. [ADMIN CONSOLE] Threaded out server-to-console communications

The Ganymede server has always communicated synchronously with any
attached admin consoles, which made the server effectively block if
the admin console became non-responsive without explicitly throwing a
RemoteException.  I noticed today that the admin console was blocking
the server from processing logins/logouts while my window manager on X
was waiting for me to place a new window, which of course is not
acceptable.

The Ganymede server now queues up notification events for the admin
console, and uses a separate thread for each admin console for
actually handling the server-to-console communications, using the new
serverAdminProxy class.  The Ganymede server will no longer sit around
waiting for the admin console when something happens, with the obvious
benefits resulting in speed and reliability on the server.

As a side benefit of the event queueing model, I've implemented event
coalescing and event replacement, so that if the Ganymede server is
generating admin console events faster than the console update thread
is getting them to the admin console, the server will combine and/or
drop old events, so that the server won't waste time sending events to
the console that are already out of date.

The admin console was changed a very little bit to support event
coalescing; the changeStatus() method in arlut.csd.ganymede.iAdmin now
depends on the server prefixing each status line with a time stamp,
rather than trying to time stamp each event from the server itself.

Good stuff, and very clean on the server.

6. [ADMIN CONSOLE] Fixed admin console shutdown process

I fixed some code in the server's GanymedeAdmin class so that an admin
console will stay open if a shutdown attempt by it was refused for
some reason.  Also fixed a bug in the admin console that made it fail
to really attempt to shutdown the server if it tried once and failed.

7. [SERVER] Cleaned up some locking and transaction logic

I went through and did a code review of all of the server's
transaction and locking classes, and removed a few bits of extraneous
stuff, fixed a couple of mistakes that could conceivably cause
problems on edge cases, and extensively commented code that I hadn't
really touched for a couple of years so that if I dive into it in 2002
I won't have to figure it all out from scratch again.

These changes probably won't be noticeable in operation unless I broke
something.

The DBLock code itself has been reworked a fair bit.  The DBLock code
no longer synchronizes directly on the Ganymede server's DBStore
object, but instead uses a DBLockSync object held in the DBStore
object.  This as well will help prevent deadlock problems, by limiting
the scope of synchronization on the DBStore object even further.

8. [SERVER] Made the database dumper task less timid

Due in part to the increased confidence I have in the server's thread
safety with the new login semaphore and my code review, I made the
database dumper task go ahead and do a database consolidation even if
users are logged into the server.

Previously, the database consolidation task could be postponed
indefinitely if users were logged in each time the dump task fired,
leading to a bigger journal file and longer startup times if the
server machine was rebooted.  Now the database dumper task will go
ahead and do the dump, even if it has to wait for transaction commits
to finish so it can assert a DBDumpLock on the database.

9. [SERVER/CLIENT] Added bulk add/remove methods to DBField

The Ganymede client has long supported moving large blocks of choices
from the unselected panel to the selected panel in vector invid and
string fields, using shift-click to do multiple selection.
Unfortunately, actually doing this with large blocks has been
*excruciatingly* slow to perform, as each item moved was a separate
round trip network call.  For Invid fields, things were even worse, as
each single item moved involved a separate transaction checkpoint to
make sure that all bind and unbind operations for the element
add/delete were done with transactional consistency.

The db_field interface now has explicit addElements() and
deleteElements() methods that support bulk adds or deletes.  The
DBField and InvidDBField classes have had these methods added as well.
For InvidDBField particularly, the speed up from using the bulk
add/delete methods is dramatic.

In the process of adding addElements() and deleteElements() on the
server, I wound up significantly increasing the efficiency of even
non-bulk vector adds where there is an enumerated list of choices that
any adds have to be chosen from.

In the process of supporting such bulk transfers, I modified a good
bit of the client and the client's StringSelector GUI component.
Everywhere the client uses the StringSelector can now handle bulk
moves in one operation.

Making this change has the potential to complicate custom DBEditObject
subclasses, which now have to be able to respond in wizardHook() to
both singleton and bulk adds and deletes, and which have to provide
separate finalizeAddElements() and finalizeDeleteElements() in
addition to the old finalizeAddElement() and finalizeDeleteElement()
methods if the customizer wishes to take special action when items are
added to or removed from a vector field.

10. [CLIENT] Cleaned up the Objects Owned panel in the Owner Group window

Despite the fact that the 'Objects Owned' panel in the Owner Group
window uses the same GUI component that the normal 'General' panel
uses, it was not possible to right-click on entries in order to view
or edit objects listed.

This has been fixed, and it should now be possible to right-click on
any object listing in the client and get at least a view/edit menu.

The object type listing pulldown menu in the Objects Owned panel will
now be sorted in the client.

Turned debugging off in the Objects Owned panel.

11. [CLIENT] Added threading to adminHistory panel

When I added a thread-out to the generic history panel in the client
(change 0.99.9#10), I neglected to do the same for the admin history
panel that is present when viewing and/or editing an admin persona
object.

The admin history panel is now threaded so that it won't block the GUI
refresh during long waits for a history summary from the server.

12. [SERVER] Fixed journal handling on start up

The DBStore load() method failed to consolidate the journal into a
clean ganymede.db dump if the journal was not in a fully consistent
state.  It would print an error message and then skip the normal
start-up dump.

Now, the error message is still printed, but since the journal loading
code will properly cope with incomplete or malformed transaction
records in the journal, the normal startup process is followed
thereafter.

13. [SERVER/CLIENT] The client now displays whether the server is doing a build

The server will now notify the client when it starts and stops
performing a build.  The client displays this information in the panel
in the very bottom left corner, where the 'Status: ' text used to be.

This is an asynchronous message sent by the server using a new
serverClientProxy class that has its own background thread for
queueing and sending asynchronous RMI calls to the client.  The
mechanism used to send this message is generic; it will be easy to
extend this mechanism if I want to add more generic server
notification features to the client.

This involved a change to several server classes, including the
GanymedeBuilderTask base class.  Any task not subclassed from
GanymedeBuilderTask will not trigger this status update.

14. [SERVER] Took out deprecated thread stop() operations

The Ganymede Server's background scheduler used the deprecated
java.lang.Thread.stop() method call to shut down running tasks when
the server is shut down.  The stop() method is deprecated in 1.2 as
unsafe, however, so the scheduler now uses the interrupted() method
instead.  The scheduler and all tasks included with Ganymede 1.0pre1
are responsive to the interrupted() call, but custom background tasks
written for use with the Ganymede scheduler may need to be reworked to
respond to the interrupt call.

The GanymedeBuilderTask base class will check for an interrupt between
phase1 and phase2 of a build, but it will be up to a builder task
writer to check java.lang.Thread.currentThread.isInterrupted() during
the two phases if they want to be able to quickly interrupt their
builder task.

With this change and a couple of unrelated deprecation fixes in the
JDataComponent and JTree packages, the Ganymede server and client
should now build with no deprecation warnings under Java 1.2 and
later.

15. [SERVER/CLIENT] Added regexp support for IP fields in Query mechanism

The Ganymede server and client did not previously support regexp
matching for IP fields.  The client now provides a 'matching' option
in the query box for performing regexp searches on IP address fields.

Gil Kloepfer (gil@arlut.utexas.edu) pointed this out.

16. [SERVER] Did a basic reworking of ganymede.db and DBStore structures

I've been laying the groundwork for support for XML database
export/import, and towards this end, I have evolved the ganymede.db
database format and the server's DBStore data structures in a
direction that may make it easier in the future to load data from an
XML file, and to make the server code simpler.

The ganymede.db file no longer includes definitions for the mandatory
fields in the schema section.  The server now implicitly creates these
field definitions whenever an object type definition is read from the
ganymede.db file or created with the schema editor.

This not only makes the schema editor simpler, but it also will lessen
bulk and possibility for error when schema importation from an XML
file is supported, as the XML file would only need to specify custom
object and field types.

And while I was mucking around with the file format, I modified server
code and the ganymede.db file format to make keeping track of the
order of fields within an object and object types within a category
simpler, and more robust.

There were also a good number of RMI interface changes to simplify
schema editing logic.  Signficant changes were made to Base,
BaseField, Category, FieldTemplate, and SchemaEdit, and the classes
that implement these interfaces.  A number of effects of these changes
are discussed in change 1.0pre1#17, below.

These changes in the server are wrapped in file version checks, so
Ganymede 1.0pre1 should work fine with ganymede.db files from older
versions of the Ganymede server.  When 1.0pre1 dumps out the
ganymede.db file, however, it will be dumped in its new form.

17. [SERVER/ADMIN CONSOLE] Made schema editing more robust

The server now supports schema editing better.  Schema changes that
would break the database are prevented (at least, much more than
before).  The server now provides feedback to the admin console (with
error dialogs, and the like) if an attempted change would cause
problems.

A lot of the interface between the graphical schema editor and the
server has been simplified and regularized, to depend less on
serialization of server-specific data structures.  The schema editing
system now uses String labels to identify objects to be manipulated in
many cases.  A number of operations during schema editing that were
performed as an interaction between the graphical schema editor and
the server are now done locally on the server, to reduce performance
problems.

18. [SERVER] The server now resolves client ip addresses to host names

When a user or admin console logs into the Ganymede server, the server
will now do a reverse DNS lookup to get the fully qualified name of
the system that the user is logging in from.

The admin console will show the fully qualified domain name for
attached clients.

19. [SERVER] Transaction email can be turned off with the starttransaction event

Formerly, the server would always send a transaction log to the admin
who commited a transaction, as well as sending email to all owner
groups that own any objects affected by the transaction.

Now, this email can be turned on or off by editing the
'starttransaction' system event object.  If the 'Event Mail' checkbox
in the starttransaction object is not checked, no transaction logs
will ever be mailed.  If it is checked, then the 'Cc: Admin' and 'Cc:
Owner Groups' checkboxes are consulted to determine whether the admin
and the owner groups of any affected objects should be notified.  In
addition, any email addresses added to the 'Mail List' vector field
will receive a complete copy of any transaction logs generated during
the server's operation.

By default, all three checkboxes are turned on and mail will be sent
out in the same fashion that it was in 0.99.9 and earlier.  Users with
pre-existing Ganymede databases will have to edit this system event
object in order to restore the previous mail behavior.

20. [SERVER] Fixed QueryResult.append()

The arlut.csd.ganymede.QueryResult class's append() method did not
properly update its invidHash and labelHash tables when multiple
QueryResult objects were coalesced.  This caused the appended
QueryResult object on the server to act as if it did not contain
Invid's that it did, which could result in a permissions violation
report when an InvidDBField tried to verify an Invid in an addElement
operation.

This only had an affect when the DBEditObject.obtainChoiceList()
method used the QueryResult.append() method to concatenate query
results, which the gasharl schema kit's emailListCustom class did for
its Internal Members field.

Thanks to Andrew Helyer (helyer@arlut.utexas.edu) for reporting this.

21. [CLIENT,SERVER] Fixed object description mail from the client

Fixed the client's object mail feature.  Added getSummaryDescription()
to the DBObject class and the db_object interface, which is now used
for sending an email description of an object from the client.

This feature is activated by using the 'Mail To...' menu item under
the 'File' menu in the client's object view/edit windows.

22. [SERVER] Fixed role cloning

The PermissionMatrixDBField class had a slight error in its
copyFieldTo() method which broke role cloning.  This was supposed to
work, but it didn't.

Thanks to Pug Bainter (pug@pug.net) for reporting this.

23. [SERVER] Changed client naming for concurrent use

Previously, if a user logged into the Ganymede server with more than
one client at a time but the same username, the Ganymede server would
report the second account as user2 in the logs and the admin console.

This can be confusing for situations where both user and user2
actually exist as valid user accounts, so the Ganymede server logic
has been changed so that if 'user' is logged in twice at the same
time, the second login will be reported as 'user[2]' rather than
'user2'.

Thanks to Pug for complaining about this.

24. [SCHEMA] Fixed gash/gasharl schema system interface deletion bug

Pug caught a bug in the gash/gasharl schema kit, by doing
something I would never have been able to make myself do.  He created
a system, set the room to a room with a single network, edited the
system's single interface, set the network, then deleted the
interface.

In doing so, he discovered that the gasharl schema kit didn't keep
track of things properly on interface deletion, and that the network
that was chosen became unavailable as a choice for that system when an
interface was created again with the 'add interface' button.

I've fixed this, so now you can delete and add interfaces all you want
without losing the ability to choose a previously chosen network.

25. [CLIENT] Fixed default owner dialog logic

Previously, the ganymede client could get confused about the proper
settings for the default owner dialog (the dialog that pops up to ask
you what owner group you want to put a soon-to-be created object into)
if you change personas.

It would be too painful right now to explain the precise situation in
which the ganymede client wouldn't do the right thing, but it doesn't
anymore.  It does the right thing.

This is another one of those things that's been bugging me for a long
time, and I'm glad to see it fixed.

26. [SERVER] Fixed case-sensitive login handling

The GanymedeServer login process was mishandling login authentication,
by canonicalizing user login to lower case, but using a case-sensitive
comparison to find matching admin privileges.

GanymedeServer.login() now uses a case-insensitive query to look up a
user's login privileges.

This is another thing that Pug caught.

27. [SERVER] Fixed mail handling for persona names

Another fix for a Pug bug.  Pug reported that attempting to log into a
server with a fully qualified persona name, like 'pug:admin', and
messing up on the password would cause the server to attempt to send
mail to 'pug:admin@mailserver.com', which his mail system ultimately
resolved to be equivalent to 'admin@mailserver.com'.

The DBLog code that handles system event mail will now truncate at the
colon, so a bad login for 'pug:admin' will be sent to
'pug@mailserver.com', as it should.

28. [PASSWORD CLIENT] Fixed PasswordClient compatibility for 1.2

The PasswordClient class used by the changePassword program attempted
to set an RMISecurityManager, which is not allowed under Java 1.2.  As
a result, running changePassword on a 1.2 JVM would cause a security
exception to be thrown.

Sun really makes it hard to be both 1.1 and 1.2 compatible, some
times.  Fortunately, the Ganymede clients work acceptably well with
the default security manager, since we package all of the RMI stub
classes with the client and password jar files.

This was yet another fine bug report by Pug.

29. [NIS PASSWORD CLIENT] Fixed rpcpass non-termination errors

There were certain conditions in which the rpcpass client proxy would
not properly exit the JVM, but would instead linger until manually
killed.

Now, the client will always terminate if something goes wrong, rather
than sticking around.

This time, for real.

30. [SERVER] The dump task now reports how many transactions it is dumping

The dump task, which typically runs every couple of hours to
consolidate the transaction journal into the ganymede.db file, will
now announce the number of transactions in the journal to the log and
admin console when it runs.

31. [SERVER] Fixed supergash admin persona renaming in DBStore.initializeObjects().

The Ganymede server wasn't properly renaming the supergash admin
persona on startup if the ganymede.properties file specified an
alternate name for the supergash persona.

I've fixed this.. the server will now always reset the supergash
persona name to that specified in the server's ganymede.properties
file.

Thanks to Alex Robinson (Alex_robinson@bigfoot.com) for reporting
this.

32. [SERVER] Improved object cloning with partial failure on vector copy

I happened to notice that when I attempted to clone a user who
belonged to one or more groups that I did not have permission to put
into a new user, Ganymede did not react well.  That is, I got a dialog
saying that certain items could not be cloned, and when I looked at my
newly cloned object, several of the fields were just totally blank.

I've now modified DBField.addElements() and InvidDBField.addElements()
so that they are able to handle partial copies, and so that the
DBField.copyFieldTo() method takes advantage of this to make object
cloning more robust.

33. [CLIENT] Added a fabulous new XML client

When installing the client, a new text-mode client program called
xmlclient is installed.  xmlclient is able to bulk load data into the
Ganymede server from an XML file.  Loading data through the xmlclient
is less efficient than the old custom runDirectLoader hacks that data
loaded depended on before, but it is far more versatile and lets you
load in any kind of data that you can specify in your Ganymede schema
definition with nothing more than a Perl script and this client.

Ultimately, I want to make the xmlclient able to not only load
arbitrary Ganymede data from an XML file, but for it also to be able
to connect to an empty Ganymede server and establish a custom schema
from an XML file.  Supporting this and converting all of the existing
schema kits to this method of server initialization is the big goal
left for Ganymede 1.0.

34. [SERVER] Made the server less picky about setting owner groups ahead of time

The Ganymede server now allows non-interactive clients (clients that
have turned wizards off) to create objects without having previously
selected one or more owner groups to be the default owners for newly
created groups.

This was only an issue if the user that was creating objects belonged
to more than one owner group, such as supergash.

Now, if supergash creates objects through a non-interactive client,
the objects will be created with null (supergash) ownership.  If a
non-supergash user that belongs to multiple owner groups creates
objects through a non-interactive client, the objects will be owned by
the first owner group found by the server.

All of this was done to simplify things for XML bulk loading.

35. [SERVER] Made the server not complain about redundant invid loading

I'm finding that I need to make a number of tweaks to the server so
that it behaves in a reasonable way in the XML client context.  In
many ways, the server was designed around interactive usage, and a
number of issues are coming up with the batch mode XML client.

In this case, the server previously complained if an object reference
(invid) was added to an vector object reference field that already
contained that object reference.

Now, the server silently discards redundant invid additions.  This
allows the XML client to have a symmetric invid relationship in which
both sides of the relationship show the link without having the server
reject the second object's invid links as redundant.

36. [SERVER] Improved error reporting on object editing contention

The Ganymede server previously returned a misleading error message if
during editing the user attempted to cause an object to be linked to
an object that was already checked out for editing by another user on
the Ganymede server.

Now, if a link to an object cannot be established because the object
is already checked out for editing by another user, the server will
report the error properly, and will identify the user that has the
object checked out for editing.

37. [SERVER, CLIENT] Fixed the deferred shutdown option

The 'shutdown the server when all users log off' mechanism was broken,
and didn't actually shutdown the server even when all active users
logged off.  This has been fixed.

Additionally, the Ganymede ClientBase class has been reworked slightly
so that if the server is in deferred shutdown mode, the login failure
message will explain this, rather than the generic 'bad
username/password?' message.

38. [SERVER] Added deleteAllElements() to the db_field interface

I added the deleteAllElements() method to the server's DBField class
and db_field interface, to allow the new XML client to remove all
values in a Vector DBField without having to do an additional
roundtrip to the server to get the list of elements currently in the
field.

39. [SERVER] InvidDBField.createNewEmbedded() now includes a db_object ref

Another change to support the XML client efficiently.  The
InvidDBField createNewEmbedded() method now includes a db_object
reference as well as a serialized Invid.  This removes the need for
the XML client to make an additional round trip request to the server
in order to get an editing handle on a newly created object.

40. [SERVER] Added support for high speed log scanning

The Ganymede server has always had support for scanning its own log
file to produce an object summary report for the Ganymede client.
Unfortunately, Java is rather inefficient at text file parsing, and
the log scanning code converts each line of the log file scanned into
a DBLogEvent object.  This conversion made coding convenient, but made
scanning a very large log file much much slower than a simple grep
would be.

The Ganymede server now supports an external Perl script which
prefilters the log file on request, so the Ganymede server only needs
to process those lines that match the object being reported on.  This
makes the object history feature much much faster, as the server
doesn't have to process unrelated log entries.

If an object (such as an owner group) actually does have a very large
number of lines in the log file, parsing and reporting on what might
be megabytes of log data will still be very slow, however.  At some
point, the log scanning code should probably be written to avoid the
object conversion step in favor of simply identifying matching lines
through textual means and appending the text of such lines to the
object history report.

The external helper program is an optional acceleration.. if the
'ganymede.loghelper' property isn't defined in the server's
ganymede.properties file, then log scanning will be performed in the
old-fashioned style.

41. [CLIENT] Fixed persona creation in user object panel

The Ganymede client allows new admin personae to be created in two
different ways.  In 0.99.9, however, creating a new admin persona from
within the user editing window threw an exception.

It is now possible to create new admin personae either directly
through the client's tree by creating a new 'Admin Persona' object, or
by editing a user object and using the Personae tab to create a new
persona.

42. [SERVER] Optimized critical database lookup code.

This one's embarassing.  A long time ago I heavily tuned and optimized
the Ganymede server's database hash structures.  A job well done, I
said to myself.  Unfortunately, the DBSession.viewDBObject() method
which everything in the server actually uses to pull objects out of
the server was doing some extremely dumb things.

In particular, before ever looking in the highly optimized database
structures, viewDBObject() looked to see if the object being sought
was already checked out for editing by the session's transaction.

Unfortunately, the DBEditSet transaction class tracked objects checked
out for editing in a linear Vector rather than a hashing structure, so
as more and more objects got checked out by a session, *all* database
access operations on behalf of that session got slower and slower and
slower.  This turns out to be the reason why the Ganymede server's
bulk loading was so very slow.

Now this transactional data structure has been converted to a
Hashtable, and viewDBObject() has been extensively optimized so that
it will only attempt to find an object in the transaction's hashtable
if an object has not yet been committed into the main database
structures.  All transactional object view, edit, delete, inactivate,
and clone operations will involve *much* less work as transactions get
bigger.

I knew there was a reason I hadn't called it 1.0 yet..

43. [SERVER] Schema Editor dumps database after making schema changes

A number of people, including Christoph Litauer
(litauer@uni-koblenz.de), reported problems resulting from the server
not consolidating its on-disk database after schema changes were
committed.  The problems arose whenever changes were made to the
server's schema in the schema editor after which transactions were
committed to the journal file before the server was abnormally
terminated, leaving a mix of object data in the journal file from
after the schema change, which is not itself recorded in the journal
file.

The server will now dump its database to disk and consolidate the
journal file when schema changes are committed, so there is no
opportunity for a mismatch between the schema in the ganymede.db file
and the object change records recorded in the server's journal file.

44. [SERVER] Object Event Mail Is Now Coalesced

Another change in response to bulk loading with the XML client.  It
used to be that people who requested mail notification when certain
operations happened to certain kinds of events (such as when a user
was created or deleted, say) will now get a single message with a
summary of all events of that type during a transaction rather than a
single message for each event.

We had a user who was subscribed to system creation objects and was
getting hundreds of messages all at once every time I ran my bulk
loader test.  Now he'll just get an easily deleted summary.

In addition, when these events do happen one at a time, the message
subject in the mail sent out by Ganymede will now include the name of
the object operated on.

45. [SERVER] Expiration/Removal dates may not be set to times past

Jeff Sims (jsims@arlut.utexas.edu) reported that Ganymede erroneously
allowed a user's Expiration date to be set to an already passed date.
The admin intended to set the Expiration some months in the future,
across a year boundary, but neglected to change the year, causing
Ganymede to expire the object at the next run of the Expiration Task.

Ganymede will now forbid any changes to an object's Expiration or
Removal date that would set either date to a time earlier than the
Ganymede server's notion of current time, as a safeguard against this
sort of mishap.

46. [SERVER/CLIENT] Server now has soft timeout option

The Ganymede server has always had the ability to time out users that
were idle in the Ganymede client.  Tom Embleton expressed
dissatisfaction with this, for the reason that having to login again
from scratch forced him to readjust his windows and was a pain in
general.

In response, I implemented a soft timeout option that can be turned on
by activating the 'ganymede.softtimeout' property in the server's
ganymede.properties file.  If this option is chosen, the server will
send a time out message to the client, which will respond by
downshifting from any admin persona to the base user login and
presenting a mandatory persona selection dialog which will insist on
being given a proper password in order to let the user continue work.

A soft timeout will still close any transactions open at the time of
the server timeout, but it will be faster to enter the user's password
into the persona dialog and get back to work than to login from
scratch again.

In addition, the server's two time limits (for idle sessions with no
objects checked out and for idle sessions with objects checked out for
editing) are now configurable in the server's ganymede.properties
file.

47. [SERVER] Added shutdown semaphore to keep shutdown from breaking builds

I have added an additional semaphore to the server so that when an
admin initiates a shutdown of the server from the admin console, the
server will wait for any builder tasks waiting on phase 2 (execution
of external scripts for NIS/DNS/etc. builds) to complete before
shutting down.

This is to insure that a builder task in phase 2 won't interrupt an
external build script or program, leaving a network service in an
inconsistent state.

48. [SERVER] Fixed namespace-optimized lookups on IP fields

The server has special logic in GanymedeSession for optimizing query
lookups on namespace-indexed data fields.  That logic was not properly
handling IP address fields when given an IP address in string form
rather than in byte array form.

Now, the query logic will properly convert IP strings to byte array
form for optimized namespace lookups.

49. [SERVER] Fixed queries on embedded objects

The GanymedeSession queryDispatch() method will now properly perform a
search on an embedded object if a Query is submitted whose requested
returnType is equal to the searched-for type.

My initial decision to include embedded objects in the Ganymede design
has made *so* many things painfully complex in the whole system.
Sigh.

This is another bug that showed up in development and testing of the
XML client.

50. [SERVER] Optimized transaction commit

The server code had a potentially quite expensive checkpoint()
operation in the DBEditSet.commit() method.  This checkpoint operation
was there to allow the server to set the historical fields (creation
and/or modification date and user identity) before the commitPhase1()
methods were called while still being able to do a rollback() if a
commitPhase1() method rejected the transaction commit.

This checkpoint and rollback pair could be enormously expensive if the
transaction gets real big, as it can with the xmlclient.

The commit() method now forces the new date and user identity
information into the transaction's edited objects in a manner that
bypasses the object value freeze put in place during an early point in
transaction commit.

This change will make committing large transactions a good deal
faster, and will make failed commit attempts even faster than that.

51. [SERVER] Improved startup sequence

The server now intelligently checks to make sure that no server is
already running at the same host and port, rather than simply ignoring
the warning from the RMI rebind process.  The server will now decline
to start up if another server is answering the phone at the same
address.

52. [SERVER] Fixed PermissionMatrixDBField loading

The server had a bug that would prevent the database from loading
if a role had a permission matrix defined with no entries defined
in it.

53. [INSTALL] Install scripts now check to make sure jars were built

All of the install scripts now double check to make sure the
jar files were made during the system build.

54. [SERVER] Reworked checkpoint handling for non-interactive transactions

I have simplified how non-interactive transactions handle
checkpoints..  basically, the DBEditSet class now automatically
ignores checkpoint activity for non-interactive transactions.  Any
rollback request made during a non-interactive transaction will result
in the transaction committing hari-kari at commit time.

By putting all the logic for this in DBEditSet, I've simplified a fair
bit of code in InvidDBField, DBEditObject, and DBSession.

I should have done this before.

55. [CLIENT] Added count indicator to StringSelector

The vector field GUI component now includes a count number, to make it
easy to see how many items are in the field.

56. [CLIENT] Windows now kept within bounds

It turns out that the default Swing window resize/move logic made it
possible to resize/pull windows above the top of the inner desktop,
such that the title bar was no longer accessible.  Richard Mach here
at ARL:UT graciously pointed out this problem, and it turned out to be
pretty easy to fix.

The Ganymede client will now block attempts to resize or pull a window
out of the visibility area to the top and left.

57. [CLIENT] Fixed owner panel on view windows

The right-menu popup menus weren't enabled on view object windows in
the owner panel.  Now, you can view any object and be able to
right-click on the items in the owner list to directly view or edit
the owner groups that own that object.

58. [SERVER] Fixed object cloning in previously broken case

The object cloning logic was not previously excluding object linkage
(invid) fields that were involved in a one-to-one or many-to-one
relationship.  The upshot of this was that when a user here cloned an
I.P. network, the cloned network had duplicated not only a list of
rooms for the network to be present in, but also the systems, thereby
unlinking those systems from the original in the process.  This caused
much confusion until the cause was detected.

Thanks again to Gil Kloepfer for pointing this out.

59. [SERVER] Fixed rollback for object deletes

This was a really critical bug.  The result of this bug was that if an
object deletion was attempted, but was refused due to consistency
problems, the system would rollback everything *except* the object's
to-be-deleted status.  This meant that if the user went on to commit a
transaction after unsuccessfully deleting an object, that object would
be deleted anyway.  Worse, all the objects that should have had their
links to the object undone as part of the deletion operation will
retain their links to the deleted object, ruining the symmetrical
pointer integrity constraint of the Ganymede database.

This bug and #58 above manifested as part of the same incident, so
thanks must go again to Gil Kloepfer for demonstrating a really ugly
problem.  This bug has been in Ganymede since 0.99.8, lurking quietly
for over 10 months.

The fix was to move one line of code in DBSession up about three
lines.  Amazing how these little touches matter.

60. [SERVER] Serialized checkpoint logic, removed client access to checkpoint()

In examining the checkpoint and rollback code, I realized that there
was nothing in the system that would prevent multiple threads from
attempting to assert and rollback checkpoints on the same transaction.
This meant that it was in principle possible for the server to
interleave checkpoint operations, leading to unpredictable and
incorrect results if multiple threads simultaneously attempted to do
things like object creation, deletion, and linking within a single
session's transaction.  I don't believe this does ever happen, in
practice, but if it ever did it would be confusing and a pain to
debug.

To avoid this, I put in a system whereby only one thread at a time is
allowed to have a checkpoint on any given transaction.  If another
thread attempts to checkpoint a transaction while it is already
checkpointed by a thread, the thread that is attempting to checkpoint
will block until the earlier checkpoint is popped or rolled back.

This scheme will guarantee safety of checkpoint operations at the cost
of an mildly increased risk of deadlock.  To mitigate the deadlock
risk, I removed the ability for remote clients to access the
checkpoint() and rollback() methods in the GanymedeSession class, and
I put a deadlock timeout on the checkpoint block, so if a checkpoint()
attempt blocks for more than 60 seconds, an exception will be thrown
and the checkpointing thread will let go.

The loss of client access to checkpoint and rollback doesn't seem to
be an unreasonable thing to do, as no client code that I have yet seen
has done anything with the intra-transactional checkpoints.  If
someone convinces me that it is critical that remote clients be able
to access intra-transactional checkpoints, I'll need to rework the
anti-interleaving code to deal with the fact that subsequent RMI calls
from a single client are not guaranteed to be handled by the same
thread on the server.

61. [SERVER] Fixed deadlock vulnerabilities

Previously, the GanymedeServer object used Ganymede.internalSession to
process database queries for user and admin console logins.  The
problem was that this was prone to deadlock with some other uses of
Ganymede.internalSession in the Ganymede server.  In particular, if
anyone ever attempted to login at the same precise moment that a
transaction was being committed, the whole server locked up.

Now, the Ganymede server uses a distinct GanymedeSession object to
handle logins.  Since GanymedeServer has its own synchronization at a
higher level, it is no longer possible to get nested monitor deadlocks
from logins.

In addition, since the Ganymede.internalSession object is often used
by random Ganymede server code to look up things in the database, I
partially de-synchronized DBSession.viewDBObject() so that a DBSession
that does not have an open transaction asssociated with it will not
incur a synchronization penalty/deadlock opportunity merely to
retrieve a reference to an object from the database.

62. [SERVER, CLIENT] Added -dump{schema|data} to the xmlclient, removed decodeDB

The xmlclient is now able to emit a copy of the server's schema
definition in xml format, using xmlclient -dumpschema, and to dump the
server's database in its entirety in xml format, using xmlclient
-dumpdata..  Due to this, I have removed the old hackish decodeDB and
permdecodeDB front ends to the Ganymede server.

63. [SERVER, ADMIN CONSOLE] Added runEmbeddedTest, runEmbeddedSweep

Bug 59 above, the deletion rollback bug, allowed some inconsistencies
into our database, wherein embedded objects were left in the database
when their owners had been deleted.  I added test features to the
server and the admin console to detect and repair this condition.

64. [CLIENT] Made Client check for MOTD message in background

The client now runs its check for a new MOTD in a background thread,
so as not to lock up the GUI thread initially.  This will get rid of
the ugly lack of refresh that was visible when the user dismissed
the select persona dialog.

65. [CLIENT, ADMIN CONSOLE] Cleaned up login boxes, console shutdown dialog

I finally went back and got the admin console and gui client to put
the focus on the username box, so people can just start typing their name
when the clients come up.  I reworked the admin console shutdown dialog
so it just uses one dialog box for the various shutdown modes.

66. [INSTALL] Removed installClient2, made installServer autodetect java version

A bunch of changes to the installation process. installClient2 is now
gone, installClient is now smart enough to autodetect what version of
Java is being used and to configure the client install appropriately.

Ditto the installServer script.

I cleaned up the gasharl schema kit's ganymede.schema file to take
care of some inconsistencies that have been in that file for a long
while.

67. [CLIENT] Tweaked right-menu clicks in StringSelector to select and popup

Mike Mulvaney (ex-Ganymede GUI guy) filed a bug report indicating that
clicking the right mouse button should select the item that the pop-up
menu is being applied to.

He was right, and I so fixed it.

68. [SERVER SECURITY] Fixed a loophole in the object permission system

The Ganymede system allows admins and users to be given editing
privileges to objects that are not considered to be owned by the admin
or user.  This is an unusual circumstance, usually only done when the
customizer wants objects to appear in the client's object tree but
doesn't want to allow certain (or even any) fields to be modified.

The problem was that the built-in fields, including the owner list
field, do not typically have their own access permissions tracked,
independently from the object's access permissions.  This allowed
any user who could edit an object to grant themselves ownership
of the object, which is bad.

The server now refuses to grant edit privilege to an object's owner
list field unless the user already owns the object, or is running with
supergash-level privileges.

69. [SERVER, XMLCLIENT] Added XML-based schema editing through the xmlclient

The xmlclient can now be used to perform schema edits on a running
Ganymede server (so long as no users are logged in, and as long as the
schema edits do not result in an incompatibility that will break
loaded data).  This makes is possible to get rid of the old
directLoader crud in the schema kits.

70. [SERVER, ADMIN CONSOLE] Fixed exceptions on drag/drop in schema editor

Fixed bug #13 in bugzilla.

71. [SERVER] Added TaskOptionStrings to Task definition

The built-in object type, 'Task' now supports an optional vector
string field, which will be used in 1.0 to represesnt builder task
options for schema kits that support more than one transaction-time
builder task.

72. [CLIENT, SCHEMA EDITOR] Fixed tree's pop-up menus on Linux

The GUI tree component was still using old-style AWT pop-up menus,
which wasn't working properly on Linux using Java 1.3.  Since almost
everything in the Ganymede client and admin console are based on Swing,
switching the pop-ups to Swing should keep things consistent and
working on various platforms.

73. [ADMIN CONSOLE] Fixed console deadlocks on Linux

I was being sloppy with my GUI thread handling in the admin console..
updates from the server were not being painted on the GUI thread,
which was leading to swift lockups when running the admin console on
the Linux 1.3 JDK.  I fixed this, the admin console should run fine
in any Java environment now.

74. [SERVER] Reworked query engine to increase concurrency

The Ganymede server no longer has to block threads from doing most
queries on the server while a transaction has portions of the database
locked for updates.  The server now makes an interal snapshot vector
of the objects in an objectbase when transactions are committed, and
most queries are run against these snapshots.

The Ganymede server will still lock the database on a standard
GanymedeSession query if the query spans more than one object type.
In practice, this means that unless you or your code attempts to do a
query that involves embedded object values (as in searching for a
system using a give i.p. address held in an embedded interface
object), the query will never block.  For this reason, it is now safe
to take advantage of the Ganymede query mechanisms during all stages
of transaction commit in DBEditObject subclasses.

The main tradeoffs for doing this are a slight increase in memory
usage (4 bytes per object in the database), to maintain a second copy
of references to all objects in the database, and a slight increase in
transaction commit time, as the new snapshot is constructed.

Another, more subtle trade-off is that support for this level of
concurrency in the server might be hard to maintain if the Ganymede
server is ever reworked to move away from its in-memory database.

75. [XML] Simplified the XML schema for integer and boolean fields

The XML file format used to require boolean field values to be
expressed as <boolean val="true"/> or <boolean val="false"/>, and
integer numeric values as <int val="123"/> and the like, which seemed
unnecessarily wordy, since boolean and numeric fields are always
scalar.

Now you can do things like

<uid>1005</uid> and <option>true</option>

rather than

<uid><int val="1005"/></uid> and <option><boolean val="true"/></option>

The Ganymede XML loader will continue to recognize the older style
for backwards compatibility..

76. [SERVER] Memory optimizations

I made a number of optimizations to memory usage in the server, which
should cut the amount of per-field overhead roughly in half.  Heap
required for objects in the database has been reduced by at least
10-20 bytes plus 10 bytes per field in each object.  Fields in the
database now have only 14 bytes of overhead each, plus whatever
overhead your Java VM needs to track garbage collection for two
objects.

To accomplish this, I reduced the number of variables held in the
DBField class to an absolute minimum and restructured DBObject so it
uses a simple array with packed hashtable access logic rather than a
separate hashtable.

Custom DBEditObject subclasses that directly manipulate the old fields
FieldTable variable will need to be reworked in order to work with the
new field storage structure.

In addition, the server will now create a lot fewer temporary
Enumeration objects during runtime, and a lot fewer PermEntry objects,
both of which will have some impact on reducing memory usage.

Schema edit commits will be faster, but otherwise the overall speed
of the server will probably not be much changed.

77. [SERVER, CLIENT] More Memory Optimizations, History Panel revisions

I reduced the size of the creator info and modifier info strings so
that they will only include the name of the user/admin and the name of
the client used to create/modify the object.  This will save about 100
bytes from each object created right off.  In addition, since the time
is no longer included in these fields, the automatic interning that
the server does to eliminate redundant strings will reduce the
overhead for these data elements by a very, very large percentage.

To compensate for the creation and modification time no longer being
included in the history string fields, I have modified the client
to display the history date fields with hour, minute, and second
as well as date.

78. [SERVER] Memory, security fix

The new system for handling DBField/DBObjectBaseField association
revealed a longstanding bug in DBObject that prevented Ganymede from
properly de-linking fields from their previous DBEditObject when a
transaction was committed.  This had the effect of preventing the last
DBEditObject constructed for a particular database object from
being garbage collected unless and until the object was actually
deleted from the database.  A small memory leak.

This bug also allowed clients the possibility of at least attempting
to continue to perform edit operations on fields after a transaction
was committed and the editing objects were checked back into the
server's database hashes.

Both conditions have been solved, which will reduce the server's
active working set somewhat during runtime and which will protect the
server against malicious clients.

79. [SERVER] Fixed logging of newly created Role objects

The PermissionMatrixDBField wasn't properly reporting the value
of a newly created Permission Matrix in the log file.

80. [SERVER] Fixed permissions xml parsing

Fixed the xml parser's handling of permission matrix field objects.

81. [SERVER] Redfined DBEditObject.canRemove() to allow dialog return

Modified the canRemove() method in the DBEditObject class to allow
a complete ReturnVal to be returned rather than the old boolean.

-------------------- Changes from 0.99.8 to 0.99.9 -------------------

RELEASE DATE: January 13, 2000

1. [SCHEMA] Fixed gasharl schema errors

a. The gasharl schema kit had a bug which made it impossible to edit a
system if that system was connected to a room that had no networks
listed.

b. The gasharl GASHBuilderTask code is designed to handle netgroups that
overflow the 1024 byte NIS line length limit, but the code that
handled netgroup folding did so very inefficiently, putting only one
netgroup entry per line after the first.

2. [SERVER] Fixed race condition in Qstmp stopThreaded() method

The Qstmp shutdown process could throw a NullPointerException in
stopThreaded() if thread timing worked out just wrong.  This has been
caught, and this race condition will no longer be able to prevent
clean server shut-down.

3. [SERVER] Fixed various deadlock possibilities in server

a. The 3 DBLock subclasses (DBReadLock, DBWriteLock, DBDumpLock) did not
properly call notifyAll() on the DBStore object if they threw an
exception in their establish() method.  Now, all synchronized blocks
in the DBLock subclasses use try..finally to guarantee that other
threads waiting on DBStore will be woken up, even if something
unexpected happens.

The DBReadLock.establish() code had code to throw an exception if a
session that possessed a write or dump lock tried to obtain a readlock
at the same time.  Unfortunately, the internal sessions used for the
Ganymede builder tasks, expiration task, and warning task were given
the same identifier as the standard 'internal' session used to process
queries in the login process, so if a login was attempted during a
builder task's phase1 execution, or during portions of the expire or
warning task, an exception could be thrown, possibly causing
threadlock if other threads had wait()'ed at that point.  All this
should be fixed now.

b. There are several ways a user can be forced off of the Ganymede
server..  the idle timer, the admin console, and the RMI unreferenced
object mechanism.  Previously, various essential objects in the server
could get locked if the client refused to respond to the server's
disconnect command.  Now the client force-disconnect code is better
coordinated so only one thread in the server will ever attempt to
force the user to disconnect, and that thread will itself spawn a
background thread to force the user to disconnect.  As a result,
although the server may wind up with a single dangling thread per
client if the RMI system blocks trying to call the client, nothing
in the server will deadlock because of it.

As a result of this change, knocking a user off the system by the
admin console is no longer a synchronous operation, so it may take a
few seconds before the admin console updates to show the results of a
user kill operation.

c. The admin console code in the server had a number of problems that
could lead to a deadlock that blocked any admin console operations
(including the stopServer script).

The server code that handles communications to/from the admin consoles
have been significantly reworked so as to not be vulnerable to a
threadlock condition, and some redundant operations have been pruned
to reduce overhead.  A number of possible race conditions have been
closed.

This will make the server faster and safer when admin consoles are
attached, but the admin console updating is still generally
synchronous.  Every admin console connected to the Ganymede server
will slow the server down a bit.

4. [SERVER/CLIENT] Added support for 'browse' mode in client

John Knutson pointed out that the Ganymede client wasn't an adequate
directory browser because it always hid things that couldn't be
edited, when in fact the user had a perfect right to see everything
for which he is granted view access.  This was intended to be a
feature of the client, that it could present a simplified tree to the
user to avoid confusion, but it needlessly restricts advanced users.

The server and client have now been modified so that the client can
display all objects and object types viewable by the user, even if
the user does not have editing permission.

This is toggled by a 'Hide non-editable objects' checkbox menu item
under the client's 'File' menu.  I expect I may move this someplace
else in the client's GUI eventually.

This involved changes to GanymedeSession.java, Session.java, and
CategoryTransport.java on the server, and gclient.java on the client,
so a 0.99.9 server will now require a 0.99.9 client.

As before, the server is really responsible for deciding what object
types are reported to the client.. the client can't choose to show
object types in the tree if the server doesn't report them, but the
client can now ask to have all visible object types reported, which it
couldn't do before.

5. [CLIENT] Reworked object folder pop-up menus in client tree display

The client's object folder pop-up menus in the tree display has been
reworked to be less ambiguous.  The old 'List editable' and 'List all'
have been changed to 'Report editable' and 'Report all', and the old
'Show All Objects' has been changed to 'Show Non-Editables', and moved
to the top of the pop-up menu.

Hopefully this should be less confusing for folks.

6. [SERVER] Fixed custom schema migration bug from 0.99.8

Change 0.99.8, #17, listed below, which modified how the server
handles non-symmetrically linked InvidDBFields, didn't properly handle
InvidDBField definitions that explicitly listed the backlinks field as
the field to be linked to.  Apparently, in 0.99.7 and before, the
schema editor presented the backlinks field as a valid symmetric field
target.

Martin Schneider reported that Ganymede 0.99.8 was complaining when it
came across such a field in his ganymede.db file from 0.99.7.  I had
the server handling invid fields with target fields set to 'none', but
I hadn't handled the equivalence case of the 'backlinks' target.

The Ganymede server will now treat an invid field with a target field
set to field 8 (the obsolete backlinks field) in precisely the same
way that it treats invid fields with no target field specified.  This
should make Martin's ganymede.db file from 0.99.7 work properly in
0.99.9.

7. [SERVER] Made bad login user notification optional

Previously, the Ganymede server would always attempt to send
notification of a bad password/login attempt to the name of the user
entered at the login attempt, regardless of the configuration of the
'badpass' system event object.

Now the Ganymede server will not do this unless the 'badpass' system
event object in the ganymede.db database has the 'Cc: Admin' flag set.

8. [SERVER] Took out unnecessary fields in DBField.java

The DBField base class had a number of redundant or unnecessary
fields, including both a scalar and vector field, even though no field
is both a scalar and a vector.

I've removed the Vector values holding field and replaced it with
accessors which will treat the single value field as a Vector as
appropriate.

I also took out the 'fieldID' short, which was redundant, since the
DBField.getFieldDef() method already calculated that information from
the field definition reference.

I also took out the 'newValue' field, which was only used in one very
specialized case in the userCustom plug-in, and which was better
handled by putting an extra field in that custom DBEditObject subclass.

This will save 10 bytes per field in the database while in memory,
with probably around 40 bytes of per-field overhead remaining for
scalar fields, rather more for Vector fields.  Not a very big deal,
but it does amount to 80-300 bytes of RAM per object in the
database. For our current database, this probably saves us a megabyte
of in-memory space, or around 3% of the total resident memory usage.
That percentage could probably get quite a bit higher when the initial
size of the server itself is amortized over larger and larger
databases.

This change doesn't affect the on-disk database file at all.

9. [STOPSERVER] Modified the stopServer code to prompt for supergash password

Previously, if the Ganymede server's supergash ("root") password was
not kept in the server's ganymede.properties file, the stopServer
client would simply fail.

Now, if it can't load the supergash password from the
ganymede.properties file, stopServer will prompt the user for the
supergash password.

10. [CLIENT] Threaded out the client's history panel to avoid locking GUI

Here at ARL, our server log is getting quite large, and the round trip
time for reporting on the history of changes to an object is getting
large.  Because of this, the client's GUI has been locked for a longer
and longer time as the client's GUI thread blocks waiting for the
server to report the object history.

Because of this, the Ganymede client has been modified to thread out
history reporting.  The client's GUI will no longer lock up while the
client is waiting to get a report on an object's history back.

-------------------- Changes from 0.99.7 to 0.99.8 -------------------

RELEASE DATE: November 23 , 1999

1. [SERVER] Included regexp info in schema html dump

The html/text schema dump facilities in DBObjectBaseField.java now
include regexp information for string fields, if defined.

2. [SERVER, CLIENT] Made it possible to do a more limited log search

Previously, the history panel in the client's "show history" function
showed the complete transaction in which the object was modified.
There is now support for showing only changes directly affecting the
object in question.

Added a new version of viewObjectHistory to the Session RMI interface.

Added explanatory tool tips to the show history buttons in the
client's history panel.

Cleaned up the historyPanel generally.

3. [CLIENT] Cleaned up object report mail/save dialog

The Ganymede client allows the user to mail and/or save a status
summary for single objects in the Ganymede database (from the File
menu in the object view/edit window), but the dialog that came up was
really lousy, and hadn't been touched for over a year.  I restructured
the dialog and made it more usable.

4. [SERVER] Made server start-up less fragile

A user here got a ganymede.db file into a state where a field had a
namespace listed, but that namespace wasn't declared in the .db file.
The server was not able to recover from this because the
DBObjectBaseField class was throwing an illegal argument exception.

I modified DBObjectBaseField so that a warning message is displayed
instead.

Hopefully I'll be able to find how the system got into that state and
prevent it, but the server can deal with that inconsistency now.

5. [SERVER] Improved Qsmtp mail-out code

Previously, the Qsmtp mailer code would abort sending a message if any
of the recipients were rejected by the mailhost.  Now the Qsmtp code
will go ahead and complete the message if any of the recipient
addresses were accepted.

This was causing the 'bad login' mail for an invalid user login
attempt not to be sent.

6. [CLIENT] Made rpcpass exit cleanly

The rpcpass client (used in conjunction with the Linux NISkit's
rpc.yppasswdd to pass NIS password changes into Ganymede) wasn't
properly terminating after execution a transaction in Ganymede,
resulting in lots of lingering rpcpass processes hanging around.

This is a result of the JVM's policy of not closing down an
application until all of its non-daemon threads terminate.  Since the
rpcpass code is an RMI server, this never happened.  I added a
System.exit(0) to take care of this.

7. [SCHEMA] Minor fix to the nisonly schema

Fixed the nisonly schema's userCustom.java's fieldRequired() method to
not attempt to call isInactivated() on the non-value-containing
objectHook instance of userCustom.java.

John Knutson here at ARL was developing a schema based on the nisonly
schema kit, and it turns out that I haven't done much in the way of
upkeep on that schema kit for awhile now.

8. [SERVER] Made GanymedeBuilderTask.backupFiles handle an empty dir ok

The backupFiles() method in the GanymedeBuilderTask class is intended
to produce a zip archive of all files in the out directory when a
builder task is run.  Unfortunately, it caused an exception to be
thrown if there are no files in the out directory, as is the case the
very first time the builder task is run after installing Ganymede.

This has been fixed.  Thanks to John Knutson for reporting this.

9. [SCHEMA] Fixes to the gasharl schema

a) The server used to throw an exception if an admin created a user
and hit 'commit' without filling in the home group field, because
some of the custom logic wasn't checking to make sure that the
field was there before trying to do calculations based on it.

b) Moving a system from a room to another which was not connected
to the network that the system had been allocated on was broken..
a 'can't free network' error was being reported.  This case is
now properly handled.

10. [CLIENT] Miscellaneous Client Improvements and Fixes

a) You can now right-click on a row in the query result table and
inactivate or delete objects shown, rather than having to do a query
to find the object you want to delete and then use the tree to find
and delete it.

b) Related to this change, the 'are you sure you want to delete this'
dialog will now appear whenever an object is deleted, whether from
the delete object dialog, the table, or the tree (which used to
not prompt for verification).  The object deletion verification dialog
is now more friendly and descriptive.

c) The client now resets its 'default owner chosen' flag when the user
changes personae. *** STILL NEED MORE HERE ***

d) The GUI component for scalar invid fields has been very
significantly revised to make it possible to enter a choice by using
the keyboard instead of having to mouse through a possibly huge list.
This was done using a custom JComboBox editing component that monitors
keystrokes to do 'choice completion' during the course of text
editing.  I'm sort of pushing the Swing JComboBox to the limit here..
hopefully with Kestrel/1.3 Swing will be more useful for real world
stuff.

e) Reworked StringSelector to put the add box on the left, and to
enable typed choice completion if the only valid choices are listed

f) Changed the layout algorithm for window placement within the client.
No more client windows being placed partially outside the client's
display area.

11. [SERVER] Implemented support for object cloning

Since the beginning, I've planned on having a clone_db_object() method
on the server that would create a new object by copying values from an
existing one, but somehow I never had gotten around to implementing
it.  The work John Knutson is doing with Ganymede would benefit from
this feature particularly, so I went ahead and got it done today.

This involved lots of changes to the server, including DBField and all
its subclasses, DBEditObject, DBObject, GanymedeSession, and possibly
other classes.

These changes included making the server's code differentiate between
inhibiting wizards for the sake of an automated field copy and
inhibiting oversight for the purpose of loading the database on server
start, etc.

I still need to document all of this in the DBEditObject subclassing
guide.

12. [SERVER] Added support for floating point fields

All of the code for this came from John Knutson
(johnk@arlut.utexas.edu) here at the laboratory.

This change bumps the DBStore revision numbers to

	major: 1
	minor: 15

13. [CLIENT] Improved speed for object editing

The client was inefficiently and redundantly sorting large choice
lists, slowing object editing down a great deal.  Now all QueryResult
objects from the server are automatically sorted one time only, on
receipt, using an efficient binary insertion sort.

14. [SERVER] Reworked Password fields to support FreeBSD-style MD5 hashing

The Ganymede has supported 'MD5 passwords' since August, but this was
actually just a simple MD5 hash of the plaintext.  FreeBSD, OpenBSD,
Linux PAM, and possibly other operating systems use a much more
involved process of generating 'MD5' passwords, including a salt field
and a thousand round trip through MD5 to make dictionary cracks much
more time-consuming.

Ganymede's PasswordDBField class now supports these fancy MD5
passwords, and the old support for a simple MD5 password hash has been
removed.

This change bumps the DBStore revision numbers to

	major: 1
	minor: 16

15. [CLIENT] Fixed tree filtering

The Ganymede client had a bug in gclient.recurseDownCategories which
could cause categories to be erroneously omitted from the client's
tree display in a specific situation.  This was detected by John
Knutson in regard to a custom schema that he did.

16. [SERVER] Took unnecessary fields out of the DBObject class

I took the 'int id' and 'DBEditSet editset' variables out of the
DBObject class.  This will save 8 bytes per object in the database in
memory.  Not a major deal, but every little bit helps, I suppose.

Added methods to compensate for the lack of the redundant, but
directly accessible variables.

17. [SERVER] Major reworking of object linking

I spent a week and did major surgery on the Ganymede server's handling
of non-symmetric object linking.  Previously, if an invid field did
not point to an explicit field in a remote object, the server would
automatically maintain the symmetric back pointer in the target
object's back links field, field 8.

This meant that whenever an object's field was set to point to an
object, that object had to be exclusively checked out by the server to
handle the association.  Only one transaction at a time could
establish such a linking, which could make for a really big and
unnecessary bottleneck in the server.

Now the server maintains at all time a separate, in-memory hash
structure for back pointers.  This hash structure is designed to allow
any number of objects to 'anonymously' link to a given object
concurrently.  The server maintains a separate system to track
deletion locking for objects, allowing anyone to forge or remove an
asymmetric link to a common object, and to allow the common object
itself to be edited, as long as no one tries to delete that common
object.

This change can drastically increase performance in cloning objects
which point to a lot of non-symmetrically linked objects, since those
targetted objects no longer need to be pulled out of the database for
editing in order to establish the link.

This change bumps the DBStore revision numbers to

	major: 1
	minor: 17

18. [SERVER] Server object accounting fix

There was an opportunity for the server's objects checked-out
accounting to get out of sync if an object creation operation was not
able to proceed to completion as a result of a failure to perform
the custom object initialization.

Now the server's object accounting should be reliable under
all circumstances.

19. [SCHEMA] Updated BSD and Linux schema kits to support MD5 hashed passwords

I have put preliminary (but functional) support into the Linux and BSD
kits for MD5-hashed passwords.

Someone will have to let me know how well this works.

-------------------- Changes from 0.99.6 to 0.99.7 -------------------

RELEASE DATE: October 13, 1999

1. [SERVER] Fixed the Ganymede server's registry binding for localhost

Michael McEniry (mmceniry@itsc.uah.edu) provided a patch to
Ganymede.java which corrects the problem on Linux systems with the
server failing on startup with a localhost RMI registry binding
permissions failure.

2. [SERVER] Fixed object deletion for multi-level nested objects

Martin Schneider (martin@ira.uka.de) wrote to report that deleting
objects which contained embedded objects which contain embedded
objects fails on transaction commit.. the problem was that the
GanymedeSession.getContainingObj() method couldn't walk the embedding
stack to find the top-level parent to determine object ownership for
logging purposes.

GanymedeSession.getContainingObj() will now check to see if it is
trying to find the parent of an embedded object which has been
processed for deletion, in which case it will check the original
version of the object in order to find the original parent.

3. [SERVER] Added support for regexp matching in StringDBField

Added support for specifying a regular expression pattern in string
fields.  Modified the server and schema editor to support regular
expression binding to string fields.

The older 'OkChars' and 'BadChars' string field parameters are still
maintained.. any change to a string field must satisfy any regular
expression specified in addition to the OkChars and BadChars
parameters.

This change bumps the DBStore revision numbers to

	major: 1
	minor: 14

Martin Schneider (martin@ira.uka.de) asked for this change, it seemed
like a good idea to me.

4. [SERVER] Fixed server-loading bug from 0.99.6.6

Change 6 to Ganymede.java in release 0.99.6 broke loading data from a
freshly dumped schema.  Fixed the server initialization so that it can
properly reset passwords if the password field is undefined in the
supergash persona object.

5. [SERVER] Fixed schema dump bugs in DBObject.partialEmit()

Since July there has been a bug in DBObject.partialEmit() which makes
it impossible to successfully dump a modified schema to disk and then
turn around and use it to load a new database.  The problem was that
when the server was dumping a ganymede.schema file, the supergash
persona was being written to disk without the link to the supergash
owner group (owner group 0) that gave it supergash privileges.

Actually, there were a cluster of bugs in the schema dump logic.  All
of them have been fixed, so schema dumping should produce a clean and
usable ganymede.schema file now.  I knew the code for schema dumping
was poorly architected, but I somehow missed the fact that it was so
thoroughly broken.

6. [SERVER] Added code to protect the essential server data structures

It used to be possible to manually delete the supergash owner group,
which would put the ganymede.db file in an unusable state.
GanymedeSession.remove_db_object() now refuses attempts to do this.

In addition, the Ganymede server now checks the database schema on
start-up to make sure that all of the essential database objects are
defined, including the supergash persona object, the supergash owner
group, the default role, and the system event objects.  If any of
these objects are undefined in the database on server startup, the
server will now recreate them.

The server still doesn't do a schema sanity check on startup, though.
Still lots more work to do to clean up schema editing as a whole.

7. [SERVER] Fixed logging for IP address changes

The IPDBField.getDiffString() method had an error that caused it to
report the old value for a changed IP address when it was meant to be
showing the new.

8. [SERVER] Added a background task to disconnect idle users

The Ganymede server now has a built-in task which runs every minute to
disconnect users that have been idle for a certain period of time.

Currently, users with no objects checked out of the database will be
disconnected after 15 minutes of idle time, and users who are in the
middle of a transaction but have not done anything for a long time
will be disconnected after 20 minutes of idle time.

9. [CLIENT, SERVER] Made client able to re-acquire server without restarting

The Ganymede client is now able to attempt to re-acquire the Ganymede
server if the server goes down while the client is running.

That is, if the server goes down cleanly while the client is running,
the client will close the main window, leaving the login box.  The
login box will now be able to attempt to bind to a new instantiation
of the Ganymede server if the server has been brought back up.

In order to do this properly, I had to add a new method to the
RMI Server interface (Server.up()), so you'll need to install
the client from Ganymede 0.99.7 for use with the 0.99.7 server,
and vice-versa.

Also modified the client to be more responsive to a server shut-down
message, even if the user was sitting on the persona selection dialog.

10. [SERVER] Server now associates embedded objects with container in log

Previously, if changes were made to an embedded object that did not
result in a modification to the state of the container, the log for
that change did not record that the container was involved in the
transaction, making such changes invisible to the history panel of the
containing object.

Now, whenever an embedded object is changed, the log record for the
change will be associated its container and the change will show up
properly in the container's history view.

11. [SERVER,SCHEMA,DISTRIBUTION] Changes to handle descriptor starvation

We put Ganymede in production at the laboratory on October 10th and
quickly discovered that the Ganymede server was running into file
descriptor exhaustion when builds are done at a high rate of speed.

It turns out that one of the problems was that the GASHBuilderTask
class in the gasharl schema kit wasn't explicitly closing the file
descriptors for the Process object used to do the external build.
Looking on the Java BugParade, I found bug item #4025386, which
describes the problem and how to fix it.

I went through all the BuilderTask classes in all the schema kits
included with Ganymede and added a finally {} clause to close the
Process file descriptors once the external build was done.

Even after this change, I still saw the server run out of descriptors,
so I put a 'ulimit -n 128' in the runServer script (64 is the default
on Solaris.. the ulimit command is commented out so that people on
Linux, etc., don't get an artificially low descriptor limit), and I
fixed a few other places in the code where the system was
unnecessarily waiting for garbage collection to close resources.

File descriptor depletion is still a big issue with Ganymede, as a
user with the client login window open is taking up a descriptor on
the server for the RMI connection, even though the user isn't doing
anything.  I'm not sure how much I can do about this, since I believe
the only way to force an RMI break is by allowing the remote object to
be garbage collected on the client.

In any case, the problem isn't so much that the Ganymede server itself
is keeping an excessive number of descriptors open, but that the
extensive Ganymede/GASH build process we initiate on transaction
commit opens dozens of files, taking up lots of the server's
descriptors along the way.  Java doesn't provide any good way to
manage this, but it may be a good idea to put another ulimit
invocation in the script run by Ganymede?

12. [SERVER] Fixed really stupid DBEditSet rollback mistake

Ugh. Ugh. Ugh.

I had broken the DBEditSet rollback code with change #0.99.6.9 below.

There was no way it could have ever worked.  It didn't come close to
working.  Not even remotely plausible code.  I am dum.

I fixed it.. it is now both efficient and correct.

13. [SERVER] Optimized DBEditSet.commit()

I reworked DBEditSet.commit() so that it doesn't need to use
rollback() to handle an incompletely filled-out object at transaction
commit time.  Basically streamlined commit() so that it does the
minimum amount of work required to check things out, without doing an
unnecessary checkpoint()/rollback() pair when one or more objects in a
transaction aren't completely filled out.

And I checked my work, and I tested the code, this time.

14. [SERVER, ADMIN CONSOLE] Added a deferred shutdown option

The server can now be told to disable logins and to shutdown the server
when all users log out of the system.  This is very useful for shutting
the server down cleanly in a production environment.

If you use the 'stopServer' script, you can now say 'stopServer
-delay' to have the server do a user-friendly shutdown.  If you do do
'stopServer -delay', the stopServer script will not wait for the
server to shut down before exiting.

If you shut the server down through the admin console, a dialog will
pop up asking if you want to make the shut down deferred or not.

This change involves an incompatible change to the
arlut.csd.ganymede.adminSession RMI remote interface used by the
server, admin console, and stopServer script.

15. [SCHEMA] Several tweaks to gasharl schema

Lots of minor fix-ups to the gasharl schema code, which we are actually
using in production at ARL now.

-------------------- Changes from 0.99.5 to 0.99.6 -------------------

RELEASE DATE: August 27, 1999

1. [SERVER DISTRIB] Fixed decodeDB

decodeDB was failing because the properties weren't being loaded and
the db load routine was attempting to find the Ganymede.journal
property.

2. [SCHEMA] More improvements to gasharl schema.

Fixed ARL maildirect file generation to only emit entries for
normal category user records, made it possible to clear the
primary user field for systems.  Fixed a minor infidelity in
hosts_info generation.

3. [CLIENT] Fixed problems with string area gui object

I fixed a couple of problems with the JstringArea class used in
the Ganymede client to display and edit multi-line string fields.

I had to make some changes to the code so that it really didn't try to
do editing when the field was not editable, and so that when errors
are returned when the JstringArea is edited, the focus-loss update
trigger won't get into an infinite loop, popping an error dialog up
over and over and over and over again.

The JstringArea is one of the least used GUI components in the client,
which is why this wasn't detected and fixed a long time ago.

4. [CLIENT] Fixed error in handling fixable transaction failure

The change I made in 0.99.5 to the client's handling of catastrophic
transaction commit failure broke proper handling of non-catastrophic,
correctable commit failure.. if such a transaction failed to commit,
gclient would reset its state, even though the transaction was still
open and repairable.

gclient.commitTransaction() now handles both circumstances properly.

5. [SERVER] Added support for MD5-hashed passwords.

Added MD5 support to PasswordDBField.java.  Bumped DBStore.minor_version
to 13 for MD5 support.

The server can now store and authenticate using MD5-hashed passwords.
Schema kits can be written to input MD5-hashed passwords into the
system as well as UNIX crypt()'ed passwords.

Neither the Linux nor the BSD schemas have yet been modified to
support MD5 passwords, but the PasswordDBField code now supports MD5.

6. [SERVER] Improved the server's password reset logic

Previously, if the -resetadmin parameter was given on the command
line, the server would always open a new transaction and reset the
supergash password.  Now the server checks to see if the password has
changed before doing the edit.  This cuts down on the redundant 'hey,
i set the supergash password!' mail that Ganymede was sending out.

7. [SERVER] Fixed adminPersonaCustom.convertAdminInvidToString()

The new DBLog transaction logic for finding an email address to mail
to was broken when the change was made by an end-user and not by an
admin persona.  I fixed adminPersonaCustom.convertAdminInvidToString()
so that it can gracefully deal with the possibility that the 'admin
invid' in an DBLogEvent produced by the DBEditSet commit() routine is
actually an end-user Invid.

8. [SERVER] Reworked DBSession.createDBObject()

The old createDBObject() call in DBSession had problems if a user
in an owner group tried to create a new object at the same time that
another user had the owner group checked out for editing.. the object
create would fail in an ungraceful manner, resulting in the server
thinking that the user had a negative number of objects checked out,
and getting very confused.

The DBSession createDBObject() code now does a transaction checkpoint
before attempting to place the newly created object in the proper
owner groups.  If one of the owner groups turns out to be unavailable,
the object create will fail and the transaction will be rolled back.

I also modified createDBObject() to return a ReturnVal so that
detailed failure information can be returned to the user if object
creation fails.

This is continuing a trend.. I've been finding the need to go through
the code and make some of the server's internal paths better about
returning detailed error information.

9. [SERVER] Improved DBEditSet.rollback() code

Minor changes to the transaction rollback code, to take advantage of
the optimized vector compare operations in arlut.csd.Util.VectorUtils.
Should make rollbacks in very large transactions noticeably faster.

10. [DISTRIBUTION] Added template schema directory, fixed installServer

In 0.99.5 and for awhile before, the installServer script would break
if you tried to install the Ganymede server without choosing a schema
kit, which caused installServer to break with a complaint about

'Couldn't write schema build config'.

installServer now goes ahead and installs an empty schema directory
with all of the build scaffolding required to develop your own schema
kit.

I am planning on significantly revising the way that schema kits are
packaged and distributed to make it easier to distribute schema kits
independently, but that is something for a future revision, I think.

Thanks to Glen Joseph <gjoseph@us.ibm.com> for reporting this.

11. [SERVER] Reworked embedded object initialization

In 0.99.5 and before, embedded objects had their custom
initializeNewObject() methods called before the embedded object
was linked into its container, making it hard for custom code
to make initialization decisions based on the container's state.

This has been fixed; initializeNewObject() for embedded object types
is now called after the object has been linked into its parent.

In addition, initializeNewObject() now returns a ReturnVal rather than
a boolean, to allow detailed error messages to be returned to the
user.  All custom DBEditObject subclasses included in the Ganymede
distribution have been brought up to date with the new
initializeNewObject() method signature.

Thanks to Martin Schneider <martin@ira.uka.de> for bringing this to my
attention.

12. [SERVER] DBEditObject.createNewEmbeddedObject() now returns ReturnVal

Another change made to provide better feedback paths for server
errors.

This method wasn't one that was ever overridden in the schema kits
provided with Ganymede, but adopters who did override this method
will need to modify their custom code to return a ReturnVal rather
than a boolean.

13. [CLIENT] Fixed string field GUI component.. now limits size properly

Way back in dev6a, I modified the arlut.csd.JDataComponent.JentryField
class so that it would employ a custom DocumentModel to make sure bad
characters couldn't be entered into a string/number/ip field in the
Ganyemde client.

Looks like I had somehow missed testing the code which was supposed to
prevent entry of characters after the size limitation on the field had
been met.  There was code there for that function, but it wasn't
correctly functional.

Modified the arlut.csd.JDataComponent.JentryDocument class to check
for the number of characters currently in the JentryField versus the
stated limit (if any) on the number of characters allowed in the
field.

14. [SERVER] Moved GanymedeServer creation, binding to end of server start

Previously there was a remote chance that a user or admin console
could attach to the server before all startup housekeeping was taken
care of.  The GanymedeServer RMI binding is now the very last thing
done by Ganymede.main().

15. [CLIENT] Fixed client behavior for Java 1.3 beta (Kestrel)

In the new Java 1.3 beta, code-named Kestrel, a lot of JInternalFrame
code has been changed.  Several things in the Ganymede client had to
be tweaked to insure internal windows in the Ganymede client work
acceptably well with all existing versions of Swing.

-------------------- Changes from 0.99.4 to 0.99.5 -------------------

RELEASE DATE: July 28, 1999

1. [SCHEMA] Several gasharl schema mods

The commitPhase2() method in userCustom for the gasharl schema now
calls out to 'directory_namer' and 'directory_maker' to handle
external actions when users are renamed or created.

Added support for generation of the ARL maildirect file, which will be
of absolutely no interest to anyone outside of ARL.

2. [CLIENT] Fixed client behavior on catastrophic transaction commit failure

The client wasn't properly getting rid of any edit windows if a
transaction commit failed catastophically.  This was occuring during
development and test of changes in 1.) above.. a NullPointerException
was causing the transaction to be aborted rather than committed, and
the client wasn't properly resetting its state to reflect this.

3. [DOCUMENTATION] Put a warning about deadlock possibilities into DBEditObject guide

The DBEditObject commitPhase1(), commitPhase2(), and release() methods
are called by the server while portions of the database are locked.
Using the GanymedeSession query mechanism will very likely bring the
server into deadlock, as I discovered.  Documented this in the DBEditObject
subclassing guide, as well as the javadocs for those methods.

4. [SERVER] Revised GanymediatorWizard spec, logic

In 0.99.4 and before, the GanymediatorWizard logic depended on the
custom wizard author writing a getStartDialog() method that had to
be smart enough to unregister() itself if no further interaction
with the user was needed.  It was very, very easy to not do this,
and so leave the client's GanymedeSession with a registered wizard
blocking further activities.

To prevent this problem, I have deprecated getStartDialog() in
favor of a processDialog0() method to start the wizard sequence.  This
has the advantage that the GanymediatorWizard's respond() method
will automatically unregister a wizard that no longer needs to talk
to the user.

This is an incompatible change in the GanymediatorWizard class that
will break existing wizard-using custom code, but fixing this now
will make wizard use far more reliable, and it would only be harder
to fix this later.

I don't expect many (any?) folks outside of ARL have written wizards
for use with custom schemas yet.  For those who are interested, the
GanymediatorWizard javadoc header now has some hopefully usable
documentation on how to do this.  (I've also written a guide on
the whole subject.. see change #18.)

Modified all wizard code in all of the included schema kits (not
counting the not-maintained ganymede.only schema) to use the modern
wizard support code.

5. [SERVER] Made transaction commit, abort clear active wizard

Another fix to help keep wizards from getting stuck.. aborting or
committing a transaction will now clear any wizard from registration
with the GanymedeSession object.

6. [SERVER] Improved lock system to prevent intra-thread deadlock

Modified the DBSession and DBEditSet lock handling code to help insure
that the server can't be deadlocked by a thread attempting to
establish a lock when it already holds one that would cause the new
lock establish() to deadlock the server.

This should only be an issue in terms of deadlock brought about during
transaction commit through errors in custom-plugin development.. the
existing system already prevented deadlocks between threads.  Now the
server will report an intra-thread lock error rather than simply
dead-locking.

Note that this deadlock prevention is in the form of an
InterruptedException being thrown, rather than permissive
lock-sharing.. the reason for this is that the operations that require
a read or dump lock (such as a query) will not be able to provide
transaction-consistent results when called from within a transaction
commit.

This change is entirely for the purpose of making the server more
friendly to customizers, and should not have any significant effect on
normal server operations.

7. [CLIENT] Fixed password handling in dialogs passed to the client

Wizards that asked the client for a new password value (as in the user
reactivation wizard in the gasharl schema) were not able to get a
password back because the password field logic in StringDialog and
JpassField was not working properly.

Wizards can now request a new password from the client properly.

8. [SERVER] Changed dump process to be safer

Previously, the server would dump the database by first renaming the
old on-disk database to ganymede.db.bak, and then write the current
in-memory database to ganymede.db.

This meant that if the server was killed while writing out the
database, there would be an incomplete ganymede.db file in place,
which would require the admin to manually recover from the
ganymede.db.bak file.  Things would always be recoverable, but it
wasn't an automatic process.

Now, the ganymede.db file is not renamed until the server has
completed dumping out its new dump, which is called ganymede.db.new
during the dump.

There are still one very narrow opportunity for stopping the server at
a point in which the server can't recover on its own at restart, but
the odds of this occuring are infinitesimally low now.. only if the
server is killed after it has renamed ganymede.db to ganymede.db.bak
and before it has renamed ganymede.db.new to ganymede.db will the
server be unable to properly recover on start-up.

This change does slightly increase the amount of disk space needed
during operations, as the system needs to have enough disk space for
three versions of ganymede.db at a time during a database dump.

Thanks to Darrell Tippe, <Darrell.Tippe@atcoitek.com>, for reporting
the problem.

9. [SERVER] Made DBObjectBase explicitly record max object id

The Ganymede server in some cases could erroneously re-use an Invid.

This could occur if an object was deleted, and the server was
restarted before another object of that type could be created.  The
server now explicitly records the maximum object id seen for
each object type in the ganymede.db file.

This change bumps the DBStore revision numbers to

	major: 1
	minor: 12

10. [SERVER] Seriously Improved Logging, Mailouts Systems

Reworked lots and lots code in DBLog.java, DBEditObject.java, and
DBEditSet.java to make all of the transaction logging stuff work
right.

--

Mail-related exceptions will no longer cause transactions to fail to
commit.. the server will report the mailer error, but continue with
normal processing.

--

Modified the Qstmp class so that it can queue messages for sending on
a background thread.. the extensive mail-outs in the revised logging
system was noticeably slowing down transaction commits.

All e-mail messages sent out by the DBLog class will be done without
blocking the logging thread.

--

The DBEditSet class now does transactional event logging after a
transaction's objects have had all of their commitPhase2() methods
called.  This change makes it possible for a commitPhase2() method to
send mail out.

--

Added variants of DBSession.viewDBObject() that will automatically
retrieve the original version of a DBObject that is being deleted by
the current transaction.  Useful for getting the original version of
an object during transaction commit from the object's Invid.

--

Added a getLabel() override to DBEditObject.java that will
automatically return the object's pre-deletion label if the
object is being deleted and the DBObject getLabel() routine
returns null.

--

Added standard methods to allow custom objects to identify email
addresses that should be notified when changes are made to them.
Useful for making sure that users get cc:'ed on any changes made to
their account, etc.

These methods are documented in the DBEditObject subclassing guide,
and are used by the logging system.

11. [CLIENT] Added tooltip support to checkboxes, other GUI field types

A number of the GUI field types supported by the clients were not
being added to containerPanel with any registered field comments set
as tooltips.  Fixed.

12. [SERVER] GanymedeScheduler could lose on-demand tasks

If the Ganymede server tried to initiate an on-demand task that was
currently running, the scheduler would 'lose' the task instead of
properly doing back-to-back execution of the task.  Big oops.

13. [SERVER] GanymedeBuilderTask didn't do overlap properly

The GanymedeBuilderTask base class has code that remembers the last
time the builder task was run, so that it can check for changes in
the various object bases that occurred since the last builder task
run.  Unfortunately, the time stamp was being recorded after the
builder task completed, even though changes could be made in the
database after builderPhase1() finishes and before builderPhase2()
completes, which left the possiblity for build tasks to not be
re-issued properly if two transactions affecting the same object
base were committed back-to-back .

The GanymedeBuilderTask base class now properly records the time
before entering builderPhase1(), so that rapid sequences of changes to
the database will always properly trigger subsequent builds.

14. [SERVER, CLIENT, CONSOLE] Cleaned up login sequence

Both the admin console and client will now give a nice dialog if login
fails.  Previously, the admin console showed an exception trace, which
did not make it clear that the problem was a bad username or password.

Made the clients include error.gif in several of their dialogs.

15. [CLIENT] Removed extraneous menu items from object windows

The object view/edit windows had a menu on them that included a number
of options that didn't always apply, as well as a 'print' option which
never worked well enough to be available to the end user.

Now, the object window's "File" menu only includes actions that can't
be done anywhere else.

The File menu will now only present the 'Set Inactivate Date' option
when the object being edited can in fact be inactivated.

16. [SERVER] Cleaned up schema dumping of persona objects

Modified DBObject and DBObjectBase's partialEmit() methods so that
supergash's locally-defined email address is not saved into the
ganymede.schema file.

17. [WEB FILES] Reworked the web access system for the Ganymede clients

Brian O'Mara did a very nice job of making a new set of web pages for
the Ganymede clients.  The new primary web page uses JavaScript to
launch the clients in their own frames, making it possible to run a
client and console simultaneously, and to be able to close the
original launch frame without killing the clients.

18. [DOCUMENTATION] Added a detailed guide to Wizard authoring

Added another long document on customizing the server with plug-in code,
this time on the topic of wizard authoring.

-------------------- Changes from 0.99.3 to 0.99.4 -------------------

RELEASE DATE: July 8, 1999

1. [SERVER] Optimizations to permissions system

The simplifications to GanymedeSession.getPerm(object,field) that
I did for 0.99.3 was in fact doing too much work, and the logic
didn't flow clearly.

GanymedeSession.getPerm(object, field) is now correct, efficient, and
easy to read.

2. [SERVER] Added new versions of anonymousLinkOK/UnlinkOK() to DBEditObject

It was previously impossible to grant anonymous linking privileges to
a field only when the other side of the link was being created in a
particular object type/field.  This posed a problem in the
circumstance where you want to allow a specific field in a given
object type to be linked to another object type's backlinks field
without having general editing permissions for the other object.

I added a couple of new versions of these methods that take additional
parameters identifying the originating object and field, so that
customizers can take the other side of the proposed link into
consideration before granting anonymous linking permissions.

See the DBEditObject.anonymousLinkOK()/anonymousUnlinkOK() javadocs
for details.  Hopefully I'll get around to writing a full-scale
technical manual for Ganymede that will go into when/where you would
use these methods.

Modified InvidDBField.bind() and InvidDBField.unbind() to call the
new versions.

3. [CLIENT] Improved StringSelector used in String and Invid fields

The StringSelector GUI component used in the client to allow editing
of String and/or Invid vector fields has been fixed so that it does
not include the text box for entering custom entries unless the
StringSelector is editable.

Fixed some minor GUI problems with the StringSelector resizing itself
to a minimum size after adding a new entry to an empty
no-choices-provided string selector, made the 'remove' button disable
itself after removing something.

4. [SCHEMA] Improved email list behavior in gasharl schema

An email list owner can now add any user, list, or redirect to his
list without having general editing privileges for that user, list,
or redirect.

This takes advantage of change #2, above.

5. [INSTALL] Various improvements to the install process

Modified installServer so that the custom loading instructions it
writes out at the end is sent to a README.custom file in the installed
server's directory rather than to screen.  No sense spewing a couple
of screen's worth of customized install instructions if the user has
to do cut and paste to save it to a file for reading.

Added an installWeb script to install the web files into a designated
directory, separate from the normal UNIX client install.  This has the
advantage that you can install the web resourcs for Ganymede without
having to have Swing installed locally (which installClient checked
for.)

Added a handy index.html file for the Ganymede web resources, got rid
of the old activator and hotjava directories, added a nice set of web
frames to allow the installer to bring up their browser and directly
visit the directory where they installed the web resources.

6. [WEB FILES] Updated plugin client files for 1.2

The plugin client files were re-generated from the 1.2 htmlconvertor
utility, and now include proper links to the latest version of
Sun's java plug-in for Solaris and Windows.

7. [SCHEMA] Fixed BSD schema master.passwd parser, Linux passwd parser

The old Linux and BSD passwd file parsers would break if they
encountered a user record with a blank password field.

Modified GanymedeServer.login() and GanymedeServer.admin() to make
sure the server won't throw a null pointer exception if the database
contains a user or admin persona with no password during the login
process.

If a Ganymede user record has no password set, that user will not be
able to login to the Ganymede server.  Unlike UNIX, Ganymede never
lets anyone log in without a password.

8. [ADMIN CONSOLE, SERVER] Reworked admin console, server's admin API

Fixed the admin console so that it will properly log itself out when
it is closed through the window system.

In prior versions, the admin console wanted to know the name of the
supergash account on the server, so that it could preemptively refuse
to perform certain operations.. this really makes no sense at all, and
was never needed for security.  The admin console now goes ahead and
attempts such operations, and depends on the server properly refusing
the operations.

The install scripts no longer prompt for the supergash account name
when installing the UNIX and web clients.

Also in previous versions, the server wasn't actually honoring the
console permissions checkboxes in the admin persona objects.. any
admin persona could log into the server via the admin console, but no
one other than supergash could do any control operations.  The admin
persona console permissions checkboxes now properly control the
server's response to admin console operations.

Admins other than supergash and monitor will need to log into the
server via the admin console using their full persona name.. i.e.,
'broccol:GASH Admin'.

9. [DOCUMENTATION] Revised, improved the install guide, added FAQ

Significantly revised and enhanced the install guide, corrected some
mistakes, included information on the use of installWeb.

The text INSTALL file is now a lynx-generated version of the HTML
file.. it was too much work and too error-prone trying to maintain
both independently.

Added a text FAQ file.  Will convert this into a nice web version at
some point.

-------------------- Changes from 0.99.2 to 0.99.3 -------------------

RELEASE DATE: June 23, 1999

1. [SERVER] Fixed value() in StringDBField.java

The value() method in StringDBField.java was improperly forcing
a permissions check, despite the fact that value() was used for
server-side code only.

This was causing a problem at transaction commit-time under certain
conditions, caused by the PermMatrix.union() bug below.

2. [SERVER] Fixed bugs in permissions system

Ugh, I had screwed up PermMatrix.union() in 0.99.2, the object->field
inheritance logic wasn't working properly.

I fixed PermMatrix.union(), simplified GanymedeSession.getPerm(), and
modified both PermEntry.union() and PermEntry.intersection() to optimize
out a lot of new PermEntry creation.

-------------------- Changes from 0.99.1 to 0.99.2 -------------------

RELEASE DATE: June 18, 1999

1. [DOCUMENTATION] Seriously Improved Javadocs

Lots more improvements to the source code javadocs.  Improving the
javadocs is an ongoing project.. I really want to make it possible to
get a very good feel for how the entire system works on a technical
level by simply browsing through the javadocs.  It's an interesting
exercise, and a continuing one.  If you find yourself puzzling about
something in the code, please let me know so I can document it better.

2. [SERVER] Improved DBLock, DBObjectBase classes slightly

Added some more explicit lock status check methods to DBObjectBase.
Modified GanymedeAdmin's schema edit status check to use the
new DBObjectBase.isLocked() method.

Pulled some methods and variables from the DBLock subclasses up to
DBLock.  Added DBLock.isLocked(Vector) to safely test a lock against a
vector of DBObjectBases.  Modified GanymedeSession.queryDispatch() to
use the new method, and to verify that the database remains read
locked at the completion of the query loop.  (the user could have been
knocked off via the admin console, etc.).

Documented all the pieces of the lock system in the Javadocs.

3. [SERVER DISTRIB] Added permdecodeDB command

permdecodeDB will take an existing ganymede.db file and report on
the permissions bits encoded by roles in the database.  Useful
for debugging permissions code problems.  See #4 below.

4. [SERVER] Corrected, simplified server's permissions system

The Ganymede server's permissions system had a complex and (as it
turned out) broken system for inheriting 'default permissions' for
fields in an object.  Previously, an object type could have a default
permissions bitset registered, and any fields in that object for which
permissions had not been explicitly set would take on the default
value.  This was intended to support reasonable behavior for newly
created fields, but the code to implement this had problems in some
odd cases.  The notion of an explicitly recorded default permission
set for fields that differed from the permissions on an object type
itself has been removed.  In keeping with this, the
setDefaultFieldPerms() methods on both PermissionMatrixDBField and
perm_field have been taken away, in favor of setFieldPerms(), which
sets permissions for all existing fields in an object type, but
doesn't involve the complex default logic.  The permissions logic has
been simplified, and better documented in the javadocs.

Now, when new fields are added to object types through the
schema editor, all Role objects will grant the permission for the
object as a whole to the new fields.  If you run the permissions
editor in the client on a Role object's permissions fields, you can
change this default.

Since the Ganymede client's permissions editor dialog had no code for
calling the setDefaultFieldPerms() methods on perm_field, this change
shouldn't affect any custom work people have done.  The gash and
gasharl ganymede.schema files did ship in 0.99.1 with a default
permission set on a few object types.  People using a ganymede.db file
derived from a gash or gasharl ganymede.schema may find that they need
to use the permissions editor to correct problems in the GASH Admin
Role object's permissions bits.

5. [SERVER] Changes to permissions fields now descriptively logged

If an administrator makes changes to the permission bits in a Role
object, the server's log will now record the precise changes, which
will be visible in the Role object's history panel.

The history recorded may be a bit deceptive in the case where a field
which previously had no explicit field-level permissions set has them
set..  if the object type has a permission of view/create/edit, and
you edit the field's permission to set it to view/edit only, the log
file will show that the field gained view and edit, when it
effectively lost create, which was previously inherited.

6. [SERVER] Miscellaneous methods added

Added debugging toString() to DBField, PermMatrix classes.
Added getFieldName(short) to DBObject, db_object.

7. [ADMIN CONSOLE] Fixed admin console

The admin console now works properly in a browser, and properly
initializes under all conditions.

The magic incantation to get the Ganymede Admin console to find the
arlut/csd/ganymede/GanymedeServer_Stub.class file was to include the
arlut/csd/ganymede/Client.class in the ganymedeAdmin.jar file.

Apparently, Sun's RMI classloading logic freaks out if it comes across
a reference to an unavailable RMI remote interface parameter in a RMI
remote method _Stub.class file, even if the freaked-out client in
question never refers to that method call.

I tried *everything* to figure this out, before finally noticing the
blurb in the Java plug-in's console about the Client.class load
attempt.. d'oh.  I wound up having to include
arlut.csd.ganymede.Client and arlut.csd.ganymede.GanymedeScheduler
because both were referenced by RMI methods or serialized objects the
admin console needed.

In addition to fixing the ganymedeAdmin.jar file, the applet code was
revised to be closer to that of the standard client, to improve its
behavior and appearance in a web browser context.

I still need to make a nice 'admin console' logo for the admin console
applet.

Thanks to Charles Adams (charles.adams@central.sun.com) and Andy
Johnson (andyjohnson@engineer.com) for reporting the problem with the
admin console not working properly with the Java plug-in.

8. [SERVER] Fixed login logic

In 0.99.1 and before, the server had an ambiguity that would allow
users to login to the wrong priviliged admin account on occasion, if
there was more than one account with the same persona name.

I.e., if there was persona account 'broccol:supergash' and
'supergash', each with different passwords, the broccol:supergash
password could allow access to the non-user-associated supergash
admin account.  This was true for both normal client login and for
admin console attachment.

9. [SERVER] Fixed DBQueryHandler's field DEFINED logic

The Ganymede server's DBQueryHandler would throw an exception if
attempting to process a "field is defined" QueryDataNode if the field
in fact did not exist.

-------------------- Changes from 0.99 to 0.99.1 -------------------

RELEASE DATE: May 27, 1999

1. [SERVER] Made decodeDB omit built-in field types by default

Did a bit of work to clean up the schema report code generated by
decodeDB.

2. [SERVER] Cleaned up, documented DBEditSet, DBEditObject

Added lots and lots of documentation to the DBEditSet transaction
class, made the commit() operation a bit more robust in terms
of relinquishing a write lock on exceptions and errors.

The DBEditObject class was modified to include a boolean finalAbort
parameter in the release() method.  Javadocs for this method were
improved to clarify and document the different ways this method can be
called.

3. [SERVER DISTRIBUTION] Added a stopServer commandline app

In order to facilitate automatic shutdown of the Ganymede server on
operating system shutdown, I've created a command-line stopServer
application that can shut the server down if you keep the server's
supergash password in its ganymede.properties file.

Yet another thing for production use.. we use this to support
automatic server shutdown when we reboot our Solaris Ganymede server.

4. [SERVER, CLIENT DISTRIBUTION] Made registry port number configurable

The Ganymede server and clients can now be configured to use an RMI
Registry on a non-standard tcp port.  This is necessary if you are
going to run the Ganymede server on the same system as other RMI
servers (such as Sun's SIMS administration console).

The server, client, admin console and password client all now accept a
property/parameter called ganymede.registryPort.  If the
ganymede.registryPort is not set, all Ganymede code will default to
the standard RMI registry port, 1099.

5. [SERVER] Added Session.findLabeledObject()

Added a convenient shortcut method in Session called
findLabeledObject() which allows the client to quickly get an invid
from an object type and name without having to use the full Query()
method.

This method is used by client/rpcpass.java.

6. [CLIENT] Added client/rpcpass.java to support Linux NIS rpc.yppasswdd

Added a program 'rpcpass' to the client distribution.  rpcpass is
designed to be used with the -x/--execute option present in versions
1.3.6.92 and later of the Linux NISkit's rpc.yppasswdd daemon
(included in the ypserv distribution), to allow users to use yppasswd
to change their password in Ganymede.

When run, rpcpass reads a single line from stdin,
which should be of the form

<username> o:<oldpass> p:<new cryptpass> s:<shell> g:<gcos>\n

Where the p:, s:, and g: fields are each optional.

After reading this line from stdin, rpcpass will connect to
the Ganymede server and change the password and/or shell.  If
this operation succeeds, rpcpass will emit a single line
to stdout,

OK\n

which is to be read by rpc.yppasswdd to signal success.  If
there are any problems, these will be described in a
report to stdout, which rpc.yppasswdd will take as a failure
signal.

Note that while the rpc.yppasswdd daemon may pass gcos information to
rpcpass, the gecos field is formatted differently on different
operating systems, and so rpcpass does not attempt to handle it at
all.

Of course, any program can use the rpcpass program to propagate
password/shell change requests to Ganymede, not just the Linux NIS
kit's rpc.yppasswdd daemon.

7. [SOURCE] Fixed a couple more PATH problems with building sources

A couple of fixes relating to doing source builds with '.' not in
one's PATH.

EXECUTING: src/server/build
  ERROR     :  buildrmi: command not found
  FILE FIXED: src/server/build
  PREV LINE :  buildrmi
  NEW LINE  :  ./buildrmi

EXCECUTING: src/classes/buildJar
  ERROR     :  sync_tree: command not found
  FILE FIXED: src/classes/buildJar
  PREV LINE :  sync_tree
  NEW LINE  :  ./sync_tree

Reported by Andy Johnson (andyjohnson@engineer.com).

8. [DOCUMENTATION] Significantly Improved Documentation

Fixed all link errors in the javadocs.  Added more javadocs to several
classes.

Reworked doc/index.html.  Added an HTML version of the INSTALL file,
cross-linked to a file glossary.  Added the log file format documentation
to the doc distrib.

-------------------- Changes from 0.98 to 0.99 -------------------

RELEASE DATE: April 16, 1999

1. [SCHEMA] gasharl schema mods

Various kinds of objects listed in an emailList object are now
shown with extra detail so that admins can see what sort of thing
they are dealing with.

Modified interfaceCustom so that the server will now check ethernet
addresses for valid form.

2. [CLIENT] Newly created object windows can now be undone.

Put the close box back on object creation windows.. now if you
close the window, the client will prompt you as to whether you want
to delete the object.  If you do, the server will drop the newly
created object from the server when the transaction commits.

Unfortunately, the code to handle intercepting the internal frame
close request in framePanel.java and windowPanel.java had to be
hacked with the VetoablePropertyChangeListener since Swing 1.1
and 1.1.1-beta1 have JInternalFrame.setDefaultCloseOperation()
totally broken.

If you click the close box on an edited object, you'll also
get a dialog explaining what that does and does not do.

3. [SERVER] Server always allows deleting a newly created object

This provides an exception to change 0.98.5 to allow the client to
always undo an object creation, even if normally they could only
inactivate the object, not delete it.

The server had always had support for DROPPED objects as well
as DELETED objects, the change above just really takes advantage
of this more.

4. [SERVER/CLIENT] Added case-insensitive regexp matching

The server and client now support performing a case-insensitive
regular expression comparison when querying on string and object
reference (invid) fields.

5. [CLIENT] Scalar invid fields are now always sorted

Scalar invid fields would become unsorted when the server instructed
the client to refresh them.  The client will now properly sort
invid lists from the server on field refresh as well as on field
construction.

6. [CLIENT] Client code simplified, documented, improved

Embedded object vector panels were seriously broken, and would break
when you tried to delete entries.  The vectorPanel refresh logic
was totally reworked and should behave properly in all situations now.
Loading embedded object vectors will be more efficient now.

The client's tracking of container panels active loading information
from the server has been simplified and better documented.. the client
does less work now to track container panels, and hitting cancel or
commit while loading a panel should still do the right thing.

Lots and lots of javadocs added and corrected in the client code.

Some of the debug output when running the client in debug mode has
been cleaned up.

7. [CLIENT] Fixed error on loading 'inactivate.gif' on HPUX JVM

The client included code to try to create an inactivate icon from a
file called inactivate.gif.  That file did not exist in the jar file
distributed with Ganymede 0.98, and the JVM on HPUX 10.20 threw an
exception on this, even though Sun's JVM's do not.

Reported by Lewis Muhlenkamp (lewism@tt.comm.mot.com)

8. [DISTRIBUTION] Build scripts, make files no longer require '.' in PATH

The Ganymede install and build scripts required '.' in the user's PATH
during install.  This should no longer be the case.

Reported by Lewis Muhlenkamp (lewism@tt.comm.mot.com)

9. [DISTRIBUTION] configure prompts for Swing jar location

Previously, src/make client would fail because the client's build
script didn't know how to find the Swing swingall.jar file if it
wasn't in the user's CLASSPATH.  Now the initial configure bootstrap
script will ask for the location of the Swing jar files so that the
build scripts in the source tree can be properly configured.

Reported by Lewis Muhlenkamp (lewism@tt.comm.mot.com)

10. [CLIENT] Fixed focus handling for IP address component

The JIPField GUI component class had problems with reentrancy that
could cause problems when dialogs were thrown up in response to
focus loss.

11. [CLIENT] Added toolbar to the Query Results window

The File menu on the Query Results window was really unobtrusive, and
didn't really emphasize to users that they could mail out the query
table.  Brian put in a spiffy mini-toolbar to show people what's up.

12. [CLIENT] JComboBoxes in client are more keyboard friendly

Previously, JComboBoxes in the client used the default
KeySelectionManager, which only allowed you to search for items by
their first letter.  You can now type in a full name of an object and
as long as you don't pause for more than a half-second between
keystrokes, the JComboBox will find the object by its full name.

Actually, this doesn't work as well as it sounds, because Swing's
KeySelectionManager stuff doesn't let you use the keystrokes to choose
an entry in a JComboBox before actually selecting it, and selecting
items in the Ganymede client will tend to be a slow operation due to
the round-trip to the server.

13. [SERVER DISTRIBUTION] Added decodeDB command

Added a new executable in the server's bin directory, decodeDB, which
takes the name of a ganymede.db or ganymede.schema file and generates
a print-out of the file's object and field type information.

This was added mainly to make it easier for me to do quick checks on
the ganymede.db and ganymede.schema files packaged with Ganymede
releases.

14. [DISTRIBUTION] Fixed linux, bsd, nisonly schema kits

In Ganymede 0.98, the linux, bsd, and nisonly schema kits had an
incompatibility that rendered them inoperable.  This had to do with a
field in the admin persona object type that was not defined in these
kits, but which the server had been modified to depend on.

I've modified the kits so that the field is defined, and modified
the server so that the lack of the field won't prevent an existing
ganymede.db file from being usable.

Reported by Nikola Nedeljkovic, nikola@carbonix.iam.tu-clausthal.de

15. [SERVER] Upgraded to gnu.regexp 1.0.8

Updated the gnu.regexp classes used in the server to version 1.0.8.

-------------------- Changes from 0.97 to 0.98 -------------------

RELEASE DATE: March 17, 1999

1. [BUILD] Ganymede build scripts moved out of db/out to db

The Ganymede server now uses a property called
'ganymede.builder.scriptlocation' to locate the builder scripts to be
run by the builder tasks registered in the server.

Previously, the builder tasks ran the build scripts in the out
directory, which wasn't really compatible with the decision in 0.97 to
have the build tasks archive the output files written by the build
tasks.  Now the build scripts are expected to be located by default
in the directory about out, which allows the zip archives to be
cleanly made.

There is still some confusion here.. in theory, you can have multiple
build tasks registered in the server, each of which will try to zip up
the out directory.  This is not optimal, but what to do about it?

2. [SCHEMA] Added emailListCustom class to gasharl schema kit.

The email list members field is an invid field that can point to
objects of different types.  It is used to allow the email list object
to have members that are users, external email listings, or other
email groups.  Previously, the email list members field couldn't really
be edited after the bulk loading process was complete.

3. [CLIENT] Made windows for newly created objects non-closeable

The client allows the user to close object windows manually as a
convenience, to avoid cluttering up the client when editing lots of
objects.

Unfortunately, if we allow the user to close a newly created object,
there isn't necessarily a good way of getting that window back if it
turns out that the user left some mandatory field unfilled when they
go to click on 'commit'.  Now object creation windows will not be
closeable, so that it's not possible to 'lose' object windows that you
need to do further editing on.

4. [CLIENT] Modified the list editor GUI component

The 'list editor' GUI component used in the client may have a text
field at the bottom to allow the user to simply type in something
to add to the list.  Previously, the 'Add' button connected to the
text field was enabled all the time, confusing users manipulating
the component.  Now the 'Add' button will only be enabled when the
user has entered something into the text button.

5. [SERVER/CLIENT] Modified removal/inactivation handling

The server now by default refuses to allow objects to be removed
that have an inactivation protocol defined.  supergash can go ahead
and remove objects directly, and custom DBEditObject subclasses can
still redefine this behavior by overriding canRemove().

In conjunction with this change, the client has been modified so
that it will always show 'delete' as an option when right-clicking
on an object, even though the object may have an inactivation
protocol defined.  This makes things simpler, since it would be
difficult to handle the delete toolbar icon properly if I just
wanted the client to prevent the user from ever trying to delete
an object that can be inactivated.  And besides, the user might
be operating with supergash privileges.

6. [SCHEMA] Lots of refinements to gasharl schema

Modified the loader code to use BufferedReader, and to take care
of a few deprecation warnings.  Loading files from GASH should
be a good bit faster now.

The GASHARL schema now includes fields in the admin persona and owner
group objects to track information used for more complete GASH
compatibility.  New users and groups will be created with their
uid and gid starting in a numeric range compatible with GASH's
permission model if the owner group has a minimum uid/gid set.

The GASHARL schema now includes support for handling ARL contract
information, which will be of no interest to anyone outside the
lab.

7. [CLIENT] Reworked the persona select handling

The client now prompts users with multiple admin personae with a
persona selection dialog on log-in.

8. [SERVER] Fixed the html schema dump code in DBStore.java

Oops.  DBStore had my home directory path hard-coded when it
should have been using the ganymede.htmldump property.  Fixed.

9. [SERVER] Fixed a bug preventing creating new object types in develop mode

The server's schema editing code had a bug that caused it to reject
the creation of new object types when the server was run with -develop
on the command line, which allowed for the creation of new universal
fields.  Not a bug that anyone else should have run into, but some
aspects of this bug had the server doing some unnecessary extra work.

10. [CLIENT] Changed date representation in the query table

The query table will now use 4 digit years when showing dates.

11. [CLIENT] Fixed Calendar GUI

Previously, viewing an object that had date fields in it was not
handled properly.. the date field in most respects did not act
properly (clear button, calendar popped up in edit mode, etc.).

Fixed bug in JPanelCalendar which caused the calendar's year field not
to be refreshed properly upon rejecting a date as out of range.

12. [CLIENT] Fixed stupid lingering 'working guy' window bug

Fixed the problem with the 'working' internal frame not being properly
disposed of, and with it being hidden behind existing windows when
created.

13. [CLIENT] Redesigned query dialog

The querybox has been redesigned for clarity and user-friendliness.

The querybox now allows the ownerlist field to be searched on.

14. [SERVER] Optimized memory handling for strings

All strings loaded into the database are now interned(), trading
database load time for memory/runtime efficiency.  Not sure that this
makes a noticeable difference, but it will be of some help.

15. [CLIENT] Fixed dialog placement and sizing errors

The JCenterDialog placement code was broken if the dialog was larger
than the frame parenting it.  All Ganymede dialogs should now always
be entirely visible on screen.

The JCenterDialog's pack() method improperly adjusted the size of
centered dialogs, causing the edges of centered dialogs to be too
small.  All dialogs should now be properly sized on NT as well as
UNIX.

16. [SERVER] Empty strings are considered undefined in StringDBField

The server's required fields logic wasn't detecting an empty string (a
string equal to "" rather than null) as an undefined field, which it
should do and now does.

17. [SERVER] Improved error reporting paths in server

Changed the verifyNewValue() methods in DBField and DBEditObject
to return a ReturnVal so that those methods can pass back detailed
error dialogs to the user.

18. [SERVER/ADMIN CONSOLE] Added more information to admin console

Added a column to the active user table to show how many objects each
user is actively editing, to help track actual use.

Added a column to the task table to show when each task was last run.

-------------------- Changes from 0.96 to 0.97 -------------------

RELEASE DATE: February 11, 1999

1. [DISTRIBUTION] Fixes to the distribution set

There was an error in scripts/sync_tree.admin.in.. it didn't include
the NameSpaceEditor class required by the admin console's schema
editor, so editing namespaces in the schema editor was broken in 0.96.

Thanks to frederick.j.dickey@boeing.com for reporting this.

2. [DOCUMENTATION] Fixes to the customization guide

The DBEditObject customization guide has lots of references to javadoc
pages, but the method links weren't properly constructed for use with
the 1.2 javadoc output format.  You should now be able to click on
method names in the customization guide and be taken directly to the
method docs.

Brought the guide's content up to date with recent changes to
DBEditObject.

3. [SERVER] Amended journal loading code for production mode

The journal loading code in the Ganymede server now simply complains
politely to stderr if the journal is truncated inappropriately, rather
than throwing an IOException.  This allows the server to recover
gracefully if the server was killed while in the middle of writing a
transaction to its journal.

The Ganymede server no longer keeps the old journal files around, but
rather just keeps a single back-up.  While it was entertaining doing a
hexdump on old journal files to see what had been done, it cluttered
the db directory more than being of any use.

4. [SERVER] Changed how file backups are handled

Previously, all of the schema kits' builder tasks copied individual
files to file.<date> when the builder task was re-run.  Now any time
a builder task is executed, the existing output files are zipped into
a file named <date>.zip in an 'old' directory in the output directory.

Actually, didn't modify all of the kits.. I didn't bother to modify
the ganymede.old kit, which is really in need of a major overhaul.

The ganymede.db file itself is now archived when the newly created
archive task is run (on command by an admin console currently), or
when the server starts up with a dirty journal.

5. [SERVER] Enhanced Task Management

The server's task scheduling system was basically replaced.  The old
'Builder Task' object type is now a generic 'Task' type, with support
for editing scheduling parameters directly from the Ganymede client.
You can now add, remove, or tweak any task's scheduling parameters
from the Ganymede client.

Previously, of course, all but the builder tasks were configured by
hand in Ganymede.java.  Now it will be possible for Ganymede adopters
to have total control over task registration and scheduling without
having to edit code.

The downside to this is that the server will not have the 'standard'
tasks scheduled unless the ganymede.db file you are using has the
tasks registered.

Most importantly, the change from Builder Task to Task object types is
an incompatible change from 0.96 and previous releases.  You will need
to start from scratch with the schema files included in the Ganymede
0.97 release in order to get the new Task object definition that the
server now requires.  Or, if you are particularly enthusiastic and
don't want to lose data already in Ganymede, you could use the schema
editor to bring your pre-0.97 datafile into compliance with the 0.97
Task object schema.

Generally, I am not committing to a freeze on the basic schema
definition until 1.0.  I don't expect any more incompatible changes to
the basic schema, but no promises until 1.0.

See doc/server.html for a guide on the new task scheduling system.

6. [CLIENT] Better taskbar icons

Brian reworked the taskbar significantly, with bigger and nicer
taskbar icons.  The dialogs brought up when the toolbar buttons are
clicked have been reworked a bit to be more distinctive.

More work coming to the client's GUI in 0.98.

7. [CLIENT] Fixes to date GUI code

The date selector dialog now properly shows the time of day set in a
date field.  You could set the time of day before, but upon reloading,
it would not be displayed properly.

-------------------- Changes from 0.95 to 0.96 -------------------

RELEASE DATE: January 29, 1999

1. [DISTRIBUTION] Fixes to the distribution set

Added the missing calendar.gif used by JdateField.  Added missing
up.gif and down.gif used by JpanelCalendar.  Added doc/buildjavadoc to
automate building of javadocs.

Added a -develop option to the configure script, to allow doing builds
in a CVS checkout directory, with the results of compiles put in a
location outside the checked out tree.  For use when developing and
maintaining Ganymede.

2. [SERVER] Fixed bug in query engine

Queries that combined embedded and non-embedded fields would cause
the server to throw an IllegalArgument exception, complaining about
a DBObjectBase being locked with an extantLock.

Made the query engine a bit more regular with respect to dump()
vs. query().  Made things a bit safer.  Improved the server's handling
of a forced disconnect of a user who is executing a query.

3. [CLIENT] Fixed bug with open object dialog and inactivated objects

The client displays inactivated objects in the tree with "(inactive)"
added to the end.  The dialog brought up when clicking on the Edit,
Delete, or View buttons on the toolbar were including the "(inactive)"
bit when brought up with a tree node selected.

4. [SERVER] Made GanymedeExpirationTask more robust

Previously, the Ganymede Expiration Task would be unable to commit
its transaction if, by removing objects from the database, other
objects were left with empty fields that are normally mandatory.

Now, the expiration task will commit successfully, possibly leaving
some objects with empty invid fields that would ordinarily not be
permitted.  This is the most straight-forward way of handling things
like systems with required user fields, which would otherwise have to
be automatically removed, a much dicier prospect.

Probably need to add a feature to scan the database for objects which
do not satisfy the required-fields check.  Any time an admin edits
an object that doesn't satisfy the check, the user will be required
to fix the object before that transaction can be committed.

5. [SCHEMA] Fixed the gasharl schema's user inactivation logic

The userCustom object in the gasharl schema was not properly
differentiating between the case of wizard-led inactivation and
automated inactivation in handling checkpointing and rollback.

Note that I'm not really maintaining the 'gash' schema at this point..
I'm concentrating on getting gasharl to the point where we can use it
here in the lab.  At some point someone is going to have to take on
the task of doing a generic DNS schema that lots of people can easily
use, but it's not me right now.

6. [SERVER] Added GanymedeValidationTask

Added GanymedeValidationTask, a task that scans through the entire
database and reports on any objects which are missing ostensibly
required fields.

7. [ADMIN CONSOLE] Added a clear log button

Added a clear log button to the admin console, to allow admins to
reset the log display on the admin console.

8. [CLIENT] Various cosmetic tweaks

Mike did a nice help system, with index and what-not implemented in
HTML, but we never wrote any content for it, so I've taken the help
menu item out of the help menu for the time being.

Put the release info and copyright info in the about.. dialog.

There are a couple of places in the client that were throwing
NullPointer exceptions due to Swing's choice of behaviors.  We were
catching it, but reporting it to stderr.  Now we just stay silent
about it.

Modified the glogin applet so that it gives a better appearance when
invoked in a web browser.

9. [SERVER] Upgraded to gnu regexp 1.06

The gnu regexp classes used in the server have been updated to version
1.06, the 18 November 1998 release.

-------------------- Changes from 0.94 to 0.95 -------------------

RELEASE DATE: January 22, 1999

1. [DISTRIBUTION] Final distribution preparation

Some more tweaks to the build scripts.  Added some more README type
files. Changed COPYRIGHT to a declaration of the GNU General Public
License.  Added GPL boilerplate to all source code.

-------------------- Changes from dev6a to 0.94 -------------------

RELEASE DATE: January 21, 1999 (internal)

1. [CLIENT] QueryBox improvements

The client was broken when it came to submitting requests for a
membership test on vector fields.  This caused the server to always
return empty results when querying on invid vector fields.

Also, the querybox now just prompts for the label of an invid to match
against, rather than forcing the user to scroll through a (possibly
huge) list of possibilities.  The user can actually use any string
comparator on simple scalar invid fields.

Regular expressions can be used on vector string and invid fields
using the 'Contain Matching' operator.

The list of available object types and fields to search on is
now sorted.

2. [SCHEMA] Fixed gasharl schema to expand group ownership privileges

The gasharl schema now allows an administrator who has privilege to
edit a group to place any user in that group, even if the
administrator would not normally be able to edit those users.

3. [CLIENT] Icons shuffled

The tree no longer uses the icons from the toolbar.

4. [CLIENT] Permissions editor now uses a hybrid table/tree GUI.

Brian O'Mara has made the permissions editor much spiffier, by using a
hybrid table/tree Swing GUI component.

5. [DISTRIBUTION] Source included

We've spent the last couple of weeks reworking our build scripts and
getting everything into CVS so that we can release ganymede with
source code and allow people to tweak and build everything themselves.

-------------------- Changes from dev6 to dev6a -------------------

RELEASE DATE: December 4, 1998

1. [CLIENT] Fixed another long-standing table rendering bug

The table code had an obscure bug that would cause the vertical lines
to be misdrawn if the table's vertical height got too big, either
through having a lot of rows, or by having a lot of word-wrapping.

2. [CLIENT] Fixed tooltips on vector panel element wrappers

The 'expand this element' and 'close this element' tooltips were
reversed after an element was first opened.

3. [CLIENT] Made the QueryBox pop down calendars

Basically took care of the problem handled in change dev6.10 in the
query dialog.

4. [SERVER] A number of fixes relating to schema editing

There was an opportunity for field ordering changes to not be kept..
it turned out to be a bug with wider implications, so a fair bit of
code was changed to make things work better.  Turned off some
debugging which should make schema editing go faster.

5. [SERVER] Changed the permissions inheritance system

Previously, the default role's Objects Owned permissions bits were
inherited into all other roles.  This is inappropriate, as it might
be desirable to have an admin who has less power over a broad set
of objects owned by a linked object group than end-users have over
themselves and other implicitly owned objects.

That is, for all admin personae other than supergash, permission to
mess with object types and fields in objects owned must be explicitly
granted by a non-default role.  Default privileges over non-owned
objects are still enjoyed by all, and end-users still get the
'default' objects owned privilege over their implicitly owned objects.

6. [CLIENT, ADMIN CONSOLE] Replaced Homer and Krusty with custom icons

Brian O'Mara has done some really nice work on the icons for Ok,
Error, and Question in the client and admin console.

7. [CLIENT] GUI Calendar fixes

Previously, the GUI Calendar object used in the client was not able to
handle a non-set date.  In addition, the calendar did not properly
reset itself if the user hit a date button that was rejected by the
server.  These have been fixed, and the client's datePanel class now
directly clears the calendar rather than displaying a 'This date will
be cleared when the transaction is committed' message.

8. [SCHEMA] gasharl schema user expiration limitation implemented

The gasharl schema will now properly restrict changes to a user's
expiration date in keeping with the limits set in the user's category.
User's with mandatory expiration dates will not be able to be cleared,
nor set too far in the future, relative to the current date.

9. [ADMIN CONSOLE] Revised console applet to simplify login

You can now hit return in the password field of the admin console
applet to log in.

10. [CLIENT] Changed code for text fields

We now use a custom DocumentModel to filter keystrokes in the client's
text fields, which is more reliable, works properly in JDK 1.2rc2, and
will properly handle cut and paste text entry.

11. [CLIENT] Rewrote permissions editor

The permissions editor is now based on the Swing JTable class,
providing better performance.  The code is much cleaner now.

-------------------- Changes from dev5c to dev6 -------------------

RELEASE DATE: November 9, 1998

1. [CLIENT] Replaced the 'click' menu on the scalar invid chooser fields

Replace the confusing 'click' menu with a pair of buttons to view and/or
create a remote target for a scalar invid field.

2. [CLIENT] Modified the status label handling

The status bar at the bottom of the client will now be automatically
cleared after 5 seconds by default, to make the status label less
confusing.

3. [CLIENT] GUI Tweaks

Some new icons, tweaks to dialogs, fixes to a few graphical glitches.

4. [CLIENT] A few major fixes to table code

The table code had a horrendous bug that made the table rendering
routine take forever with large datasets.  All table operations on
large tables should be very much faster now.

A number of memory and speed optimizations were made along with the
bug fix, probably of minor impact, but maybe not.

Fixed a couple of edge-case rendering problems in both the table and
the tree that could screw things up if the vertical or horizontal
scrollbar was set to max and then the table/tree was grown to a larger
size, but not large enough to get rid of the scrollbars.

5. [CLIENT] Fixed date bug in QueryBox

The JdateField and JpopUpCalendar classes had a problem such that
if the JdateField had no callback set, the JpopUpCalendar would
always complain that the date entered was unacceptable, which broke
things in the QueryBox.

6. [SERVER/CLIENT] Made the DumpResult class more memory efficient

In what is probably at best a fairly minor aid to garbage collection
on the server, the DumpResult class used to pass tabular query results
from the server to the client has gained the ability to self-dissolve,
making GC go quicker if things get tight while handling a query.

7. [CLIENT] Fixed re-entrancy problem with JstringField, JnumberField

The GUI fields used in the client's object editing windows had a
problem wherein they could trigger a callback to the server several
times for a single change, if the focus is taken back and forth from
the GUI field whose changed triggered the callback.  This was a
particular problem with changes to fields that triggered wizards,
because the focus transition between the wizard dialog and the main
client window could cause the callback to be re-issued repeatedly.

The focus/callback handling code in JstringField and JnumberField have
been made safe for re-entrancy.

8. [SERVER] Fixed admin persona rename on user rename

Fixed a bug in the adminPersonaCustom class so that the label for
adminPersona objects are properly refreshed when the attached user
is renamed.

9. [SERVER/CLIENT] Added regexp capability to the server's query engine

The dev6 server takes advantage of the gnu.regexp package to provide
regexp based string match capability in the server's query engine.

10. [CLIENT] Cleaned up auxiliary windows on frame close in client

The client now closes any date field calendar windows when the
containerPanel held in a object editing window is closed.

11. [SCHEMA EDITOR] Made some dialogs more descriptive

For instance, when you go to delete a field in the schema editor,
the confirmation dialog mentions what field it thinks you are trying
to delete.

12. [INSTALL SCRIPTS] Added installClient2.perl for installing the client under 1.2

The installClient2.perl script is provided to install the client on
systems using the 1.2 JDK, which includes the Swing GUI library
required for the operation of the Ganymede client and admin console.

-------------------- Changes from dev5b to dev5c -------------------

RELEASE DATE: November 2, 1998

1. [SERVER] Fixed bug in GanymedeWarningTask

The GanymedeWarningTask class had a bug such that when it went to
report on an object's incipient removal, it would throw a
NullPointerException.  The task scheduler did properly clean up after
itself, but no warnings about incipient removal were sent.  Fixed.

2. [CLIENT] Fixed buttons embedded object wrapper

Previously, the buttons on the blue embedded object wrapper did this
ugly flashing thing when you clicked on them.  No more.  Brian O'Mara
did nice pressed/not-pressed versions of the expand/contract icon, and
figured out how to make them not be surrounded by the brown yuck.

3. [CLIENT] Set menu hot keys for all the menus

Some of the menus in the client had no hot-keys associated with them.
I set hot keys for most of the menus.. the version of Swing in
JDK 1.2 RC 1 has a bug that makes it impossible to give an item in
a menu the same hotkey as the menu itself has, so I didn't set menu
items where it would naturally lead to a conflict.

4. [CLIENT] Improved open/close box handling in tree, rendering

Previously, the tree open/close boxes were handled by the tree when it
received the mouseClicked() signal.  Now the boxes are handled on the
mousePressed() signal, which happens faster.  It seems that the
mouseClicked() and mousePressed() handling were confusing each other
occasionally before.

Also, the tree would only render rows that it could fully display,
leaving a bit of empty space at the bottom of the tree, depending
on where the scrollbar was with respect to the division point between
rows in the tree.  This has been fixed.

5. [CLIENT] Reinstated applet.destroy(), made the client respond to disconnect

Change dev5b.8 turned out not to be practical with the JDK 1.2
plug-in, as it will halt the plug-in if the browser leaves the
applet's page, regardless of whether the destroy() method does
anything or not.  So, I reinstated the destroy() clean-up method to
help insure a clean logout.  This is unfortunate for users with the
1.1 plug-in, but the 1.2 behavior is quite arguably correct, so
we'll just request users running Ganymede in a browser not to leave
the page while they're running it.  At least now they'll get very
clear feedback when they do.

Made the client able to respond properly to disconnect commands from the
server, even when running as an applet.  This change makes the disconnect
works properly under the Java 1.2 security model.

6. [CLIENT] Intercepted client's main window closing

Now, when you click on the window manager (or Windows') close box to
shut down the main client window, it will handle that in the same way
that it handles the logout menu item.

7. [CLIENT] Fixed title for created object panels

This is just a tweak to the dev5b.3 change.

8. [CLIENT] Made the persona combo not show up if the user has no personae

The persona selector combo box will no longer appear if a user logs
into the Ganymede client who has no admin personae attached to his
user account.

9. [CLIENT] Made the cancel button close all windows in the client

Before, clicking on cancel would close just the editing windows in the
client.  This could be potentially misleading to the user, as view
object windows can be left displaying the status of things as they
were during the transaction which was since aborted.  The commit
button will continue to leave up the non-editable windows.

-------------------- Changes from dev5a to dev5b -------------------

RELEASE DATE: October 28, 1998

1. [SCHEMAS] gasharl schema fixes

The gasharl schema included in dev5a did not properly have the custom code
to manage the user category object specified.

The system object's fieldRequired() logic did not check the associated
system type to see if the associated user field was required.

Both fixed.

2. [SERVER] Took out limit variable from GanymedeServer

The arlut.csd.GanymedeServer class had a 'limit' parameter in its
constructor, ostensibly for allocating space for a specific maximum
number of concurrent clients.  In fact, this parameter didn't actually
do anything and so it was removed.

3. [CLIENT] Titlebars in all windows now include type of object

There were a number of cases in which object panels would pop up
without any indication of what type of object was being displayed or
edited.  This has been fixed.

4. [SERVER] Fixed permissions checking for object viewing

In dev5a and before, the server could throw an
IllegalArgumentException if the user tried to view an object that
contained fields that the user did not have permission to see.  This
has been fixed.  The server will now just omit those fields from the
FieldInfoVector passed back to the client.

5. [CLIENT] Added ARL logo

I spent some time playing with gimp, to fancy up the login window,
and to let people know where ganymede came from.

6. [SERVER/CLIENT] Optimized memory usage for query handling

In dev5a and before, the client can take an enormous amount of
memory in processing a set of query results from the server, as
a result of inefficient memory usage in java.lang.StringBuffer().

The code used by the client to process query results now uses a
modified StringBuffer() designed to be far more memory efficient,
so the client should be able to handle much larger queries.  Truly
large result sets may still be a pain to handle in the table, speed-wise,
but the client shouldn't just crap out trying to digest the results
from the server.

In addition, the client now traps any errors in query processing and
will clean up the 'guy at work' window if the query fails for some
reason.

7. [SERVER/SCHEMAS] Fixed embedded object creation logic, pulled it into DBEditObject

Previously, the gash and gasharl schemas included code in the
userCustom and systemCustom objects to handle creating embedded
objects.  That code was broken, as it did not go through the normal
permissions checking to see if the user had permission to create the
object.  Now, in order to create an embedded object of a given type,
the user must be granted privilege to create that kind of object.
Having editing authority over the embedding field is no longer
adequate.  This change makes it possible to grant permission to edit
or delete entries in an embedding field without necessarily granting
privilege to create new entries in the field.

In addition, the code to do the creation of embedded objects was
essentially the same in the gash and gasharl userCustom and
systemCustom classes, so I pulled the code up into
DBEditObject.createNewEmbeddedObject().  Customizers can still
override createNewEmbeddedObject() in their custom DBEditObject
subclasses, but the normally expected behavior will be handled by
DBEditObject if they do not.

8. [CLIENT] Made the client applet not log out when the browser leaves the page

Previously, the client would log out as soon as the browser that it
was running in left the applet page, even though the client's other
windows were left up.  Now the client won't disconnect from the server
until forced to by the browser closing.  This was done by having the
glogin applet ignore both the stop() and destroy() methods in JApplet.
This probably wouldn't be the right thing to do if the client were run
directly out of a browser, but when using the Java plug-in, this lets
us keep going until the browser is shut down.  The server is able to
automatically disconnect clients that go inactive, so it isn't
strictly necessary that the client do a clean logout.

9. [CLIENT] Fixed the StringDialog class

The dialog used for entering a persona password when changing personas
was some broken, in that if you entered your password and hit return
it would fail, and in that on Windows, the dialog would come up with a
big scrollbar across the dialog.  Both of these have been fixed.

This change will also make it possible to hit enter in the last field
in a wizard to progress to the next.

One possible detriment to this change is that the main dialog class
used in the Ganymede client no longer includes a ScrollPane.. this
means that if a wizard dialog is given too much stuff, the dialog
might not be able to fit on the user's screen.. but no wizard dialog
should have that much stuff anyway.

Also made it so that if the StringDialog window is closed without
cancel being hit, the dialog treats that as a cancel request.

10. [SERVER] Fixed embedded object query handling

In dev5a and prior, the server would not return proper results if a
equality test query was made on an namespace-managed field in an
embedded object.  This caused corrupted results if you tried to
search for a system by IP address in the gash and gasharl schemas.

-------------------- Changes from dev5 to dev5a -------------------

RELEASE DATE: October 19, 1998

1. [SERVER/CLIENT] Added support for date, multiline text fields in dialogs

The dialog system used to pass wizards from the server to the client
has been radically simplified, and now depends on the Java
serialization system rather than its own serialization /
de-serialization code.

The server can now pass multiline text fields and date fields to the
client for use in wizards.

The GUI calendar/date field class has been reworked a bit so that
the system is more robust when dates outside of the defined range
are entered.

2. [SERVER] Reworked database field handling

Made the server more robust in the face of changes made with the
schema editor.  Converting fields in use between vector and scalar
will no longer cause the schema/db dump to fail, but changing a
field between scalar and vector will cause any instances of that
field loaded in the database to be dropped.

All of the DBField subclasses now use explicit isDefined() and
setUndefined() methods to handle tracking whether a field is
defined or not, rather than using a defined flag.  This makes
things much more flexible and robust, as the isDefined() method
can check things out at any time and report on the status of the
field, rather than having to try to properly update the defined
flag everytime anything could possibly affect it.  Like changes
to the schema, say.

3. [SERVER] Made DBEditObject.{getSession(), getGSession(), getOriginal()} public

This change allows custom wizard code to have more access to the
Ganymede server for examining state and making changes, etc.

4. [SERVER] Fixed GanymedeWarningTask

The Ganymede warning task had an error in it that caused it to throw a
NullPointerException when it came time to warn about an impending
object removal.

5. [SERVER] Added support for generic transactional mailouts

I added support in the Ganymede transaction class (DBEditSet) for
doing generic mailings associated with a transaction, using the
logMail() methods.  Some changes made to DBLog and DBLogEvent to go
along with it.

This code isn't the cleanest, as the existing event handling code has
been adapted for handling generic mailouts rather than reworking the
logic to use different kinds of events with a base class and
subclasses, but will have to do for now.

6. [CLIENT] Substantially simplified, reworked icon tracking for nodes in tree

Previously, the client maintained an extensive set of hashes to try to
synchronize changes to objects made by the client to the graphical
representation of nodes in the tree.  This has been changed.  Now the
client just asks the server for the status of objects after a transaction
has been committed, rather than trying to track the changes by guesswork.

The point of these changes is to make the client much more robust in its
synchronization with the server, and to make it easier to maintain.

The cost is that whenever an object is manually inactivated or
reactivated, the client will do a round-trip to the server to verify
and update the status of the object, and whenever a transaction is
committed, the client will spend time checking the status of all the
objects touched by the transaction.

All of this was due to the need to have the client be able to
react appropriately if server-side wizards set an expiration date
on a user, etc.

The client still uses a client-side cache to handle reverting the icon
for deleted/created objects on transaction cancel.

7. [SERVER] Added Session.queryInvids()

This provides a high-speed, non-blocking path that the client can use
to get a status update on a specific list of objects.  This is to
support #6, above.

8. [SCHEMAS] Fixed writeUserLine in builder task

In the Linux and BSD schemas, the system would throw a null pointer
exception when it went to build the passwd file if a user had
a home phone number but no office phone number registered.

9. [SERVER] Fixed InvidDBField.unbindAll()

The object binding logic erroneously assumed that all invid fields
potentially targeting a backlinks field were vector fields.  This
could lead to problems when deleting objects with a non-empty
backlinks field.

10. [SCHEMA] Added 'gasharl' schema

This is a variant of the gash schema that includes support for some
of the extended user_info fields we use here at the lab, including
user categories, user expiration dates, and social security number
handling.

-------------------- Changes from dev4f to dev5 -------------------

RELEASE DATE: October 8, 1998

1. [SERVER/SCHEMA EDITOR] Enhanced the password field

Password fields can now be configured in the schema editor to remember
plain text passwords while the server is up.  That is, a password
field can be configured such that when a password is entered into
that password field and, in the case of the password field in the
user account object, whenever a password is authenticated for
Ganymede login, the field will remember the plaintext password in
memory.

This is intended to be used to support NT and UNIX integration.
Currently, ARL is synchronizing our UNIX passwords to our NT primary
domain controller by running a Perl script on NT via RSH.  The Perl
script is fed a list of changes, including the passwords for the NT
users.  Because NT can't use UNIX-crypt()'ed passwords, it is
necessary to include the plaintext of any passwords that have changed.

Since the beginning, the Ganymede server was able to handle passwords
using either UNIX's crypt() password hashing function or plain text
passwords.  Now, a password field can store both in memory (if both
crypted and plain-text are selected for a password field in the schema
editor), but plaintext passwords will never be saved to disk if the
crypted option was also selected.  That is, Ganymede uses crypted
passwords for its own authentication and unless it has NO OTHER
CHOICE, it will only save the encrypted passwords to disk.  In addition,
the Ganymede server provides no means for a client to get access to
the plain text of a password stored in the server.  Only local code
directly linked with the server (built tasks) can get access to the
plain text of the passwords.

The idea is that the NT primary domain controller is perfectly capable
of remembering the NT-hashed passwords once set, and Ganymede need not
risk security by keeping the plaintext passwords around on disk.

By default, all Ganymede passwords are encrypted using the UNIX crypt
function.  An adopter must explicitly configure a password field to
have the server retain plaintext passwords in memory during execution.

2. [EVERYTHING] Updated all code to work with JDK 1.1.7, Swing 1.1b3

Swing 1.1 beta 3 has had all of the Swing classes moved from
com.sun.java.swing to javax.swing.  For compatibility with the future
releases of Swing (beta and otherwise), all of the Ganymede clients
and admin console code has been moved over to the javax.* package
naming.  This means that from this point on, the Ganymede client will
require Swing 1.1 beta 3 or later.

3. [CLIENT] Fixed invid/string vector selector so that pop-up menus work again

At some point the right mouse button stopped selecting items in the
Swing JList class.  This broke the popup menu in the client's
string/invid selector.  The problem was that the lack of selection
caused the right-mouse-button pop up menu to not be able to identify
the item in the list that it needed to pass up to the containerPanel.

4. [CLIENT] Fixed scalar invid chooser refresh

The JInvidChooser GUI component's refresh logic, as changed in dev4f,
wasn't taking into account the possibility that an invid field's
choices() might not include the value currently set in the invid
field.  This left the invid chooser blank after a change was made, as
the server immediately requests a refresh of the field.

5. [CLIENT] Tweaked the open object/create object dialogs

The Swing libraries, up to and including Swing 1.1 beta 3, have some
serious problems with pop-up menus in dialogs.  That is to say, Swing
is currently incapable of displaying a pop-up menu that goes outside
of the dialog.  In dev4f and before, we tried to encourage Swing to
try to do this, with the result being that the pop-up menu just gets
truncated in the dialog.  In dev5, we have modified our dialogs with
pop-down menus so that they don't try to extend the menus past the
bounds of the dialog.

All of which is to say, we've made it look a _little_ bit better now,
but we're going to have to remember to keep an eye on later Swing
releases and try to make this a bit better when we can.

6. [SERVER/CLIENT] Field choice listing improved

Previously, the list of choices provided when one edited an invid
field only listed the set of objects that could be directly edited,
and did not take into account the DBEditObject anonymousLinkOK()
method that can provide for an exception to the normal editability
determination.  Now, objects that can be linked by virtue of the
remote object type's anonymousLinkOK() method will also be shown
in the client's list of possible choices.

The server and client have been elaborated a bit so that the client's
results caching system won't be confused by this.

7. [SERVER/SCHEMAS] Removed the 'throws RemoteException' from DBEditObject constructors

Part of change dev4b.11 (the big optimization) was to make it so that
the DBObject and DBField classes no longer subclassed from
java.rmi.UnicastRemoteObject, which automatically exported them on
creation.  Since this change, it has not been necessary to have the
'throws RemoteException' any more.  Since the optimization has been
working perfectly for over a month now, I've gone ahead and gone
through the code and removed these vestigal code bits.

I also removed the try..catch.. that was surrounding code in the
server that was responsible for creating instances of DBEditObject,
as well.

The upshot of this is that you'll want to use the new custom jar
file corresponding to whatever schema kit you've been testing with,
and if you've written any custom DBEditObject subclasses, you'll
want to go ahead and lop the 'throws RemoteException' off of your
constructor definitions.

The documentation for DBEditObject customization has been amended to
reflect the dropping of the 'throws RemoteException' from the
constructors.

8. [SERVER] adminPersona custom code modified

The plug-in for the admin persona object has been modified so that the
user field is not presented for editing for the supergash and monitor
personas.

9. [SCHEMA] Fix-ups to the GASH schema

The GASH schema was not correctly updated in dev4f.  The
ganymede.schema file included with the GASH schema kit now has all of
the fields required for the new admin persona schema.

10. [PASSWORD] Brought the password client(s) up to date

The password client has not been up-to-date with changes to the server
over the last several releases.

-------------------- Changes from dev4e to dev4f -------------------

RELEASE DATE: September 28, 1998

1. [SCHEMA EDITOR] Minor fixes

Made it so you can't set a category's title to the empty string.  Made
it so the category name string field will prevent entering a '/'
character.

2. [CLIENT] Client Fixes/Enhancements

The client will now put up the same 'guy working' query progress
window while the client is working on responding to a 'List editable'
or 'List all' command in the tree, to emphasize that listing is using
the same mechanisms as querying, and will result in the same kind of
report window.

3. [SERVER] Fixed Bug In Owner Group Handling

Ugh.  The server code was accidentally messing with the list of owners
in an owner group outside of the normal change process, creating an
improper self-reference in owner group's owner list.  This basically
put an invalid self-reference in a newly created owner group's owner
list.

4. [SERVER/CLIENT] Modified admin Persona object

The admin persona object class now includes a hidden label field which
it uses to maintain uniqueness among persona objects.  The server no
longer requires you to enter <name:descrip> for an admin persona.
Instead, you choose the user with a combo box and type in a brief
description.

The client once again allows personas to be created directly from
the tree and/or toolbar.

5. [CLIENT] Fixed JComboBox Update Logic, Rescan Logic

Previously, if the client ever got it into its head that it needed to
update the contents of a very large JComboBox, it would take a very
long time.  As in, on the order of half an hour for a list of 800
users.  It'll now happen pretty much instantaneously, as it ought.

Removed a couple of redundant client updates that could happen on an
invid chooser. This should speed up by a good margin operations that
required the client to update fields.

6. [SERVER/SCHEMAS] DBEditObject methods changed

The finalizeSetValue(), finalizeSetElement(), finalizeAddElement(),
and finalizeDeleteElement() methods in DBEditObject have been
redefined.  Now, instead of returning a boolean indicating approval
or disapproval of the operation, they return a ReturnVal.

This allows the finalize*() methods to return error dialogs to the
client, and to pass back rescan information from contingent operations
that the finalize methods may perform (for cascading changes, etc.)

All of the custom code included in the schemas distributed with dev4f
have had their finalize methods changed to return the appropriate
return type.

-------------------- Changes from dev4d to dev4e -------------------

RELEASE DATE: September 21, 1998

1. [CLIENT] Tooltips Not Set If Blank

In 4d and prior, the client would set tooltips on all field
components, even if those components did not have comments registered.
This left mysterious little blue boxes floating around the field.  Now,
the client will only set tooltips if the comment for that field is
actually defined.   Thanks to Matt Bush for pointing this out.

2. [CLIENT] Toolbar added, persona menu moved, dialogs added.

The client now has a nifty toolbar, both to facilitate users creating
new objects intuitively, and to allow for easier single-button
operation on Macs and the like.

3. [CLIENT/SERVER] adminPersonae may no longer be created out of context

Previously, it was possible to use the tree in the client to create
new admin Persona objects.  This was an oversight.  Admin Persona
objects should only be created in the context of a user, using the
create button in the Personae panel, which takes care of linking
the persona to the user, behind the scenes.

4. [SCHEMA] Parser, script fixes for linux and bsd schemas

Fixed up the sample builder scripts, and modified the parser for Users
so that trailing commas in the gcos field won't break things.

5. [CLIENT/SERVER] Added support for a message of the day, other messages

We added the property ganymede.messageDirectory to the server.  This
property is set by default to <installdir>/db/text.  Files placed in
this directory can be retrieved by the client using Session.getMessage()
and Session.getMessageHTML().  So far, the client only ever looks for
motd.txt and/or motd.html in this directory.

6. [CLIENT] Pruned Client's Category Tree Display

The client will now omit category nodes in the server's category tree
if the category has no siblings.  That is, the 'Categories' node will
no longer show up in the client, and if the user is logged in without
privileges to see the Admin Level Objects or Configuration categories,
they will just see the user level object subtrees at the top level.

-------------------- Changes from dev4c to dev4d -------------------

RELEASE DATE: September 9, 1998

1. [SCHEMA] Added Linux Schema, Revised BSD Schema

We have added a simple Linux schema, which is designed for basically
managing users and groups on a single system.  The user
inactivation/reactivation logic from the GASH schema has been left
out of the linux schema.  Unlike the BSD schema, the Linux schema
does not currently support the use of shadow passwords.

The linux and bsd schemas both have sample linux/bsd builder scripts
that you can install.  We've tested the linux_builder.sample script on
RedHat Linux 5.1, the bsd_builder.sample has not yet been tested.

Using the linux schema with the linux_builder.sample script installed,
we have got Ganymede actually creating real user accounts on RedHat
5.1.

2. [INSTALL] Made the installServer script smarter.

The installServer.perl script will now refuse to continue if it
couldn't find java where you told it to look.  installServer will now
give you a brief, customized outline of what you need to do to get
your schema up and running when it finishes installing everything.

-------------------- Changes from dev4b to dev4c -------------------

RELEASE DATE: September 2, 1998

1. [SERVER] System Event Records Initialized On Database Bootstrap

More work to get the database bootstrapping complete.  Now, if you
run Ganymede without a database, in addition to everything else, it
will create the system event objects in the database.

2. [CLIENT] Tree Wasn't Handling Fonts Correctly

The arlut.csd.JTree.treeControl GUI component we're using in the
client and the schema editor was consulting the font specification
passed in by the user of the tree in order to calculate the width
of the tree, but it didn't actually bother setting that font to
render with.  This made the tree width calculations not match the
on-screen rendering, confusing things as to when scroll bars were
supposed to appear.

3. [CLIENT] Number Fields were broken

In dev4b and dev4a, Number fields just didn't work.  This broke
things in the Schema editor, mainly, since number fields don't
tend to show up a lot in the schemas we've been working with,
except for uid's/gid's, which are fixed.

4. [CLIENT] Viewing Persona Panel With No Persona Fixed

In dev4b, viewing (not editing) a user who had no persona attached
would cause a NullPointerException to be thrown on the client.

5. [ADMIN CONSOLE] Schema Editor GUI Changes

The Schema Editor's layout has been improved a bit.

6. [SERVER] Invid and String fields sort values for getValueString().

Invid and String fields will now show up sorted when Doing queries or
lists on the server.

7. [CLIENT] Save File Dialog Expects Swing 1.1 beta 2 or better

The query / object list table in the client gives the user the option
of saving the table to disk in ASCII or HTML mode if the client is
run as a command-line application rather than as an applet in a
web browser.  When this option is chosen, the Ganymede client will
bring up the Swing file chooser to select the filename and location
to save to.  We've made the Ganymede client require the 1.1 beta 2
version of Swing to do this function.  If you are running an older
version of Swing, you won't be able to save files to disk from the
Query Result table successfully.

8. [CLIENT] Improvments to the table code

The table now changes the cursor while you are dragging a pole.
Improvements were made to the optimize column logic.

9. [CLIENT] Query Box Can Be Used To List All Objects Of A Type

The Query Box now has a 'view all objects' check box that you can
use to get a full list of objects that you do/do not have editing
authority over, without having to make up a selection criteria
that all objects should match (User uid > 0, etc.).

10. [DISTRIBUTION] base_classes included again

In dev4b, schemas/base_classes was inadvertently omitted.  This
pre-release includes the source to DBEditObject and GanymedeBuilderTask
to help people interested in writing custom subclasses of these
classes.

-------------------- Changes from dev4a to dev4b -------------------

RELEASE DATE: August 27, 1998

1. [CLIENT] View-only StringSelectors are double width

The StringSelector GUI components are now twice as wide as before
when they are not being edited.  That is, the single StringSelector
panel will expand to take up the space that would have been given
to the list of choices in an editing context.

2. [SERVER] Owner Groups Now Own Themselves, Implicitly

This change allows owner groups to be self-administering, so that
group administrators (possibly with a special role) can edit the mail
delivery options for object event notification for a group.  This
change also prepares the way for group administrators with a special
role to be able to create sub-groups, and to put their administrators
in these subgroups.

The permissions logic in GanymedeSession can now handle recursive
ownership hierarchies.

3. [SERVER] Roles can be created by non-supergash admins

But the roles created cannot exceed the privileges granted to the
admin that creates the role.

4. [SERVER] Permission Bits Cleaning Implemented

The permission bits system (used by PermissionMatrixDBField) is only
loosely bound to the object/field definitions held in the schema.
Previously, the permission matrix fields would not properly drop
permission entries recorded for object or field definitions that had
been removed by the schema editor.  Now, the database-writing code for
the PermissionMatrixDBField will check all entries for validity, and
omit those entries that no longer have any application to the revised
schema. Permission entries that are redundant or pointless are
filtered out as well.

5. [SERVER] CategoryTransport fixed

The CategoryTransport object passed from the server to the client was
supposed to carry only entries for object bases that were editable by
the end user, and the categories containing such objects.  In dev4a
and prior, the CategoryTransport code that was supposed to make this
determination did a Visibility test when choosing to include Category
definitions and an Editability test when choosing to include Base
definitions.  This made for a confusing set of results in the tree for
non-supergash users/admins.

6. [CUSTOM] Miscellaneous GASH Schema Fixes

Fixed network selection error in interface custom class.  This was
preventing users from being able to create more than one interface
in a system.

7. [CLIENT] Miscellaneous Client Fixes

Fixes in some of the GUI components, and their interaction with the
server to present choices, etc.

8. [INSTALL] Changed the install script to support new schema development.

It has always been possible to install the Ganymede server without
specifying one of the schemas that we provide with the Ganymede
distribution.  Now, if the installer does not choose one of bsd,
nisonly, or gash, the installServer.perl script will go ahead and
install the scaffolding upon which custom code for a custom schema can
be built.

9. [DOCUMENTATION] Documentation on DBEditObject subclassing added.

We've added a lengthy guide to the operation of the DBEditObject
class, and how you can subclass it to implement custom logic
on the server.  A book should probably be written on this topic,
but this guide should prove a useful start on things.

10. [SERVER] Important Fixes In Invid Binding Code

The object linking code now handles 'untargeted' object reference
fields properly, both in binding and in unbinding.  The InvidDBField
code will also properly catch ambiguous link operations resulting from
improperly specified link fields in the schema.

11. [SERVER] *** Major Memory/Speed Optimization ***

Previously, all objects and all fields in the server were subclassed
from java.rmi.server.UnicastRemoteObject, which meant that as each
field and object was created, it was registered as a remotely
accessible object through the RMI system.  Now, all objects and fields
in the server are no longer subclasses of UnicastRemoteObject, but
instead are exported through the RMI system only at the time the
object or fields are accessed by the client.  This reduces the size of
the Ganymede server in RAM by a lot.  With the full ARL GASH database
loaded, the server used to take up anywhere from 93 to 105 megabytes.
After this change, it's down to 40.  Likewise, database loading time
for our GASH database is less than half of what it was.

12. [SCHEMA] Recompiled all schema files to take advantage of 11.

You will need to recompile any custom classes you may have authored,
as they are no longer subclasses of UnicastRemoteObject.

-------------------- Changes from dev4 to dev4a -------------------

RELEASE DATE: August 11, 1998

1. [SERVER] Deleting admin personae now works properly

Before, deleting an admin persona object would cause a null pointer
exception to be thrown, as the code that checked to verify the new
name wasn't expecting a null name.

2. [SERVER] Added more rescan info for client.

In many cases, the server would not properly notify the client of
fields that Wizard code had changed on behalf of the client.  Now,
anytime anyone calls setValue(), setElemet(), addElement(), or
deleteElement() on a field, the ReturnVal returned will include an
instruction to the client to rescan that field.  This is primarily
useful for Wizard code that changes fields on the client's behalf.

3. [SERVER] User names forced to lowercase on login

Previously, it was possible to log in with mixed case or all-uppercase
and have Ganymede not recognize that the user was the same as another
login by the same user for the purpose of tracking the client's name.
Now, whenever a user logs in, Ganymede will track that user by an
all-lowercase version of their name.

4. [SERVER/CLIENT] Server tells Client its assigned id

If a user logs into the server more than once concurrently, the server
assigns each instance a unique name.  The client now asks the server
what that name is, so that you can see which instantiation of the
client matches which entry in the admin console's active user list.

5. [SERVER/CLIENT] String Fields can optionally be multi-line

The schema editor and the client now support multiline string fields,
for cases where an object needs more than a simple TextField can
reasonably provide.

6. [SERVER/CLIENT] CategoryTransport, BaseListTransport fixed

CategoryTransport and BaseListTransport are serializable classes that
transport data from the server to the client.  Both of these classes
included transient (non-transported) references to the GanymedeSession
class for use on the server.  Because we are not using the
RMISecurityManager any longer, the JVM is not able to download the
GanymedeSession class definition when the object is deserialized.  To
get around this, these classes now simply hold a generic transient
Object reference, and do a cast whenever the methods intended to be
run on the server are run.  This allows us to defer the class-loading
on the client until a server-side only method is called on these
objects, which should never happen anyway.

7. [CLIENT] Viewing The Ownership Panel Fixed

In dev4, clicking on the 'ownership' tab on a viewed (not edited)
object would throw a null pointer exception.

8. [SERVER/CLIENT] ReturnVal Rescan Information Reworked

In dev4 and before, the ReturnVal object returned by the server to the
client in response to most operations included information on what
fields and objects should be rescanned.  The design was ad hoc and
unwieldy, however, and has been changed. [...]

9. [CLIENT] String, Number Data Fields Limit Entry Properly

For awhile now, the GUI fields that allow String and Number entry
in the client weren't properly restricting keystrokes to enforce
string length and acceptable character restrictions.

In addition, String fields in the GUI will now size themselves to
reflect the size limit for size-limited fields.  Fields with large
sizes will be limited to 40 characters wide.

-------------------- Changes from dev3c to dev4 -------------------

RELEASE DATE: July 30 1998

1. [SERVER] DBEditObject.finalizeInactivate() no longer assumes removal

DBEditObject.finalizeInactivate() used to throw a NullPointerException
if the inactivated object didn't have its removal date set in the
removalField field.  While this is arguably appropriate, I made the
code resistant to that circumstance.  I may want to change it back
to catastrophic failure.

2. [SERVER/CUSTOM] Radically Simplified Wizard Construction

I moved a lot of the wizard bookkeeping from the individual
GanymediatorWizard subclasses up into GanymediatorWizard.  It's much
simpler to make wizards for DBEditObject customization now.

3. [SERVER] Modified DBEditObject.finalizeRemove() to track rescan.

The server should now properly provide field/object rescan to the
client when the client deletes an object.

4. [SERVER] Rescheduled Garbage Collection Task in Ganymede.

The server used to schedule the Garbage Collection task for 5 minutes
after startup, which always hit us right when we were testing whatever
feature we restarted the server for.  Ganymede will now schedule the
Garbage Collection task for 6 hours after server startup.  The server
will initiate garbage collection when it needs it anyway.. I had just
made it a task to have something to test the scheduler with.  I may
take the GC task out entirely.

5. [SERVER] Fixed bugs in Journal file processing

We ran into some problems with the Journal file logic as implemented
in dev3b.  The journal file format has been reworked a bit, both to
fix some vulnerabilities to failure and to further improve file size
and processing time.

6. [SERVER] Server Properly Reports Inactivated Objects

The server wasn't properly reporting the inactivation, expiration,
and removal of objects.  Now whenever the client does a query on
the server (including for the building of the tree), it will get back
the proper status of the objects.

7. [SERVER] DBEditObject now has default reactivation logic

Previously, the DBEditObject reactivate() method would simply refuse
action, forcing this method to be overridden by subclasses if
inactivation was to be supported.  Now DBEditObject.reactivate() will
go ahead and clear the Removal date if called on an inactive object.
Customizers are still encouraged to override this method to provide
intelligent reactivation.

8. [CUSTOM] Various refinements to the GASH schema

You can now rename a user to a name the user previously had as an
email alias.  You can now inactivate a group in good conscience.  The
fields in the user object have been re-ordered to make entering data
for a new user more straightforward.  The mapEntryCustom code checks
to see if it is being deleted before complaining about an
auto.home.default pointer being messed with.

Added preliminary code in userCustom.commitPhase2() to perform
external actions when a user is created, deleted, renamed, or when
their home directory is moved from volume to volume.  Right now, these
are just stubs, but the logic is there.  Turn on the debug flag and
recompile userCustom.java to get messages about when various external
actions would be taken.

9. [SERVER] Set serialVersionUID on serializable classes

This change will allow the server and client to communicate even if
they are using slightly different versions of the data objects used to
pass data back and forth between the client and server.  It will also
make for a (probably very) slight speed-up in server initialization.

10. [CLIENT] The client will now terminate when killed by the server

In recent versions, the client would not get the hint and die when the
admin console knocked a user off.  Instead, it would display a dialog
reporting the boot, but would keep running, getting all kinds of
exceptions from the server.  Now, the dialog will show up, and a timer
is started.  When the timer counts down to zero, the client will
self-destruct.  The user can, of course, acknowledge the dialog to
shut it down sooner.

11. [SERVER] original field in DBEditObject is now protected

This change allows custom DBEditObject subclasses (out of the
arlut.csd.ganymede package) to get easy access to the original state
of a changed object for comparisons.

12. [SERVER] Server now re-sorts fields after the schema is edited

The server will now properly report the correct order of fields after
that order has been changed by the schema editor.  This worked before,
but only if the server was restarted.

13. [SERVER] Object creation and deletion is now logged

Before, only changes to existing objects were explicitly logged.. now
object creation and deletion is also logged, and can be emailed.
Right now, there is no straightforward way to specify that different
users should get mail when different kinds of objects are created, as
all object creation falls under a single event category.  This is one
problem with trying for the super-generic, customizable event
model.. to handle this properly would involve having a whole sub-tree
under the object creation/deletion/change events, with each object
type being able to have its own notification list, like GASH has.

14. [CLIENT] Embedded objects in newly created objects are opened

Before, when you created a user in the GASH schema, you had to manually
open up the auto.home.default entry to set the user's home volume.  Now
all embedded objects in newly created objects start out opened by
default.

15. [SERVER/CLIENT] The log/history system has been optimized

All of the log code has been heavily optimized, and is now both a
great deal faster and a great deal more memory efficient on the
server.

We're intending to support more general (i.e., non admin or
object-limited) log searching by the client in the next pre-release.

16. [SERVER] Event/Email notification system has been totally revised

GASH's event/mail/logging model had separate event codes for the
creation, inactivation, and editing of objects by their type.  So it
was possible to specify that whenever a user was created, someone in
personnel could be notified.  The code in GASH that manipulated users
was custom-coded to use those event codes.  Ganymede uses a more
generic model, in which the event logging/email system is designed to
work with any kind of object handled by Ganymede.  However, there is
still a need to be able to specify that certain users be notified
whenever an object of a specific type has had a specific operation
performed on it.  So there is now a second kind of event object listed
under Admin-Level Objects, an 'Object Event' object, which lets the
adopter create special email for whenever an object of a specific type
is created, edited, inactivated, deleted, reactivated, etc.  The orignal
event object type has been renamed 'System Event'.

Admin Persona objects now include a field to specify an email address
where email to that administrator should be sent.  If Ganymede is
managing your email database, you won't need to set this field,
but it's there otherwise.

Owner Groups and System Events now have an 'external email list' field,
used when you want email related to those objects to go to email
addresses not contained within Ganymede.

17. [TREE] Significant tree fixes/enhancements

The JTree code had an error in its drag/drop logic when both
tween-dropping and on-dropping were enabled, that made it impossible
for on-dropping to ever occur.  This has been fixed, which corrects a
problem in the schema editor.  I added a moveNode() method to allow
the schema editor to do the right thing in moving shuffling categories
and objects around within the tree.

Made treeCanvas extend JComponent instead of JPanel.. this seems to
fix some responsiveness problems that the tree was exhibiting under
Java Plug-In 1.1 on Win32. (ref. changes dev3 #5)

18. [SCHEMA EDITOR] Fixed lots of bugs in schema editing

Moving object types and categories between categories now works
reliably.  This involved both adding a new primitive to the server,
Category.moveCategoryNode() and to the tree, moveNode().  The
server-side schema editing context does a better job of mapping remote
Base references supplied by the client-side schema editor to the
server-side local DBObjectBase objects.  Basically, schema editing
should be robust and reliable now.  Or at least, much more so.

Forced the schema editor tree to be black on white instead of
black on SystemColor.background.. testing the admin console on a
friend's FreeBSD system showed the ugliness that results if a system
had SystemColor.background set to black.

19. [SERVER/CUSTOM] Pulled classes for critical objects into ganymedeServer.jar

Previously, all subclasses of DBEditObject were packaged along with
the custom schema code.  This was inappropriate, as the server depends
on those classes for proper authentication and logging, and there is
little to be gained by forcing adopters to include those classes in
their customization efforts.  Worse, if an adopter did not select a
schema at install time, no support classes for the permission system
were installed.  Now, all the code that the server _depends_ on for
its own logic is kept safe and bundled.

20. [CLIENT] Fixes to the tree icon logic

There were some circumstances in which an object with more than one
exceptional condition (both checked out for editing and marked for
inactivation, say..) would not have the most sensible icon displayed
for it in the tree.  The client now takes the possibility of multiple
conditons into account and takes the relevance of the various
conditions into account when assigning an icon for the object's node
in the tree.

21. [CLIENT/ADMIN CONSOLE/SERVER] Tweaks for 1.2b4 compatibility

The new security system supported in the 1.2b4 and (presumably) later
releases of the JDK forces a new security model on code that installs
a SecurityManager, in such a way that the code is not compatible
across 1.1 and 1.2, at least as of 1.2b4.  To get around this, the
Ganymede client and server code no longer install the
RMISecurityManager.  There is no practical downside to this, as the
only thing that the RMISecurityManager enabled was the transfer of
executable classfiles between the client and the server, which
Ganymede did not do anyway.

22. [SERVER] Various fixes

a. The fully-privileged internalSession GanymedeSession that the
server uses for its internal operations has had oversight turned off,
to speed things up somewhat, and as an attempt at debugging stuff.

b. Object reference (invid) fields can now lookup a remote object's
label in a transaction, even if the remote object has had its fields
cleared in preparation for deleting that object at transaction commit
time.  This comes into play when the details of a transaction is being
logged to disk at transaction commit time.  One side effect of this is
that if you (say) rename a user and then delete the user in one
transaction, the logging system will report that user <newname> was
deleted, but that <oldname> was removed from account groups, and the
like.  Fixing this would be rather more involved than the payoff would
be worth at this time, I think.

c. Boolean fields are considered to be undefined if they are set to
false.  This makes it possible to edit the schema to remove boolean
fields if all instances of a boolean field are false.  Previously, the
boolean fields would be undefined until someone set them, at which
point they would remain defined, regardless of value.  One consequence
of this is that the logging for boolean toggles may seem
counter-intuitive, with true-to-false transitions recorded as the
field being removed.

d. The database dumper, expiration processor, and expiration warning
tasks won't fire if the schema is being edited.

e. The Ganymede server will never explicitly record an object as
being owned by supergash, since by definition supergash implicitly
owns all objects.  This change reduces the size and memory loading
of the database, but will increase the time it takes to do initial
bulk loading.

23. [ADMIN CONSOLE] Admin Console will exit when server is shutdown

If you use the 'shutdown server' menu option in the Ganymede Admin
Console, the console itself will shut down once the server finishes
its shutdown sequence.

24. [SERVER/CUSTOM] anonymousLinkOK and anonymousUnlinkOK expanded

In order to properly secure privileges to add or remove objects from
owner groups on the server, we have added a new parameter to the
anonymousLinkOK and anonymousUnlinkOK methods in DBEditObject.  This
is the GanymedeSession parameter, so the code can determine whether to
allow the link or unlink based on the identity of the user attempting
the link.  If a custom subclass author does not override the expanded
version of these methods, the old methods will be called, without
the GanymedeSession reference.

25. [SERVER] Ownership security hole fixed

Previously, the Ganymede system depended on the client being
trustworthy when it came to restricting the ability to grant ownership
of objects to an owner group.  The server now checks this operation
against the admin's owner group membership to prevent abuse.

A related change fixes a similar vulnerability in all object-linking
logic.  The server will no longer allow values to be placed in
InvidDBFields if the field is choice-restricted and if the value was
not a choice presented to the client.

26. [CUSTOM] Added bsd schema

Added a simple passwd/group BSD 4.4 compatible schema, analagous to
the nisonly schema, but with support for the BSD 4.4 master.passwd
file format.

-------------------- Changes from dev3b to dev3c -------------------

RELEASE DATE: 17 June 1998

1. [CLIENT] Improvements in the view/edit/delete/inactivate object dialog

 Previously, the view object dialog would only search for objects that
the user had permission to edit.  The dialog is smarter now about what
is appropriate for the different operations.

2. [CUSTOM] GASH, nisonly loader code fixed.

 Ugh.  In dev3b, the GASH and nisonly loader code was simply broken..
the change in create_db_object() and edit_db_object() broke the
directLoader class in both schemas.  I had caught the changes in the
custom_src directory, but not in loader/source.  The loader code
for both work now.

3. [SERVER] Made InvidDBField checkpoint/rollback code issued selectively

 In dev3b, all Invid binding operations was checkpointed, which made
doing bulk-loading far, far too slow.  The bind/unbind checkpointing
is now only done when GanymedeSession.enableOversight is true.  That
is, not during bulk-loading.

-------------------- Changes from dev3a to dev3b -------------------

RELEASE DATE: 15 June 1998

1. [SERVER] Memory, Performance Optimizations

 By using custom-crafted DBObjectTable, DBFieldTable, and
DBBaseFieldTable classes instead of java.util.Hashtable's for the
primary Ganymede server data structures, we have reduced the memory
loading for the server by over 20% and eliminated the creation of a
lot of unnecessary temporary objects.  A number of operations that
were previously synchronized unnecessarily are now unsync'ed, and a
number of operations that required explicit casts are now type
specific, leading to further performance improvements.

2. [CLIENT] Swing Deadlock Prevented

 The Swing GUI library is not thread safe, and the client had a risk
of locking on updating the status label at the bottom of the client
window if the user attempted to open up several object windows
simultaneously.  The client now uses the Swing thread synchronization
primitives to update the status label, eliminating the potential for
client thread lock.

3. [CLIENT] Object Window ProgressBar Improved

 The object view/edit window progress bar has been improved so it
shows progress across vector fields, rather than simply halting while
a large number of vector components are loaded.  This gives a better
indication of the true progress when loaded an object with many
embedded objects.

4. [CLIENT] Query Dialog Allows Matches Against Non-Editable Objects

 Before, the query box would allow you to do queries on invid fields,
searching for objects whose invid field pointed to a listed object.
The choices listed, however, only included objects that the admin had
edit privilege on.  Now the list of options that is included in the
pull-down list for such fields will show all objects of the given type
that the admin has read privileges for.

5. [CUSTOM] Fixes to GASH Schema

 The system interface code was not properly handling the display of
the name and alias fields.. this was due to a broken canSeeField()
fix in dev3a.

6. [SERVER/CLIENT] edit/create/view/clone_db_object now return ReturnVal

 The server now has the ability to return error dialogs in response
to requests to view, edit, create, or clone objects.  The server will
now provide a detailed report explaining why an operation could not
be successfully completed.  If a user attempts to edit an object that
has already been checked out for editing by another user, the server
will report this, including the identity of the user who has the object
checked out.

7. [CLIENT] Query issue is now threaded

 Before, the client issued queries to the server on the GUI thread,
blocking the client's GUI refresh until the server responded.  The
client now handles queries in a separate thread, and displays a
window with an animated gif while waiting for the server to complete
the query.

8. [SERVER] Journal file optimized

 Before, whenever an object was changed in a transaction, the entire
state of the object would be written to the journal file.  Now, the
server only writes a record of the changes made to objects to the
journal, in some cases dramatically reducing the size of the journal
file and the time required to process it.

9. [SERVER] All Invid bind/unbind operations checkpointed

 The Ganymede server is now more robust in the face of errors in the
object linking logic.

10. [CLIENT] Fix in client refresh logic

 The client wouldn't properly display items in vector panels when
ordered to change the vector panel's visibility in certain cases.

11. [CUSTOM] Changes to gash, nisonly custom source

 Small changes to the gash and nisonly custom source to cohere with
change #6, above.  Since Session.edit_db_object() now returns a
ReturnVal, the custom code for the two schemas had to be changed where
it called edit_db_object().  No changes were made to NISBuilderTask.java
and GASHBuilderTask.java.

-------------------- Changes from dev3 to dev3a --------------------

RELEASE DATE: 8 June 1998

1. [CLIENT] Notes, History Panels reorganized.

 The object created, object modified timestamps are now on the history
panel.

2. [CLIENT] Can now do vector search operations on embedded invid fields.

 Previously, it wasn't possible to do things like search for systems
with multiple interfaces, as the client-side query code just displayed
the fields contained under the embedded interfaces.  Now it is possible
to query based on the status of the interfaces vector itself.

3. [INSTALL SCRIPTS] Fixed install scripts to work with Perl5.003

 It turns out that the Copy module in perl 5.003 is less friendly than
the one in perl 5.004_4, which we used to develop the install scripts.
We have fixed the install scripts to run under 5.003.

We also made the installServer and installClient scripts check to
prevent relative pathnames.

4. [CUSTOM] A number of fixes in the GASH/nisonly schema/custom code.

 Fixed interfaceCustom.canSeeField(), which was breaking the
viewing (not editing) of multi-interface systems.

 Made the directLoader code a bit more liberal, and made it able
to handle admins in admin_info with supergash-level privs.  Made it
able to handle alias_info lines with program invocation targets.

  Both the GASH and nisonly schemas now allow single-character
usernames.  Previously, the schemas were configured to require
a minimum of 2 characters for usernames, which was unnecessarily
restrictive.

5. [SCHEMA EDITOR] Turned off debug output.

-------------------- Changes from dev2c to dev3 --------------------

RELEASE DATE: 28 May 1998

IMPORTANT NOTE: The GASH schema and custom code have been very
significantly revised since dev2c.  You will not be able to port
an existing ganymede.db file from 2c or prior for use with the
dev3 GASH schema module.  As a general matter, we are not committing
to a lock-down on either schemas or classes until the initial
'production' release.

1. [CLIENT] Double-clicking on objects in the tree opens them for viewing.

 It is now more straight-forward to use Ganymede without a
multi-button mouse.

2. [CLIENT] Simplified ContainerPanel Menus.

 The menus attached to Object windows have been reduced in number and
made more intuitive.

3. [ADMIN CONSOLE] Fixed Initialization Exception on JDK 1.1.6 / Swing 1.0.2.

 The Admin Console used to throw an IllegalArgumentException on
startup when run under JDK 1.1.6 and Swing 1.0.2.

4. [SERVER/CLIENT] Fixed a lot of things regarding the query system.

 The client is now a lot smarter about handling non-String query
types, and will provide a list of choices for restricted object
reference fields.  Dates are handled with a pop-up calendar, and the
client properly implements 'same day as', 'same week as', and 'same
month as' logic.  A single query can handle searching on top-level
fields as well as fields in embedded objects.  The whole system is
significantly reworked and improved.  The GUI was reorganized.

5. [CLIENT] Custom Table Code Improved.

 Previously, the table code used in the admin console and query
results windows was subclassed from the Swing JPanel class.  This
caused a lot of mouse events to not get propagated to the table canvas
for some reason.  The table code now subclasses directly from
JComponent and works more reliably now.  Menus and row selection
occurs reliably on the first mouse click.

6. [CLIENT] Object Panel Loading Speedups.

 The manner in which certain GUI components load what may be large lists
of objects has been improved, leading to possibly significant speedups
in bringing up an object window.

7. [CUSTOM] The GASH schema has been significantly revised and completed.

 The GASH schema and custom code has been significantly revised.
Instead of there being a separate NISBuilderTask and DNSBuilderTask,
there is now just a single GASHBuilderTask, which generates all the
files supported by the GASH make process. The hosts_info file emitted
does not contain a list of administrators, so you will lose
information if you import GASH files into Ganymede, modify them, and
write them back out via the GASHBuilderTask.

 The GASH schema has been revised so that the systems database closely
adheres to the features supported by GASH.  The following changes
were made in this regard:

  1. Single IP address and DNS record per interface
  2. Rooms are bound to a list of networks, and the interface objects
     for each system provide a list of IP network choices from the
     room the system is placed in.

 In addition, the GASHBuilderTask now emits an aliases_info file for
the first time.  In theory, you should be able to plug an installation
of Ganymede with the GASH schema loaded into the back-end GASH makefiles
you may already have.

8. [SERVER] Client code can now use the server to send mail.

 It is now possible to write client-side code that can send mail out,
even if that code is running as an applet in a restricted browser
context.  The mail is sent by the Ganymede server rather than the
client, and includes a signature identifying the mail as coming from a
Ganymede client.  The server supports sending both plain ASCII (rfc
822) mail, and mixed HTML/ascii MIME mail.

9. [CLIENT] Object windows and query result tables now save and mail reports.

 The object windows and the query result table windows now have a menu
allowing you to save and/or mail their contents in either text or HTML
format.  If the client is running in a restricted applet context, the
save option will not be available, but the client can still email
reports.

 The save function uses the version of the JFileChooser provided in
Swing 1.0.2, it may not work properly with earlier versions of the
Swing GUI libraries.

10. [SCHEMA EDITOR] Fixed a few schema editor bugs.

 Made the schema editor generally more robust when removing objects
types that have fields in other object types pointing to them.

11. [CLIENT/ADMIN CONSOLE] Put in some work-arounds for bug in Swing 1.0.2

 There is a significant bug in the JComboBox code in Swing 1.0.2.. see
bug #4132528 in the Javasoft Bug Parade at
http://developer.javasoft.com/developer/bugParade/bugs/4132528.html.

 We put a few workarounds in the code, but there may still be times
when JComboBox operations will throw IllegalArgumentExceptions.  I
think folks are still better off using Swing 1.0.2 over previous
releases.

12. [SERVER] Added a reserve() method to the DBNameSpace class.

 Previously, a transaction could only allocate a unique name/number by
actually placing that value in a field of an object.  Now custom code
can 'pre-allocate' a set of values which can then be shown to the
client as a list of available options.  The newly revised GASH schema
uses this code to pre-allocate IP addresses for the networks available
in a particular room.

13. [SERVER] Fixed various elements relating to IP address handling.

 The java.util.Hashtable class does not work properly for the array of
Byte representation used in Ganymede to handle IP addresses.
GHashtable was modified to wrap bare Byte arrays with a wrapper class
that enables proper Hashtable functioning.

 The IPDBField class was not properly invoking a DBEditObject subclass'
wizardHook() method.

14. [DOCUMENTATION] Added more documentation.

 We have added doc/editobject.html, doc/server.html, and
doc/client-custom.html.  Each of these documents should be
significantly expanded in the next dev release.

15. [SERVER] Added default field permissions support for object types.

 In modifying the directLoader to properly initialize permissions for
the GASH schema, it became clear that we needed a way to set the
default permissions for an object's fields to something other than the
top-level permissions that apply to the object.  We modified
perm_field and PermissionMatrixDBField to add setDefaultFieldPerms().

16. [SERVER] invid_field.createNewEmbedded() returns a ReturnVal

 Previously, there was no way to get a detailed error report or rescan
information from a createNewEmbedded call.  We added a getInvid()
method to ReturnVal and made the client able to handle rescan requests
when adding a new edit-in-place object to a vectorPanel.

17. [SERVER] Various changes in Invid handling.

 Fixed/altered a number of things involving invid binding to make
things more robust across a wider range of circumstances.

18. [SERVER] Dumped schemas now retain builder task information.

 Previously, a raw schema file dump would not retain any information
about builder tasks associated with the schema, forcing end-users to
manually configure builder tasks after initially populating the
database from their data.  Ganymede now saves the builder task
information in the schema file, and users implementing Ganymede no
longer need to register the default builder task.

19. [DOCUMENTATION] Added CREDITS file.

20. [INSTALL SCRIPTS] Made the install scripts provide schema-specific options.

 The installServer.perl script has been made a bit smarter, and will
not prompt for configuration variables that are not applicable to the
schema selected.

-------------------- Changes from dev2b to dev2c --------------------

RELEASE DATE: 5 May 1998

1. [SERVER] DBSession.createDBObject() checkpoints DBEditObject.initializeNewObject().

 The server now performs a checkpoint/rollback when it calls
DBEditObject.initializeNewObject() if enableOversight is turned on.
That is, as long as the system is not doing bulk loading, the
DBEditObject.initializeNewObject() method is no longer responsible
for doing its own checkpoint/rollback.

2. [CUSTOM] Bulk loading Streamlined for nisonly, gash Schemas.

 The nisonly and gash userCustom and groupCustom classes no longer try
to automatically allocate uid's and gid's in initializeNewObject()
during bulk loading.  The gash schema previously tried to create
automounter entries at user creation time, which could confuse the
gash directLoader code.  Both the nisonly and gash schemas now only
attempt to do user and group initialization if the system is not
in bulk-load mode (enableOversight is true).

3. [CUSTOM] nisonly, gash schema code cleaned up.

 I went through and cleaned up the NISBuilderTask source in the
nisonly and gash schemas.  They now use schema constants rather
than numbers for field id's.  Removed a bunch of code from the
nisonly NISBuilderTask class that was no longer relevant.

4. [SERVER] Added a DBObject Parameter to anonymousLink methods in DBEditObject.

 I added a DBObject parameter to anonymousLinkOK() and anonymousUnlinkOK()
in DBEditObject.  This allows DBEditObject subclasses to enable or
disable anonymous linking and unlinking to a given field based on the
state of the object.

5. [CUSTOM] The GASH Schema Always Allows Linking, Unlinking from auto.home.default.

 In dev2b, it was not possible to create a new user as a GASH administrator
without specifically setting the permission matrix privileges to allow all
GASH administrators to edit all automounter maps.  With the new parameter
added to anonymousLinkOK(), we have now made the mapCustom object always
allow remote linking to auto.home.default.

6. [SERVER] Fixed InvidDBField.getEncodedValues() To Show Custom Labels on Viewing.

 In dev2b and before, Invid fields were supposed to be able to generate
custom labels for objects referenced through the lookupLabel() method
in DBEditObject subclasses.  This was only occurring with editable objects,
and not when such objects were being viewed.

7. [CLIENT] Fixed Objects Owned Panel.

 The client was not enabling the commit button when ownership of objects
was changed in the objects owned panel when an administrator edited
an owner group.

-------------------- Changes from dev2a to dev2b --------------------

RELEASE DATE: 4 May 1998

1. [SERVER] Creating New Objects With Embedded Objects.

 In dev2a, creating objects whose initializeNewObject() methods
created embedded objects would fail with a NullPointerException.  This
was due to the object being initialized before being added to the
transaction's working set, and caused NullPointerExceptions when
creating users with the GASH schema.

2. [CLIENT] Queries Were Not Attached To Tree Nodes Before Loading.

 In the client, if a user tried to issue a query on an object base
folder in the tree prior to opening the folder, a null query would
be issued, and no results would come back.

3. [CUSTOM] Groups Field Broken In nisonly User Object.

 When using the nisonly schema, adding groups to the user's group
list was not working properly.  The home group choice box was not
updated properly, and the change to the group list would not be
committed properly.

4. [CLIENT] Editable/non-editable Choice GUI Component Logic Fixed.

 The release of Swing 1.0.2 from Javasoft has revealed a logic error
that would invert the editability status of a choice list.  It is now
possible to manually enter arbitrary strings for the shell field in
user objects in the gash and nisonly schemas.

5. [CLIENT] Tab Pane In Object Windows No Longer Transparent.

 Another bug that showed up when running the Ganymede client against
the 1.0.2 Swing release.  With Swing 1.0.2., the tab panes in dev2a
were transparent, allowing obscured windows to show through.

-------------------- Changes from dev2 to dev2a --------------------

RELEASE DATE: 1 May 1998

1. [SERVER] Required Fields Not Checked For Deleted Objects.

 DBEditSet (the Ganymede transaction class) no longer checks required
fields for objects that are being dropped or deleted.

2. [HTML] HTML for Java Activator/Plug-In Updated for Plug-In 1.1 Release

 We have updated the HTML template for use with the Java Activator to
work with the official release of the 1.1 Java Plug-In.

3. [SERVER] Added GanymedeSession.enableOversight(boolean) Method

 It is now possible for server-side code to turn off required field
checking and all wizard activity.  This is intended to be used
by the offline bulk loader code.

4. [CUSTOM] Updated nisonly, gash Loader Code

 Both the gash and nisonly loader code now uses enableOversight() to
turn off wizards and required field checking during bulk loading.  The
nisonly loader hadn't been recompiled since the delete bit was added
to the PermEntry class, so the nisonly loader code as broken.


-------------------- Changes from dev1b to dev2 --------------------

RELEASE DATE: 27 April 1998

Following are the most significant changes since dev1b.  There are a
number of additional small performance and behavior improvements
throughout.  We have continued to evolve the ARL Ganymede schema
and custom code, which has prompted us to enrich the customization
plug-in classes to allow for needed functionality.  The ARL schema
is not yet ready for release, but our work on it is driving the
improvement of the system generally.

1. [SERVER] Deadlock Prevention.

 There were a number of cases in which the server could become
deadlocked due to a nested-monitor condition.  We have corrected a
good number of these, and the server should be far less susceptible to
deadlocked threads.

2. [SERVER] Transactions Can Fail Without Mandatory Abort.

 The server now has the ability to have a transaction fail without
having to throw out the entire transaction.  This is built on the
checkpoint logic in DBEditSet and DBEditObject.

3. [CUSTOM] DBEditObject Customization Class Enriched.

 We have added a method to DBEditObject that will allow custom
plug-ins on the server to specify what fields must be filled in at
commit time.  The server's commit engine uses (2.) above to return an
explanatory dialog to the client, allowing the user to fill in the
additional required fields before trying to re-commit the transaction.

 In addition, the DBEditObject class now has the ability for an object
to determine for itself when it should be considered owned by the
current Ganymede administrator, as well as the ability to return
expanded or restricted permissions as needed.

4. [CLIENT/SERVER] Client Automatically Refreshes Object Pointers.

 The server now returns to the client a list of objects and fields
that need to be updated in response to an action by the client.  The
basic object linking code automatically handles this, so whenever you
change an object reference pointer with the client, you will see
affected references in other windows update.  There are still a couple
of cases in which this is not yet handled correctly, including when a
change to an object causes the object's label to change, and in
certain cases when objects are deleted.

5. [SERVER] String Relative Compares Work In Query.

 Before, if you attempted to use >, >=, <, or <= in a string field in
the Ganymede query mechanism, you would not get any results back.
Relative string comparisons is now working properly.

6. [CLIENT] Several GUI Fixes.

 The calendar panel now shows up correctly.  The GUI logic for the
vector of embedded objects has a number of bug fixes.  More.

7. [CUSTOM] Now Using *Schema.java Files To Enumerate Fields.

 The custom code associated with the provided schema files now use
symbolic constants to refer to fields in the customized object types.
This should make that code easier to read and maintain.

8. [CLIENT/SERVER] Permissions System Now Has Delete Bit.

 The permissions system now maintains a delete permission that is
distinct from an edit permission.  It is now possible to allow an
administrator permission to edit an object while denying that
administrator deletion privileges.  The permissions editor currently
shows the delete bit as applying to object fields as well as objects,
but currently the server only consults the delete bit when an attempt
is made to remove, inactivate, or reactivate objects.

9. [CUSTOM] GASH Schema Uses Properties.

 We have made the GASH schema custom code use a couple of properties
to specify the default home directory location and default email
address suffix.  You will be prompted for the values for these
properties when you run the installServer.perl script.  We may
want to move these out of the ganymede.properties file and into the
database proper to make it possible to change these values without
having to stop and restart the server.

 installServer.perl should be smart enough not to prompt for these
properties if the user is not installing the GASH schema.


-------------------- Changes from dev1a to dev1b --------------------

RELEASE DATE: 6 April 1998

1. [SERVER] Tasks Loading / Server dependency fix fixed.

 In amending the arlut.csd.ganymede.QueryResult class to not be dependent
on server-side classes, I broke a method that was required for the
server to load any registered builder tasks.  As a result, installers
were not able to get the DNS/NIS builder tasks to load.  This problem
was considered serious enough to precipitate this release.

2. [SERVER] Default Builder Tasks Will Now Call External Build Scripts.

 I have modified the builderPhase2() method in the builder tasks
included in the gash and nisonly schemas to automatically call a
script, "nisbuilder" or "dnsbuilder" in the db/out directory whenever
a transaction is committed that resulted in changes to the object
bases monitored by the installed build scripts.  With this change,
it is now possible for the nisonly schema install to be used for
actually managing passswd and group without much hassle.

 We are still not including any actual NIS or DNS build scripts.

3. [SERVER] Permissions Failures No Longer Throw Server Exceptions.

 Pretty much everywhere the server previously threw exceptions on
permissions violations are now properly caught and an error dialog
is returned to the client.

4. [CLIENT] Persona Panel Works Properly.

 In dev1a, the client could freeze up when the user attempted to carry
out an action that they did not have permissions for.  The persona
panel is more robust now.

5. [CLIENT] Miscellaneous Client Fixes.

 A small number of minor blemishes were cleaned up in the client.

-------------------- Changes from dev1 to dev1a --------------------

RELEASE DATE: 1 April 1998

1. [DOCUMENTATION] INSTALL

  We have slightly revised this file to better reflect the state of
things after the install scripts have run.

2. [DOCUMENTATION] Javadocs improved

  We used the 1.2beta3 JDK's javadoc to produce the class documentation.
The 1.2b3 javadoc includes a lot more information about the classes,
and will give you a better idea of how things fit together.

3. [INSTALL] Install Script Fixes.

  The dev1 prerelease installServer script was broken when it
came to properly configuring the loader code.

4. [CONSOLE] Schema Editor Fixes.

  Check boxes were not working properly in the schema editor,
making it impossible to configure certain aspects of new fields.

  Dragging newly created object types around between categories
was throwing an exception.

5. [CLIENT] Permission Editor Fixes.

  When the user attempted to edit permissions in a permission
object, it wasn't possible to change the top-level permissions
for an object type, only for individual fields.

6. [CLIENT] Reworked client architecture.

  We broke out the basic client logic into a new class,
arlut.csd.ganymede.client.ClientBase.  This change makes it easy
to write new clients without having to restart the server.
Previously, the server couldn't talk to a client unless it had
the custom RMI stub for that client.  Now, all clients simply use
the ClientBase class to connect to the server.  The server jar file
has this stub in place, so any clients written to use ClientBase will
work without needing to restart the server.

  We rewrote the Password client to demonstrate the use of ClientBase.

7. [SERVER/CLIENT] Fixed client dependency on server class.

  The client referred to a class, arlut.csd.ganymede.QueryResult,
which in turn contained a reference to a server class,
arlut.csd.ganymede.DBObject.  This made the client crash out
with a class not found exception when run as configured in the
client install.

8. [SERVER] Expanded DBEditObject to allow permission customization.

  We added permOverride() and permExpand() methods to
arlut.csd.ganymede.DBEditObject to allow custom objects to be able to
customize permission authentication for special cases.

9. [SERVER] Fixed initialization bug in DBStore.initializeSchema().

  This code that is supposed to create the standard object types
for a newly created ganymede.db file had an error that caused an
exception to be thrown.

10. [CLIENT] Object reference GUI field changed.

  When an object has a single reference to another object in a field
(as opposed to a vector of references), the client will now allow you
to link to the referred object.  The editable version of this field
type will be a bit more polished in the next release.  Currently, you'll
see a simple 'click' button.  We're going to come up with some sort of
icon for this by the next release.

			 --------------------

The remainder of our coding efforts have been in the development of a
new schema set, called 'ganymede' for now, which is being tailored for
our laboratory environment.  Our initial DNS support will be in this
schema, but we are not yet ready to release this set of
customizations.