Skip to content

Commit

Permalink
#61 merge master
Browse files Browse the repository at this point in the history
  • Loading branch information
michael-conway committed Apr 28, 2015
2 parents 0649afb + 7ee0f26 commit d83084a
Show file tree
Hide file tree
Showing 270 changed files with 6,063 additions and 3,922 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ nbactions.xml
testing.properties
*/test-output
JargonVersion.java
**/${test.option.mount.basedir}
92 changes: 4 additions & 88 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,17 @@ Jargon Core API
Work for milestone https://github.com/DICE-UNC/jargon/milestones/4.0.3.1%20second%20tier%20features%20branch

# Project: Jargon-core API
<<<<<<< HEAD

### Date:
### Release Version: 4.0.3.1-SNAPSHOT
### git tag:

=======
#### Date:
#### Release Version: 4.0.2.1-SNAPSHOT
#### git tag: 4.0.2.1-SNAPSHOT
#### Developer: Mike Conway - DICE

## News

Release Candidate for milestone: https://github.com/DICE-UNC/jargon/milestones/idrop%20for%20jargon%204.0.2%20release%20support%20-%204.0.2.1
>>>>>>> master

## News
https://github.com/DICE-UNC/jargon/issues?q=is%3Aopen+is%3Aissue+milestone%3A%22features+branch+4.0.3%22


Please go to [[https://github.com/DICE-UNC/jargon]] for the latest news and info.

Expand Down Expand Up @@ -48,83 +42,5 @@ Jargon-core uses Maven for dependency management. See the pom.xml file for refe

Note that the following bug and feature requests are logged in GitHub with related commit information [[https://github.com/DICE-UNC/jargon/issues]]

## Bug Fixes

<<<<<<< HEAD
## Features

=======
#### fix display version of file length for rounding #54

Fix display of file size to not round down for CollectionAndDataObjectListingEntry

#### Overwriting a file with IRODSFileOutputStream deletes file metadata #52

IRODSOutputStream now has extended support for open flags, including truncation/overwrite handling. In the immediate case this prevents deletion of AVUs on
overwrite of a file via OutputStream. Extended operations are available

READ, WRITE, READ_WRITE, READ_TRUNCATE, WRITE_TRUNCATE, READ_WRITE_CREATE_IF_NOT_EXISTS, WRITE_FAIL_IF_EXISTS, READ_WRITE_FAIL_IF_EXISTS

#### #62 [iROD-Chat:12888] JargonException: java.io.IOException: read length is set to zero when copying file

Clarified the usage of the default storage resource, if set, in IRODSAccount so that it is propagated to move() and copy() operations if a specific resource was not set in the move or copy call.
Note that if no resource is set, it will defer to default resource settings in iRODS. Additionally, the move() method was enhanced, so that, if a move is being done from a source file to the same target
file, but with a different resource, it will delegate to a physical move. This seems like a 'least surprise' sensible default.

#### #68 display of sharing when group assigned does not show up in shared w me

Updated specific query to not select user, and added view of 'public' in sharing listings. This is in the jargon-user-tagging specific queries

#### appending path in dataObjectAOImpl causes duplicate file name #73

Fix path munging in removeAccessPermissionsInAdminMode in DataObjectAOImpl

#### untracked error leaves idrop in a processing state #71

Tightened up parallel transfer processing to avoid sending oprcomplete if parallel transfer threads fail. This will now forcefully abandon the agent connection and properly return an error. This was falling through the cracks and leaving clients like iDrop in an indeterminate state.

#### Output streaming did not trigger post proc for put in iRODS4 #74

Fixes that work with 4.0.3 protocols for streaming data to iRODS

#### rule processor rules deleting a parameter #76

Fix Stream2StreamAO so that streaming from a stream into an iRODS file replaces all file contents, this was causing erroneous rules to be written back to iRODS

## Features

#### Setting inheritance on collection as admin #55

Add ability to set inherit/noinherit in CollectionAO as admin

#### Add create/mod date/data size to DataObjectAOImpl.findMetadataValuesByMetadataQuery #60

Include additional information on data size, create and modify dates to MetaDataAndDomainData,
useful in metadata query based virtual collections

#### Connection tester #67

Add connection tester package to jargon-data-utils to test out and measure put/get performance

#### Parallel file transfer performance #72

Additional tweaks to improve parallel file transfer performance. This includes an expansion of TCP configuration parameters for both the primary and parallel transfer thread to configure window size, quality of service parameters, timeouts, nodelay and other aspects. This also includes adjustemt to various buffering parameters based on benchmark testing.

This version includes much greater configurability of TCP/IP parameters for the primary and parallel socket connections. Currently these parameters are set to an aggressive window size, and bandwidth-favoring parameters, but can ba adjusted in the jargon.properties file.

#### Path interpolation when browsing across federations #78

Improved ability of Jargon to at least get some view into an iRODS tree starting at root, down into the user directory, when StictACL is enabled. It uses a heuristic to drill down into a user home and public directoy in the home and federated zone when it cannot actually access the tree. This might be configurable at a later point to get 'hints'.

Forgot what this was!

#### Socket renewal for long running get and put tranfers and large file transfer restart #77

Added a new jargon.properties value for socket connection and renewal that will, on get an put recursive operations, optionally renew the socket by closing the agent and starting a new one.

This may help in situations where intervening firewalls or other network considerations may otherwise forcefully terminate the connection.

Added a new AbstractRestartManager and a default in-memory restart manager in the IRODSSession. This can be configured to support large file restarts for get and put operations (work in progress!)
## Changes

####
>>>>>>> master
Original file line number Diff line number Diff line change
Expand Up @@ -467,7 +467,7 @@ public void overallStatusCallback(final TransferStatus transferStatus)
try {
if (transferStatus.getTransferState() == TransferStatus.TransferState.OVERALL_INITIATION) {
// potentially handle any pre proc for operation flow specs
log.info("handle overall initiation");
log.info("handle overall initiation");
handleOverallInitiation(transferStatus);
} else if (transferStatus.getTransferState() == TransferStatus.TransferState.OVERALL_COMPLETION) {
log.info("overall completion...updating status of transfer...");
Expand All @@ -486,14 +486,17 @@ public void overallStatusCallback(final TransferStatus transferStatus)
log.error("transfer cancelled, this will be handled by the conveyor execution service, and the callback here will be ignored");
processOverallCompletionOfTransferWithCancel(transferStatus);
} else {
log.info("unhandled transfer status of:{}", transferStatus);
}
log.info("unhandled transfer status of:{}", transferStatus);
}
} catch (ConveyorExecutionException ex) {
log.error("conveyorExecutionException during handling of overall status callback:{}", transferStatus, ex);
log.error(
"conveyorExecutionException during handling of overall status callback:{}",
transferStatus, ex);
doComplete = true;
throw new JargonException(ex.getMessage(), ex.getCause());
} finally {
if (conveyorService.getConveyorCallbackListener() != null) {
log.info("sending overall status callback to client");
log.info("sending overall status callback to client");
conveyorService.getConveyorCallbackListener()
.overallStatusCallback(transferStatus);
}
Expand Down

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

This file was deleted.

4 changes: 4 additions & 0 deletions jargon-core/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@
<parent>
<groupId>org.irods</groupId>
<artifactId>jargon</artifactId>
<<<<<<< HEAD
<version>4.0.3.1-SNAPSHOT</version>
=======
<version>4.0.2.2-SNAPSHOT</version>
>>>>>>> master
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>org.irods.jargon</groupId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,14 @@
/**
* Abstract base class for classes that compute and return a checksum based on
* some algorithm, based on as Strategy pattern
*
*
* @author Mike Conway - DICE
*
*
*/
public abstract class AbstractChecksumComputeStrategy {

/**
*
*
* @param localFileAbsolutePath
* <code>String</code> with the absolute path to a local file
* @return {@link ChecksumValue} with a checksum in digest (string) form.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,16 @@
/**
* Interface representing code to handle negotiation and determination of
* checksum type
*
*
* @author Mike Conway - DICXE
*
*
*/
public interface ChecksumManager {

/**
* Using the values in the jargon properties, as well as info on the target
* server, determine the type of checksum to use
*
*
* @return {@link ChecksumEncodingEnum} used for the server
* @throws JargonException
*/
Expand All @@ -25,7 +25,7 @@ public abstract ChecksumEncodingEnum determineChecksumEncodingForTargetServer()
/**
* Using a value from iRODS describing a checksum on a file, determine the
* type of encoding used
*
*
* @param irodsChecksumValue
* <code>String</code> with the checksum value from iRODS,
* potentially including a prefix
Expand All @@ -35,6 +35,6 @@ public abstract ChecksumEncodingEnum determineChecksumEncodingForTargetServer()
*/
public ChecksumValue determineChecksumEncodingFromIrodsData(
String irodsChecksumValue)
throws ChecksumMethodUnavailableException;
throws ChecksumMethodUnavailableException;

}
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@
/**
* Service to determine, based on local and remote server properties, the
* correct checksum hash type.
*
*
* @author Mike Conway - DICE
*
*
*/
public class ChecksumManagerImpl implements ChecksumManager {

Expand All @@ -45,7 +45,7 @@ public ChecksumManagerImpl(final IRODSAccount irodsAccount,

/*
* (non-Javadoc)
*
*
* @see org.irods.jargon.core.checksum.ChecksumManager#
* determineChecksumEncodingForTargetServer()
*/
Expand Down Expand Up @@ -107,9 +107,9 @@ public ChecksumEncodingEnum determineChecksumEncodingForTargetServer()

/*
* Negotiation:
*
*
* DEFAULT - use MD5 pre consortium and SHA256 post
*
*
* STRONG - use MD5 pre 3.3.1 and SHA256 after
*/

Expand Down Expand Up @@ -152,21 +152,21 @@ public ChecksumEncodingEnum determineChecksumEncodingForTargetServer()

private void cacheEncoding(final ChecksumEncodingEnum checksumEncoding) {
irodsAccessObjectFactory.getDiscoveredServerPropertiesCache()
.cacheAProperty(irodsAccount.getHost(), irodsAccount.getZone(),
DiscoveredServerPropertiesCache.CHECKSUM_TYPE,
checksumEncoding.toString());
.cacheAProperty(irodsAccount.getHost(), irodsAccount.getZone(),
DiscoveredServerPropertiesCache.CHECKSUM_TYPE,
checksumEncoding.toString());
}

/*
* (non-Javadoc)
*
*
* @see org.irods.jargon.core.checksum.ChecksumManager#
* determineChecksumEncodingFromIrodsData(java.lang.String)
*/
@Override
public ChecksumValue determineChecksumEncodingFromIrodsData(
final String irodsChecksumValue)
throws ChecksumMethodUnavailableException {
throws ChecksumMethodUnavailableException {

log.info("determineChecksumEncodingFromIrodsData()");

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@

/**
* Represents a checksum and the type of checksum
*
*
* @author Mike Conway - DICE
*
*
*/
public class ChecksumValue {

Expand Down Expand Up @@ -83,7 +83,7 @@ public void setChecksumTransmissionFormat(

/*
* (non-Javadoc)
*
*
* @see java.lang.Object#toString()
*/
@Override
Expand All @@ -108,4 +108,65 @@ public String toString() {
return builder.toString();
}

/*
* (non-Javadoc)
*
* @see java.lang.Object#hashCode()
*/
@Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime
* result
+ ((checksumEncoding == null) ? 0 : checksumEncoding.hashCode());
result = prime
* result
+ ((checksumStringValue == null) ? 0 : checksumStringValue
.hashCode());
result = prime
* result
+ ((checksumTransmissionFormat == null) ? 0
: checksumTransmissionFormat.hashCode());
return result;
}

/*
* (non-Javadoc)
*
* @see java.lang.Object#equals(java.lang.Object)
*/
@Override
public boolean equals(final Object obj) {
if (this == obj) {
return true;
}
if (obj == null) {
return false;
}
if (getClass() != obj.getClass()) {
return false;
}
ChecksumValue other = (ChecksumValue) obj;
if (checksumEncoding != other.checksumEncoding) {
return false;
}
if (checksumStringValue == null) {
if (other.checksumStringValue != null) {
return false;
}
} else if (!checksumStringValue.equals(other.checksumStringValue)) {
return false;
}
if (checksumTransmissionFormat == null) {
if (other.checksumTransmissionFormat != null) {
return false;
}
} else if (!checksumTransmissionFormat
.equals(other.checksumTransmissionFormat)) {
return false;
}
return true;
}

}
Loading

0 comments on commit d83084a

Please sign in to comment.