Voyager extract scripts, newest to be used

[patrickzurek]

JIRA issue created by: rcook
Originally opened: 2012-07-23 11:54 AM

Issue body:
Just want to make sure that 1. Ralph's extract scripts (daily, initial, deletes, etc.) are documented and 2. that the CARLI staff knew about them and were using them. Specifically a question came up today about whether a data set used in testing contained suppressed records (as a possible reason why a single institution database contained so many match points in Aggregation within itself).

I know Ralph has quite an involved process to handle suppressed records and thought it worth having Gordon touch base.

[~cdelis] [~admin] [~jbrand]

[patrickzurek]

JIRA Coment by user: fellows
JIRA Timestamp: 2012-08-01 02:47 PM

Comment body:

No, I am not using any XC supplied scripts to export records from voyager for the OAI toolkit. I am using a script I wrote myself. The only scripts I can find on the WebSite (http://code.google.com/p/xcoaitoolkit/downloads/list) are from May 2010. Are these the ones you are referring to?

The script I wrote exports the bib and MFHDs separately and does two marcexports for bibs (one for updates/new, one for suppressed bibs). It turns the suppressed bibs to deletes, combines them with the true deletes, and sends a delete file to OAI.

[patrickzurek]

JIRA Coment by user: rcook
JIRA Timestamp: 2012-08-01 02:57 PM

Comment body:

[~admin] can you direct/point us on this? Are all doing the same basic thing? Are there separate needs that require two processes? If so, great, but otherwise maybe we can consolidate better and be more transparent.

[patrickzurek]

JIRA Coment by user: admin
JIRA Timestamp: 2012-08-01 03:24 PM

Comment body:

I'm in the process of updating my documentation for the export process as the scripts in the OAI wiki are out of date. We're doing something very similar- exporting four different files daily (updates/new & suppressed for both bib and MFHDs) then a weekly deleted export that contains separate bib & MFHDs files. We could actually pull out the deletes daily as well if we wanted to.

[patrickzurek]

JIRA Coment by user: fereira
JIRA Timestamp: 2012-11-16 07:23 AM

Comment body:

I've been working on something for extracting bib, mfhd, and authority records from a voyager database that can be run more frequently than a daily update. I'm having a heck of time trying to get something working that will read the CLOB in the database such that it produces marc that the marc4j library reader can handle. If anyone has any code that is accessing the MARC data from the voyager database directly I'd love to take a look at it.