Using rwp for archived content

[HadrienGardeur]

A draft of EPUB/a was recently announced by ISO/NISO/CLOCKSS: https://clockss.org/new-draft-international-standard-for-the-preservation-of-ebooks-published-in-the-epub-format/

EPUB/a stands for EPUB archive, a profile of the EPUB spec designed with long term preservation in mind.

The draft is available at: https://clockss.org/wp-content/uploads/2024/06/EPUB-3-Archiving-for-Preservation.pdf

On the W3C mailing list, @iherman did a great job summarizing what makes this profile different from "normal" EPUB:

• Use only core media types (section 3.2)
• Use of exempt resources is not encouraged (section 6.2)
• Use only UTF-8 (section 6.3)
• Publication resources shall be located in the EPUB container (section 7.3)
• EPUB container shall not be encrypted (section 7.4)
• Publications must not include multiple renditions until such time as the note becomes a stable W3C Recommendation or is replaced by another technology. (section 7.6)
• Publications require the following additional metadata (beyond those that are required by EPUB 3.3) (section 8.3):
  • Information about any compression methods or any reading systems for which the file is optimized
  • Creator
  • Creation Date or Start Date/End Date
  • Publisher/producer/distributor
  • A standard unique manifestation/work identifier (e.g. ISBN or ISSN)
  • An item or instance identifier, if available (e.g. a DOI)
  • Metadata format and its versions
  • Schema identifiers for metadata records (e.g. Dublin Core or PREMIS)
  • Administrative metadata (e.g. nature and formats of embedded media)
  • Technical metadata (e.g. file formats and versions, digital signatures and checksums)
  • Rights metadata
  • Structural metadata
• Although the EPUB 3 specification requires a media type be specified for each resource, it does not require the correct media type be specified. For EPUB/A publications, each resource shall have the correct media type. (section 8.4)
• As EPUB/A publications do not allow remote hosting of resources, the remote-resources value shall not be used with this attribute (section 8.4)
• For collections (in the package document) only use collection types that are recognized as standard (none at this moment) (section 8.6)
• Scripts in EPUB content documents (section 9.3):
  • shall not include scripts that retrieve resources or information from outside the EPUB container (e.g., using JavaScript APIs such as XmlHttpRequest and Fetch).
  • shall not include HTML forms that submit data to external servers.
• Do not use features that are considered as deprecated in EPUB 3. (section 11)

In a similar vein, @atomotic mentioned his interest in extracting external links from an EPUB, to check if they're still alive and return a 2xx HTTP code.

How does this group feels about adding support for such potential features to rwp and the go-toolkit in general?

[atomotic]

I was unaware of the EPUB/a initiative; it's good to see progress in this area.
This report on eBook preservation from several years ago still contains many applicable ideas.

Ideally, the rwp CLI could provide a verify-archive subcommand that would:

1. Analyze an EPUB file against preservation best practices
2. Generate a compliance report

The report could potentially be saved inside the EPUB container, though this is not recommended as it would alter the checksum of the original file.

Preserving HTTP links within book content is a separate challenge, as these links decay over time like any others. It would be useful to:

1. Extract and check these links
2. Report on their current status

This feature could be particularly valuable in scholarly publishing.

Related Resources

1. EPUB Link Rot
2. EPUB Link Checker
3. Lychee EPUB Support
4. Using the Perma.cc API to Check Links
5. When Good Links Go Bad: Link Rot in eBooks

[chocolatkey]

In addition to what @atomotic is mentioning, which does not modify the EPUB file, I think there is also a use case for lossless optimizing of an EPUB file.
This includes recompressing resources at level 9, setting STORE compression mode for media assets that are already using a compressed file format (which many EPUBs do not do), removing unused files etc.

[HadrienGardeur]

Preserving HTTP links within book content is a separate challenge, as these links decay over time like any others. It would be useful to:

• Extract and check these links
• Report on their current status

This feature could be particularly valuable in scholarly publishing.

It's a separate but complementary challenge:

• organizations archiving EPUB may want to archive their external resources as well which means potentially:
  • archiving pages referenced by those links (some of them might already have a Web archiving program or collaborate with a third party for that)
  • and/or archiving remote resources (video, audio, images mostly)
• and in order to be compliant with EPUB/a, they might also be tempted to package these remote resources back into an EPUB

In addition to what @atomotic is mentioning, which does not modify the EPUB file, I think there is also a use case for lossless optimizing of an EPUB file.
This includes recompressing resources at level 9, setting STORE compression mode for media assets that are already using a compressed file format (which many EPUBs do not do), removing unused files etc.

I would treat that one differently, it would be mostly interesting for optimizing EPUB files in a distribution workflow than archiving them. We might want to open a separate discussion, where we identify other means of optimizing an EPUB (unused styles for example).