Keep legacy tapes readable during a migration.

Suppose an archive changes software before replacing a tape library that still has useful life. Existing cartridges were written by the old application; new cartridges use the new format. The drives can handle both sets of media, but each application also moves cartridges, loads drives, positions tape, reserves devices, and keeps its own account of what is mounted where. They cannot safely do that independently to the same hardware.

A library can be split into logical partitions, with drives dedicated to each application. That prevents collisions, but one side cannot borrow an idle drive from the other. An integrated deployment with a reader for the legacy format could leave those drives in one pool. This helps in this particular arrangement; most migrations will use separate old and new hardware.

This example depends on media compatibility. IBM documents, for instance, that an LTO-9 drive can read and write both LTO-8 and LTO-9 cartridges; other generation combinations differ. IBM also states that a drive assigned to one logical library cannot be shared with another. HPE's MSL3040 guide also assigns at least one drive to each partition. A format reader changes none of these hardware rules.

Remanence's SCSI and library layers know about drives, slots, filemarks, tape records, and position. They do not know what archive format is in those records. Format-specific code lives above that seam. A reader can probe a source, describe what it can do, and expose the files it finds through one normalized reader interface.

The word plug-in is useful as a mental model, with one qualification. The current reader is a compile-time Rust module selected when the binary is built, rather than an executable discovered at runtime. This is a code boundary, not a general plug-in registry.

The foreign-format reader interface is read-only. It normalizes what a reader returns, whether the source is physical tape records or a dump stream. It does not define a way to write another vendor's format.

probe, scan, and stream_all are operations in the public Rust interface. Damage reporting is part of scan and stream, not a separate fourth operation. An implementation may support indexed single-file reads; our current reader does not. The present command line also requires the operator to name the format and source.

Much of our existing tape archive was written by a proprietary application that we no longer run. We still had the tapes, but no supported version of the software that could recover them on the systems we use now.

We reverse-engineered the on-tape format and wrote a read-only reader. An open specification would have spared us much of this work. Instead, we had to learn the format from the evidence left behind.

A reader solves the tape side. The application above Remanence can import the old catalog and put legacy and native copies in the same search. When someone requests a file, that application chooses the copy and asks Remanence to use the corresponding reader. The user does not have to know which archive program wrote the tape.

The catalog, database, and search interface are application features, not part of Remanence. Our application above Remanence is internal and is not published. If the old software has a dependable API, the application could send legacy restores there instead. A direct reader is useful when that connection is unavailable, or when the archive should no longer depend on the old software. An imported catalog says what should be on a tape; it does not prove that the tape is readable.

Remanence has a public, read-only reader interface. One compatibility reader exists today, but the current tools still require an operator to name the format and source. Automatic format selection and shared-drive scheduling are not implemented.

A reader cannot overcome a drive or cartridge mismatch, and it cannot supply a missing encryption key. A reverse-engineered reader may also encounter a format variation it has never seen. Restored and migrated files still need to be checked before old tapes are retired.

Inspect the reader interface → ·  Read the command reference →

Implementation note: the compatibility reader we used for our migration handles BRU and BRU-PE archives written by software from TOLIS Group. We regard it as auxiliary migration tooling for this archive, rather than part of the Remanence core. TOLIS ceased operations and support in 2020; OWC later acquired the product technology and customer base. See the contemporary notice and acquisition announcement.