LTFS compared with tar and the REM formats.

Legibility we inherited. POSIX tar is published, decades old, and read by a tool already installed on every Unix-like system. An archive written as tar was never exposed to that failure; we chose a base that already had the property.

Location both solve, by different means. LTFS keeps a filesystem index in its own partition, updated as the cartridge is used. Remanence puts a manifest in every object — each file's path, size, checksum and position — and repeats bootstrap records along the cartridge, so the map is rebuilt from a few short reads rather than the whole reel.

Which mechanism is better depends on who is doing the reading.

For manual access, LTFS is the better model. Once mounted, a cartridge is a filesystem: a person can list it, browse it and copy a file out using the tools already on the machine, with nothing project-specific to install or learn. That is a real advantage, and it is why LTFS is pleasant to work with at a single drive.

For automated access the same presentation is overhead. A scheduler picks the cartridge and a program extracts a known byte range; what it needs is an offset and a length, which a manifest gives it directly. The mount and unmount cost is paid on every cartridge to produce a view no person will look at — what that costs is measured further down. Our own archive runs this way almost entirely, and as more of the work of an archive is handled by software rather than by hand, that advantage matters less than it once did.

Survival LTFS does not address. Media degrade, and in a library of any size some cartridges develop blocks the drive cannot read well before their rated life is out. LTFS assumes a conforming implementation is available and the medium reads back correctly, both reasonable for interchange. Remanence assumes neither: parity stored apart from the data, so unreadable blocks are reconstructed from the surviving ones, and a SHA-256 for every file inside the format, so what comes back can be shown to be what went in rather than assumed to be. The checksums also catch faults that never touched the tape — in software, memory or transfer.

Solving the third is only worth its cost if an archive expects to outlive its own software and to meet media that have degraded. Many working archives reasonably conclude otherwise. What follows is the property-by-property detail, with the measurements and vendor documentation behind our own decision.

One distinction runs through what follows: the LTFS specification, a vendor's implementation of it, and the workflow built around it are three different things, and several of the difficulties below belong to the second and third rather than the first. The sections below say which.

Hard disks are random-access devices. A filesystem can read and update file metadata — names, paths, sizes, dates — anywhere on the disk at any time, which is why filesystems sit so naturally on them.

Tape is a sequential-access device. Data is written as a stream of blocks, one after another. LTFS presents a random-access view of that medium by splitting each cartridge into two partitions that run side by side along the length of the tape: one for data, and one for a metadata index that is rewritten when the tape is used. The filesystem presentation is the point — it is what lets an operator and ordinary tools navigate a cartridge directly.

That presentation has a cost. A filesystem is machinery for managing a space that changes: files are created, grow, shrink, and are deleted; the holes left by deleted data must be found and reused; free space must be tracked and fragmentation kept under control.

An archive of the kind we operate uses very little of that. Archival data is written once and not deleted, so the workload is stacking objects end to end, which is what sequential media does well. None of the rest is needed for it.

The simpler the on-tape structure, the fewer states it can be left in, and the easier recovery becomes when something has gone wrong.

TAR — the tape archive format — predates LTFS by decades and is available by default on every Linux and Unix-like system. Any tar writes an archive as a single stream — a simple metadata header, then the file's bytes, file after file: no filesystem, no mounting, no index to keep synchronized. Tar has accumulated several dialects over its long life, so for standardization we use the POSIX-specified pax form (POSIX 1003.1-2001) — one precisely defined variant that any modern tar reads.

Aspect	TAR	LTFS	REM formats
Primarily designed for	Streaming files to sequential media	Interchange — a cartridge readable without the writing application	Automated archives that must survive lost catalogs and partial damage
Open specification	Yes (POSIX)	Yes (SNIA LTFS Format, ISO/IEC 20919)	Yes — published, with DOIs
On-tape structure	One append-only stream	Two partitions; an index updated as the cartridge is used	One append-only stream, plus parity and bootstrap records
Mount/unmount overhead	None	30–60 s each way measured; vendor docs allow minutes	None
Usable tape capacity	~98%	≈95% structural; 80–88% in our measured use	~98%, less the parity level you choose
Self-describing tapes	Achievable (see below)	Yes, natively	Yes — a manifest in every object, bootstrap records on every tape
Per-file checksums	No	No	Yes — SHA-256 for every file
Damage recovery (parity)	No	No	Yes — Reed–Solomon, stored apart from the data
Partial file retrieval	Yes	Yes	Yes — any byte range, by arithmetic
Software needed to read a tape	tar — a standard tool on every Linux and Unix-like system	Vendor or third-party LTFS application, installed separately	Standard tar for the files; open-specification tools for verify and repair

Third-party corroboration: HPE's StoreOpen best-practices guide notes that large indexes lead to "an extended delay in mounting the file system" (≈1 kB of index per file); Fujitsu's LTFS guide allows about ten minutes for background processing at eject. On capacity, the index partition and its guard wraps consume about 5% structurally (≈75 GB on LTO-5), before index growth in the data partition; tools such as YoYotta additionally reserve ~100 GB of headroom.

An LTFS tape is mounted before use and unmounted afterward, so that the index can be read and updated. If a cartridge is removed, or the drive loses power, before unmounting completes, the volume can be left in an inconsistent state. That is the ordinary consequence of maintaining mutable state on the medium, and the vendors document it. From the Quantum LTFS User's Guide:

"Tape activity may continue for several minutes after un-mounting an LTFS volume… Attempting these actions before tape activity has finished may cause corruption of the tape format."

And on power loss:

"…if power is removed from the drive without unmounting… the volume will be left in an inconsistent state and all files added since the last dismount operation will be inaccessible. The ltfsck utility may be able to recover the file data… but the metadata (such as filename, access dates) will be lost."

The second half of that passage is what mattered to us: recovery may restore the data, but without filenames and dates. For an archive whose descriptive metadata is much of its value, that is a partial recovery.

Implementations mitigate this with periodic index writes and write-on-close updates, and a workflow that unmounts promptly will rarely meet the failure. The exposure is a function of how the format is operated.

Our requirement was to remove the exposure rather than manage it. With tar, an interrupted write is repaired by seeking back to the last valid header block and rewriting from there, with no recovery utility involved. The REM formats inherit that property: an interrupted write costs at most the object in flight, never the volume.

Mounting and unmounting an LTFS cartridge takes time, because the index must be read and written. In our measurements it was 30–60 seconds each way, and vendor documentation allows for several minutes. A tar tape has no equivalent step.

Whether that matters depends on scale. For an archive whose cartridges are mounted occasionally it is irrelevant. In one representative month our restore requests touched dozens of different tapes a day, and the mount and unmount cycles alone accounted for the better part of an hour of drive time daily. Quantum's own StorNext documentation advises against configuring LTFS for the copy used for file retrieval, which is consistent with what we measured.

The LTFS index partition consumes tape capacity. The structural cost is about 5%; in our measured use the total ran between 12% and 20%. The gap between the two figures has a cause: the index is not only kept in its own partition but is also written again into the data partition at syncs and at unmount. Through format 2.4 each of those writes was a complete copy of the index, so consumption grew with the number of files and the number of sessions — these are the versions behind our measurements. Format 2.5 (2019) added incremental indexes that record only changes, which reduces this growth on implementations that use them. TAR's metadata overhead is about 2%, including a safety buffer.

LTFS is an open, published standard, and an open-source reference implementation exists. The friction we met was in implementations and packaging rather than in the specification:

• Some vendor LTFS tools check the manufacturer of the attached drive and decline to work with another vendor's hardware. On one occasion we compiled the open-source reference implementation from source in order to read LTFS tapes written by a third-party archiving application. The specification does not ask for that behaviour; particular products do.
• The specification has revised over time (2.5 is standardized as ISO/IEC 20919), and a cartridge written under a newer revision may not be readable by an older implementation. That is ordinary for a maintained standard, and manageable when the reading environment is known. It is harder to reason about across several migration cycles.

A tar tape can be read on any Unix-like system with a tool that is already installed. That was one of the reasons we chose tar.

One dependency neither format removes is the drive. Since LTO-8 a drive reads only its own generation and the one before it; earlier generations read two back. A cartridge outside that window cannot be read at all, whatever is written on it. So the practical horizon is a migration cycle, not a lifetime: archives move to current media every several years, and a tape left much longer than that is a hardware problem before it is a format problem. We design for that window rather than for a notional fifty years. Migration is also when the format matters most: every tape is read end to end, a bad patch surfaces then, and a per-file checksum is what shows the new copy is faithful.

LTFS records no per-file checksums and provides no parity. Neither belongs to the problem it set out to solve — a format for interchange has no business dictating an organization's fixity strategy.

For a preservation workflow the consequence is still real. Fixity and damage tolerance have to come from somewhere, so they are supplied by the layer above: a checksum manifest in a database, a second copy on other media, a validation script run after every write. That scaffolding works, but it lives outside the format and outside the cartridge.

We moved those controls into the format instead, so they travel with the cartridge. REM-OBJECT carries a SHA-256 for every file; REM-PARITY adds Reed–Solomon parity stored apart from the data. A cartridge separated from its institution can then still prove what it holds.

TAR's one genuine gap is that it is not natively self-describing. Closing it costs one extra write. For years our production archive appended a full catalog of the tape's contents — the logical equivalent of the LTFS index, but written once, at the end of the tape, when the cartridge was finalized. No second partition, no mount cycle, and nothing rewritten as the tape was used. Even with the archive database lost, every tape could describe itself.

The REM formats formalize that idea: the catalog moves inside each object as a manifest, and bootstrap records repeat along the cartridge, so a bare unlabelled tape can be mapped, verified and repaired from the tape alone.

LTFS's self-describing index does not eliminate the need for a database in practice — any archiving application still needs one for cross-tape search.

The choice follows from what an archive has to survive.

• Where cartridges leave the building — interchange with another institution, delivery to a client, a colleague with a standalone drive and no relationship to your system — LTFS is doing the job it was designed for. A published filesystem view means the recipient needs nothing from you but the cartridge.
• Where the archive is written and read by one automated system, and the requirement is to reconstruct and prove its contents after the catalog and the software are gone, we chose the tar family. Plain tar is simple, open, and readable with tools already present; the REM formats keep that foundation and add per-file checksums, parity, and catalog-free recovery.
• Whatever the format, do not put every copy in one. Implementation defects have shipped in tape software of every kind; ours is not exempt. If every copy of an archive shares a single format, one format-level defect can reach all of them at once. This is the recommendation we would make most insistently.

Fixity needs the same discipline whichever format is used. Neither tar nor LTFS performs write-time integrity checking of its own. Our practice has been to calculate a checksum for every file, store it away from the tape, and verify it, including a full restore and re-checksum of every archive we write.