An archive in daily use, built to outlive its software.

Storage is not the same as preservation. An archive may survive physically and still become difficult to recover.

The original software may disappear. The catalog may be lost. The vendor may stop supporting the product. The people who built the system may no longer be available.

A long-term archive should be designed for those conditions from the beginning — while still allowing people to retrieve useful material without reading an entire cartridge.

Remanence is for people responsible for digital material that must remain recoverable for decades. It may be relevant if:

• You manage large collections of video, audio, photographs, research, or institutional records.
• You need material back today — a single clip from a master, not a whole cartridge.
• You know that backup is not the same as long-term preservation.
• You want restored files to be independently verifiable.
• You do not want permanent dependence on one vendor or application.
• You believe the archive itself should carry the information needed for recovery.
• You would rather judge a design on its merits than on how many others already use it.

You do not need to accept the design on trust. The specifications, implementation, test vectors, and recovery procedures are published so the claims can be examined directly.

Three layers, three responsibilities. Each solves a separate problem, so archive identity, tape recovery, and encryption are not locked inside one application.

Optional off-site protection: REM-ENCRYPT seals sensitive copies stored outside your control, without changing the durable object underneath.

Remanence does one thing: it writes archives to tape and reads them back, verifiably. Everything it does is exposed through a complete API and command line. It is not a hierarchical storage manager, and it does not want to be one. The catalog people search, the policies that decide what is copied where, the scheduling, and the other media of a real archive belong to a larger system above it.

Archive software is typically bought as one application that does everything, and therefore owns everything. That is the exact dependency this project exists to avoid. A component is open on both sides: an open format below it, open interfaces above it. It can be wrapped in a larger system, and it can be replaced, without touching the tapes. The tapes can also outlive it.

We use it this way in our own evaluation environment, where Remanence sits beneath an orchestration layer that manages copies across media against a central catalog and policy database. That layer holds an institution's own rules, so it is each operator's own affair. Ours is internal and is not published.

A self-describing archive package. REM-OBJECT packages files and their preservation information into an openly documented archive object. A standard pax-aware tar program can extract the files without the original archive application.

Each file carries a SHA-256 checksum, so restored data can be checked against what was originally written. The object stores files uncompressed, each beginning on a fixed grid, so the position of any file — or of any byte range within one — follows by arithmetic, and a reader retrieves just what it needs without reading the rest of the object.

Why it matters: the archive remains useful even if the original software or operational database is unavailable. Read the REM-OBJECT details →

Recovery information for damaged tape. REM-PARITY arranges archive objects and separate recovery data across the cartridge. If part of the tape becomes unreadable, the recovery data can be used to reconstruct the damaged region.

Bootstrap records stored on the cartridge describe its layout, so the contents can be mapped without depending on the central catalog.

Why it matters: a lost catalog or a damaged stretch of tape does not automatically cost you the data on that cartridge. Read the REM-PARITY details →

Optional protection for copies outside your control. Encryption is not automatically an advantage for long-term preservation: a lost key can make a perfectly intact archive permanently unreadable.

REM-ENCRYPT is intended primarily for sensitive copies stored off-site, or where contractual requirements make encryption necessary. It preserves the object structure needed for partial restoration while keeping encryption separate from the core archive format.

Why it matters: sensitive copies can be protected without making encryption mandatory for every archived object. Read the REM-ENCRYPT details →

The format is specified, not trapped inside one program. The specifications are openly published; the reference implementation, test vectors, and independent reader are available for inspection and testing.

A future implementation can be written from the specification even if the current software, supplier, or institution is no longer available. The design does not ask the archive to depend forever on one vendor, one executable, or one development team.

The project publishes evidence that can be checked independently:

• Formal specifications.
• A Rust reference implementation.
• A separate Python reader written from the specifications.
• Test vectors with pinned SHA-256 values.
• Deliberately malformed test objects.
• Defined errors that a correct implementation must produce.
• Mathematical proofs of selected pieces of the format logic, checked by the Lean theorem prover, with a public inventory of what is and is not proved.
• Source code and recovery procedures.

The objective is not to trust the author. It is to make the technical claims reproducible.

The practice this format formalises — tar with a full catalog appended to each tape — has run in our production archive for years with zero format-related corruption. The REM formats extend that practice with deterministic encoding, parity, and catalog-free recovery; the implementation has been developed against a Quadstor virtual tape library and tested on an HPE MSL3040 library with LTO-9 drives. It is not yet in production deployment, here or elsewhere.

That is real evidence for the practice, but the implementation is young, and one library family and one drive generation is a narrow basis for a format meant to last decades. More independent implementations, hardware environments, and institutional trials are still needed.

Early users should retain an independent archive copy and regularly restore and verify real material from physical cartridges. How and why it was built is on the About page.

• Understand how the architecture works →
• Inspect the specifications and source →
• Test it →

The first leads to conceptual understanding. The second leads to technical proof. The third leads to implementation.