Your Backup Hasn't Been Tested in 18 Months. Is It Still a Backup?

You have 18 months of successful backup jobs. Every morning, the dashboard is green. The backup repository is growing exactly as expected. Retention is working. Nothing has complained.

Then someone asks a simple question: when was the last time you actually restored from one of those backups?

Silence. Maybe the answer is 18 months ago. Maybe it was never.

That does not automatically mean your backups are unusable. It means something more precise:

Your evidence about recoverability may be 18 months old.

A successful backup job tells you that data was copied according to the backup system's process. It does not, by itself, prove that the data can be restored, that the restored system will work, or that the business can recover within its required timeframe.

NIST SP 800-209 recommends periodically testing backups to verify their integrity and ability to be restored, and recommends end-to-end test restores that simulate real-life recovery scenarios for applications with strict restoration-speed requirements.

That leads to a useful distinction:

Backup success is an event. Restore success is evidence.

1. A Backup Job Is Not a Recovery Test

Consider a backup job that reports "completed successfully." What does that actually tell you?

It may tell you that the backup process ran, the source data was read, data was transferred, the backup target accepted the data, and the software recorded a successful job. Those are useful facts.

But they are not the same as proving that the required recovery point can be found, the backup data is intact, the data can be restored, the restore target is available, the required credentials work, encryption keys are available, the restored system can operate, the application can use the restored data, or the recovery can finish within the required timeframe.

The gap matters: a backup job succeeding only tells you a backup exists. Whether it can be restored, whether the restored data is correct, whether the system or application works, and whether the business service actually works are all separate, unproven steps further down the chain.

A normal backup job mainly gives you evidence near the top of this chain. A recovery test exercises more of it.

2. What a Successful Backup Actually Proves

It is easy to overinterpret the word successful. Suppose a backup application reports "job completed in 47 minutes." That is evidence about the backup operation. It is not automatically evidence about the recovery operation.

Backup and restore are different workloads. A system can be very good at writing data and still have problems reading it back under recovery conditions. The recovery path may depend on components that the backup job never exercises: recovery credentials, encryption keys, backup catalogs, recovery servers, network access, compatible software versions, storage capacity, application dependencies, DNS or identity services, licensing, and operator procedures.

This is why NIST SP 800-209 separates monitoring backup execution from periodically testing backup integrity and restore capability. For applications with strict restoration-speed requirements, it specifically calls for end-to-end testing in a sandbox recovery environment.

The important question changes from "did the backup succeed?" to:

"What part of the recovery process have we actually demonstrated?"

3. Restore Testing Turns an Assumption Into Evidence

Without a restore test, the statement often sounds like "the backup should work." That may be a reasonable expectation. But it is still an expectation.

After a successful restore test, the statement becomes more concrete: "we restored this recovery point, to this environment, on this date, in this amount of time, and verified the result." That is evidence. It has a timestamp. It has a scope. It has a result. It can also have exceptions and lessons learned.

This distinction is the real value of restore testing. You are not merely checking whether a backup exists. You are reducing uncertainty about recovery.

4. A Restore Can Succeed and Recovery Can Still Fail

Imagine restoring a 500 GB dataset. The restore finishes successfully. The backup software reports "restore completed." Is the recovery complete?

Not necessarily. The application may still fail because its database was not restored, a required service is missing, credentials are unavailable, an encryption key cannot be recovered, a dependency is still offline, the application version is incompatible, permissions were not preserved as expected, the restored data is incomplete, or the recovery environment is not ready.

NIST CSF 2.0 goes beyond simply restoring data. Its recovery guidance (subcategories RC.RP-03 and RC.RP-05) calls for verifying the integrity of backups before restoration, verifying the integrity of restored assets, restoring systems and services, and confirming normal operating status.

That gives us an important principle:

A successful data restore is evidence of one part of recovery — not automatically evidence of business recovery.

5. Test More Than One File

Not every restore test needs to rebuild the entire business. But not every restore test proves the same thing either. A useful way to think about testing is by depth.

Level 1: Does the backup exist? You locate the recovery point. This tests availability and discoverability. Useful, but limited.

Level 2: Can the data be restored? You restore selected files, folders, or datasets into a controlled environment. Now you have evidence that the backup can actually be read and restored. Better.

Level 3: Can the system or application operate? You restore enough of the environment to test the actual workload — start the database, connect the application, open representative records, perform a critical query, verify permissions, test an important workflow. Now the test is exercising more of the recovery path.

Level 4: Can the business function return? The final question is not "did the files come back?" It is "can the business perform the critical function again?" For a file server, that may mean users can access required shared data. For an accounting system, it may mean the application starts and critical financial records are available. For a database-backed application, it may mean the service can accept a representative transaction.

The deeper the test, the more assumptions you remove — moving step by step from "backup exists" to "data restores" to "system restores" to "application works" to "business function works." This is not a formal NIST maturity model. It is a practical way to understand what your restore test actually proves.

6. Test Against the Recovery Requirement

There is another trap: a restore can succeed and still fail the recovery requirement.

Suppose your business has an RTO of four hours. Your restore test takes nine hours. Technically, the restore worked. Operationally, it did not meet the recovery requirement you established.

NIST SP 800-209 explicitly connects restoration testing with required restoration speed and recommends end-to-end testing for applications with strict restoration-speed requirements.

So a meaningful restore test should record more than pass/fail. It should also ask: what data was restored? From which recovery point? How long did restoration take? What dependencies were required? Was the data usable? Did the application work? Did the test meet the required recovery timeframe?

This changes the purpose of the test. You are no longer testing only whether the technology works. You are testing whether the recovery capability is good enough for the requirement.

7. Your Last Test Is Evidence From a Point in Time

Now return to the 18-month example. Suppose the backup was successfully restored in January 2025. Since then, the NAS firmware changed, backup software was upgraded, credentials were rotated, the application was updated, the database version changed, storage capacity increased, retention policies changed, the recovery administrator left, and the business added new data sources.

The backup jobs are still green. Does the 2025 restore test prove that the 2026 recovery path works? Not completely. It proves that the tested recovery path worked under the conditions that existed when the test was performed. That is still valuable evidence. But it is not a permanent certificate of recoverability.

NIST's OT Backup Quick Start Guide makes a similar operational connection: effective backup management should integrate backups with change management, create them regularly, test them, and review them during recovery exercises. The guide is written specifically for operational technology, so its recommendations should not be treated as a universal SMB schedule, but the underlying principle is useful: backup reliability has to keep pace with system change.

A restore test is evidence of a recovery system at a point in time — not a permanent certificate of recoverability.

8. How Often Should You Test?

There is no single testing frequency that makes every organization safe. A business with highly critical data, strict RTOs, rapidly changing applications, complex dependencies, and frequent infrastructure changes may need more frequent and deeper testing than a small environment with simple file-based workloads.

Factors to consider include data criticality, RPO and RTO, rate of system change, recovery complexity, number of dependencies, previous test failures, business or regulatory requirements, and changes to backup and recovery infrastructure.

NIST SP 800-209's current Revision 1 draft specifically recommends periodic testing at least monthly for critical data, in addition to end-to-end test restores for applications with strict restoration-speed requirements. The 2020 final edition already establishes the broader principle — periodic testing of backup integrity and restore capability — though the specific "at least monthly" cadence appears in the newer draft language.

The important point is not to turn "monthly" into a magic number regardless of its source. The point is:

Critical recovery capability should be tested often enough that the evidence remains relevant to the environment you would actually have to recover.

9. A Restore Test Should Produce Evidence

A test that ends with "looks good" is difficult to use six months later. A better test leaves behind a record. For example:

This record does something a green dashboard cannot. It tells you what was actually tested, under what conditions, and what happened. That becomes valuable during an incident. Instead of saying "the backup should work," you can say "this recovery path was successfully tested against this workload, from this recovery point, under these conditions." That is a much stronger position.

10. What Should Count as a Failed Test?

A restore test does not only fail when the restore command returns an error. Consider these cases.

The restore completes, but data is incomplete — the software reports success, but expected files or records are missing. Recovery test: failed.

The data restores, but the application cannot use it — the files exist, but the database or application cannot open them. Recovery test: failed.

The restore works, but required credentials are missing — the data is available, but nobody can authenticate the recovered service. Recovery test: failed.

The restore works, but the encryption key is unavailable — the backup exists, but the organization cannot make usable data from it. Recovery test: failed.

The restore works, but it takes nine hours against a four-hour RTO — the technology worked, the recovery requirement did not. Recovery test: failed against the stated objective.

The restore works, but nobody knows what to do next — the technical procedure succeeded, but the operational recovery process stops there. Recovery capability remains incomplete.

This is why restore testing should be evaluated against the recovery objective, not merely against whether the backup software displayed a green checkmark.

11. Test the Recovery Path, Not Just the Repository

By now, the architecture may look familiar: a backup depends on a repository, which depends on credentials, which depend on an encryption key, which feeds into a recovery host over a network, serving an application, which finally supports the business service.

Earlier in this series, we looked at how multiple copies can share failure domains and dependencies. Restore testing asks a different question:

Have we actually exercised the path we are depending on?

A recovery diagram can show that a path exists. A restore test can provide evidence that the path works. That distinction matters.

A recovery path that exists on paper is an architecture. A recovery path that has been tested is evidence.

And the more important the workload, the more important it becomes to test something close to the recovery scenario you would actually face. CISA's ransomware guidance likewise recommends maintaining offline, encrypted backups and regularly testing their availability and integrity in a disaster-recovery scenario. It also recommends exercising backup procedures regularly.

12. Not Every Test Has to Be a Full Disaster

There is a temptation to think "if we cannot rebuild the entire business every month, we cannot test properly." That is unnecessary. Testing can happen at different scopes.

Small, frequent tests are useful for representative files, recent recovery points, basic restore procedures, and integrity checks. Deeper workload tests are useful for databases, virtual machines, critical applications, large datasets, and dependencies. End-to-end recovery exercises are useful when RTO is strict, the workload is highly critical, recovery dependencies are complex, major infrastructure changes have occurred, or the organization needs stronger evidence of full recovery capability.

The goal is not to make every test enormous. The goal is to make the evidence appropriate to the risk.

13. The Test Should Change When the Environment Changes

One of the easiest ways to create stale recovery evidence is to test only on a calendar.

Suppose you successfully tested a recovery environment six months ago. Then you migrate the application, replace the NAS, change the backup software, change encryption, change credentials, modify retention, move the repository, or change the recovery network.

A calendar reminder may still say "next restore test: six months from now." That is not enough. A significant infrastructure or application change can invalidate assumptions immediately.

This is why change management belongs in the backup conversation. A useful operational rule is:

A significant change to the recovery path should trigger a review of recovery evidence.

That does not mean every minor configuration change requires a full disaster-recovery exercise. It means the organization should know which changes can invalidate its recovery assumptions.

14. What Does "Passed" Actually Mean?

Before running a restore test, define what success means.

For a simple file restore, success might be: expected files are present, files open correctly, permissions are correct, and restore completes within the target time.

For an application, success may require that the system starts, the database is consistent, the application connects, critical records are available, a representative workflow succeeds, required dependencies are available, and recovery finishes within the required timeframe.

For a business-critical service, the final validation may belong to the business owner rather than the backup administrator. That distinction matters. The person who can prove that a database started is not necessarily the person who can prove that the business can operate again.

NIST CSF 2.0 reflects this broader recovery view by requiring verification of restored assets, restoration of systems and services, and confirmation of normal operating status.

15. The 18-Month Question

So, is a backup that has not been tested for 18 months still a backup? The precise answer is:

Probably — but you have less current evidence that it is recoverable.

The problem is not the number 18. The problem is the uncertainty. If your last successful restore test was 18 months ago, ask: what has changed since then? Is the backup software the same? Is the recovery environment the same? Are the credentials still valid? Are the encryption keys available? Is the application version compatible? Is the recovery target still available? Can the current administrator perform the procedure? Can the workload be restored within the current RTO? Has the business's recovery requirement changed?

If the answers are unknown, the restore test has revealed something important. Not necessarily that your backup is broken. It has revealed that your evidence is incomplete or stale.

16. A Simple Restore-Test Record

For each important workload, you should be able to answer, grouped into five areas.

Recovery point — which backup did we test, and why did we choose that point?
Recovery environment — where did we restore it, was it isolated or production, and what dependencies were required?
Data — was the expected data present, was integrity verified, were permissions and metadata correct where required?
Application — did the application start, could it access the restored data, did a representative business workflow succeed?
Time — how long did the recovery take, what was the required RTO, and did the test meet it?
Evidence — who performed the test, when, what failed or required manual intervention, and what changed afterward?
Next test — when should it be repeated, and what changes would trigger an earlier test?

This turns backup testing from a checkbox into an operational record.

17. The Real Value of Restore Testing

The purpose of restore testing is not to make a report look good. It is to reduce uncertainty.

Before the test: "we believe the backup can be restored." After a successful test: "we have evidence that this recovery path worked under these conditions." After repeated tests: "we have a history showing whether the recovery capability continues to work as the environment changes." And after a failed test: "we discovered the recovery problem before the disaster did."

That last outcome is not a wasted test. It may be the most valuable result you can get. A failed restore test gives you a problem while you still have time to fix it.

Final Takeaway: Your Backup Is Only as Trustworthy as Your Evidence

A green backup dashboard is useful. A successful backup job is useful. A repository full of recovery points is useful. But none of these automatically proves that the business can recover.

The evidence becomes stronger as you exercise more of the recovery path — from a completed backup, to a confirmed existing backup, to a backup that restores, to data that proves correct, to an application that works, to a recovery that meets RTO, to a business function that actually returns. The deeper the test, the more of the assumption you replace with evidence.

That is the real purpose of restore testing.

Backup success is an event. Restore success is evidence.

And that evidence has a date. It has a scope. It has limitations. It should be repeated when the recovery environment changes. Most importantly, it tells you something the backup dashboard cannot: whether the recovery path you are depending on has actually worked.

A backup that has never been restored may still contain your data. But you do not yet have strong evidence that you can recover from it.

A successful backup job tells you that the backup process completed. A restore test tells you much more: whether the recovery point can be used, whether the data is intact, whether the recovery environment works, whether dependencies are available, whether the application can operate, and whether the recovery meets its required timeframe.

NIST recommends periodic backup testing — its current draft guidance specifies at least monthly for critical data — and end-to-end testing where restoration-speed requirements are strict. CISA likewise recommends regularly testing the availability and integrity of backups and backup procedures.

So don't ask only "did the backup succeed?" Ask:

"When did we last produce evidence that we could actually recover from it?"

That is a much harder question. It is also the question that matters when the NAS is already dead.

A restore test can prove that one recovery point works. But what happens when your backup rotation keeps overwriting, replacing, disconnecting, or aging the copies you depend on? That is the next problem to examine: #32 — The Backup Rotation Problem.

Sources

• NIST SP 800-209 — Security Guidelines for Storage Infrastructure. Backbone source for periodic backup testing, restore capability, integrity, and restoration-speed alignment with RTO.

• NIST SP 800-209 Revision 1 (Initial Public Draft, 2026). Specific "at least monthly for critical data" testing cadence and end-to-end restoration-scenario simulation language.

• NIST Cybersecurity Framework 2.0, RC.RP — Incident Recovery Plan Execution. Verification of restored assets, restoration of systems and services, and confirmation of normal operating status (RC.RP-03, RC.RP-05).

• NIST SP 1339 — OT Backup Quick Start Guide. Backup testing, recovery exercises, and change-management integration, in an operational-technology context.

• CISA — #StopRansomware Guide. Regular testing of backup availability and integrity in disaster-recovery scenarios, and ransomware/disaster-recovery context.

Scope note
The specific "at least monthly for critical data" testing cadence cited in this article comes from NIST SP 800-209 Revision 1, currently an initial public draft (2026) rather than the finalized standard; the 2020 final edition of SP 800-209 establishes the broader principle of periodic restore testing and end-to-end testing for strict restoration-speed requirements, but this article does not claim the specific monthly figure appears in that final text. NIST SP 1339 was written for operational-technology environments, not general small-business IT; its testing and change-management principles are cited here because they generalize well, not because the guide targets a NAS or SMB environment. The four-level testing-depth model and restore-test record template in this article are this publication's own editorial framework, not a structure defined by NIST or CISA.

Series 5 — Backup Architecture

#21 — The 3-2-1 Backup Rule: What It Actually Means
#22 — Is 3-2-1 Enough for a Small Business?
#23 — Backup Independence: Are Your Copies Really Independent?
#24 — One NAS, Two Drives, One Cloud: Map Your Failure Domains
#25 — Local Backup vs Off-Site Backup: Which One Should Recover You?
#26 — Why Your Backup Architecture Has a Single Point of Failure
#27 — Who Is Actually Responsible for Your Backups?
#28 — The Backup Dashboard Is Lying to You
#29 — What Should You Monitor in a Backup System?
#30 — How Long Can a Backup Fail Before You Notice?
#31 — Your Backup Hasn't Been Tested in 18 Months. Is It Still a Backup?

The next article turns from testing to the recovery copies themselves: what happens when the backup rotation keeps overwriting, replacing, disconnecting, or aging the very copies your recovery plan depends on? That is #32 — The Backup Rotation Problem.