NARA's Web Harvest Program to End

A recent article in dotgovwatch.com reports that for the first time in the history of the internet, the National Archives and Records Administration (NARA) has decided to abandon its Web Harvest Program where it records snapshots of Executive Branch websites at the end of each presidential administration. A recent NARA memo (NWM 13.2008) states:

“After considering our other records management program priorities for FY 2008, availability of harvested web content at other "archiving" sites (e.g., www.archive.org ), and the resources required for conducting and preserving a government-wide web snapshot, NARA has determined that we will not conduct a web harvest or snapshot at the end of the current Administration.”

The dotgovwatch.com article says that the "last Executive Branch web harvest that NARA conducted preserved 75 million web pages, many [of] which will be valuable records for historians in the coming decades. The Internet Archive may cease to exist in 10 years, but the archives will only grow more valuable with time. Not capturing federal web sites now may mean losing millions of web pages authored under the Bush administration when leadership changes in January 2009." The Sunlight Foundation comments that "The fact that digital preservation is done by others outside NARA isn't an excuse for NARA to abdicate their responsibility, but an argument that they should be capable of fulfilling it." (Digital Preservation Under Threat? by John Wonderlich, April 9, 2008)

This move is contrary NARA’s mission to enable people to personally inspect the record of what the government has done. It is not clear what prompted this decision. A comment suggests limited funding by Congress and the Administration is the cause. The NY Times addressed the issue this week in an article by John Schwartz In Storing 1's and 0's, the Question Is $. It is not a technological issue; it is an issue of funding and policy and control.