Tasmania Tackles Thousands of Duplicate Files Clogging Public Image Archives

Tasmania's state government is moving to clean up a sprawling problem buried inside its own digital infrastructure: tens of thousands of duplicate images spread across agency servers, cultural institutions, and tourism databases, accumulated over more than two decades of uncoordinated digitisation work.

The problem matters now because the Liberal government is preparing to consolidate agency IT systems ahead of planned budget cuts to the public sector digital portfolio in the 2026-27 financial year. When auditors began cross-referencing holdings across agencies earlier this year, the scale of the duplication became impossible to ignore. Cultural and heritage organisations are particularly exposed, having independently scanned the same historical photographs multiple times through different grant-funded projects with no central registry to flag conflicts.

How the Duplication Built Up

The roots go back to the early 2000s, when the Tasmanian Archive and Heritage Office, based in Davey Street, Hobart, launched its first major digitisation push under Commonwealth funding tied to the centenary of federation. At roughly the same time, Libraries Tasmania, then still operating as part of a separate structure, began its own scanning program for colonial-era photographic collections held at the State Library on Murray Street. Neither program used a shared metadata standard, so the same glass-plate negatives of Launceston's Paterson Street in the 1880s, for instance, ended up as separate digital files in at least two institutional repositories.

The issue compounded through the 2010s. Tourism Tasmania ran successive campaigns that drew on historical imagery from multiple suppliers, often purchasing or commissioning scans of photographs already held publicly. The Huon Valley Council digitised local heritage photographs through a regional grant in 2014. The Port Arthur Historic Site Management Authority developed its own image library for interpretive displays and online content. Each project was legitimate in isolation. Together, they produced a fragmented ecosystem with no single point of truth.

A 2019 review commissioned by the then-Department of Premier and Cabinet estimated that Libraries Tasmania alone held more than 180,000 digital image files, with internal assessments suggesting a meaningful proportion shared visual content with holdings elsewhere in the system. The review recommended a whole-of-government image registry but the recommendation stalled, overtaken by machinery-of-government changes and the operational demands of the COVID-19 period from 2020 onward.

What the Consolidation Project Actually Involves

The current remediation effort, managed through Service Tasmania's digital infrastructure branch, involves running perceptual hashing software across participating agency servers to identify visually identical or near-identical files regardless of filename or metadata. Where duplicates are confirmed, a triage process determines which version carries the most complete provenance record, creator, date, copyright status, and that version is designated the canonical file. Others are flagged for archival or deletion.

The Tasmanian Museum and Art Gallery on Dunn Place is one of the first institutions to complete a pilot round of the deduplication process across its digitised photographic collections. The exercise is also informing a broader push to align metadata standards across cultural institutions with the Atlas of Living Australia's biodiversity image standards, a cross-sector alignment that has been discussed at the national level for several years but rarely implemented at the state level in Tasmania.

Budget figures for the full consolidation project have not been made public. However, the government's broader digital infrastructure program, announced in the 2025-26 state budget, set aside funds under a whole-of-government technology modernisation envelope. Agencies participating in the image deduplication work are drawing on that allocation rather than seeking separate appropriations.

For institutions and local councils that manage their own image holdings outside the state system, the practical advice from Libraries Tasmania's digital preservation team is straightforward: before commissioning any new digitisation work, check the state's existing online catalogue at the Libraries Tasmania portal and contact the Tasmanian Archive and Heritage Office directly to confirm whether the material has already been digitised. The cost of a phone call is considerably less than the cost of scanning the same photograph for the third time.