Tasmania Spent Millions Eliminating Duplicate Digital Archive Files

Tasmania's state and local government agencies are sitting on digital image libraries containing tens of thousands of duplicate files, a problem that crept up over nearly two decades of piecemeal digitisation and has now forced a coordinated remediation effort across at least a dozen public bodies. The Libraries Tasmania repository at 91 Murray Street in Hobart, the central custodian of the state's photographic and documentary heritage, flagged the scale of the problem internally in late 2024, triggering a broader audit that is still under way as of this week.

The timing matters because the state budget, handed down in May 2026, allocated $3.2 million over three years toward digital asset management reform across the Tasmanian State Service. That figure, confirmed in the published budget papers, is now the benchmark against which agencies are measuring what it actually costs to identify, tag and retire redundant image files. For some departments the duplicate problem is administrative nuisance; for others, particularly those managing sensitive material linked to Aboriginal cultural heritage repatriation or development applications near places like Macquarie Point, it carries real legal and reputational weight.

How the Mess Was Made

The roots of the problem go back to the early 2000s, when individual councils and state departments began digitising paper photo archives with little coordination. Funding came in fragments: a $480,000 federal grant under the Regional Partnerships Program in 2003 enabled the Launceston City Council to scan its civic records collection, while Parks and Wildlife Service Tasmania used separate Natural Heritage Trust funding to digitise coastal survey photographs around the same period. Neither project used a shared file-naming convention or a common metadata schema. Files were often saved multiple times, once in working folders, once in backup systems, and again when staff emailed attachments that were later archived locally.

The shift to cloud storage after 2015 accelerated the problem rather than solving it. Agencies migrating legacy servers to platforms such as Microsoft SharePoint or AWS S3 buckets frequently uploaded entire folder trees without first deduplicating them, effectively tripling or quadrupling the footprint of already-bloated archives. The Tasmanian Planning Commission, which handles image evidence submitted alongside development applications for contested sites from the Huon Valley to Circular Head, had by one internal estimate more than 40 percent of its stored images flagged as probable duplicates during a 2023 pilot audit, though that figure has not been publicly released and The Daily Tasmania could not independently verify it.

Libraries Tasmania is not alone in trying to fix this. The Tasmanian Museum and Art Gallery on Dunn Place, which digitised roughly 180,000 collection items between 2010 and 2022, has been running its own deduplication project since mid-2025 using open-source tooling built on perceptual hashing algorithms. The work is slow. A single batch processing run across 30,000 image files can take several days of staff time to review, because automated tools flag near-duplicates, slightly different crops of the same photograph, or the same image saved at different resolutions, that require human judgement to resolve.

What Comes Next for the Cleanup

The $3.2 million budget allocation is meant to fund a shared Digital Asset Management System that would give agencies a single repository with deduplication built into the ingest workflow. Procurement is expected to begin in the third quarter of 2026, with a tender published through TenderLink, the state's standard procurement portal. Agencies will then face a transition period of up to 18 months to migrate existing archives into the new system.

For Tasmanians with a practical stake in public image records, heritage property owners lodging development applications, researchers using the Tasmanian Archive and Heritage Office collections, or Aboriginal community groups working with the Tasmanian Aboriginal Centre on cultural material, the immediate advice is to document what copies of government-held images you have submitted or received, and to confirm with the relevant agency which version is the authoritative record. Once migration begins, version histories in legacy systems may be harder to retrieve. The window to raise questions with agencies directly is now, before the procurement tender locks in the architecture of what comes next.