DAM migration: planning, metadata mapping and validation checklist

Why it matters

Moving information from one data structure and application to another can lose content, context or access, so that people can no longer find, open, work with, understand or trust it R1. The UK National Archives, writing about records systems, gives failure modes that apply directly to a DAM:

• Exports are incomplete. It is rare for a system's export to capture everything it holds, and the target's import may not accept everything either. Files can become separated from their metadata or from related objects R1.
• Metadata changes meaning or value. Fields that look alike, such as "date" or "author", may be used differently in the two systems. Dates are especially at risk: the creation date can be reset to the migration date, and partial dates can be blanked or padded with invented days R1.
• Relationships do not travel. Versions, renditions, attachments, shortcuts and links between items are managed internally and there is no standard way to export them R1.
• Access controls are lost. If permissions are not replicated, people can end up with access they should not have R1.

In a DAM these failures show up as assets nobody can find, campaign images with no rights information, broken images on websites and in emails, and integrations that fail silently because an ID changed. Practitioners consistently report that migration effort is underestimated and grows with the age of the source system R15 Unverified (2014 article; the observation is qualitative). No reliable, freely available industry statistics on migration duration or failure rates were found, so none are quoted here.

Migrations also expose earlier schema debt. Every overloaded field, free-text keyword and undocumented convention has to be interpreted during mapping. This is one reason the metadata schema & governance guide recommends documenting fields before you need to move them.

Core concepts

Recommended practices

1. Decide what to migrate, and whether to migrate at all

Migration is costly and risky, so confirm the business need first R1. For each collection, choose one route:

Typical candidates for disposal in a DAM are exact duplicates (found by checksum, see duplicate detection), expired assets with no archival value, abandoned uploads and orphaned renditions. Decide this with the business owners and record it; "we didn't migrate it" must be a decision, not an accident.

2. Turn user needs into acceptance criteria before choosing a method

The National Archives frames requirements around information, not features: what must remain findable, openable, workable, understandable and trustworthy R1. In DAM terms:

• Found: which fields, facets, collections and folder paths do people actually search and browse with?
• Opened: which formats and renditions must be viewable? Are there formats the target cannot accept or preview?
• Worked with: downloads, crops, templates, sharing links, integrations to CMS, PIM or creative tools.
• Understood: descriptions, captions, campaign and product context, version history.
• Trusted: rights and licences, approval status, provenance, audit trail.

Separate critical requirements from desirable ones, because systems rarely match feature for feature R1. If a supplier carries out the migration, agree acceptance criteria and documented exceptions with them in advance, and ask for the schema mapping and a proof of concept before the live run R1.

3. Build the inventory from data, not from memory

Export or query the source and measure it. As a minimum, record:

• Asset counts and total size by file type, plus the largest files (target upload limits matter).
• File formats, identified with a profiling tool rather than by extension. The National Archives' free DROID tool profiles formats and helps find duplicates R1.
• Every metadata field with its type, number of filled values, number of distinct values and examples of odd values.
• Vocabularies and taxonomies, versions, renditions, derivatives, collections, saved searches, comments and annotations.
• Users, groups, roles and permission rules, including rules driven by metadata.
• Rights, expiry and embargo data, model and property releases, and where these documents live.
• Integrations and published URLs: which websites, apps and feeds embed or call the source.
• Audit logs and usage statistics you may need to keep.

Stanford Libraries, migrating nearly 5 million repository objects, built reporting tools on a cached copy of the source to find missing MIME types, missing dimensions and unusual values before mapping R4. The same approach scales down: a spreadsheet of field fill rates and value counts will show most problems before they reach the target. Also assess the source's export capabilities (schema, volumes, speed) and the target's import capabilities R2.

4. Map metadata in a documented crosswalk

Write a crosswalk at field level and, for controlled fields, at value level. Getty's Introduction to Metadata lists the usual mismatches R3:

Other rules that save rework:

• Check meaning, not labels. Look at how the target uses and displays each field before mapping into it R1.
• Fix date formats explicitly. Decide what happens to partial dates, time zones and empty dates, and test that the import does not overwrite them R1. Normalising to YYYY-MM-DD before import was a step in Oregon State University's journal migration R5.
• Carry the past in read-only fields. Keep the source ID, the source system name, the original file name (see file naming conventions) and, where helpful, the source folder path. The folder path often carries context that is not in any field R1.
• Expect embedded and DAM-only metadata to differ. Files may carry XMP or IPTC values that are older or newer than the values in the source database. Decide which one wins for each field, and whether the target should re-read embedded metadata at ingest.
• Watch the exchange format. CSV has no single formal specification, so quoting, delimiters and character encoding vary between tools R12. State the encoding (UTF-8) and the delimiter for multi-value fields in the crosswalk, and test with values that contain commas, quotes, line breaks and accented characters.

5. Clean before you move, not during the move

Data quality problems should be fixed before migration to reduce cost and risk R1. Clean in the source, or in an intermediate staging copy, so that you can re-run the extract. Normalise vocabularies, merge synonyms, fix obvious typos and fill required fields that the target will enforce. Two limits apply:

• Do not redesign the whole information model during the migration. Agree the target schema, freeze it for the duration and log any change requests for later.
• Do not let cleanup hide information. Keep raw legacy values where a transformation loses detail, as Oregon State did when it moved rich issue-level metadata into a description field because the target had no equivalent R5.

6. Preserve identity: IDs, links and relationships

The target will assign its own IDs. Plan for what points at the old ones.

• Identifier map. Export a permanent old-ID → new-ID table. Integrations, analytics, archived campaign records and support tickets will need it.
• Published URLs. Public links, embeds and CDN URLs are used in places you do not control. "Cool URIs don't change" argues that a URI should outlive the system behind it and should not contain things that change, such as file extensions, status or organisational structure R9 Unverified (1998 document; the principle is still the reference). Where URLs must change, answer old URLs with permanent redirects: 301 or 308 (the latter keeps the request method) R10. Use 410 Gone for assets you deliberately retired R10. Boston College kept its legacy repository identifiers in new persistent links resolved through a handle server, so that old references kept working R6.
• Relationships. List the relationship types the source holds (versions, renditions, derivatives, parent-child, related assets, collections) and test each one after import. Check that version history stays attached to the current version R1. For renditions, decide whether to migrate them or regenerate them in the target; regeneration is often simpler R1, except for hand-made crops or approved variants that cannot be recreated.

7. Carry rights, restrictions and permissions explicitly

Rights data is the part of a migration where a mapping mistake creates legal exposure. Treat it as its own workstream:

• Map every usage rights field, expiry date, embargo, territory and channel restriction, and the link to release forms and licences. See Digital rights & usage management for the underlying model.
• Map permissions and roles, including any that depend on metadata. If sensitivity or restriction markers cannot be mapped, restricted assets may become visible to everyone R1.
• Test with deliberately restricted, expired and embargoed sample assets, logged in as users of each role.
• Decide what happens to audit trails. If the target cannot import them, keep them in an accessible store outside the DAM for as long as you may need them as evidence R1.

8. Move files with integrity checks

• Compute checksums at the source for every original before transfer, or record those that the source supplies, and verify them at the target. The Digital Preservation Coalition notes that MD5 is enough to detect accidental damage, while SHA-256 gives stronger protection against deliberate alteration R7.
• Package large transfers with manifests. BagIt (RFC 8493) is a simple folder layout in which a manifest lists every payload file with its checksum. A bag is "valid" only when it is complete and every checksum verifies R8. Even if you do not adopt BagIt, copy the idea: a manifest per batch, verified on arrival.
• Do not rely only on platform hashes. DAM APIs differ: some expose an MD5 or CRC32, some none, and some change the value after metadata edits (see How DAM platforms support this). If the target or your tooling writes metadata into files at ingest, the file bytes, and therefore the checksum, change. In that case, keep the pre-ingest checksum as a field and document why the values differ.
• Record what happened. The PREMIS preservation metadata standard, maintained at the Library of Congress, defines event types such as migration, fixity check, filename change and metadata modification R13. A DAM does not need full PREMIS, but a migration log per asset (extracted, transferred, verified, imported, with dates) is what lets you answer "what happened to this file?" years later.

9. Engineer the load for the target's API and bulk paths

Most migrations into SaaS DAMs end up as API calls. Plan for:

• Rate limits. HTTP 429 Too Many Requests signals rate limiting and may carry a Retry-After header saying how long to wait R11. Throttle the loader, honour Retry-After, back off exponentially and log every rejected call. Check whether limits are per key, per user, per IP address or per tenant. A tenant-wide limit means the migration competes with live integrations, so schedule bulk loads accordingly.
• The vendor's bulk path. Some platforms provide a bulk import, fetch or migration route that is preferred over the everyday upload API for large volumes. Use it where it exists and still validate its output.
• Pagination on export. Extract with stable sort orders or cursors, and check that the exported count matches the source count, so that pages are not skipped or duplicated.
• Idempotency. Key each import on the source ID (stored in a target field), so that a re-run updates or skips rather than creating duplicates. Fotoware describes migration as importing the binaries with new primary keys and then patching metadata onto them, which only works if the link between old and new keys is kept R17.
• Large files. Use the target's chunked or resumable upload where available, and record failures for retry rather than stopping the whole batch.

10. Pilot, then rehearse

• Develop and test the process outside the production system R2.
• Pilot on representative data and representative volumes R2: every file type, the oldest and newest assets, the largest files, assets with many versions, restricted assets and assets with unusual characters in names or values. Oregon State tested on three issues chosen from different periods before migrating the whole run R5.
• Test the mapping against real records, not only against the crosswalk. Stanford found that unit tests alone were insufficient because the data was so varied, and ran round-trip tests against cached real objects, in the end against all of them R4.
• Repeat the pilot until the reconciliation report (practice 12) is clean, then run a full rehearsal to measure the real duration of each step.

11. Plan the cut-over and the rollback

Choose the cut-over model deliberately:

Stanford ran a rolling migration: all creates and updates went to both the old and the new datastore, reads preferred the new one and fell back to the old, and a parallelised script moved the rest, over 4 million objects in under a week R4. Whatever the model:

• Announce a content freeze or a delta process for changes made after the main extract.
• Have a tested rollback plan in case the migration fails R1, and agree in advance who decides to roll back, and by when.
• Switch integrations and redirects in a planned order, and keep a list of every system that was repointed.

12. Validate, accept, then decommission

Validation should compare the source and the target, not just look at the target:

• Counts: assets, versions, renditions and collections, by type and collection.
• Fixity: 100% of originals verified against source checksums.
• Metadata: fill rates per field before and after; sampled field-by-field comparisons, with extra samples for dates, rights and multi-value fields.
• Access: role-based tests on restricted and embargoed assets.
• Findability: the most frequent searches and browse paths return what users expect. Fotoware suggests checking the top use cases with employees before go-live R17.
• Links: a sample of published URLs and embeds resolves or redirects correctly.

Involve users in acceptance testing R1. When results fall short, the National Archives' checklist offers three responses: refine and re-run the migration, fix the data in the target, or accept low-impact issues and document them R2. Only then dispose of the source data and decommission the system R1 R2. Keep the source read-only for a defined, limited period if some information could not be migrated, update information registers and ownership, and archive the crosswalk, identifier map, manifests and migration log alongside the target R1.

Common pitfalls

• Treating migration as a file copy. Files arrive; metadata, relationships and permissions do not. The National Archives warns against IT-led migrations that do not engage the business, and against business-led ones that ignore technical constraints R1.
• Mapping by label. Two fields called "Date" or "Owner" can mean different things R1.
• Losing the old IDs. Without an identifier map, every downstream system that stored a source ID breaks, and nobody can trace an asset back.
• Breaking published links. Images in emails, PDFs and partner sites point at the old URLs. Plan redirects or keep URLs stable R9 R10.
• Dates reset to the migration date. Check created, captured and modified dates explicitly in the test plan R1.
• Restricted assets becoming public. Permissions and restriction metadata were not mapped or tested R1.
• Trusting the vendor hash. Platform checksums may be missing, use different algorithms, or change after metadata edits. Compute your own.
• Ignoring rate limits. A loader that ignores 429 responses can block itself or starve production integrations R11.
• Redesigning everything mid-flight. A changing target schema invalidates the crosswalk and the tests. Freeze it.
• Decommissioning too early. Switching off the source before acceptance testing removes the rollback option R1 R2.

How DAM platforms support this

The wiki's API pages (rate limits, uploads, data model, limits & quirks) and the Ingestion & storage sections of the Features pages hold the facts that shape a migration plan. The examples below show the kind of difference to look for; they are not a ranking, and they cover only a few of the platforms documented on the wiki. Check current behaviour through each solution's sources page.

Several vendors also describe assisted or paid migration offers on their features and integrations pages. The wiki records these as facts and does not evaluate them. For platform-level facts, see the comparison table. The guide Choosing a DAM: requirements & RFP covers import and export criteria.