New software projects are usually judged on features, but they are often won or lost on data. If customer balances are wrong on day one, product codes do not match, or five years of order history disappears, trust in the new system collapses quickly. Legacy data migration is the disciplined process of moving information from old systems into a new platform accurately, securely and with minimal disruption. This guide sets out a step-by-step approach that experienced teams use, and the mistakes to avoid.
Why legacy data migration is harder than it looks
Old systems accumulate quirks over years:
- Fields used for purposes they were never designed for ("notes" holding phone numbers).
- Duplicates, for example the same customer entered three times with different spellings.
- Codes whose meaning lives only in someone's memory.
- Data spread across a database, spreadsheets, email folders and paper.
- Inconsistent formats for dates, currencies, units and Persian, Arabic or English text.
The technical transfer is rarely the hardest part. Understanding, mapping and cleaning the data is.
Step 1: Inventory every source
List each place business data lives today. For each source, record:
- What it contains and roughly how many records.
- Who owns it and who uses it.
- The format and how it can be exported.
- Whether it is still being updated.
- Any legal retention requirements.
Include the unofficial sources, such as the spreadsheet the sales manager maintains. They often hold data the official system lacks.
Step 2: Decide what to migrate
Not everything needs to move. Sort data into categories:
| Category | Typical approach | Examples |
|---|---|---|
| Master data | Migrate, cleaned | Customers, suppliers, products, staff |
| Open transactions | Migrate | Open orders, unpaid invoices, active contracts |
| Balances | Migrate as opening balances | Customer and supplier balances, stock on hand |
| Recent history | Migrate if used for operations or reporting | Order history for a defined recent period |
| Old history | Archive, read-only | Older transactions kept for reference or legal retention |
| Obsolete data | Do not migrate | Discontinued test records, abandoned drafts |
Agree this with business owners and your accountant, not just the technical team.
Step 3: Map fields from old to new
Create a mapping document for each entity, listing every source field, its target field, the transformation rule and the person responsible for signing it off. Typical transformations include splitting full names, normalizing phone numbers, converting units, mapping old status codes to new ones, and converting date formats.
Where the new system has required fields the old one lacks, decide on defaults or a manual completion process before migration starts.
Step 4: Cleanse the data
Cleansing is best done before migration, in the source or in a staging area:
- Remove duplicates using matching rules on names, emails, tax numbers or phone numbers, with human review for close matches.
- Standardize formats: addresses, phone numbers, units, product naming.
- Fill critical gaps, such as missing product categories or customer emails needed for notifications.
- Validate references, ensuring every order points to an existing customer and product.
- Flag and fix outliers, such as negative stock or impossible dates.
Assign data owners in each department. They know which records matter and which can be retired.
Key takeaway: Most migration problems are data problems, not code problems. Time spent on inventory, mapping and cleansing pays back many times over at cut-over.
Step 5: Build repeatable migration scripts
Migrations should be automated and repeatable, never a one-off manual copy. Good migration tooling:
- Extracts from sources into a staging database.
- Applies transformation rules from the mapping document.
- Loads into the new system using its own validation rules where possible.
- Logs every rejected record with a reason.
- Can be re-run from scratch at any time.
Repeatability is what makes rehearsals possible.
Step 6: Rehearse, then rehearse again
Run the full migration into a test copy of the new system using a recent export. Then validate:
- Record counts per entity match expectations.
- Financial totals reconcile: receivables, payables and stock value agree with the old system at the same date.
- Spot checks: business users verify a sample of customers, products and orders screen by screen.
- Process tests: create an order, invoice it and receive payment using migrated data.
- Timing: how long the migration takes, which determines the cut-over window.
Fix issues, update the mapping, and rehearse again until results are clean.
Step 7: Plan the cut-over
A cut-over plan covers, minute by minute where needed:
- Freeze date and time for the old system.
- Final export and migration run.
- Validation checks and sign-off by named business owners.
- Go or no-go decision, with a clear rollback plan.
- Opening the new system to users.
- Read-only access to the old system for reference.
Choose a quiet period, such as a weekend or a low season, and make sure key people are available. For website platforms, coordinate with redirects and SEO; our guide to SEO site migration without traffic loss covers that side.
Step 8: Support and validate after go-live
The first weeks matter. Keep a short daily check-in, track reported data issues in one place, and fix root causes rather than patching individual records where possible. After the first month-end, reconcile key figures again.
Security and privacy during migration
Migration files often contain the most sensitive data a company holds, all in one place. Protect them:
- Restrict access to named people.
- Encrypt files in transit and at rest.
- Use masked or synthetic data in development where practical.
- Delete temporary exports and staging copies when the project closes.
- Log who accessed what.
Privacy laws such as PIPEDA, BC PIPA and the UAE PDPL continue to apply during migration. For health or financial data, get professional advice on your obligations.
Special cases worth planning for
Some data types need extra attention during a legacy data migration:
- Files and attachments. Scanned contracts, product images and documents are often stored on file shares or embedded in the old database. Plan storage, naming, permissions and link updates in the new system.
- Passwords. You usually cannot migrate plaintext passwords, and you should not want to. Either migrate the existing hashes if the new system can verify them, or ask users to set a new password through a secure link at first login.
- Multilingual text and encodings. Older databases may use legacy character encodings. Persian, Arabic and accented characters can turn into unreadable symbols if encoding is not handled explicitly. Test with real samples early.
- Calculated values. Decide whether to migrate stored totals or recalculate them in the new system, and reconcile the difference.
- Identifiers referenced externally. Invoice numbers, order numbers and customer IDs may appear on printed documents and in emails. Preserving them, or keeping a lookup from old to new, avoids confusion.
- Website URLs. Product and article pages need redirects from old addresses to new ones.
Roles in a migration project
Clear ownership prevents decisions from stalling:
| Role | Responsibility |
|---|---|
| Executive sponsor | Approves scope, resolves conflicts, makes the go/no-go call |
| Data owners | Define cleansing rules and sign off mappings for their area |
| Migration lead | Plans rehearsals, tracks issues and runs the cut-over |
| Developers | Build and maintain the extraction and transformation scripts |
| Business testers | Validate migrated data and processes after each rehearsal |
Common mistakes
- Starting migration work late, after the new system is built.
- Leaving mapping decisions to developers without business sign-off.
- Migrating everything "just in case".
- Skipping rehearsals because the first test "mostly worked".
- No rollback plan.
Next steps
Start your inventory of data sources now, even before choosing a new system; it will sharpen your requirements. DigiVort plans and executes platform and data migrations through our website migration service, including moves from WordPress and older custom systems to Laravel. Tell us about your current systems through our project wizard.