Most data migration tool evaluations focus on the wrong things: connector counts, UI polish, vendor reputation. The questions that actually predict whether a tool will survive contact with your data are more specific than that. These six cut through the marketing and get to what matters.
| Question | Red flag answer | What you want to hear |
|---|---|---|
| Does it handle my sources? | “Yes” with a connector list but no proof | A working demo against your actual source |
| What happens on schema change mid-run? | “The job fails and you restart it” | Drift detected, you’re alerted, stream continues or pauses cleanly |
| Can I see lag in real numbers? | A green status indicator | Actual seconds/rows of lag, updated in near real-time |
| How does it recover from a half-failed load? | “Restart the job from the beginning” | Checkpointed resume, idempotent reloads, no duplicate rows |
| Can I cap usage? | No, or yes but only at the billing level | Pipeline-level limits you control before costs escalate |
| Does it scale 1 to whole team? | “Just spin up another instance” | Roles, audit, multi-user controls on the same platform |
Data migration tool evaluations tend to go wrong in the same way: you compare feature lists, watch a demo on clean sample data, and pick the tool with the best UI. Then it meets your actual data — your legacy sources, your mid-run schema changes, your half-failed loads at 2am — and the evaluation didn’t test any of that.
These six questions are the ones worth asking before you commit. They’re not comprehensive — there are plenty of other legitimate evaluation criteria — but they’re the ones that most reliably separate tools that work from tools that work in demos.
Does It Handle Your Sources, Specifically?
A connector checkbox is not the same as working support. Most tools will tell you they support Postgres, MySQL, Snowflake, and Redshift. That’s table stakes. The question that matters is whether the tool handles your source — its version, its configuration, its quirks — with the depth required for a production migration.
This distinction matters most with less common or older sources. If you’re migrating from IBM Netezza, for example, the relevant questions aren’t just “is Netezza on the connector list?” but: does the tool know that Netezza has no COPY command? Does it handle external table extraction? Does it manage the CSV quoting issues that corrupt downstream loads? A checkbox says nothing about any of this.
The same applies to on-premise SQL Server with non-standard collations, legacy Oracle with unusual CDC configurations, or any database that’s been heavily customised over years of use. Depth of support is invisible on a feature matrix.
What to do instead: ask for a proof-of-concept against your actual source, not sample data. Any serious vendor will agree to this. If they won’t, that’s your answer. Pay attention to how they handle the first edge case you throw at them — that’s a preview of what production support will look like.
What Happens When a Schema Changes Mid-Run?
Schema drift is not an edge case — it’s a certainty on any migration of meaningful length. Columns get renamed, types get widened, tables get added or dropped. The question is not whether this will happen to your pipeline; it’s what the tool does when it does.
The failure modes range from catastrophic to merely annoying. At the bad end: the tool silently continues processing rows with the old schema, writing malformed data into your target without raising an error. You discover it days later when a report looks wrong. At the annoying end: the job fails with a schema mismatch error, everything stops, and you spend an hour diagnosing and restarting.
What good looks like: the tool detects the schema change, alerts you, and either adapts automatically (for safe changes like new nullable columns) or pauses cleanly and tells you exactly what changed and what action is required. The stream doesn’t corrupt data and doesn’t disappear silently.
Ask the vendor: “Walk me through what happens, step by step, if I rename a column in the source while a CDC job is running.” The specificity of the answer will tell you whether they’ve actually solved this or are hand-waving.
SDF has built-in schema-drift detection. When drift is detected, the pipeline pauses and surfaces the change — it doesn’t silently corrupt data or fail without explanation.
Can You See Replication Lag in Real Numbers?
A green status indicator tells you nothing. “Running” and “healthy” are the two most useless pieces of information a pipeline dashboard can show you. They tell you the process is alive — not whether it’s keeping up.
Replication lag is the gap between when a change happens in your source and when it’s reflected in your target. For CDC pipelines feeding live dashboards or compliance systems, lag is the metric that matters. A pipeline can be “running” with 45 minutes of lag and your dashboard will look fine until someone notices the numbers are wrong.
The number you need is simple: how many seconds (or rows) behind is the replica right now? Updated continuously, visible without digging into logs. You also want to know the trend — is lag stable, growing, or recovering after a spike?
Secondary metrics worth having: rows moved in the last run, last successful sync timestamp, and whether any pipeline is approaching its configured limits. These three numbers, available on demand, are the difference between managing your pipelines and being managed by them.
If the vendor’s demo dashboard shows you a green dot and a “last synced” timestamp, ask where the lag number is. If they have to open a log file to find it, the observability isn’t built into the product — it’s bolted on.
How Does It Recover From a Half-Failed Load?
Every migration will have a failed job at some point. The network drops, a source goes briefly unavailable, a target warehouse hits a quota. What separates tools is what happens next.
The naive approach — and the one many tools take — is to restart the job from the beginning. On a small table, that’s a minor inconvenience. On a 500-million-row table that was 80% through when it failed, restarting from scratch means hours of wasted work and, worse, the risk of duplicate rows in your target if the load isn’t idempotent by design.
What you’re looking for is checkpointing: the tool records progress periodically, and on failure it resumes from the last checkpoint rather than the beginning. Paired with this: idempotent loads, meaning re-running a load (or a section of it) produces the same result rather than creating duplicates. These two properties together mean a failed job is a recoverable inconvenience, not a data integrity incident.
Questions to ask: “If a load job fails at 80%, what exactly happens when I restart it? How does it know where to resume? What guarantees do I have against duplicate rows?” If the answers are vague, the recovery story probably is too.
Can You Cap Usage Before Costs Escalate?
Uncapped pipelines are a cost control problem waiting to happen. Cloud data warehouse pricing is consumption-based, and a runaway pipeline — one that enters a retry loop, processes the same data repeatedly, or grows faster than expected — can generate significant cost before anyone notices.
The governance question is whether you can set limits at the pipeline level: maximum rows per run, maximum runs per day, maximum data volume. Not at the billing level (a credit card cap that cuts off your entire account), but at the individual pipeline level, where you can contain a problem without taking down everything else.
This matters more as you scale. One unchecked pipeline run at 3am shouldn’t blow your monthly budget. And when you’re onboarding new team members or running experimental pipelines, you want guardrails that don’t require manual oversight of every run.
The honest version of this question to ask a vendor: “If a pipeline enters a retry loop at 3am and processes ten times the expected data volume, what happens? How would I know? What would stop it?” If the answer is “you’d see it in your cloud bill at the end of the month,” that’s not governance — that’s hope.
SDF includes configurable pipeline limits as a first-class feature — you set caps before you run, not after you’ve been surprised.
Does It Scale From One Person to a Whole Team?
Most data migration tools are built for a single operator. That’s fine when you’re starting out — one engineer, a handful of pipelines, no access control required. The problem comes when the team grows and the tool doesn’t.
Scaling a single-operator tool to a team typically means one of two things: everyone shares the same credentials (a governance and audit nightmare), or you buy a completely different enterprise product and rebuild everything on it. Neither is acceptable for a system that’s supposed to be your data infrastructure.
What you want is a platform where adding team members is a configuration change, not a migration. That means: role-based access control (who can create pipelines vs. who can only monitor them), audit logs (who did what, when), and multi-user pipeline ownership. These features should be in the product roadmap from the start, not afterthoughts bolted on in an “enterprise” tier that requires a separate contract.
It’s also worth asking about pipeline definitions: are they portable and reusable, or locked to a single user’s workspace? A team that can share and version pipeline definitions is a team that can actually operate at scale.
SDF’s pipeline definitions are reusable across users and environments. The platform runs the same engine from solo operator to full team — you grow into it rather than out of it.
The Underlying Test: Does It Survive Contact With Real Data?
All six questions have the same underlying purpose: they test whether a tool was built by people who’ve run migrations in production, or by people who’ve built software that works on clean data in demos. The difference is usually obvious once you know what to look for.
Smart Associates has been running data migrations — including Netezza migrations, CDC pipelines, and multi-source warehouse projects — for over 20 years. SDF (Smart Data Frameworks) was built from that experience. If you’re in the early stages of a migration tool evaluation, talk to the team — we can tell you quickly whether SDF is the right fit, and if not, what questions to ask whoever is.
Smart Associates is a B2B data engineering consultancy specialising in data migration, CDC pipelines, and warehouse projects. SDF is our data migration and pipeline platform.
