The question
Two method transfers are closed on the same argument: a t-test found no significant difference between the sending and receiving sites. Does that show the receiving site is producing equivalent results?
The analysis
| Transfer | Mean difference | 90% CI | t-test p | TOST at ±5% |
|---|---|---|---|---|
| Transfer 1: 6 lots x 3 replicates | -1.63 | -3.55 to 0.30 | 0.16 | Equivalent |
| Transfer 2: 2 lots x 3 replicates | -3.50 | -13.08 to 6.08 | 0.52 | Not shown equivalent |
Both t-tests fail to find a difference, and both would pass a "no significant difference" criterion. Only the first transfer is equivalent. The second has so few results and so much spread that its confidence interval runs well past the margin: the test did not find a difference because it could not have found one.
Why it matters. "No significant difference" rewards small, noisy studies. Two one-sided tests (TOST) flip the burden: the transfer passes only if the whole 90% interval sits inside a margin set before the data, so a weak study cannot pass by accident.
What to do
- Set the equivalence margin in the transfer protocol, from the method's validated precision and the specification, before any testing.
- Size the comparison so the interval can realistically fit inside the margin; the sample size tool gives a starting point.
- If a closed transfer relied on a t-test alone, re-evaluate it with TOST before relying on the receiving site's release results.
How it was done
Synthetic relative potency results for two transfers. Welch two-sample comparison; 90% confidence interval for the difference in means, which is equivalent to two one-sided tests at alpha 0.05. Download the data (CSV).