My cold email subject line A/B testing process for finding winners in under 30 days
split_test_marcus · 2026-06-08 · 1,110 views
I've run A/B tests on cold email subject lines for three years. Here is the process that consistently finds winners within 30 days without burning inboxes on large test batches.
The setup. I test two subject lines per campaign, split evenly across prospects. Minimum 200 emails per variant before I call a winner. Anything under 200 is statistically noise. I measure by reply rate only. Reply rate is the only number that matters.
What I test.
Round 1: Question vs statement. Does a subject line framed as a question get more replies than a statement? Varies by ICP. Finance personas respond better to statements. Growth personas respond better to questions. Test before assuming.
Round 2: Specific vs vague. Something like 37% fewer cold email bounces vs improve your deliverability. Specific almost always wins. But I still test it because sometimes vague outperforms and I need the data to understand why.
Round 3: Short (under 5 words) vs medium (6 to 10 words). Short wins on mobile more often. Medium wins when the ICP reads on desktop. Know your buyer.
The winner selection process. Once a variant hits 200+ sends and shows a 20% relative difference in reply rate, I call it. I don't wait for statistical perfection. Cold email moves fast. A winner that holds for 4 weeks gets rolled out to full volume.
The failure mode to avoid. Testing too many variables at once. Subject line, first line, and email body all different between variants tells you nothing. Change one thing at a time. Know what you changed.
Current best performers: subject lines referencing the prospect's company name or a specific outcome from their industry. Generic subject lines die every time regardless of how well the body is written. Send from pre-warmed Google Workspace inboxes through Instantly or Smartlead so your split tests measure copy quality, not deliverability variance.