Strategy

Why I stopped testing 50 subject lines and started winning

icp_first · 2026-04-01T00:00:00Z

A/B testing subject lines constantly. Reply rates stayed the same. Realized: subject line matters 20 percent, copy matters 30 percent, ICP matters 50 perce

icp_first · 2026-04-01 · 2,890 views

For about 8 months I was A/B testing subject lines constantly. Every campaign, 5 variations, hundreds of sends per variation, statistical significance testing. I thought I was being a rigorous growth marketer. My reply rates stayed the same.

The realization. I mapped out the actual contribution of each factor to reply rate based on data across 50 campaigns. Subject line mattered about 20 percent. Email copy mattered about 30 percent. ICP targeting mattered about 50 percent. All my testing was happening on the smallest lever in the system.

What ICP actually means here. I had been defining ICP broadly: "B2B SaaS companies." That's not an ICP. That's a market. When I tightened to "B2B SaaS companies, 50 to 200 employees, Series A or B funding, targeting sales operations pain, decision maker is VP Sales or RevOps leader," my reply rate jumped from 2.1 percent to 3.8 percent with no changes to subject line or copy.

Why ICP matters so much. A mediocre email to the perfect prospect outperforms a perfect email to a wrong prospect every time. If the person receiving your cold email has the exact problem you solve, they'll read past an okay subject line to understand what you're offering. If they don't have the problem, no subject line in the world will make them care.

My new priority order. 1. ICP refinement: who exactly are you targeting, what's their specific pain, what role makes this decision. 2. Email copy: does your email speak to that specific pain in their specific context. 3. Subject line: last 20 percent of optimization, only worth tuning once ICP and copy are dialed in.

The 50 subject line tests I wasted time on. Probably cost me 3 months of progress. All that testing effort could have gone into ICP refinement or copy improvements that moved the needle 10x more. Lesson learned: optimize the biggest lever first. Subject line testing is useful polish but it's never the difference between a 2 percent and 4 percent reply rate.

Stop obsessing over subject lines until your ICP and copy are definitely working. That's the order. Fixing ICP first is the single most impactful change most teams can make.