DEV Community

Discussion on: When Not To A/B Test

ben profile image
Ben Halpern

Nice post Elena, I'd add this one:

  • Feature success is too hard to actually measure.

I see a lot of time wasted on trying to bend over backwards to measure something when you really just need to trust your gut, or trust the expertise involved. You're only testing because it will deliver good return on investment in the first place. If your investment to try and measure the thing is too great, you might want to take a different approach.

Depending on your implementation, split testing can also have pretty terrible consequences in terms of performance and software complexity. In general I think we need to proceed with humility in this area.

ice_lenor profile image
Elena Author

Hi Ben, thank you, you raise a great point.
Actually, even two points: hard to measure and complexity. I should've thought about it myself, I've seen consequences of both!
Thank you again!