Sample Size Estimation

Continuing about the testing section (A/B, Multivariate) “Sample Size” is very crucial in the analysis. From a statistics point of view, the aim is to demonstrate with 95% certainty that the true value of a parameter is within a distance an error range of the estimate, typically the value of error range is referred to as the 95% confidence interval. OR in simple English, the aim is to demonstrate with 95% certainty that the result (example: Conversion rate, Click rate etc) would hold true 95% of the time.
It would be good to explain with a simple example. If there is a test for a click-through rate on a banner, there should have a big enough sample size to estimate if x% CTR would be true 95% of the time with 95% certainty. One more example is if there is an A/B test for a homepage test measuring clicks on a call to action button it is important to have enough samples or views of the page to be able to estimate their conversion or click rate with 95% certainty.
As a rule of thumb, the lower the response rate, the higher the sample size needed for accurate estimation.
For someone who is interested in know the math behind the test, please follow the links below:
http://en.wikipedia.org/wiki/Sample_size
http://www.surveysystem.com/sscalc.htm
http://www.isixsigma.com/library/content/c000709a.asp