When it comes to surveys, having good quality data encompasses many things like having a high response rate (see blog post here). But it also encompasses having representative sampling. The data should be like a chip off the old block; it should be just like your users as a whole in composition but on a smaller scale.
Some people believe that “bigger is better” when it comes to sample size – the more survey respondents you have, the more trustworthy your results.
True, a bigger sample gives you more precise estimates, which is necessary for your results to be trustworthy. It also gives you more statistical power to detect differences between estimates and a benchmark, or differences between control vs. treatment.
But a bigger sample is only necessary and not sufficient for results to be trustworthy. You also need to correct for nonresponse error, or the bias in survey results due to non-respondents having different characteristics from survey respondents.
