Different types of bias 2
During data collection, expectations by experimenters, observers, or research participants could also create biases in the direction of these expectations.
Observer bias, experimenter bias, and the placebo effect might all create the inflate the beneficial or causal effects of a particular experimental manipulation or treatment. Such biases are often dealt with by blinding procedures (including the use of placebo treatments) during the data collection. Biases can also emerge during the quantitative or qualitative analysis of data.
Notably, outcome reporting bias refers to the selection of desirable results among a set of outcome measures, often based on whether these outcome measures show positive (often significant) results. Similarly, the selection of results based on the specification of the analysis (e.g., based on which set of analyses happens to show significant results) might create biased estimates that are not expected to re-emerge in novel samples of the same population (a problem referred to as overfitting in statistics). For instance, a researcher might try out a large set of alternative analyses and only report the result that showed the largest (and hence most desirable) effect of a particular treatment or the strongest association between particular variables in the data. In a qualitative study, the coder of transcripts could select only instances that conform to a particular hypothesis, thereby creating the false impression of consistency in the transcripts.