We should be aware of the possible limitations of item statistics and various data collection designs before applying results of item analysis. The better studies use larger samples (>30) composed of individuals with the same characteristics as the targeted examinee/respondent group. Classical item statistics are all group-dependent. We should make sure that appropriate criterion groups were used in the studies. We should remind ourselves that conducting an item analysis will not automatically improve an instrument unless we use the results to delete, rewrite, or otherwise alter a flawed item. Follow-up work on items and the overall instrument should be continued until we obtain the desired item properties.

Page 6