Some authors seem to consider alpha as a measure of the internal consistency of the test. Although the most obvious source for such standards would be the Standards for Educational and Psychological Testing (American Educational Research Association, American Psychological Association, & National Council of Measurement in Education, 1999), this text does not specify standards for reliability values. Authors may use a combination of the arguments above, may not specify a standard value, or may avoid discussing the problem altogether. Many psychometric books devote no more than a few paragraphs to reliability standards and use rather intuitive arguments. However, a standard cannot be derived simply on the basis of common practice, because such reasoning is circular. 29) stated that some standards are based on the observed reliabilities of tests that are regarded as acceptable by most. 70 are usually presented without comment and without further attempts to improve the scale.

The common-practice standard is often used implicitly, as indicated by Cortina (p. Similar remarks could be made with respect to standards for effect sizes. Experience with the literature gives one some general idea of what an acceptable alpha is, but there is usually little else to go on.

One reason for the misuse of alpha in applied psychology is that there seems to be no real metric for judging the adequacy of the statistic. So, the question is which allocation rule maximizes the power of a t test, given the budget. Thus, there are two sample sizes, and both increase the power, but if one sample size increases, then the other must decrease-otherwise, the budget will be exceeded. For fixed costs, one has to allocate one part of the budget to the recruitment of subjects and another part to test administration (e.g., paying the raters). Now, consider the goal of maximizing the power under the restriction that the costs are fixed. Increasing the number of subjects will also increase the power. In the present situation, increasing the number of measurements will increase the reliability of the test, and thus decrease the standard deviation and increase the power. In a traditional power analysis, one would consider the number of subjects, the difference of the means, and the standard deviation. Their effects on power are summarized in Fig. Two types of sample size are included in this design: the number of subjects and the number of measurements (e.g., test items, raters, or time points).

Test of reliability in research