Reliability in psychometrics: what it is and how it is estimated in tests

If you have studied psychology or other related careers, surely the concept of reliability is familiar to you. But... What exactly does it consist of? Reliability in psychometrics is a quality or property of measurement instruments (for example tests), which allows you to verify if they are accurate, consistent and stable in their measurements.

In this article we tell you what this property consists of, we name some examples to clarify the concept and we explain the different ways to calculate the reliability coefficient in psychometrics.

• Related article: "Psychometrics: what is it and what is it responsible for?"

What is reliability in psychometrics?

Reliability is a concept encompassed within psychometrics, the discipline in charge of measuring the psychological variables of the human being through different techniques, methods and tools. Thus, reliability in psychometrics, redundancy be valid, consists of a psychometric property, which implies the absence of measurement errors of a certain instrument (for example, a test).

It is also known as the degree of consistency and stability of the scores obtained in different measurements through the same instrument or test. Another synonym for reliability in psychometry is "precision.". Thus, we say that a test is reliable when it is accurate, it does not present errors and its measurements are stable and consistent throughout repeated measurements.

Beyond reliability in psychology, in what fields does this concept appear and be used? In different fields, such as social research and education.

Examples

To better illustrate what this psychometric concept consists of, consider the following example: we use a thermometer to measure the daily temperature in a classroom. We take the measurement at ten o'clock in the morning of each day, for a week.

We will say that the thermometer is reliable (it has a high reliability) if, by doing more or less the same temperature every day, the thermometer indicates this (that is, measurements are close to each other, there are no large jumps or large differences).

Instead, if the measurements are totally different from each other (the temperature being more or less the same every day), it will mean that said instrument does not have a good reliability (because its measurements are not stable or consistent over time).

Another example to understand the concept of reliability in psychometry: let's imagine that we weigh a basket with three apples every day, for several days, and we record the results. If these results vary a lot over successive measurements (that is, as we repeat them), this would indicate that the reliability of the scale is not good, since the measurements would be inconsistent and unstable (the antagonists of reliability).

Thus, a reliable instrument is one that shows consistent and stable results in repeated measurement processes of a certain variable.

The variability of the measures

How do we know if an instrument is reliable? For example, starting from the variability of your measurements. That is, if the scores we obtain (measuring the same thing repeatedly) with said instrument are highly variable between yes, we will consider that its values ​​are not accurate, and that therefore the instrument does not have a good reliability (it is not reliable).

Extrapolating this to psychological tests and the responses of a subject to one of them, we see how the fact that he answered the same test under the same conditions, repeatedly, would provide us with an indicator of the reliability of the test, based on the variability in the scores.

• You may be interested: "Types of psychological tests: their functions and characteristics"

The calculation: reliability coefficient

How do we calculate reliability in psychometrics? From the reliability coefficient, which can be calculated in two different ways: from procedures that involve two applications or just one. We are going to see the different ways to calculate it, within these two large blocks:

1. Two applications

In the first group we find the different ways (or procedures) that allow us to calculate the reliability coefficient from two applications of a test. Let's get to know them, as well as their disadvantages:

1.1. Parallel or equivalent forms

With this method, we obtain the measure of reliability, in this case also called “equivalence”. The method consists of simultaneously applying the two tests: X (the original test) and X '(the equivalent test that we have created). The disadvantages of this procedure are basically two: the fatigue of the examinee and the construction of two tests.

1.2. Test-retest

The second method, within the procedures for calculating the reliability coefficient from two applications, is the test-retest, which allows us to obtain the stability of the test. It basically consists of apply an X test, allow a period of time to pass, and re-apply the same X test to the same sample.

The disadvantages of this procedure are: the learning that the examined subject may have acquired in that lapse of time, the evolution of the person, which can alter the results, etc.

1.3. Test-retest with alternative forms

Finally, another way to calculate reliability in psychometrics is to start from the test-retest with alternative forms. It is a combination of the two previous proceduresTherefore, although it can be used for certain cases, it accumulates the disadvantages of both.

The procedure consists of administering the X test, allowing a period of time to pass, and administering the X 'test (that is, the equivalent test created from the original, X).

2. A single application

On the other hand, the procedures to calculate the reliability in psychometrics (reliability coefficient) from a single application of the test or measurement instrument, they are divided into two subgroups: the two halves and the covariance between items. Let's see it in more detail, so that it is better understood:

2.1. Two halves

In this case, the test is simply divided into two. Within this section, we find three types of procedures (ways to divide the test):

• Parallel forms: the Spearman-Brown formula is applied.
• Equivalent forms: the Rulon or Guttman-Flanagan formula applies.
• Congeneric forms: Raju's formula is applied.

2.2. Covariance between items

The covariance between items involves analyzing the relationship between all the test items. Within it, we also find three methods or formulas typical of psychometry:

Croanbach's alpha coefficient: its value ranges from 0 to 1. Kuder-Richardson (KR20): applied when the items are dichotomous (that is, when they only acquire two values). Guttman.

3. Other methods

Beyond the procedures that involve one or two applications of the test to calculate the reliability coefficient, We found other methods, such as: inter-rater reliability (which measures the consistency of the test), Hoyt's method, etc.

Bibliographic references:

• Kaplan, R.M., & Saccuzzo, D.P. (2010). Psychological Testing: Principles, Applications, and Issues. (8th edition). Belmont, CA: Wadsworth, Cengage Learning.
• Martínez, M.A., Hernández, M.J. and Hernández, M.V. (2014). Psychometry. Madrid: Alliance.
• Martínez Arias, R. (2006). Psychometry. Madrid: Anaya.
• Morales Vallejo, Pedro (2007). Statistics applied to the social sciences. The reliability of tests and scales. Madrid: Comillas Pontifical University. p. 8.
• Prieto, Gerardo; Delgado, Ana R. (2010). Reliability and validity. Roles of the psychologist (Spain: General Council of Official Associations of Psychologists) 31 (1): 67-74.