The 4 differences between reliability and validity (in science)

Since in colloquial language they have very similar meanings, it is easy to confuse the terms reliability and validity when we talk about science and, specifically, psychometrics.

With this text we intend to elucidate the main differences between reliability and validity. We hope you find it useful to clarify this common question.

• Related article: "Psychometrics: studying the human mind through data"

What is reliability?

In psychometrics, the concept "reliability" refers to the precision of an instrument; Specifically, the reliability coefficients inform us of the consistency and stability of the measurements taken with said tool.

The higher the reliability of an instrument, the fewer random and unpredictable errors that will appear when using it to measure certain attributes. Reliability excludes predictable errors, that is, those that are subject to experimental control.

According to classical test theory, reliability is the proportion of the variance that is explained by the true scores. Thus, the direct score in a test would be made up of the sum of the random error and the true score.

The two main components of reliability are temporal stability and internal consistency. The first concept indicates that the scores change little when measured on different occasions, while the Internal consistency refers to the degree to which the items that make up the test measure the same construct. psychological.

Therefore, a high reliability coefficient indicates that the scores in a test fluctuate little internally and as a function of time and, in short, that the instrument is free of measurement errors.

• You may be interested in: "Types of psychological tests: their functions and characteristics"

definition of validity

When we talk about validity, we refer to whether the test correctly measures the construct it intends to measure. This concept is defined as the relationship between the score obtained on a test and another related measure; the degree of linear correlation between both elements determines the coefficient of validity.

Likewise, in scientific research, high validity indicates the degree to which the results obtained with a certain instrument or in a study can be generalized.

There are different types of validity, which depend on the way in which it is calculated; This means that it is a term with very different meanings. Basically we can distinguish between content validity, criterion (or empirical) validity and construct validity.

Content validity defines to what extent the items of a psychometric test are a representative sample of the elements that make up the construct to be evaluated. The instrument must include all the fundamental aspects of the construct; For example, if we want to carry out an adequate test to measure depression, we must necessarily include items that evaluate mood and decreased pleasure.

Criterion validity measures the ability of the instrument to predict aspects related to the trait or area of ​​interest. Lastly, construct validity aims to determine whether the test measures what it purports to measure, for example from the convergence with the scores obtained in similar tests.

Differences between reliability and validity

Although these two psychometric properties are closely related, the truth is that they refer to clearly differentiated aspects. Let's see what these differences consist of.

1. The object of analysis

Reliability is a characteristic of the instrument, in the sense that it measures the properties of the items that compose it. On the other hand, validity does not refer exactly to the instrument but rather generalizations made from the results obtained through it.

2. The information they provide

Although it is a somewhat simplistic way of putting it, in general terms it is usually affirmed that validity indicates that a tool psychometric actually measures the construct it purports to measure, while reliability refers to whether it measures it correctly, without mistakes.

3. The way they are calculated

Three main procedures are used to measure reliability: the method of the two halves, the method of parallel forms and the test-retest. The most widely used is the procedure of the two halves, in which the items are divided into two groups once the test has been answered; then the correlation between the two halves is analyzed.

The method of parallel or alternative forms consists of creating two equivalent tests to measure to what extent the items correlate with each other. The test-retest is simply based on passing the test twice, in conditions that are as similar as possible. Both procedures can be combined, giving rise to the test-retest with parallel forms, which consists of leaving a time interval between the first form of the test and the second.

On the other hand, the validity is calculated in different ways depending on the type, but in general all the methods are based on the comparison between the score in the objective test and other data from the same subjects in relation to similar traits; the objective is that the test can act as a predictor of the trait.

Among the methods used to assess validity we find factor analysis and the multi-method-multi-trait matrix technique. Likewise, content validity is frequently determined by rational, not statistical, analysis; for example, it includes face validity, which refers to the subjective judgment of experts on the validity of the test.

4. The relationship between both concepts

The reliability of a psychometric instrument influences its validity: the more reliable it is, the greater its validity will also be. Therefore, the validity coefficients of a tool are always lower than the reliability coefficients, and validity indirectly informs us about reliability.