The Ministry of Education and Science of Russia has published a test, according to which it is planned to test schoolchildren and students for drug addiction. Psychologist Yevgeny Osin explains why this test confused the professional community.
In early March*, the Ministry of Education and Science proposed a good initiative: to examine all Russian schoolchildren and students for «early detection of non-medical use of narcotic drugs and psychotropic substances» through psychological testing. However, this prospect has worried many parents: what if the test is not entirely accurate, and the child accidentally falls into a group of drug addicts? The questions of this test, posted by the ministry in the public domain, are already actively laughed at by users of social networks: indeed, what does the answer to a question like this, “Despite the danger, would you like to hunt a tiger, have to do with drug use?” But for any professional psychologist, this story does not cause laughter, but a real shock. It seems that the authors of the project are not fully aware of the differences between a professional psychological test and a joke test. What is this difference?
A responsibility. If a comic or entertaining test is designed to have a good time, then serious decisions are made based on the results of a professional test: whether to hire a person, what grade to give him in the certificate (USE), whether he needs the help of a psychologist. Therefore, a professional test is not just a set of questions, but a measuring tool similar to a water meter or a speedometer. By analogy with a water meter, which must have a certificate and be verified from time to time, the quality of a psychological test must also be checked using special studies.
- Julia Gippenreiter: “Parents should not take everything upon themselves”
Psychometrics is a special area of science that conducts research on the development and quality control of tests. The first psychological test, developed by Frenchmen Alfred Binet and Theodore Simon in 1905 to measure the intelligence of children, has recently turned XNUMX years old. Over this century, psychologists have developed tens of thousands of tests and questionnaires to measure a variety of individual characteristics: abilities, character traits, emotional states. And during this time, very clear criteria were developed that any test must satisfy (in Russian, these criteria are perfectly described in the recently published book by Professor A. G. Shmelev «Practical Testology»). There are three main criteria:
- Reliability. To understand how much we can trust the measurement results, we first need to know how accurate the instrument is: what is its error? If Ivanov scored 100 on an intelligence test and Petrov scored 105, can you be sure that Petrov’s intelligence is indeed higher? We don’t usually think about this question, but the answer to it depends on the reliability of the test: the lower it is, the more likely it is that Petrov got a higher score purely by chance.
- “Passing the Luscher test on the Internet is pointless”
Therefore, the reliability of any test is extremely important and is first tested in scientific research. A test with low reliability does not measure anything at all: it is like a water meter, the speed of rotation of the needle of which does not depend on the flow of water, but on some extraneous factors. Perhaps no one would like to pay for water using such a meter.
The first problem with the so-called «drug test» from the Ministry of Education and Science is that its reliability is unknown: there are no references in the document to scientific studies in which it would be tested.
- Let’s stop putting pressure on our children!
- Validity. When a test is reliable (meaning it measures something), psychometrics ask the following question: Does the test really measure what it is supposed to? With a water meter, everything is simple: you can determine exactly what it measures, knowing its design and the laws of physics. With psychological tests, everything is a little more complicated: this issue is studied specifically, because the test can be reliable, but not valid. For example, if very rare words are used in the tasks of the intelligence test, the test will measure not so much the intelligence of the subjects as their vocabulary.
The question of validity is especially important when a test results in a diagnosis: say, whether a person has a certain infection or uses drugs. In medicine, any diagnostic criterion relies on research data on how well it distinguishes people who are known to be sick from those who are definitely healthy. No one wants to spend money on an analysis that will often result in healthy people being diagnosed as sick, or vice versa, which is even worse.
The second problem with the methodology proposed by the ministry for “identifying students with signs of addictive behavior” is that there is no research data that would allow us to understand whether the criteria proposed by officials really distinguish adolescents with drug addiction from those who definitely do not use drugs. .
- Take your child to a psychologist?
- Reliability. Even if a test is both reliable and valid but misused, its results may be unreliable. For example, a water meter is suitable for measuring the flow of exactly water, and at a certain temperature (it’s not without reason that the meters for cold and hot water are different). If you put it on a pipeline with some other liquid, say, gasoline, then the consumption figures in cubic meters can be very different from the real ones. And if using a water meter in this way, perhaps, no one comes to mind, then psychological tests, unfortunately, are too often used at all in the wrong conditions for which they are intended. A test designed for people who seek to answer questions sincerely may give incorrect results in a group of people who consciously or unconsciously seek to distort information about themselves (for example, in a job selection).
If a test is used in settings other than those for which it was designed, then the reliability of its results under those conditions must be checked separately, and separate criteria are needed to interpret the results. Anyone who has ever measured the temperature of a cat or dog must have been surprised to learn that 38 degrees for these animals is a normal temperature. For psychological tests, the criteria that allow you to answer the question “A lot, a little or normal?” Are called test norms. Occupational tests usually include different norms for men and women, for different age groups.
- How to help study without stress
The third problem with the test proposed by the Ministry of Education and Science is that no one has checked the reliability of its results in schoolchildren who are afraid (justifiably or not) of being suspected of using drugs. The proposed methodology does not contain any test norms necessary to take into account the peculiarities of the reaction to questions of boys and girls. So, reliability, validity and reliability are three key requirements that any professional test must meet. And if in relation to people in any civilized country the principle of the presumption of innocence is used, then the presumption of unreliability, invalidity and unreliability should be applied to psychometric tools that are used in responsible situations. No test can be considered reliable, valid, and reliable until these characteristics are substantiated by scientific methods.
How are things going with the Cattell questionnaires offered by the ministry? These are questionnaires based on the 16-factor model of personality (16PF), which was developed by the American psychologist R. B. Cattell in the 1940s. Modern English versions of these questionnaires can be considered reliable, valid and reliable tools for diagnosing personality traits. However, the versions of these tools that exist in Russian and are presented in the document of the Ministry were developed decades ago, before the advent of modern methods of psychometrics. You can draw an analogy with a water meter that has not been verified for 20 years. But if the physical characteristics of water should not change over time, then the personality traits of Soviet schoolchildren and students, on whose groups these questionnaires were developed in the 1970s and 80s, and students in modern Russia can differ in the most radical way. Individual questions of the test by modern teenagers can be understood differently than they were once understood. Therefore, without a repeated psychometric review of this test, it cannot be considered reliable.
But even for the American version of the test, there are no clear and unambiguous criteria by which one could reliably say whether a person is prone to the use of psychoactive substances: they are different for different substances, people from different social groups, different testing conditions. And in principle, the Cattell test, aimed at diagnosing the personality as a whole, is not intended to solve such a problem. It’s not that you can’t use it in this way, but it’s like taking the temperature of a sick person with a thermometer designed to measure the temperature outside in the range from -50 to +50 ° C. This test cannot be considered valid and reliable without special studies of what its indicators and with what accuracy make it possible to draw a conclusion about the propensity to use certain substances among Russian schoolchildren and precisely in conditions of mass testing.
Thus, it is simply impossible to consider the Cattell test as a scientifically based tool for diagnosing drug addiction in Russian schoolchildren. You can rely on its results with the same success as on the results of divination on coffee grounds or an astrological forecast. Not to mention the fact that, in addition to the test itself, there are certain standards in world psychometrics for how test materials and results should be stored and applied . These standards are adopted by the International Testing Commission (ITC). Some of these standards are seriously violated in the methodology of the ministry.
1) Awareness. Each subject must be informed not only about the purpose of testing, but also about the consequences of refusing it. In the written form of voluntary informed consent, the receipt of which from students and their parents is «organized» by the head of the educational institution, this information is not available.
2) Competence. Tests should only be used by a person who has received special training, understands the limitations of test methods and follows the rules for their ethical use. However, there is no information about the special training of people «from among the employees of the educational organization» who will conduct testing and process the results, in the procedures.
3) Safety of test materials. In no case should professional tests be published in full in open sources. Imagine what would happen to the exam if all the options for tasks with the correct answers became known long before the day of testing? The ministry’s document published not only all the questions, but also the procedures for calculating and interpreting test scores. After reading it, it is easy to memorize the number of questions (there are only 10 of them) and the sequence of answers that you need to give in order not to fall into the “risk group” for sure.
- Teenager smoking a hookah
Summarizing all of the above, children are proposed to be tested for their propensity to use drugs with an instrument, about which it is completely unknown whether it can measure this propensity at all and how well. It is not clear either the level of qualification of the people who will do this, or how they will use the results obtained.
And one can imagine the worst possible scenario: a certain Marya Ivanovna «with the best of intentions» will violate the anonymity and confidentiality of testing. She will independently calculate the test scores of students using public keys and call her parents: “And our test from the Ministry showed that your Vovochka is a drug addict!” One can only hope that this prospect is unrealistic, but such a possibility cannot be ruled out.