Reliability estimation

Before estimating the reliability of the questionnaire, it is necessary to use a suitable scale (such as Likert scale) to convert various subjective or objective alternative answers in the questionnaire into digital form, and then score the questionnaire on this basis (including single score, related topic group score and total score, etc.). ).

There are four common concrete methods for reliability analysis: retest reliability, repeated reliability, split-half reliability and reliability coefficient method (the latter two can be classified as internal consistency reliability). Test-retest reliability, also known as test-retest reliability, refers to the difference between the two survey results of the same group of respondents using the same questionnaire at different time points. Test-retest reliability reflects the influence of random error.

The error source of retest reliability is the random influence brought by time change. When evaluating the reliability of retest, we must pay attention to the interval of retest. For personality tests, it is appropriate to retest between two weeks and six months.

When evaluating the test-retest reliability, we should also pay attention to the following two important issues: (1) The test-retest reliability generally only reflects the changes caused by random factors, but does not reflect the long-term changes of the subjects' behavior. ⑵ Random errors have different effects on different behaviors.

Disadvantages: There is a dilemma in retest reliability. Shorten the time interval between the two tests, so that the examinee can recall the test questions more easily; However, if the time interval between two tests is prolonged, the testee is more susceptible to external influences. Parallel form reliability, also known as equivalence coefficient, is a kind of equivalence reliability, which refers to the degree of variation of one questionnaire survey result relative to another identical questionnaire survey result. It is to use two questionnaires with the same content but different topics to investigate the same group of respondents, and then compare the correlation between the two groups of data.

It is more workload than reliability test, because the same measuring tools (questionnaire, psychological scale, etc. ) You need to build two copies with the same quantity, type, content and difficulty. To evaluate the reliability of repetition, we need to test the same group of subjects with two repetitions, and then estimate the correlation coefficient of the measurement scores of the two repetitions. The larger the correlation coefficient, the smaller the variation caused by the composition of the two copies, which is different from the variation caused by considering time in the retest stability reliability, that is, the correlation coefficient reflects the equivalence of the measured scores, so the reliability of the copies is also called equivalence reliability.

The main advantages of replica reliability are: (1) it can avoid some problems of retest reliability, such as memory effect and practice effect; (2) It is suitable for long-term follow-up study or investigation of the influence of some interference variables on test scores; (3) Reduce the possibility of tutoring or cheating. ☆ The limitation of replica reliability lies in: (1) If the measurement behavior is easily influenced by practice, the replica reliability can only be reduced but not eliminated; (2) The nature of some tests will change due to repetition; (3) Some tests are difficult to find suitable copies. It mainly reflects the relationship between the types of questions in the test questions, and examines whether the contents or characteristics of each type of test questions are the same. The reliability of internal consistency can be divided into split-half reliability and homogeneous reliability.

1, split-half reliability (split-half reliability) refers to the degree of variation of the survey results of two and a half questions in a survey. It is the reliability coefficient obtained by dividing the test into two halves and calculating the correlation between the two halves. The longer the test time, the higher the reliability coefficient. The modified formula is Spearman-Brown formula. Spearman-Brown formula is an empirical formula to modify split-half reliability (rtt = 2HH/( 1+RHH). Generally speaking, if the test is divided into two equal halves, the split coefficient RTT is 0.5).

Its assumption is that the variance of the two test scores is equal. When the hypothesis fails, one of Flanagan formula or Coulomb formula can be used to directly obtain the reliability coefficient of the test.

2. Homogeneous reliability refers to the degree to which each question examines the same content in the test. When the reliability of homogeneity is low, even though each test seems to measure the same trait, it is actually heterogeneous, that is, it measures more than one trait. Homogeneity analysis is similar to internal consistency analysis in project analysis. Several formulas for calculating homogeneity reliability: (1) Kuder-Richardson formula (2) Kehlenbach α coefficient. For some complex and heterogeneous psychological variables, it is impossible to use a single homogeneity test, so several relatively heterogeneous subtests are often used. Refers to the consistency when different raters rate the same object. The simplest estimation method is to randomly select several answers, score them by two independent raters, and then find the correlation coefficient of the two evaluation scores of each answer. This correlation coefficient can be calculated by product moment correlation method or Spearman rank correlation method.