What Is Agreement In Statistics
Written by Staff on December 20, 2020
A recent study  examined the Inter-Rater agreement for specific magnetic resonance imaging (MRI) in 84 children who, for one reason or another, underwent full-body MRI in a large public hospital. Two radiologists who blinded each other reported all the lesions they identified in each patient. A third radiologist linked these independent surveys and identified all the unique lesions and therefore consistent and contradictory diagnoses. A total of 249 different lesions were detected in 58 children (the remaining 26 had normal MRI scans); 76 disagreed and 173 agreed (Table 2). For the three situations described in Table 1, the use of the McNemar test (designed to compare coupled categorical data) would not make a difference. However, this cannot be construed as evidence of an agreement. The McNemar test compares the total proportions; Therefore, any situation in which the total share of the two examiners in Pass/Fail (for example. B situations 1, 2 and 3 in Table 1) would result in a lack of differences. Similarly, the mated t-test compares the average difference between two observations in a single group. It cannot therefore be significant if the average difference between unit values is small, although the differences between two observers are important for individuals. As mentioned above, correlation is not synonymous with agreement.
The correlation refers to the existence of a relationship between two different variables, while the agreement considers the agreement between two measures of a variable. Two sets of observations, strongly correlated, may have a poor agreement; However, if the two sets of values agree, they will certainly be strongly correlated. For example, in the hemoglobin example, the correlation coefficient between the values of the two methods is high, although the agreement is poor [Figure 2]; (r – 0.98). The other way of looking at it is that, although the different points are not close enough to the dotted line (least square line; , indicating a good correlation), these are quite far from the running black line that represents the perfect chord line (Figure 2: the black line running). If there is a good agreement, the dots should fall on or near this line (of the current black line). Suppose you analyze data for a group of 50 people applying for a grant. Each grant proposal was read by two readers, and each reader said “yes” or “no” to the proposal. Suppose the data for the counting of disagreements were the following, where A and B are drives, data on the main diagonal of the matrix (a and d) the number of chords and non-diagonal data (b) and c) the number of disagreements: Weighted kappa is a version of kappa used to measure agreement on ordered variables (see section 11.5.5 de Agestir, 2013).