Kellgren and Lawrence measured how much readers disagree when grading radiographs for osteoarthrosis and how that disagreement shifts prevalence estimates in population surveys.
Reliability study of 510 radiographs of 11 joints from 85 randomly chosen adults aged 55 to 64 in Leigh, Lancashire, read four times by two observers.
Methods
Design
Inter- and intra‑observer reliability study
Part of a rheumatic disease survey in Leigh, Lancashire
Patients
85 randomly chosen urban adults aged 55 to 64
510 radiographs of 11 joints
Basis
Marginal osteophytes, or tibial spine osteophytes at the knee
Periarticular ossicles, mainly at the interphalangeal joints
Joint space narrowing with subchondral sclerosis
Subchondral pseudocysts with sclerotic walls
Altered bone‑end shape, especially the femoral head
Standard reference films for each grade at each joint
Reliability
Combined reading by both observers in consultation, setting agreed grading standards
Independent readings by each observer 2 years later
Repeat reading by one observer 1 month later
Correlation between readings and spread of prevalence estimates
Key Findings
The two observers' grades correlated significantly at every joint but the wrist, yet their accumulated bias left prevalence estimates varying by ±31%.
At the metacarpophalangeal joints, Observer A diagnosed definite osteoarthrosis about twice as often as Observer B (27 vs 14 joints).
Wrist agreement was so slight it may have been chance, and the authors traced most disagreement to wrists that also had severe rheumatoid arthritis.
When one observer reread the films, the two prevalence counts were close (153 vs 169 joints), but both fell below the mean of all readings (195 joints).
A combined reading by both observers in consultation (179 joints) came closer to the mean of all readings (195) than any individual reading.
The authors concluded that prevalence estimates from different readers may be of little value to compare in population surveys.
Board pearl
Kellgren and Lawrence counted osteoarthrosis as definitely present from Grade 2 upward, with Grade 0 meaning definite absence of radiographic change.
Kellgren‑Lawrence radiographic gradesKellgren, Ann Rheum Dis 1957
0
None: definite absence of x‑ray changes of osteoarthrosis
1
Doubtful
2
Minimal: osteoarthrosis definitely present
The diagnostic threshold. Grades 2 to 4 count as definite osteoarthrosis.
3
Moderate
4
Severe
The paper lists osteoarthrosis signs but no criteria per grade.
Authors' conclusion
It is suggested that, where possible, in all population studies which it is desired to compare, the x rays should be read by the same observer or preferably by two observers in consultation.
Kellgren and Lawrence measured how much readers disagree when grading radiographs for osteoarthrosis and how that disagreement shifts prevalence estimates in population surveys.
Reliability study of 510 radiographs of 11 joints from 85 randomly chosen adults aged 55 to 64 in Leigh, Lancashire, read four times by two observers.
Methods
Design
Inter- and intra‑observer reliability study
Part of a rheumatic disease survey in Leigh, Lancashire
Patients
85 randomly chosen urban adults aged 55 to 64
510 radiographs of 11 joints
Basis
Marginal osteophytes, or tibial spine osteophytes at the knee
Periarticular ossicles, mainly at the interphalangeal joints
Joint space narrowing with subchondral sclerosis
Subchondral pseudocysts with sclerotic walls
Altered bone‑end shape, especially the femoral head
Standard reference films for each grade at each joint
Reliability
Combined reading by both observers in consultation, setting agreed grading standards
Independent readings by each observer 2 years later
Repeat reading by one observer 1 month later
Correlation between readings and spread of prevalence estimates
Key Findings
The two observers' grades correlated significantly at every joint but the wrist, yet their accumulated bias left prevalence estimates varying by ±31%.
At the metacarpophalangeal joints, Observer A diagnosed definite osteoarthrosis about twice as often as Observer B (27 vs 14 joints).
Wrist agreement was so slight it may have been chance, and the authors traced most disagreement to wrists that also had severe rheumatoid arthritis.
When one observer reread the films, the two prevalence counts were close (153 vs 169 joints), but both fell below the mean of all readings (195 joints).
A combined reading by both observers in consultation (179 joints) came closer to the mean of all readings (195) than any individual reading.
The authors concluded that prevalence estimates from different readers may be of little value to compare in population surveys.
Board pearl
Kellgren and Lawrence counted osteoarthrosis as definitely present from Grade 2 upward, with Grade 0 meaning definite absence of radiographic change.
Kellgren‑Lawrence radiographic gradesKellgren, Ann Rheum Dis 1957
0
None: definite absence of x‑ray changes of osteoarthrosis
1
Doubtful
2
Minimal: osteoarthrosis definitely present
The diagnostic threshold. Grades 2 to 4 count as definite osteoarthrosis.
3
Moderate
4
Severe
The paper lists osteoarthrosis signs but no criteria per grade.
Authors' conclusion
It is suggested that, where possible, in all population studies which it is desired to compare, the x rays should be read by the same observer or preferably by two observers in consultation.