In preparation for this weeks #PHTwitJC I thought it would be a good idea to 1) read the paper in it’s entirety, then 2) critically appraise the paper using the CASP framework for case-control designs. My theory is the more familiar I am with the paper, the easier it will be to chair the discussions.
(Also – critical appraisal will be tested for within the Part A exam, plus it’s relevant to LO 2.14… so one piece of work can tick a few boxes. And I’m hoping that practice will make it easier and quicker to do…). Now for a prostate screening related cartoon:
Critical appraisal of Vickers et al (2010) Prostate specific antigen concentration at age 60 and death or metastasis from prostate cancer: case-control study, BMJ 2010;341:c4521
1 – Did the study address a clinically focused issue?
Yes. Population = Swedish men (data taken at age 60 and followed-up outcomes at age 85). Risk factor studied = PSA concerntration at age 60. Study assessed association between PSA concerntration and outcome of death from prostate cancer, or prostate cancer metastatis.
2 – Did the authors use an appropraite method?
Prostate cancer in men is relatively high (estimate is that close to 40% of 70 year old men have detectable cancer in the prostate at autopsy), yet death from prostate cancer is relatively low. Therefore the outcome of ‘death from prostate cancer’ is suitably rare for a case-control study design, as used in this study. The key question of this paper was whether prostate cancer is likely to cause symptoms or shorten life, not whether a man has prostate cancer.
3 – Were the cases recruited in an acceptable way?
Cases were obtained from the ‘Malmo Preventive Project‘ Cohort (subjects were invited to participate in a broad health-screening programme, including a physical examination and a panel of laboratory tests – the objective of the project was to identify individuals with high-risk of cardiovascular disease and alcohol abuse). As a large representative sample of the population, with health information and stored samples, on face value this appears to be a convenient and appropriate recruitment method. The paper described the cohort as:
The cohort reported here includes 1167 men born in 1921, constituting 71% of the eligible population, who gave blood and completed medical history and lifestyle questionnaires in 1981-2
According to the MPP website link and numbers there was a large drop-out rate (as often the case in cohort studies); only 40% had completed the study in full. I’m not sure what ‘eligible population‘ refers to – men born in Malmo in 1921? Cohort population who had completed health data? Is ‘cohort reported here’ different from MPP? Important to know precisely what population is being referred to. Also – there is likely to be a healthy-participant selection bias as participants would have agreed to tests to be taken and to participate in the topic of the research (CVD and Alcohol).
Case selection: cases were defined precisely and verified by blinded/independent clinicians. Number of cases selected was all meeting case definitions within cohort. Case identification was reliant upon Cancer Registry records, which are reported to be very reliable in Sweden. No power calculation was apparent.
4 – Were the controls recruited in an acceptable way?
We used a nested case-control design to analyse data from the cohort. Separate matches were conducted for each of the three study events: clinically diagnosed prostate cancer, prostate cancer metastasis, and death from prostate cancer.
Cases were matched 3:1 (which is excellent!) from the same cohort population (therefore the same healthy-participant selection bias applies). For each case, controls were selected at random (precise randomisation method omited) from a group of men who were alive and event-free at the time of the event in the case. They were matched on date of birth and date of base-line blood sample (with a 3 month window).
I guess three questions arise here. 1 – what impact would a healthy-participant selection bias have upon the study? Is this group different to the general population? This population is probably more likely to engage in any screening programmes, but do we know whether the incidence of life-threatening prostate cancer is comparable from this group to the general population? 2 – (and this may be a silly question, but) why did cases need to be alive at the time of the case-event? Does this not contradict the point of controls; for them to be exactly the same (date of birth, blood sample) apart from their outcomes (e.g. prostate cancer or metastatis is cases and diseases-free or death in controls). What effect does exclusion of dead controls (at time of case-event) have upon study? 3 – (again, this may be a silly question, but) would it be realistic and/or useful at all to match cases and controls to PSA levels?
5 – Was the exposure accurately measures to minimise bias?
Case-control studies normally have a disadvantage of reliying upon historical records and memory/recall for exposure measurements. However, this study design overcomes such problems as the exposure (PSA) was measured prior to the events. Furthermore, the exposure samples were assayed in anticoagulated blood plasma (stored at −20°C) and re-tested and ‘determined as previously reported’. Identical objective methods were used to assess PSA concerntration levels for both cases and controls. It was not mentioned whether the re-test were done by someone blinded or independent to this investigation, but that would’ve increased the reliability of such concerntration measures.
The use of PSA as a valid marker for prostate cancer was discussed by the authors.
6 – What confounding factors have the authors accounted for? Were these accounted for in the design/analysis?
Main factors which concern me relate to the original cohort recruitment. Although the authors of this study have reported the age and clinical exposure and outcomes of participants (both cases and controls), no consideration seems to have been given to their socio-economic status, ethnicity (men with african-heritage are more at risk), family history (risk doubles when prostate cancer present in immediate family), and other health conditions (such as men with diabetes mellitus appear to have a lower risk of developing prostate cancer). (For further information on Epidemiology of Prostate Cancer go here)
None of these potential confounders were acknowledged or accounted for in this paper.
7 – What were the results of this study?
There were 43 cases of metastasis and 35 deaths from prostate cancer. Concentration of prostate specific antigen at age 60 was associated with prostate cancer metastasis (area under the curve 0.86, 95% confidence interval 0.79 to 0.92; P<0.001) and death from prostate cancer (0.90, 0.84 to 0.96; P<0.001)… Although only a minority of the men with concentrations in the top quarter (>2 ng/ml) develop fatal prostate cancer, 90% (78% to 100%) of deaths from prostate cancer occurred in these men. Conversely, men aged 60 with concentrations at the median or lower (≤1 ng/ml) were unlikely to have clinically relevant prostate cancer (0.5% risk of metastasis by age 85 and 0.2% risk of death from prostate cancer).
8 – How precise are the results?
Can’t fault reporting of ROC curves, tables, 95% confidence intervals with associated p-values. I found the results quite straightforward and easy to understand.
*However*… one cannot rule out the effect of counfounders; the participant characteristics were not reported, acknowledged or discussed, nor adjusted for.
As well as considering the influence of participant characteristics within cohort, important to also consider charactertistics of non-responders/drop-outs… this is particularly important when drop-out is high (i.e. 60% in the MPP). The author should not assume that all readers are familiar with the MPP study, and moreover they should report the charactertistics of their participants as this is highly relevant to the interpretation of their reported findings.
9 – Do you believe the results?
The results appear to be positive – it is easy to be swayed due to the positive correlation between PSA concerntration and prostate cancer outcome. The author has reported a significant p-value, therefore the results are unlikely to be due to chance. The effect of bias and confounding cannot be ruled out…
The study design and methods are well reported and objective, but the study has been let down in it’s reporting of participant (and non-participant) charactertistics. Therefore even if the results were valid, one could not be sure whether these results are applicable to local and general populations.
10 – Can the results be applied to the local population?
Insufficient description of cohort population (aside from nationality and age) to make comparisons to local population. Authors acknowledge further research on PSA and race would be useful, yet no description of the ethnic classifications is provided in this study.
11 – Do the results of this study fit with other available evidence?
Screening is associated with considerable overdiagnosis, and many men need to be screened to save one life. PSA is a common screening tool for prostate cancer and this study seeks to stratify risk of adverse-event prostate cancer with PSA as meansured at age 60.