In this week’s study session we (5 registrars in the region who are preparing for the January exams) we discussed our critical appraisals of the following paper: Chilvers et al (2001) Antidepressent drugs and generic counselling for treatment of major depression in primary care: randomised trial with patient preference arms, BMJ
Q1 required candidates to write a structured abstract, which is pretty difficult to do after reading the whole paper, including the abstract… I shall attempt this next week!
Q2: Critically appraise the paper, paying particular attention to: the methods (including statistics), choice of outcomes, results, and conclusions drawn
- Although GP practices were randomised, only patients who self-selected to join the randomisation group were randomised. These patients tended to be less severely depressed and their characteristics may have differed from those refusing randomisation (such as levels of motivation, previous experience…etc)
- GPs recruited participants which may have provided an element of recruitment bias. Did GPs have preference for a particular treatment? No details provided or discussion of ways to avoid recruitment bias.
- Authors acknowledged recruitment to be difficult (and even adjusted their power calculation and study numbers as a result). No baseline figures of number of patients approached/refused were provided… not assured this is a representative sample of the general population with depression.
- Statistics: not all baseline characteristics were reported in table 1. Chi-square and Fisher’s Exact test both appropriate for small sample size.
- Paper title suggests an investigation of patients with diagnosed major depression, whereas throughout article reference is made to mild/moderate depression. Furthermore depression is diagnosed by ‘research criteria’ for the study which limits it’s application to the real-world
- Problematic use of the term “generic counselling” – what does that mean? Full treatment detail is really important, especially if this paper hopes to inform commissioning of services. Again – lack of information limits the real-world application
- Also, although study details that GPs were provided with protocol for anti-depressents, no attempt to assess the efficacy of this, or monitor how GPs managed patients on anti-deps…etc GP consultations are in themselves interventions. Many unknowns regarding treatment process that have important implications for considering the paper’s conclusions…
- Paper assumed that if patient was well at 8 weeks and again at 12 months, they were in remission throughout that period. Given the nature of depression, this is an unfounded assumption to make as most patients will experience a rise and fall of symptoms throughout their recovery. Authors refers to MANY assumptions, which is suggestive that they did not have appropriately defined periods of follow up (nor, as mentioned above, did they assess frequency of GP consultations as part of treatment).
- Reports no systematic different between proportion of patients asigned to different trial arms – however no test of hetergeneity for the baseline charactertics (age, sex, ethnicity)
- Patients choosing counselling reported to do better compared to those randomised to counselling – but 95% confidence interval reported as 0.0 – 9.2 ….. this is fairly wide considering a mean difference of 4.6, plus a lower limit of 0.0… is this considered as ‘crossing zero’ and therefore increase the likelihood that this result could be due to change?
- Reported conclusion suggested that “12 months after starting treatment, generic counselling is as effective as anti-depresants”… however I disagree as firstly) the term ‘generic’ counselling is meaningless, secondly) not clear what severity of depression was being treated through this trial, and thirdly) the authors have omited to specify which type of anti-depressants were used… the lack of specific details make it difficult to agree with this conclusion
- Not confident that confounding factors were well accounted for e.g. counselling has a gender bias, self-selection vs. willing to be randomised may have an effect on time to remission due to personal motivation…
- Article further concludes that both treatments (again, lacking specific details of what treatments were?) are effective (again, lacking specific details of treating what type of depression? and defined using criteria which is not applicable to general practices).
(NB: Ideally… I should be preparing my critical appraisal under timed conditions, and will do in future.)
Q3 A review group is being pressed to fund the expansion of counselling services. What would be your response; what additional information is needed?
As a group, we struggled to identify a framework to structure the answer to this question… any ideas?
In order to review a funding application it would be important to have access to the following sources of information:
- Defintion of the condition being considered (is the expansion specifically for patients with depression? Counselling can be used to address a number of problems)
- Epidemiology of the condition (including local incidence and prevalence, so trends and local needs can be considered) and local projections
- Literature review / NICE guidance to help inform how best to commission services for the condition and ensure the evidence base is known and applied
- Full details of the current services available (including numbers attending services for this condition, waiting lists if available (as this would evidence unmet need))
- Funding application ought to include cost-effectiveness analysis whereby a return on investment can be demonstrated, with clear timelines as to when that ROI would be expected
My response would be to firstly identify information needed (as above) and collate this into a report which has a particular focus on the evidence-base, local need + projections, and cost-effectiveness of counselling to address this condition. It would be important to involve stakeholders in this process of collating information, but also to present the report findings to the group reviewing mental health services to ensure that any altered commissioning arrangements are based on evidence of need and consider the bigger public health picture