Sign in to follow this  
Followers 0
ac97dmc

Best 1 out of 5 ?

27 posts in this topic

Dear everybody in the same boat

How do you feel about the concept of best one out of 5 type of questions for psychiatry? Things are pretty nebulous, and even the simplest concepts confuse if one tends to read between the lines e.g. this is THE ONLY sample question available from rcpsych -

An evaluation of a new screening questionnaire for depression in primary care is conducted. The most important single feature of this questionnaire that would encourage you to use it is:

a) 94% of patients given the questionnaire completed it *

b)it has Positive Predictive Value of 0.7

c)it has sensitivity of 0.7

d) it has specificity of 0.7

e) it is validated in different languages

whilst i feel inclined to go for c; a & e also appeal. :-? what do you think?

Share this post


Link to post
Share on other sites

The answer that is given as correct is (a) - that's why there is an * by it. I would go for that one... Plus its the one that makes the most sense.

Share this post


Link to post
Share on other sites

I think we have to adjust- especially those of use who are used to practising the old T/F questions. Instead of getting 50% right by chance, and therefore needing 85-90% out of a practice paper for a definite pass, we will only get 20% right by chance. So it doesn't matter so much if you don't know the exact answer- the more you know, the more options you can rule out in the best of 5 and the greater your chance of getting the right answer. One hopes that over the whole paper, this means that those who know the most are most likely to pass........but it is very different psychologically from T/F.

Share this post


Link to post
Share on other sites
The answer that is given as correct is (a) - that's why there is an * by it. I would go for that one... Plus its the one that makes the most sense.

thanks for pointing out... if not, i would have gone for c perhaps... since its a screening tool, i would have wanted a high specificity (dont ask me why, even though 0.7 is not high enough)...

but reflecting on it much more, it does make sense that in a PCT setting its imperative that the maximum number of people respond to it..

but again, does a high response rate really mean that i will be inclined to use it, even though its sensitivity and specificity are not upto scratch ?

my dissonant mind still tells me to go for option 'c'.. what do other ppl think ?

Share this post


Link to post
Share on other sites

http://www.medscape.com/viewarticle/511167

Depressive disorders are common, costly, and treatable, but they often go unrecognized. A 2001 World Health Organization report projected that by 2020, depression will account for more lost disability-adjusted life-years than all other conditions, except for ischemic heart disease

The focus on screening for depression in primary care makes sense, since almost two thirds of patients with depression receive care in that setting

Despite the frequency of depression, diagnosis by nonspecialist practitioners is often haphazard. Studies show that primary care physicians who provide usual care fail to recognize depressive symptoms in 30% to 50% of patients with depression.

So, surely you would want a screening tool for depression to pick up as many cases as possible - sensitivity needs to be good.

e.g. Screening tool of 2 questions:

During the past month, have you often been bothered by feeling down, depressed, or hopeless?

During the past month, have you often been bothered by little interest or pleasure in doing things?

The use of these 2 screening questions alone showed a sensitivity and specificity of 97% and 67%, respectively, when tested in a primary care setting on patients not receiving psychotropic drugs.

For a system of screening to be viable it must fulfil certain criteria.

http://www.patient.co.uk/showdoc/40024566/

The condition must be sufficiently common to merit screening. This does not necessarily mean common in the whole population unless there is universal screening. It means common in the target group for screening.

There must be an effective intervention for the condition that is being sought.

Screening must result in the condition being recognised at an earlier stage when intervention is more effective.

There must be high specificity (low rate of false positives) and a very high sensitivity (very low rate of false negatives), although this is difficult to assess when evaluating a screening tool for depression.

The screening test must be relatively cheap or else the cost per case detected is prohibitively expensive.

It must be safe, easy to use and acceptable to the patient.

Share this post


Link to post
Share on other sites

I think the issue here is that 0.7 is just too low in terms of sensitivity - the test will miss out too many people who may be depressed.

A is the best fit - the more people who fill out the tool the more useful the test as a screening tool.

It would be c if the sensitivity was .85 or higher (my arbitrary guess) .

Roll on MRCPsych and an end to pointless MCQs and BOFs.....

Share this post


Link to post
Share on other sites
A is the best fit - the more people who fill out the tool the more useful the test as a screening tool.

As far as I understand the test will pick up the same proportion of people as being positive or negative for depression regardless of how many people fill it out, be that 100 or 10000. I think the key word in the sentence that makes it answer A is completed. 94% of people completed it which means that it's nice and simple and not confusing. For a first line screening test it's good because screens are no use if they are too complicated for people to fill in.

Share this post


Link to post
Share on other sites

Yep wasn't arguing with you just using your post as an example of why I thought it was A.

Share this post


Link to post
Share on other sites

Hi, Can anybody please advise, on the College website, among sample questions, this one appears:

. The odds of an event if it happens 80% of the time. G

Now answer G is 9. How can it be 9??? Can anybody please tell me?? I am sure their answer can't be wrong, then what is it? please help. Thanks

Share this post


Link to post
Share on other sites

It might not be much consolation regarding this particular hypothetical screening instrument, but unaided clinical diagnosis in primary care only has a sensitivity of 0.65.

Additionally, a PPV of 0.7 is better than the WHO-5 (0.34), the PHQ-9 (0.51), the GHQ-12 (0.31), and unaided clinical diagnosis (0.34). Since, clinical diagnosis of depression in primary care doesn't appear particularly good you could therefore argue that a PPV of 0.7, or a sensitivity/ specificity of 0.7 are pretty good attributes.

In reality, the utility of a screening questionnaire depends on a combination of all of these attributes, and a 94% completion rate for a questionnaire that might have a sensitivity of 0.1 and a specificity of 0.1 isn't particularly compelling just by itself.

One could argue that these are the sorts of questions that are best suited to short-answer questions rather than MCQs. The person who commented that one has to shift your mind-set is probably right.

Henkel, V., Mergl, R., Kohnen, R., et al (2003) Identifying depression in primary care: a comparison of different methods in a prospective cohort study. BMJ, 326, 200-201.

http://www.bmj.com/cgi/content/full/326/7382/200

Share this post


Link to post
Share on other sites
Hi, Can anybody please advise, on the College website, among sample questions, this one appears:

. The odds of an event if it happens 80% of the time.  G

Now answer G is 9. How can it be 9??? Can anybody please tell me?? I am sure their answer can't be wrong, then what is it? please help. Thanks

http://www.rcpsych.ac.uk/exams/about/mrcpsychpaperiii.aspx

I'm confused too. I thought the answer would be 4...

Probability/risk of the event happening= 8/10=0.8

Odds of it happening= chance of it happening/ chance of it not happening=8/2=4

?????What does anyone else think? :-?

Share this post


Link to post
Share on other sites

Thanks Ros, it confirms my doubt. I assume that the answer actually is 4, which is option C. Probably the secretary wrote G instead of C, when typing the questions on the website. What do you think the probability of this is?:)

I think it is more likely than the answer being 9, which is just not possible I think.

Share this post


Link to post
Share on other sites

In reality, the utility of a screening questionnaire depends on a combination of all of these attributes, and a 94% completion rate for a questionnaire that might have a sensitivity of 0.1 and a specificity of 0.1 isn't particularly compelling just by itself.

One could argue that these are the sorts of questions that are best suited to short-answer questions rather than MCQs. The person who commented that one has to shift your mind-set is probably right.

yep.. my point exactly.. and i agree with u one hundred percent !

Share this post


Link to post
Share on other sites

Agree with Ros and SL, by my calculation the answer is indeed four.

Share this post


Link to post
Share on other sites

this one is borrowed from the College's site..

The most important component in the development of a clinical guideline is:

a) All stakeholders are consulted.

B) Economic factors are considered

c) Meta-analysis of the available data

d) Systematic development

e) Systematic literature search  *

i was wondering why systematic seach over a meta analysis ? is it cos a systematic search would cover more literature as compared to a meta analysis, since meta analyses often take into account positive studies and that lot.. ?

Share this post


Link to post
Share on other sites

Maybe because a systematic literature search is a prerequisite to a half-decent meta-analysis?

Share this post


Link to post
Share on other sites

Doing a meta analysis would exclude lots of research cos it wouldn't fit in and be analysable. A plain literature search means you can include all sorts of stuff which might be relevant

Share this post


Link to post
Share on other sites

Choose the correct statement:

a) The person in charge of the question publication isn't incompetent.

B) The college have gone to loads of effort in preparing paper 3.

c) Hardworking candidates won't come unstuck by the new examination.

d) The fee is worth every penny.

e) The questions are ill-conceived.

Share this post


Link to post
Share on other sites
Choose the correct statement:

a) The person in charge of the question publication isn't incompetent.

B) The college have gone to loads of effort in preparing paper 3.

c) Hardworking candidates won't come unstuck by the new examination.

d) The fee is worth every penny.

[highlight]e) The questions are ill-conceived.[/highlight]

that is the correct answer from the lot :D

Share this post


Link to post
Share on other sites
Choose the correct statement:

a) The person in charge of the question publication isn't incompetent.

B) The college have gone to loads of effort in preparing paper 3.

c) Hardworking candidates won't come unstuck by the new examination.

d) The fee is worth every penny.

[highlight]e) The questions are ill-conceived.[/highlight]

that is the correct answer from the lot :D

Well done Sag, it appears you have the right approach to this style of question. :lol:I should've starred, '*', the wrong answer to further make my point.

Share this post


Link to post
Share on other sites

hahaa.. fair point Magna !!!

i dont know what i might end up doing in the exam though.. which reminds me, the College has sent me two letters confirming receipt of my application and has allocated me exam seats in B'ham and Manchester..

i have written back stating that i wont be able to attend both the centres for the exam on the same day and same time !!!

still waiting for their response .. shud get it by tomorrow ;)

Share this post


Link to post
Share on other sites
hahaa.. fair point Magna !!!

i dont know what i might end up doing in the exam though.. which reminds me, the College has sent me two letters confirming receipt of my application and has allocated me exam seats in B'ham and Manchester..

i have written back stating that i wont be able to attend both the centres for the exam on the same day and same time !!!

still waiting for their response .. shud get it by tomorrow ;)

Exactly the same has happened to my mate.

I told him to ring the College pronto to sort it- and don't believe anyone who tells you it will be sorted without checking back a week later to make sure. I wouldn't put any money on the College failing you for not turning up at the space they had allotted you in Birmingham, even if you went to the Manchester one and handed your paper in. I'm not trying to scare anyone, but it is really important- I have had to deal with them every single job change until MTAS cos I have to get each of my flexible posts approved separately by them. Without going into traumatic details, the words p*ssup, brewery, organise and couldn't spring to mind....

Share this post


Link to post
Share on other sites

well, it was an organisational problem by the College.. they cudnt manage to book enough seats in Manchester to begin with.. but now they have got confirmation for more seats.. hence my centre has been moved to Manc.. he said i can destroy the B'ham letter and prepare for the exam that will be held in Manchester :D

Share this post


Link to post
Share on other sites
Sign in to follow this  
Followers 0