Skip to main content
  • Research article
  • Open access
  • Published:

Quality and safety of medication use in primary care: consensus validation of a new set of explicit medication assessment criteria and prioritisation of topics for improvement



Addressing the problem of preventable drug related morbidity (PDRM) in primary care is a challenge for health care systems internationally. The increasing implementation of clinical information systems in the UK and internationally provide new opportunities to systematically identify patients at risk of PDRM for targeted medication review. The objectives of this study were (1) to develop a set of explicit medication assessment criteria to identify patients with sub-optimally effective or high-risk medication use from electronic medical records and (2) to identify medication use topics that are perceived by UK primary care clinicians to be priorities for quality and safety improvement initiatives.


For objective (1), a 2-round consensus process based on the RAND/UCLA Appropriateness Method (RAM) was conducted, in which candidate criteria were identified from the literature and scored by a panel of 10 experts for 'appropriateness' and 'necessity'. A set of final criteria was generated from candidates accepted at each level. For objective (2), thematically related final criteria were clustered into 'topics', from which a panel of 26 UK primary care clinicians identified priorities for quality improvement in a 2-round Delphi exercise.


(1) The RAM process yielded a final set of 176 medication assessment criteria organised under the domains 'quality' and 'safety', each classified as targeting 'appropriate/necessary to do' (quality) or 'inappropriate/necessary to avoid' (safety) medication use. Fifty-two final 'quality' assessment criteria target patients with unmet indications, sub-optimal selection or intensity of beneficial drug treatments. A total of 124 'safety' assessment criteria target patients with unmet needs for risk-mitigating agents, high-risk drug selection, excessive dose or duration, inconsistent monitoring or dosing instructions. (2) The UK Delphi panel identified 11 (23%) of 47 scored topics as 'high priority' for quality improvement initiatives in primary care.


The developed criteria set complements existing medication assessment instruments in that it is not limited to the elderly, can be implemented in electronic data sets and focuses on drug groups and conditions implicated in common and/or severe PDRM in primary care. Identified priorities for quality and safety improvement can guide the selection of targets for initiatives to address the PDRM problem in primary care.

Peer Review reports


Systematic reviews have demonstrated deficits in the quality and safety of medication use in primary care to an extent sufficient to constitute a public health threat. Three to four percent of all unplanned hospital admissions are due to preventable drug related morbidity (PDRM), with the majority attributed to high-risk prescribing and inconsistent monitoring [14]. Antiplatelets, diuretics, non-steroidal anti-inflammatory drugs (NSAIDs) and anticoagulants account for almost half of preventable drug-related admissions to hospital, with opioid analgesics, beta-blockers, drugs affecting the renin angiotensin system and anti-diabetic agents also frequently implicated [1]. In addition, safety alerts have been issued for drugs less commonly implicated in PDRM but associated with preventable deaths, such as prescribing and monitoring of methotrexate [5] and use of antipsychotics in older people with dementia [6]. These figures are likely to underestimate PDRM caused in primary care, since the negative consequences of under-use of effective guideline recommended drugs have not consistently been considered by the hospitalisation studies included in systematic reviews [14].

The 'Data-driven Quality Improvement in Primary care (DQIP)' research programme is designing and testing a complex intervention to improve the quality and safety of medication use in UK primary care. It is based on encouraging and facilitating primary care medical practices to systematically and continuously identify, correct or otherwise manage drug therapy risks that are potential pre-cursors to PDRM [7]. The DQIP approach requires explicit medication assessment criteria which can (1) be operationalised in existing UK electronic data sources in order to (2) identify patients at risk of common or severe PDRM in primary care.

A number of explicit medication assessment tools have been developed in recent years. The Beers criteria set [8] lists potentially inappropriate drugs in the elderly and can be relatively easily implemented in electronic data sets. However, a large proportion of listed items are not licensed or rarely used in the UK and many of the drug groups frequently associated with preventable harm are not considered. More recently published tools that also focus on the elderly, such as 'Assessing care of vulnerable elders' (ACOVE) [9], 'Screening Tool of Older Person's Prescriptions (STOPP)' and 'Screening Tool to Alert doctors to Right Treatment' (START) [10] have a broader scope, but many of the included criteria require manual record review and/or clinical judgement, which are barriers to routine or large scale applications. Other instruments that have been implemented in electronic records and target the primary care population at large [1113] cover a limited spectrum of medication use issues, especially with respect to medication safety.

The study had two aims. First, we aimed to develop and classify by clinical importance a set of up-to-date medication assessment criteria that can be implemented in routine primary care clinical datasets to identify instances of (a) sub-optimally effective medication use for conditions commonly encountered in primary care and (b) high-risk use of drugs that have been shown to either commonly cause harm and/or cause severe harm in primary care. Second, we aimed to elicit the extent to which thematically-related medication assessment criteria, subsequently referred to as topics, are perceived to be priorities for quality improvement by professionals working in UK primary care.


Study design

The study was conducted in three stages. First, an extensive list of candidate medication assessment criteria was generated based on a structured literature review. Second, an expert panel participating in a modified RAND/UCLA (University College of Los Angeles) Appropriateness Method (RAM) study scored these items by clinical importance based on a summary of research evidence and their clinical judgement. Candidate criteria with high importance scores were translated into a final criteria set by removing redundancies (see below). Final criteria were characterised by the type of medication use targeted, informed by available taxonomies [1315]. Third, thematically related final criteria were clustered into medication improvement topics and those derived from candidates with high importance scores were presented to a larger Delphi panel of clinicians working in UK primary care for prioritisation. The study was approved by the Tayside Committee on Medical Research Ethics A (reference no. 09/S1401/54).

Literature review

Prescribing is a ubiquitous feature of medical care which makes a systematic evaluation of the literature on prescribing quality or safety unfeasible in a single research project. We therefore focussed on medication use for conditions commonly encountered in primary care and drugs with clear evidence of significant benefit or harm. The literature review drew initially on UK national clinical guidelines, prescribing advice, and safety alerts, supplemented by European or other clinical guidelines and targeted primary literature review in selected areas as detailed below.

Candidate medication assessment criteria either described potentially beneficial medication use ('quality') or the use of potentially harmful treatments ('safety'). Candidate 'quality' criteria targeted common conditions where there are compelling indications for drug therapy based on UK and European guidelines [1625]. Common conditions that may or may not require drug treatment for adequate management (depression, anxiety, dyspepsia, acute infections) or where we anticipated that undertreated patients would not be reliably identifiable from UK electronic data sets (chronic pain, chronic obstructive pulmonary disease, rheumatoid arthritis, thyroid disorders, epilepsy) were not considered. The following conditions were selected: primary and secondary prevention of vascular disease [26, 27], management of diabetes [28], heart failure [29], atrial fibrillation [30], asthma [31] and osteoporosis [32, 33].

In order to identify candidate safety criteria, the drug groups reported to be most frequently implicated in PDRM hospital admissions were identified from systematic reviews and large scale studies [14, 34]. For each drug or drug group identified, a more extensive literature search was conducted in order to identify patient and/or treatment related risk factors that make patients particularly vulnerable to drug-related toxicity by virtue of age, medical history, co-prescription, treatment duration and/or dose. Standard medicines information resources [3539] and the primary research literature were considered in addition to selected previously published medication assessment instruments [810, 40]. Safety alerts in the British National Formulary [36], the UK National Prescribing Centre [38] and the Medicines and Healthcare products Regulatory Agency [39] were examined to identify prescribing that was less commonly reported to be implicated in drug-related hospital admissions but associated with severe harm. Candidate safety criteria targeting potentially harmful prescribing in vulnerable groups were identified drawing on the above literature sources (children and young adults, the elderly) as well as current clinical practice guidelines (heart failure [22]). Potentially important aspects of high-risk prescribing that relied on data items which are not consistently recorded in UK primary care electronic data sets (monitoring or achievement of international normalised ratio targets, monitoring of blood glucose in patients co-prescribed drugs known to enhance sensitivity to insulin or oral anti-diabetics, medication use in pregnancy/lactation) were excluded.

RAND/UCLA Appropriateness Method (RAM) study

The RAND/University of California Los Angeles (UCLA) appropriateness method is a rigorous way of combining research evidence with expert opinion [41], and has previously been applied to develop explicit criteria for the assessment of a range of health care procedures including medication use [42]. A panel of ten members was selected with clinical, public health or academic expertise in medication use in UK primary care. The panel was composed of four general medical practitioners (of whom two had National Health Service prescribing improvement roles) and six pharmacists (including two academics with a special interest in primary care, two working in medicines governance at health board level, and two working directly with general practices). All ten participants completed two rounds of scoring.

The questionnaire aimed to classify candidate medication assessment criteria derived from the literature as either 'necessary' or 'appropriate' care (table 1). 'Necessary' is a more stringent rating standard than 'appropriate', because it represents care that would be 'improper' not to be offered or avoided, whereas 'appropriate' is a more neutral balancing of net benefit or harm [4345]. Following the RAM recommendations, ordinal scales of 1 to 9 were used for all ratings [43, 46].

Table 1 Definitions of rating categories used in the modified RAM study [55]

All candidate quality and safety assessment criteria were scored for 'appropriateness'. Candidate criteria with a median rating of 4 to 6 ('uncertain') or disagreement (three or more ratings of 7 to 9 and three or more ratings of 1 to 3) on the appropriateness scale were rejected. Those items with median ratings of 7 to 9 were accepted as 'appropriate' and those with median ratings of 1 to 3 as 'inappropriate'.

Candidate quality assessment criteria were additionally scored on a 'necessary to do' scale, where items with a median rating of 7 to 9 (= clearly necessary to do) were accepted. Candidate safety assessment criteria were additionally scored on a 'necessary to avoid' scale, where items with a median rating of 1 to 3 (= clearly necessary to avoid) were accepted. Candidate criteria with median ratings of < 7 on the 'necessary to do' and > 3 on the necessary to avoid scale and those showing disagreement (defined as above) were rejected. The concept of 'necessary to avoid' was an extension to the original RAM method to differentiate between prescribing that is 'generally not worthwhile' from 'improper' in safety terms (see box 2).

The ten RAM panel members were emailed the first round questionnaire and a summary of the supporting evidence base. Panellists were asked to rate each item with reference to an 'average' patient consulting an 'average' primary care clinician in 2009 based on both the evidence summary and their clinical judgement [44]. Panellists subsequently met for a full day, where a summary of the first round ratings was fed back to panellists anonymously. This formed the basis for a moderated discussion of each item before the second round ratings were placed. All findings reported in this paper are based on second round ratings.

Delphi study

A random sample of general medical practitioners (GPs) and eligible pharmacists in Scotland and England was invited to participate by e-mail. In order to be eligible, pharmacists had to have experience of working in medicines governance, as a prescribing advisor or as a practice pharmacist. Twenty three (64%) GPs and 13 (36%) pharmacists agreed to participate.

The Delphi questionnaire listed the medication improvement topics to be scored together with a short summary of the scientific rationale for each topic. For each item, panellists were asked to state their level of agreement with the statement 'The described topic is a priority for collaborative quality improvement in primary care'. The term 'collaborative' was used in order to emphasise that the intended purpose of this study was to identify priority topics for quality improvement rather than measures for judging practitioners or practices as part of performance management.

As in the RAM study, all ratings used an ordinal scale of 1 to 9 (1 = strongly disagree and 9 = strongly agree). Panellists were instructed to rate topics in relation to primary care in general, rather than their own practice. The first round ratings were summarised and returned to participants by email for a second round of scoring. Topics with second round median ratings of 7 to 9 without disagreement (30% or more ratings of 1 to 3 and 30% or more ratings of 7 to 9) were accepted as 'priority', with median ratings of 8 or 9 defined as 'high priority'. All findings reported in this paper refer to second round ratings.


Literature review and RAM study

The questionnaire listed 389 (100 quality and 289 safety) candidate assessment criteria. Upon completion of the second rating round, 318 (82%) candidates (93 quality and 225 safety) were accepted at the 'appropriate' and 275 (71%) items (73 quality and 202 safety) at the 'necessary' level. A number of candidate criteria were duplicates, in the sense that they were designed to determine thresholds beyond which care was judged appropriate and necessary. For example, 18 candidate quality assessment criteria related to glycated haemoglobin (HbA1c) levels beyond which treatment intensification was appropriate or necessary. Removing redundant candidate criteria yielded 52 quality and 124 safety assessment criteria to be included in the final set. Forty (77%) final quality assessment criteria and 107 (86%) final safety assessment criteria were derived from candidates accepted at the 'necessary' level. The results of the RAM study are summarised in tables 2 and 3 and the final list of quality and safety assessment criteria is presented in tables 4 and 5.

Table 2 Summary of final quality assessment criteria designed from candidates accepted in the modified RAM study as appropriate (App) or necessary to do (NecDo)
Table 3 Summary of final safety assessment criteria designed from candidates accepted in the modified RAM study as inappropriate (InApp) or necessary to avoid (NecAv)
Table 4 Quality assessment criteria generated from candidates that the RAM panel classified as 'appropriate but not necessary to do' (A) and 'appropriate and necessary to do' (N)
Table 5 Safety assessment criteria generated from candidates that the RAM panel classified as 'inappropriate' (I) or 'necessary to avoid' (N)

Table 2 shows the number of accepted quality assessment criteria categorised (1) by medical condition and (2) by four medication quality categories (MQ 1 to 4) referring to 'need (indication)', 'selection' or 'intensity' of drug treatment that were informed by available taxonomies [1315]. The majority (87%) of the final 52 quality assessment criteria focus on the prevention (including diabetes mellitus) or management of vascular disease with lower proportions addressing asthma (8%) and osteoporosis (6%). Over half (52%) of final quality criteria target patients with unmet indications for drug therapy (MQ1) and 43% focus on treatment intensity (MQ3 and MQ4) for effective disease management with the remainder (8%) targeting selection of first line agents within a therapeutic class (MQ2).

Similarly, table 3 categorises the number of accepted safety assessment criteria generated (1) by high-risk drug or patient group targeted and (2) by eight medication safety categories (MS 1 to 8), referring to 'need (indication)', 'selection', treatment 'intensity', 'compliance' issues and 'monitoring'.

The majority of safety assessment criteria are drug-focussed (74%), either targeting drugs reported to be frequently implicated in PDRM hospital admissions (54% - section A) or others implicated in severe preventable harm (20% - section B). The remainder (26% - section C) target medication use in particularly vulnerable groups, namely the elderly (15%), patients with heart failure (8%) and children (4%). Over a third (36%) of final safety assessment criteria focus on potentially harmful use of NSAIDs, antiplatelets, anticoagulants and diuretics, the drug groups most frequently implicated in PDRM hospital admissions [1].

Over half (52%) of safety criteria target the selection of high-risk drugs (MS2 to 4), either for indications where safer (and equally effective) alternatives exist (MS2) or in patients particularly susceptible to adverse reactions because of age/co-morbidity (MS3) or co-prescription (MS4). A further 15 (12%) criteria target omissions of drugs indicated to mitigate the risk of adverse events from high-risk treatments (MS1), while twenty (16%) criteria target inconsistent laboratory monitoring (MS8). Two (2%) criteria focus on prescribing that may jeopardise patient compliance with methotrexate dosing schedules (MS7).

The majority of quality (81%) and safety (71%) assessment criteria are not restricted to the elderly (patients aged 65 years or older).

Delphi study

Grouping of thematically related assessment criteria that were derived from candidates accepted at the 'necessary' level yielded a total of 47 (18 quality and 29 safety) medication improvement topics to be rated by the Delphi panel. Thirty-six Delphi study participants completed a first round and 26 (73%) a second round questionnaire (table 6). Fifteen (83%) quality and 23 (79%) safety topics were accepted as 'priorities for quality improvement in primary care'. Eleven (7 quality and 4 safety) topics were classified as 'high priorities' and nine (3 quality and 6 safety) topics were rejected because of lower than stipulated median ratings (table 7). There were no differences between pharmacists and GPs with respect to the assignment of priority levels, and no panel disagreement on any topic (defined in terms of variation in scoring as detailed in the methods section above).

Table 6 Delphi study: Demographics of the 26 panellists, who completed both rounds of ratings
Table 7 Delphi study priority ratings by the 26 panellists


This paper reports the development of a set of 176 explicit assessment criteria to identify patients at risk of PDRM from electronic data sources routinely held in UK primary care. The criteria set targets suboptimal selection, intensity or omissions of beneficial drug treatments (medication use quality) and high-risk use, inconsistent monitoring or patient instructions for drugs implicated in preventable harm (medication use safety) in primary care. All items are classified by clinical importance (appropriateness and necessity) as the output of an extended RAM process. Key professionals in UK primary care identified eleven clusters of thematically related medication assessment criteria (topics) as 'high priority' for quality improvement initiatives. The three highest rated topics related to methotrexate dosing instructions, high-risk prescribing of NSAIDs and antiplatelets and underuse of corticosteroids in asthma.

Development process of the DQIP criteria set

The RAM approach had advantages over the Delphi technique as an initial step in the criteria development process, because the face-to-face meeting ensured the necessary commitment of panellists to place ratings on an extensive and thematically broad list of candidate criteria that were grounded in the evidence base. The original RAM approach was extended in this study by introducing the concept of 'necessary to avoid', in order to distinguish between inappropriate ('not worthwhile') and 'improper' medication use in safety terms (see table 1). As for the distinction between 'appropriate' and 'necessary', panellists required examples to apply and reason the concepts, but the absence of paradoxical 'appropriateness' and 'necessity' ratings is consistent with a reliable rating process.

A limitation of consensus methods such the RAM is that ratings may depend on panel composition [42]. The chosen panel combined clinical, public health and academic expertise in primary care medication use in general, rather than specialist expertise in the management of each medical condition covered. It is possible that generalists underestimate the implications of suboptimal medication use because they do not individually see relatively rare PDRM events that have significant impact at population level. Conversely, specialists tend to overestimate the importance of practices that fall within their own specialty [47, 48]. However, since relatively few candidate criteria (22%) were rejected, it seems unlikely that including specialists would have substantially altered the results.

Scope and focus of the DQIP criteria set

Consistent with the intended use of the DQIP criteria set, our literature search targeted commonly encountered medical conditions and drug groups implicated in PDRM events in primary care rather than exclusively focussing on the elderly. As a consequence, only 27% of all developed criteria are restricted to patients over 65 years with the majority of generated assessment criteria covering aspects of medication use which are not or not exclusively relevant to the elderly [810, 49], such as primary prevention of vascular events, use of anti-diabetics in renal impairment [36] and treatments that are potentially harmful in children [36]. The fact that all topics identified as 'high priority' by the Delphi panel are age independent additionally underlines the relevance of not restricting a criteria set to be used in primary care to the elderly as is the case with many existing criteria sets [810, 49].

A limitation of the medication assessment criteria developed for this study is that several established and potentially important criteria were not considered because the study focused on those that could be applied routinely to existing UK electronic clinical data. For example, international normalised ratio (INR) results in the UK are often held in bespoke systems which hinder the implementation of meaningful measures for monitoring anticoagulant use [14]. Similarly, although a broad spectrum of medication use categories are covered, the criteria set is mainly focussed on the prescribing and monitoring stages of the medication use process with minimal coverage of patient education and compliance. In the future, the increasing sophistication of clinical information systems and the ability to link clinical datasets with laboratory systems and dispensing data would make an even broader set of assessment criteria feasible.

Although the DQIP criteria set has been developed for application in UK primary care, the drug groups reported to be implicated in PDRM events in primary care are similar internationally [5052], and we would expect the areas focused on to be relevant in other countries and health care settings. Nevertheless, some local adaptation may be required in order to account for differences in drug licensing, available resources, and clinical guidelines.

Implications for quality improvement initiatives

The Delphi approach allowed stakeholders in primary care to prioritise the chosen medication use topics for improvement initiatives in UK primary care. The Delphi panel was deliberately chosen to include both day to day prescribers (GPs do almost all primary care prescribing, especially of the more complex kind being assessed in this study, but pharmacists prescribe for some patients and conditions) and those involved in prescribing governance and improvement (predominately pharmacists but including GPs with a more strategic role). A limitation is that our focus on professionals involved in primary care prescribing meant that we did not seek to include either specialist or patient/public perspectives in the Delphi panel. Since there is evidence that practitioners' perceptions of a targeted behaviour as meaningful is a pre-requisite to changing behaviour [53] we aimed to identify medication improvement topics which met this condition to inform the design of an intervention targeting primary care professionals.

It is important to note that even those topics that were not considered to be priorities (3 quality - and 6 safety topics) contain individual criteria that were agreed to be 'necessary' to do or avoid by the RAM panel. Examples are 'inadequate dose titration of ACEI, ARBs and beta blockers in chronic heart failure', and the 'using of warfarin without a compelling indication in atrial fibrillation with low risk of stroke'. These should therefore not be neglected. Lower priority ratings nevertheless indicate that changing and improving the corresponding medication use aspects may require targeted effort (or resources) in order to influence prescribing behaviour.


The DQIP medication assessment criteria set presented here has been developed using established consensus methods and complements existing medication assessment instruments by not being limited to the elderly and by targeting a wide spectrum of medication use practices implicated in common and/or severe PDRM events in primary care. As all previously published explicit medication assessment tools, the criteria set presented here does not, however, provide comprehensive coverage of all situations that put patients at risk of PDRM, reflecting the large scope and high complexity of medication use in primary care and the limitations of current UK clinical information systems. The best choice of criteria set will therefore depend on the main purpose to be addressed and will be guided by local priorities. Informed by the priority ratings of a panel of UK primary care professionals, we have selected a subset of the DQIP criteria to serve as outcome measures in a cluster randomised trial evaluating the effectiveness of a complex intervention to improve prescribing safety (Trial registration number NCT01425502).

The DQIP criteria were primarily developed to facilitate the identification of patients at risk of PDRM from routine electronic data sets for a targeted review of their medication. However, we anticipate that they could also serve a range of other purposes, for example by informing the design of clinical decision support systems, where the classification of criteria by 'appropriateness' and 'necessity' may guide the selection of alerts that should or should not be interruptive to clinicians' workflow. Performance feedback is a further potential application, but in order not to overwhelm practitioners, the developed criteria are likely to require further prioritisation and/or the design of meaningful composites, for example by aggregating items that address the same topic [54] or medication use category [13].

An inherent limitation of explicit assessment criteria is that they cannot fully account for clinical factors that may justify deviations from what is considered to be best practice in an 'average' patient. The extent to which patients identified to be at risk of PDRM are judged by practitioners to represent actual opportunities for improvement (concurrent validity) and the extent to which any improvements in prescribing or monitoring translate into improved patient outcomes (predictive validity) therefore deserve further study.



Angiotensin converting enzyme inhibitor


Assessing care of vulnerable elders


Acute coronary syndrome


Adverse drug event

AF :

Atrial fibrillation


Angiotensin receptor blocker

BB :

Beta blocker

BP :

Blood pressure


Calcium channel blocker


Coronary heart disease


Coronary Heart Disease


Chronic heart failure


Chronic kidney disease


Chronic kidney disease


Chronic obstructive pulmonary disease




Cardiovascular disease

DM :

Diabetes mellitus


Disease modifying antirheumatic drug


Data driven quality improvement in primary care

eGFR :

Estimated glomerular filtration rate


Full blood count


Gastro-intestinal protective agents

GP :

General practitioner

HbA1c :

Glycated haemoglobin


Human immunodeficiency virus


Hormone replacement therapy




International normalised ratio

IT :

Information technology

MI :

Myocardial infarction




Non-steroidal anti-inflammatory drug (includes non-selective and COXII selective agents unless stated otherwise)


Non-selective NSAID NYHA:New York Heart Association


Preventable drug related morbidity


RAND appropriateness method


Short acting beta 2 receptor agonist

SD :

Standard deviation


Screening tool to alert doctors to right treatment


Screening tool of older person's prescriptions


Tricyclic antidepressant


Transient ischaemic attack

U&E :

Urea and electrolytes


University College of Los Angeles

UK :

United Kingdom

Z-drug :

zopiclone, zolpidem or zaleplone.


  1. Howard RL, Avery AJ, Slavenburg S, Royal S, Pipe G, Lucassen P, Pirmohamed M: Which drugs cause preventable admissions to hospital? A systematic review. British Journal of Clinical Pharmacology. 2006, 63 (2): 136-147.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Pirmohamed MJ, Meakin S, Green C, Scott AK, Walley TJ, Farrar K, Park BK, Breckenridge AM: Adverse drug reactions as a cause of admission to hospital: prospective analysis of 18 820 patients. BMJ. 2004, 329: 15-19. 10.1136/bmj.329.7456.15.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A: Systematic Review of the Incidence and Characteristics of Preventable Adverse Drug Events in Ambulatory Care. The Annals of Pharmacotherapy. 2007, 41: 1411-1426. 10.1345/aph.1H658.

    Article  PubMed  Google Scholar 

  4. Winterstein AG SB, Hepler CD, Poole C: Preventable drug related hospital admissions. Ann Pharmacother. 2002, 36: 1238-1248.

    Article  PubMed  Google Scholar 

  5. National Patient Safety Agency (2006): Improving compliance with oral methotrexate guideline. Available at

  6. Medicines and Healthcare Products Regulatory Agency CoHM: Antipsychotics: use in elderly people with dementia. Drug Saf Update. 2009, 2 (8):

  7. Howard R, Avery A, Bissell P: Causes of preventable drug-related hospital admissions: a qualitative study. Qual Saf Health Care. 2007, 17: 109-116.

    Article  Google Scholar 

  8. Fick DM, Cooper JW, Wade WE, Waller JL, Maclean JR, Beers MH: Updating the Beers Criteria for Potentially Inappropriate Medication Use in Older Adults: Results of a US Consensus Panel of Experts. Arch Intern Med. 2003, 163 (22): 2716-2724. 10.1001/archinte.163.22.2716.

    Article  PubMed  Google Scholar 

  9. Shekelle PG, MacLean CH, Morton SC, Wenger NS: Acove quality indicators. Annals of Internal Medicine. 2001, 135 (8 Pt 2): 653-667.

    CAS  PubMed  Google Scholar 

  10. Gallagher P, Ryan C, Byrne S, Kennedy J, O'Mahony D: STOPP (Screening Tool of Older Person's Prescriptions) and START (Screening Tool to Alert doctors to Right Treatment). Consensus validation. International Journal of Clinical Pharmacology & Therapeutics. 2008, 46 (2): 72-83.

    Article  CAS  Google Scholar 

  11. Instituut vor Verantwoord Medicijngebruik (IVM): Rapport. Benchmark Voorschrijven 2010 (in Dutch). Available at

  12. National Prescribing Service Limited: Indicators of Quality Prescribing in Australian General Practice (2006). A manual for users. Available at

  13. Wessell AM, Litvin C, Jenkins RG, Nietert PJ, Nemeth LS, Ornstein SM: Medication prescribing and monitoring errors in primary care: a report from the Practice Partner Research Network. Quality & Safety in Health Care. 2010, 19 (5): e21-

    CAS  Google Scholar 

  14. Strand LM, Cipolle RJ, Morley PC, Frakes MJ: The impact of Pharmaceutical care Practice on the practitioner and the patient in the ambulatory care setting: 25 years of experience. Current Pharmaceutical Design. 2004, 3987-4001.

    Google Scholar 

  15. Martirosyan L, Voorham J, Haaijer-Ruskamp FM, Braspenning J, Wolffenbuttel BH: A systematic literature review: prescribing indicators related to type 2 diabetes mellitus and cardiovascular risk management. Pharmacoepidemiol Drug Saf. 2010, 19 (4): 319-334.

    PubMed  Google Scholar 

  16. The American College of Cardiology and American Heart Association Task Force on practice guidelines and the European Society of Cardiology Committee for Practice Guidelines (Writing Committee to Revise the 2001 guidelines for the management of patients with atrial fibrillation): ACC/AHA/ESC 2006 guidelines for the management of patients with atrial fibrillation. Developed in collaboration with the European Heart Rhythm Association and the Heart Rhythm Society. Europace. 2006, 8 (9): 651-745.

    Article  Google Scholar 

  17. National Collaborating Centre for Chronic Conditions: Atrial fibrillation: National clinical guideline for management in primary and secondary care (update). National Institute for Clinical Excellence, London. 2006, Available at

    Google Scholar 

  18. National Collaborating Centre for Chronic Conditions: Type 2 Diabetes: National clinical guideline for management in primary and secondary care (update). National Institute for Clinical Excellence, London. 2008, Available at

    Google Scholar 

  19. National Institute for Health and Clinical Excellence: Alendronate, etidronate, risedronate, raloxifene, strontium ranelate and teriparatide for the secondary prevention of osteoporotic fragility fractures in postmenopausal women. NICE Technology appraisal guidance number 161. National Institute for Health and Clinical Excellence, London. 2008, Available at

    Google Scholar 

  20. Scottish Intercollegiate Guidelines Network: Acute coronary syndromes. SIGN publication number 93, Edinburgh. 2007, Available at

    Google Scholar 

  21. Scottish Intercollegiate Guidelines Network: Management of stable angina. SIGN publication number 96, Edinburgh. 2007, Available at

    Google Scholar 

  22. Scottish Intercollegiate Guidelines Network: Management of chronic heart failure. SIGN publication number 95, Edinburgh. 2007, Available at

    Google Scholar 

  23. Scottish Intercollegiate Guidelines Network: Risk estimation and the prevention of cardiovascular disease. SIGN publication number 97, Edinburgh. 2007, Available at

    Google Scholar 

  24. Scottish Intercollegiate Guidelines Network: British guideline on the management of asthma. SIGN publication number 101. Edinburgh. 2008, Available at

    Google Scholar 

  25. The Task Force for the diagnosis and treatment of acute and chronic heart failure of the European Society of Cardiology (ESC): ESC guidelines for the diagnosis and treatment of acute and chronic heart failure 2008. Developed in collaboration with the Heart Failure Association of the ESC (HFA) and endorsed by the European Society of Intensive Care Medicine (ESICM). European Journal of Heart Failure. 2008, 10 (10): 933-989. 10.1016/j.ejheart.2008.08.005.

    Article  Google Scholar 

  26. Komajda M, Drexler H: Lessons from the European heart survey. Circulation. 2006, 113 (7): f25-26.

    PubMed  Google Scholar 

  27. Anselmino M, Bartnik M, Malmberg K, Ryden L: Euro Heart Survey I: Management of coronary artery disease in patients with and without diabetes mellitus. Acute management reasonable but secondary prevention unacceptably poor: a report from the Euro Heart Survey on Diabetes and the Heart. European Journal of Cardiovascular Prevention & Rehabilitation. 2007, 14 (1): 28-36. 10.1097/01.hjr.0000199496.23838.83.

    Article  Google Scholar 

  28. Leiter LA, Betteridge DJ, Chacra AR, Chait A, Ferrannini E, Haffner SM, Kadowaki T, Tuomilehto J, Zimmet P, Newman CB, Hey-Hadavi J, Walkinshaw C: AUDIT study. Evidence of global undertreatment of dyslipidaemia in patients with type 2 diabetes mellitus. The British Journal of Diabetes & Vascular Disease. 2006, 6 (1): 31-40. 10.1177/14746514060060010401.

    Article  Google Scholar 

  29. Cleland JGF, Swedberg K, Follath F, Komajda M, Cohen-Solal A, Aguilar JC, Dietz R, Gavazzi A, Hobbs R, Korewicki J, Madeira HC, Moiseyev VS, Preda I, van Gilst WH, Widimsky J, Freemantle N, Eastaugh J, Mason J: The EuroHeart Failure survey programme--a survey on the quality of care among patients with heart failure in Europe. European Heart Journal. 2003, 24 (5): 442-463.

    Article  CAS  PubMed  Google Scholar 

  30. Nieuwlaat R, Capucci A, Camm AJ, Olsson SB, Andresen D, Davies DW, Cobbe S, Breithardt G, Le Heuzey J-Y, Prins MH, Levy S, Crijns H: Atrial fibrillation management: a prospective survey in ESC member countries: The Euro Heart Survey on Atrial Fibrillation. European Heart Journal. 2005, 26 (22): 2422-2434. 10.1093/eurheartj/ehi505.

    Article  PubMed  Google Scholar 

  31. Turner S, Thomas M, von Ziegenweidt J, Price D: Prescribing trends in asthma: a longitudinal observational study. Archives of Disease in Childhood. 2009, 94 (1): 16-22. 10.1136/adc.2008.140681.

    Article  CAS  PubMed  Google Scholar 

  32. Feldstein AC, Elmer PJ, Nichols GA, Herson M: Practice patterns in patients at risk for glucocorticoid-induced osteoporosis. Osteoporosis International. 2005, 16 (12): 2168-2174. 10.1007/s00198-005-2016-z.

    Article  CAS  PubMed  Google Scholar 

  33. Elliot-Gibson V, Bogoch ER, Jamal SA, Beaton DE: Practice patterns in the diagnosis and treatment of osteoporosis after a fragility fracture: a systematic review. Osteoporosis International. 2004, 15 (10): 767-778.

    Article  CAS  PubMed  Google Scholar 

  34. Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, Cadoret C, Fish LS, Garber L, Kelleher M, Bates DW, Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, Cadoret C, Fish LS, Garber L, Kelleher M, Bates DW: Incidence and preventability of adverse drug events among older persons in the ambulatory setting. JAMA. 2003, 289 (9): 1107-1116. 10.1001/jama.289.9.1107.

    Article  PubMed  Google Scholar 

  35. Stockley's drug interactions. [online]. Edited by: Baxter K. 2009, Pharmaceutical Press, London

  36. Joint Formulary Committee (British Medical Association and The Royal Pharmaceutical Society of Great Britain): British National Formulary. 2009, London, 57:

    Google Scholar 

  37. BMJ Clinical evidence [online]. BMJ Publishing Group Limited, London. Edited by: Minhas R. 2009, Available at

  38. National Prescribing Centre: Evidence based therapeutics. MeRec monthly. 2009, Available at

    Google Scholar 

  39. Medicines and Healthcare products Regulatory Agency (MHRA) and Commission on Human Medicines: Drug Safety Update. Available at

  40. Morris CJ, Cantrill JA: Preventing drug-related morbidity-the development of quality indicators. Journal of Clinical Pharmacy & Therapeutics. 2003, 28 (4): 295-305. 10.1046/j.1365-2710.2003.00496.x.

    Article  CAS  Google Scholar 

  41. Campbell SM, Braspenning J, Hutchinson A, Marshall MN: Research methods used in developing and applying quality indicators in primary care. BMJ. 2003, 326 (7393): 816-819. 10.1136/bmj.326.7393.816.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Shekelle PG, MacLean CH, Morton SC, Wenger NS: Assessing Care of Vulnerable Elders: Methods for Developing Quality Indicators. Annals of Internal Medicine. 2001, 135 (8 Part 2): 647-652.

    Article  CAS  PubMed  Google Scholar 

  43. Kahan JP BS, Leape LL: Measuring the necessity of medical procedures. Med Care. 1994, 32 (357): 365.

  44. Naylor CD: What is appropriate care?. N Engl J Med. 1998, 338: 1918-1920. 10.1056/NEJM199806253382612.

    Article  CAS  PubMed  Google Scholar 

  45. Shekelle PG: Are appropriateness criteria ready for use in clinical practice?. NEngl JMed. 2001, 344 (9): 677-678. 10.1056/NEJM200103013440912.

    Article  CAS  Google Scholar 

  46. Brook RH CM, Fink A: A method for the detailed assessment of the appropriateness of medical technologies. Int J Technol Assess Health Care. 1986, 2: 53-63. 10.1017/S0266462300002774.

    Article  PubMed  Google Scholar 

  47. Leape LL, Park RE, Kahan JP, Brook RH: Group judgements of appropriateness: the effect of panel composition. Quality Assurance Health Care. 1992, 4: 151-159.

    CAS  Google Scholar 

  48. Campbell SM, Cantrill JA: Consensus methods in prescribing research. Journal of Clinical Pharmacy & Therapeutics. 2001, 26 (1): 5-14. 10.1046/j.1365-2710.2001.00331.x.

    Article  CAS  Google Scholar 

  49. Morris CJ, Cantrill JA, Hepler CD, Noyce PR: Preventing drug-related morbidity-determining valid indicators. International Journal for Quality in Health Care. 2002, 14 (3): 183-198.

    Article  CAS  PubMed  Google Scholar 

  50. Bigby J, Dunn J, Goldman L, Adams JB, Jen P LC, Komaroff AL: Assessing the preventability of emergency hospital admissions. A method for evaluating the quality of medical care in a primary care facility. Am J Med. 1987, 83: 1031-1036. 10.1016/0002-9343(87)90938-7.

    Article  CAS  PubMed  Google Scholar 

  51. Chan M, Nicklason F, Vial JH: Adverse drug events as a cause of hospital admission in the elderly. Intern Med. 2001, 31: 199-205. 10.1046/j.1445-5994.2001.00044.x.

    Article  CAS  Google Scholar 

  52. Courtman BJ, Stallings SB: Characterisation of drug-related problems in elderly patients on admissions to a medical ward. Can J Hosp Pharm. 1995, 48: 161-166.

    CAS  PubMed  Google Scholar 

  53. Ajzen I: The theory of planned behaviour. Organizational behaviour and human decision processes. 1991, 50: 170-211.

    Article  Google Scholar 

  54. Guthrie B, McCowan C, Davey P, Simpson CR, Dreischulte T, Barnett K: High risk prescribing in primary care patients particularly vulnerable to adverse drug events: cross sectional population database analysis in Scottish general practice. BMJ. 2011, 342: d3514-10.1136/bmj.d3514.

    Article  PubMed  Google Scholar 

  55. Fitch K, Bernstein S, Aguilar M, Burnand B, LaCalle J, Lazaro P: The RAND/UCLA appropriateness method user's manual. RAND Corporation. 2003

    Google Scholar 

Pre-publication history

Download references


The study was funded by the Chief Scientist Office of Scottish Government Health Directorates programme grant ARPG/07/02. The authors would like to thank all participants in the RAM and Delphi panels and Debby O'Farrell for administrative support for both consensus studies.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Tobias Dreischulte.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

The study is part of the Data-driven Quality Improvement in Primary Care (DQIP) research programme, which is led by BG. TD led the literature review, conduct of RAM and Delphi studies, data analysis and wrote the first draft of the manuscript. All co-authors contributed to subsequent drafts. All authors have read and approved the final manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Dreischulte, T., Grant, A.M., McCowan, C. et al. Quality and safety of medication use in primary care: consensus validation of a new set of explicit medication assessment criteria and prioritisation of topics for improvement. BMC Clin Pharmacol 12, 5 (2012).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: