Skip to main content

Cluster Planning Support Portal

The Cluster Planning Support Portal (CPSP) indexes resources from across the system in Wales that may assist in producing and refreshing cluster/ pan-cluster planning group (PCPG) annual plans. It is structured to reflect the key elements of a planning cycle, providing an overview of considerations/ methodologies for assessment and response to local needs, linking to sources of further information, and supporting skill development. CPSP is aimed at cluster, PCPG, and professional collaborative members for awareness and oversight, and at primary care/ health board staff supporting them with integrated planning requirements.

How does this index work?

Click to expand each grouped topic heading to reveal topic-specific content, which may include in-line synopses, direct signposting links, or links to further content on sub-pages (hyperlinks to be added as topic sub-pages become available bilingually). The index will develop with the support of portal users; please provide feedback here.
 

CPSP index

Clusters should be able to access appropriate and timely expert planning advice and support, reference current planning guidance, and access suitable training/ resources to develop in-house planning capabilities.

National planning guidance
National planning guidance is available to inform cluster annual plans in the following guises:

  • NHS Wales planning frameworks: Guidance to help health boards and trusts complete their integrated medium term plans.
  • Cluster annual plan submission requirements: A plan template and supporting materials, co-developed by SPPC and PHW with input from Directors of Primary and Community Care (DPCCs) and Welsh Government, is issued separately:
  • Pan-cluster planning group (PCPG) planning requirements: There will be a Transition Year to update engagement and planning arrangements and to strengthen existing needs assessments and integrated medium term plans (IMTPs):
    • 2022-23 No guidance issued
    • 2023-24 TBC
  • Ministerial PCMW/ transformation milestones/ priorities: The Minister for Health and Social Services may issue in-year priorities by letter in the form of Primary Care Model for Wales (PCMW)/ transformation milestones, or wider priorities, with the expectation that cluster (and health board) plans reflect these:

National training and learning resources
Opportunities to develop in-house planning capability and capacity may be sought. National resources include the following:

  • NWSSP Planning Programme for Learning (PP4L): A planning academy has been established to strengthen planning skills, with good liaison between the NHS Wales planning community and Welsh Government. It encompasses a Diploma in Healthcare Planning (here), workshops, master classes, and learning events.
  • ​Planning and promoting learning resources (hosted by SCW): Insights into planning, commissioning and co-production under the Social Services and Well-being (Wales) Act 2014; toolkit supporting Regional Partnership Boards (RPBs) to devise Area Plans.
  • Health Education and Improvement Wales (HEIW): Resources to integrate and grow expertise and capability in planning, developing, shaping, and supporting the health workforce.
  • Academi Wales: A range of learning resources, courses and events and networks aimed at leaders and managers.
  • Welsh NHS Confederation: Topics (e.g. finance, population health, etc.), networks, leadership support, publications, events, and news.
  • Health service development and planning (part of the Public Health Textbook): HealthKnowledge is a ‘one stop shop’ that will provide you with all the public health learning materials, whatever your current competency.

Local planning advice and support
Advice and support to clusters may be available from your local primary care team, including via cluster development support officers (CDSOs).

Local public health advice and support
Public health advice can be wide-ranging and will typically advocate for planning focus on improving population health outcomes, prioritising prevention/ early detection of avoidable harms, and reduction of inequalities in access and health outcomes. Advice and support to clusters may be available from your local public health team:

Developing plans broadly entails understanding local population health needs and patterns of service utilisation (this section), identification of evidence-based options for improvement action (2c) and agreement of priorities for implementation (2d). Traditional health needs assessment looks at populations through the "lens" of aggregate data focussed on specific diseases (in contrast to population health management; see 2b below). It often depends largely upon routinely-collected national data derived partly from non-clinical sources, but will also incorporate local clinical data where this is available.

Assessment and planning footprints in Wales
Multiple directives require pairing of population health needs assessments (HNAs) and planning outputs across various geographic and/ or structural footprints in Wales. An overview of these is provided on the following page:

Data-centric approaches
Traditional needs assessment involves collating and presenting relevant health intelligence data available from national and health board sources. Where this is the only approach utilised, it may be termed “desk-based needs assessment”. These data can take several forms, but typically address topics/ conditions in isolation; take poor account of co-morbidities; and lack insights from linkage between datasets. Indicators relating to process/ performance or outcomes of interest can be considered as follows:

  • Epidemiological data: Describe disease characteristics by person, place and time—or other related characteristics, such as sex, ethnicity, local area deprivation, etc. Often these data will be analysed and presented as condition prevalence (perhaps by age or sex, on a specified geographic footprint and ideally with time trends) or incidence (e.g. new cases per 100,000 population, during a one year period).
  • Service activity data: Describe how people with given or grouped conditions utilise available services (e.g. telephone consultation frequency or hospital admission rates).
  • Comparative data: Where available, data from elsewhere can be juxtaposed with local prevalence/ incidence or service activity data. For example, measured prevalence may be contrasted with expected prevalence (often estimated via research literature) to explore potential gaps in case ascertainment, or suggest a locally-excessive burden of disease. Differing rates of attendance for screening interventions could indicate update barriers such poor access; differences may be noted over time as well as place or other characteristics (e.g. seasonal pressures).

Health intelligence data sources
Health intelligence in Wales is provided by a number of organisations, each typically producing a number of distinct products, each of which may present information in unique ways. Accessing such a diverse array of specialist resources can be intimidating and interpreting them challenging—particularly for those needing to do so infrequently.  

See also Population health information by topic which incorporates topic-specific signposting to data analyses relating to local needs.

Asset-informed approaches
The Welsh Government definition of prevention is broad: “working in partnership to co-produce the best outcomes possible, utilising the strengths and assets people and places have to contribute”. The Covid pandemic spawned “pop-up” health and well-being services within community assets, considered “vital provision within geographically isolated communities where deprivation and traditional barriers to inclusion require easy to access services, close to where people live” (BCT, 2020). Consider any relevant local assets or potential partner organisations that might facilitate co-production. The following local asset indexes may help identify these:

  • Dewis Cmyru: A searchable (by keyword, category, local authority, outcome, post code) database containing signposting to over 6,000 local and national well-being services.
  • Map of Community Assets in Wales: A clickable map interface from the Building Communities Trust, plotting assets grouped by category that are owned or run by the community.

Participatory approaches
Incorporating the views of stakeholders on questions of service redesign is sometimes termed a “corporate” approach. Participation and engagement may reference any rung on the “participation ladder”, typically ranging from informing (“doing for”) to co-producing (“doing with”). Co-production could position stakeholders as integral partners involved in the design, conduct and interpretation of local needs assessments. Methods for gathering views can include surveys, feedback analyses, focus groups, key informant interviews, etc. Broadly, two distinct groups may contribute views:

  • Community/ service users: Identify opportunities for co-production when engaging with local communities to ensure plans reflect their voice and input, and use language that speaks to and for them.
  • Professionals: Engage with healthcare professionals (including via ACD professional collaboratives and public health) over their views on local priorities (“normative needs”) and potential solutions.

See also ACD Toolkit which covers communication and engagement.

Vulnerable and marginalised groups
Some groups have less opportunity to participate/have their voice heard during the conduct of needs assessments or in conversations around service redesign.  While there is overlap between point-of-care focus on vulnerable and marginalised groups and the much broader social inequalities agenda, it can be helpful to clarify the approach taken as follows:

  • We measure inequalities in populations and large segments (defined groups) within populations
  • We generally take whole population approaches to reducing inequities (inequalities that are unjust and avoidable), which mostly involves addressing the wider determinants of health and action by larger institutions, such as government and national or regional organisations
  • We generally take targeted approaches to meeting the specific needs of individuals, families, and smaller communities who are identified as vulnerable or marginalised; actions could be at the patient or cluster level and tailored to address specific concerns e.g. improved access to screening services for a given disadvantage (rather than more universally)
  • For vulnerable or marginalised groups, the “gentle” slope of inequality (e.g. depicted as a plot of outcome vs. deprivation quintile) can seem more akin to a cliff edge 
  • Primary care services advocate to reconfigure services so they deliver improvements for all, but should recognise the additional effort needed for those groups with greater unmet needs (this is the concept of proportionate universalism).

For signposting to professional collaborative actions, refer to the following:

Approaches to data synthesis
Data gleaned from multiple sources will need collating and/or analyses and formatting for presentation. For example, you might triangulate findings from the international literature, local profiling and community/ professional voices into an evidence-informed narrative. You might supplement a technical tome with more accessible infographics. There are many possible ways to approach data synthesis; a helpful structure to support well-informed commissioning decisions is as follows:

Understanding variation
Quantitate data and other information may indicate variation in access to or outcomes from care, which may or may not correspond to the unmet needs of known vulnerable groups. Needs assessments should actively seek and discuss such variation:

  • Highlighting variation is not criticism; it is a natural phenomenon and can be healthy. For example, it can be a deliberate result of innovation in primary care settings that seeks to test improvements in processes or deliver better care outcomes.
  • Variation that is observed (or more precisely, measured) in a healthcare context may be referred to as inequality.
  • Inequality that is judged to be both avoidable and socially unjust is termed inequity, which is sometimes alternatively described as unwarranted variation.
  • Data profiles do not provide an explanation for what may account for this at the indicator level. There are many potential reasons for variation—both positive and negative, such as demographic make-up; geography and its effect on access to services; quality improvement activities; programme implementation characteristics; resource constraints; knowledge of best practice; availability of quality improvement expertise; strength of evidence for effective improvement intervention; sustainability-related issues; etc.
  • Those providing local primary care services are well placed to reflect and consider (within a supportive peer review environment) the relevance of potential explanations for any variation documented by local needs assessment.
  • Where variation is apparent, the first step should always be to verify the accuracy of the analysis by consulting the original data source, in case there are unintended errors. Understand that variation shown by any tool is a “best guess” at a single point in time. Variation is more fluid that a snapshot suggests, so measurement at another time point could show a different picture (including trends).

Support resources
General guidance, training and advice may support clusters with needs assessment requirements. This could include the following:

  • Approaches to the assessment of health care needs, utilisation and outcomes, and the evaluation of health and health care (part of the Public Health Textbook): HealthKnowledge is a ‘one stop shop’ that will provide you with all the public health learning materials, whatever your current competency.
  • Population assessment toolkit (hosted by SCW): The Social Services Improvement Agency has developed this Population Assessment Toolkit, alongside Welsh Local Government Association, Public Health Wales and partners, to support the Social Services and Well-being (Wales) Act. It is designed to support local authorities and local health boards with undertaking a population assessment and publishing the results.
  • Local public health advice and support (see CPSP section 1).

Population health management is an alternative “lens” to traditional needs assessment. It involves looking at the same population using patient-level data arranged into needs-based segments or clusters. It explores resource utilisation based on commonality of risk to describe care needs, facilitating optimisation of care provision and resource use in line with prudent healthcare  principles.

Population health management as an approach in Wales has been piloted in Cwm Taf Morgannwg (CTM) University Health Board and comprises the closely-linked components of segmentation, risk stratification and case-mix adjusted variation analysis. Importantly, traditional needs assessment and population health management should be seen as complementary as they address distinct planning and healthcare delivery needs; they both share the challenge of converting data into actionable intelligence.

Segmentation
Segmentation has the following features:

  • Segmenting the population based on a range of factors can identify groups by their holistic need and ability to benefit from anticipatory care.
  • Segmentation involves linkage of primary, secondary and, where available, community and social care datasets to take account of socio-demographic variables, morbidities, care utilisation (e.g. elective inpatient admissions, non-elective inpatient admissions, outpatient first & follow-up attendances, Emergency Department visits, GP practice visits and prescriptions), cost and risk factor information.
  • Segments (groups of patients) are derived on the basis of shared needs profiles.
  • Segmentation results in CTM revealed that significant and complex healthcare need was a feature across age groups and was driven more by deprivation and behavioural risk factors than by age and functional limitation.
  • To support planning improvements in population health, evidenced-based actions need to be identified and implemented on a per-segment basis. The tailoring of interventions to specific segments is considered the best way of ensuring the most effective use of healthcare resources.

Risk stratification
Risk stratification has the following features:

  • To support clinical decision-making, risk stratification of individual patients is based on the Johns Hopkins ACG model (so there are both clinical/ individual-level and population-level benefits to the approach).
  • Risk scores are compiled from various variables (e.g. age, sex, medication, disease, etc.) and can be applied to various predictive models (e.g. probability of emergency hospital admission).
  • These scores can be used at a GP practice, cluster and health board level to identify individuals or groups of patients within the highest risk groups and to enable the management and reduction of risk through targeted and anticipatory care.

Case-mix adjusted variation
Case-mix adjusted variation analyses build on the segmentation and risk stratification data and have the following features:

  • Case-mix adjusted analyses yield both crude and adjusted healthcare utilisation indices for practices, clusters and health boards.
  • Case mix adjustment allows for a true comparison between providers or areas.
  • A reporting platform allows comparison of case-mix adjusted rates with other anonymised GPs in the cluster, the cluster average, the health board average, and potentially across clusters nationally.
  • Once again, evidenced-based interventions must be identified to inform management based around anticipatory and preventive care.

Local implementations
There is no all-Wales programme approach to population health management. A proposal sponsored by Directors of Public Health to develop the approach with a small number of clusters in all health boards was made via the National Primary Care Board; this secured support in principle from Welsh Government but is yet to be funded, partly due to the Covid pandemic. For an update on current activity and to determine whether these tools are available to support needs assessment, clusters should contact their primary care and/or local public health teams.

The benefits of evidence-informed decision making are well established. Not using evidence to inform decision making (where evidence is available) risks doing the wrong thing—leading to sub-optimal or no intended outcomes, opportunity costs of wasting money and effort, and perhaps avoidable harms). Relying on common sense, expert opinion or experience can be problematic (the classic example being Dr Spock’s fatal advice to sleep infants on their fronts). Evidence is not the only influence on decision making (see section 2d).

Defining the question
The importance of articulating a clear question as a prelude to finding appropriate evidence cannot be overstated. A good question helps ensure the evidence fits the question, rather than the question morphing to fit the evidence. It:

  • Guides your search strategy (approach to selecting sources and syntax).
  • Guides whether evidence you found is applicable to the issue/ problem at hand (i.e. selection and appraisal).
  • Guides how you might approach evidence synthesis (i.e. make coherent sense of what you deem applicable).
  • Strikes a balance across the dimensions of clarity, focus and complexity.

The PICO mnemonic is a helpful device (there are other variations) for teasing out your question’s bespoke anatomy and designing your search strategy:

  • Patient, Population or Problem e.g. Age 65+ in Wales or Type 2 diabetes
  • Intervention e.g. exercise referral or lifestyle advice
  • Comparator(s) e.g. usual care or prescribed Wonderstatin
  • Outcome(s) e.g. 10% weight loss, CHD risk score

Finding evidence
Evidence may be sought on what works for keeping people well or to support responses to (illness related) needs. Evidence refers to findings from research and information from other sources that have value in helping to reach decisions. Evidence:

  • Requires bespoke judgement as to whether the type/ source is proportionate to the decision it will inform (no formula, sorry). What constitutes relevant evidence will also depend upon the question being asked (see above).
  • May be lacking, in which case there is an option to innovate (section 5) and evaluate (section 4). This differs from having evidence of no effect; in that case, don’t do it!
  • Should ideally describe what works, for whom, and in what context. This requires elucidation of factors that might influence whether an intervention that is predicted to work under ideal conditions (efficacy) will actually do so in practice (effectiveness)—including within the NHS Wales context.
  • May be characterised as a hierarchy by study design or, with a population health lens, from (generally) best to least reliable: evidence-based guidelines (e.g. NICE); systematic reviews of research (i.e. secondary sources); primary research (single studies of different designs serving different purposes); and “not research”. However, robust service evaluations can be highly valuable.

Aspire to be transparent about how evidence is sought, selected and appraised and acknowledge the limitations of your chosen approach.

Sources of evidence
Evidence may be obtained from one or more of the following source categories:

  • Summaries that integrate evidence from lower in the hierarchy and across multiple sources (see below).
  • Bibliographic databases, such as MEDLINE, which can be accessed via the NHS Wales e-library.
  • Websites belonging to organisations containing relevant policy or professional care standards e.g. RCGP.
  • Subject-specific journals, which can be searched for relevant articles online or by hand.
  • Search portals e.g. Trip or NHS Knowledge and Library Hub (both permit PICO term entry)
  • Search engines e.g. Google or Google Scholar can be skilfully integrated to improve the specificity of results; application of the CRAP test is advised (Currency; Reliability/ Relevance; Authority/ Audience; and Purpose/ Point of view).

Sources of pre-digested summaries (secondary evidence) for improvement actions include:

  • NICE Guidance: Evidence-based recommendations developed by independent committees, including professionals and lay members, and consulted on by stakeholders.
  • NICE Quality standards: Setting out priority areas for quality improvement; highlighting areas with identified variation in current practice.
  • NICE Clinical knowledge summaries: Providing primary care practitioners with a readily accessible summary of the current evidence base and practical guidance on best practice.
  • NICE Update for primary care newsletter: Subscribe to receive monthly news and guidance for GPs and primary care staff.
  • Reviews within the Cochrane Library: A collection of databases that contain different types of high-quality, independent evidence to inform healthcare decision-making.
  • BMJ Best Practice: An evidence-based generalist point of care tool, uniquely structured around the patient consultation with advice on symptom evaluation, test ordering and treatment approach.
  • PHW Observatory Evidence Service (OES): Systematic reviews, evidence maps and rapid summaries; collections cover Covid-19, health behaviours, and wider determinants of health. See also the glossary and list of sources for robust secondary evidence, as used by OES to create evidence maps.

See also Population health information by topic  which incorporates topic-specific improvement action options.

Selecting and appraising evidence
Critical appraisal is “the process of carefully and systematically examining research evidence to judge its trustworthiness, its value and relevance in a particular context” (Burls 2009). As peer review is not a guarantee of quality, some level of critical appraisal should be performed for all evidence you have selected as relevant, however published. Academics/ professional appraisers typically apply study design-specific checklists to journal articles (e.g. CASP); you can ask the following critical questions to help sift the wheat from the chaff (see also BMJ’s How to read a paper collection):

  • Is it of interest? Scan the title/ abstract.
  • Why was it done? Scan the introduction.
  • How was it done? Scan the methods section.
  • What was found? San the results section.
  • What are the implications? Scan the abstract/ discussion and think about contextualisation (Can this be adapted to fit the local/ Welsh context?).
  • Who funded it? Does the funding source/ declaration of interests suggest potential for bias?

Shared learning and expertise
The benefits of sharing knowledge and experience (which includes both successes and failures) should speak for themselves. Collaborative working is one way to facilitate this, however, we are generally poor at sharing knowledge (e.g. evaluation reports) and implementing prior learning, both within and across healthcare organisations in Wales. The same may be said of sharing with and from communities/ service users. Options to strength planning use of local experiential evidence may include:

  • Cluster yearbooks, which can be a source for discovery of what has worked elsewhere in Wales (2019; TBC)
  • Presentations/ workshops at the National Primary Care Conference
  • Ad hoc learning events
  • Networking via participation in various primary care fora (e.g. Cluster Leads Network)
  • Undertaking to evaluate and publish on innovative approaches (see section 5)
  • Actively seeking partnership opportunities (see asset-informed approaches, section 2a)
  • Service user engagement/ co-production (see participatory approaches, section 2a)
  • Implement learning from the Pacesetter Programme (see section 5).

A simple template to support wider sharing of learning from elsewhere in Wales could be framed around the following questions:

  • What problem was being addressed?
  • What was done to address it?
  • How does this evidence good practice?
  • What key learning can be shared?
  • Who did it or who can be contacted in the event of queries?

Supporting resources
Development opportunities for cluster teams/ supporting staff may include:

  • Becoming familiar with the “evidence hierarchy” and considering the fitness-for-purpose of evidence sources/ synthesis methods.
  • Being able to use Google (or other search tools) more effectively.
  • Learning about what an OpenAthens account from the NHS Wales e-Library offers (authenticated access to electronic resources).
  • Local public health teams (while still part of PHW) have access to suite of evidence guides (intranet only), for use by PHW staff only. These cover asking the question; finding the evidence; evidence reviewing; critical appraisal; and evidence into action. However, “Under no circumstances should they be replicated or shared with external organisations unless this has first been discussed with the Evidence Service.”

Agreeing priorities for implementation should reflect the primacy of prevention (to reduce the avoidable burden of disease in Wales) and take account of various strategic directions to facilitate planning alignment, such that joint action produces measurable improvements in population health.

Key strategic direction
Alignment of cluster activities to key strategy documents will help ensure the priorities of other local agencies and partnerships (e.g. Public Service Boards) and national bodies (e.g. Public Health Wales) influence planned cluster actions—in addition to priorities identified locally. Key strategic direction can be found within the following:

  • Well-being of Future Generations (Wales) Act 2015: Legislation that aims to improve the social, economic, environmental and cultural well-being of Wales.
  • A healthier Wales: Published in response to the Parliamentary Review of Health and Social Care in Wales (here), it sets out the long-term vision of a whole system approach to health and social care through a model is focussed on health, well-being, and preventing illness. It also sets of the Quadruple Aim of delivering improved population health and well-being; better quality and more accessible health and social care services; higher value health and social care; and a motivated and sustainable health and social care workforce.
  • Programme for government: Incorporates the Welsh Government commitment to deliver the well-being objective to provide effective, high quality and sustainable healthcare.
  • Ministerial priorities: For 2022-23 these are Covid-19 response; NHS recovery; Working alongside social care; A Healthier Wales; NHS finance and managing within resources; Mental health and emotional well-being; Supporting the health and care workforce; and Population health
  • Strategic Programme for Primary Care (SPPC): Key programmes for 2022-23 are Accelerated Cluster Development; Urgent Primary Care; Community Infrastructure; and Mental Wellbeing.
  • National Clinical Framework: Describes how clinical services must be planned using whole-system, whole lifespan pathways with nationally agreed outcome/ experience measures and standards, but locally delivered according to population and workforce characteristics.
  • Our strategy to 2024 (Welsh Value in Health Centre): Enabling a whole system approach to value-based healthcare for Wales.
  • Regional Partnership Board (RPB) area plans: Each RPB (listed here) produces a regional area plan.
  • Public Service Boards (PSB) well-being plans: Each PSB (indexed here) produces an annual local well-being plan.
  • Local health board (LHB) annual plan/ IMTP: Each LHB (listed here) produces and annual or integrated medium term plan (IMTP) in accordance with the annual NHW Wales planning framework.
  • Public Health Wales (PHW) annual plan/ IMTP: As the national public health institute for Wales, PHW also produce strategic plans that identify priorities for population health improvement.

See also Population health information by topic  which incorporates topic-specific strategic context.

Types of decision making
The Stacey matrix is a means to understand different types of decision making; this is described further in Resources to help you develop your cluster (part of Cluster working in Wales):

  • Rational decisions result from the presence of high levels of both agreement and certainty
  • Political decisions are characterised by high levels of certainty, but with lower levels of agreement over the best course of action
  • Judgemental decisions are characterised by high levels of agreement in the presence of less certainty around the best course of action.

Political decisions can be influenced by factors including:

  • Technical/ professional advice, based on evidence (which will ideally indicate a high return on investment, and/ or value-for-money relative to potential alternatives i.e. an acceptable opportunity cost)
  • Personal beliefs
  • Perceptions about public/ service user opinion
  • Party/ organisational politics (and local and national levels).

Developing a business case
The combination of data describing unmet needs, evidence for remedial action, alignment to existing strategic direction, and proposal for prioritisation may be captured within an outline business case to support formal decision making. The business case will ordinarily be revisited in preparation for implementation (see section 3) and evolve as needed in response to influences on it—which could even invalidate it.

Other prioritisation resources
The following additional resources may support cluster decision making:

  • Cluster decision making: Cluster Governance: A Guide to Good Practice covers principles, systems and processes for cluster decision making (Appendix 5). See also samples of a decision-making framework, prioritisation framework, and decision tracker.
  • Options appraisal: Although in reference to cluster models, the generic principles of options appraisal as outlined in Cluster Governance: A Guide to Good Practice Appendix 9) can be applied to other types of decision, including prioritisation.
  • Working in Wales: Summaries of key policy and strategies influencing health and well-being in Wales (part of Cluster working in Wales).
  • Themes from the Clinical Governance Practice Self-Assessment Tool (CGPSAT) and Welsh Information Governance Toolkit reviews may also provide additional intelligence for wider service planning, ensuring that learning from incidents and concerns informs service development.

Having decided on a course of action—perhaps informed by an outline business case—the case for change may need refining/ updating to reflect more detailed operational planning considerations as these emerge. Project plans will need clearly-articulated ambitions, ongoing active management, identification of funding and workforce requirements, and sense-checking against population health thinking.

SMART outcomes and logic models
Outcomes describe what we are trying to achieve (for service users or others) by doing the activity; they are not the same as outputs—the things we produce (such as a report) in the process of carrying out the activity. It may help to think of outcomes as aims, and of outputs as project objectives or products:

  • Outcomes should ideally be SMART (specific; measurable; achievable; realistic; time-bound). In addition, don’t forget to describe the desirable patient/ service user experience.
  • Logic models (see section 4) can help sense-check the elements that must come together to successfully plan, deliver and evaluate a project—providing clarity on intended outcomes.

Project management
Project management can be defined as “the discipline of applying specific processes and principles to initiate, plan, execute and manage the way that new initiatives or changes are implemented within an organization” (Axelos). A project is a “temporary venture that exists to produce a defined outcome” (Axelos) that requires initiation, planning, execution/ delivery, monitoring (see also section 4), and closing down. Various methodologies and templates exist to support each of these stages:

  • Cluster project planning: Cluster Governance: A Guide to Good Practice covers starting up; business cases; project boards/ steering groups; project manager role; project initiation; expertise recruitment; project plans; recording and monitoring; reporting and accountability; and evaluation (Appendix 12).
  • The Cluster Governance: A Guide to Good Practice also provides templates for project proposal; goal-directed plans; action plans; risk management; highlight reporting; general reporting (SBAR); and for post-project review and summary.
  • Resources to help you develop your cluster: Includes a collection of project management resources (part of Cluster working in Wales)
  • PRINCE2® glossary of terms: Project management terminology from Axelos, who provide PRINCE2 certification.
  • See also “Developing a business case” in section 2d above (a business case is part of the project management toolkit).

See also ACD Toolkit  which provides project management templates.

Funding sources
Funding available to support cluster-based initiatives may come from a variety of sources, encompassing one or more of the following:

  • Cluster funding
  • Prescribing incentive scheme
  • Integrated Care Fund (ICF)
  • Transformation Fund
  • Mainstream (core health board) funding
  • Strategic Programme for Primary Care Fund (replaces Pacesetter funding from April 2022)
  • Research funding (e.g. via Health and Care Research Wales; PRIME Centre Wales; NIHR to support robust service evaluation/ academic publication)

Financial stipulations and support
Clusters will need to be mindful of the need to comply with legal requirements and demonstrate due diligence with regard to procurement processes:

Workforce planning and support
Consult the following guidance, templates, and workforce sustainability tools:

  • Primary care cluster workforce planning (HEIW): The step-by-step approach to workforce planning in primary care provides a simplified methodology for practices and clusters to use to create their workforce plans.
  • Primary care roles in Wales (PHW): Signposting links to various resources, including the Compendium of emerging roles and models in primary care (here).
  • Primary care clusters: staff recruitment and training: Cluster Governance: A Guide to Good Practice covers values and behaviours; recruitment to cluster teams; workforce planning; education and training of cluster staff; and professional support and supervision (Chapter 11).
  • The Cluster Governance: A Guide to Good Practice provides sample job descriptions for cluster lead, practice manager, cluster co-ordinator, and project support officer; it also provides a protocol for staff employed by the local health board working in GP practices and a template for a knowledge, skills and training survey.
  • Resources to help you develop your cluster: Includes a section on workforce planning (part of Cluster working in Wales)
  • Primary and Community Care Allied Health Professions (AHP) Workforce Guidance: Organising principles to optimise utilisation: This paper by the Strategic Programme for Primary Care is a call to action for the whole health and social care system to implement the recommended organising principles required to optimise utilisation of the AHP workforce in primary and community care. Also available in short read format.
  • All Wales Locum Register: The first point of contact for practices to identify Locum GPs registered on the All Wales Locum Register, Medical Performers List and ensuring that locum GPs benefit from the Scheme for General Medical Practice Indemnity.
  • Wales National Workforce Reporting System (WNWRS): A primary care workforce tool providing necessary identification of all GPs and health professionals employed in GP practices to be covered by General Medical Practice Indemnity and enabling greater workforce planning.
  • GP Wales: A single point to advertise and monitor permanent and locum GP vacancies (incorporating Locum Hub Wales).

Population health perspectives
The PACE checklist reflects the values that public health specialists aspire to bring to conversations about how services could be reconfigured to best effect. It doesn’t have to be applied systematically, but could serve as a mental prompt to help clusters sense-check emerging plans against population health thinking:

  • PACE: A population health perspective checklist for cluster planning

Monitoring refers to setting targets and milestones to measure progress and achievement, and check whether the inputs are producing the planned outputs i.e. it determines whether implementation is proving consistent with design intent—implying we can tweak our approach during the monitoring period. Evaluation is not just about demonstrating eventual success; it also provides insights into why things don’t work (as learning from mistakes has equal value). Monitoring and evaluation are not about finding out about everything (which is intimidating), but are focused on the things that matter.

Project monitoring
In a generic project management context, monitoring involves “oversee[ing] the progress of project work and updat[ing] the project plans to reflect actual performance (Axelos). In the Welsh healthcare context, the Quality and Safety Framework (WG; 2021) describes a universal duty of “quality management” to ensure that care meets the six domains of quality (care that is safe, effective, patient-centred, timely, efficient and equitable). It describes a system that continuously connects quality assurance, planning and improvement activity. Periodic measuring and monitoring permits:

  • Assurance of implementation progress, in keeping with delivery expectations around scale and pace
  • A mechanism to capture and share emerging learning at local level (especially where a contemporaneous lessons log is maintained) and on a regional or national basis (typically via interim reports), thus deriving maximal value from early and ongoing implementation experience
  • Ensuring projects get the resources they need for successful delivery (it may identify additional support requirements/ reconfigurations to address any planning gaps)
  • Recording and management (ownership, mitigations, etc.)  of issues and risks—whether anticipated or emergent
  • Remedial course correction, or unscheduled project termination in the face of insurmountable risks, actual harms, or resource constraints etc. that depreciate the business case.

Project evaluation
Evaluation refers to the structured process of assessing the success of a project or programme in meeting its aims and for reflecting on the lessons learned. The key difference between monitoring and evaluation is that evaluation places a value judgement on the information gathered during a project (Research Councils UK; 2011), including the monitoring data. The assessment of a project’s success (its evaluation) can be different depending upon whose value judgement is used. Evaluation permits:

  • Assessment of whether a project has achieved its intended goals
  • Understanding how the project has achieved its intended purpose, or why it may not have done so
  • Identifying how efficient the project was in converting resources (funded and in-kind) into activities, outputs (objectives) and outcomes (goals or aims)
  • Assessment of how sustainable and meaningful the project was for participants
  • Informing decision makers about next steps.

Service evaluations may evolve into research proposals (perhaps aiming to resolve unanswered questions) or lead to review of the existing business case (see section 3), resulting in a decision to scale up a successful, innovative project (see section 5), continue as-is or with improvements, or to stop the project. Evaluation is:

  • Often falsely viewed only in terms of a visible endpoint product (such as an evaluation report), but is more robust when it is implemented as a “before, during and after” activity that runs alongside the project itself.
  • Best planned prospectively, with contemporaneous data collection (i.e. monitoring) during implementation for summative assessment of pre-defined outcomes; retrospective evaluations can still be worthwhile, but are subject to additional bias.
  • Best carried out or overseen by someone from outside the project (again to reduce bias) and with representative contributions (e.g. with both service provider and user participation).
  • Enhanced by inclusion of both quantitative (numbers e.g. costs) and qualitative (narrative or “lived experience” e.g. via interviews) data.
  • Aided by the use of logic model and/ or evaluation plan templates (see below).

Logic models
Logic models can help sense-check the elements that must come together to successfully plan, deliver and evaluate a project. They can be integrated into project plans from the outset, or inform a bespoke monitoring and evaluation plan by teasing out the following:

  • Inputs: The key things we need to invest/ have in place to support the activity
  • Activities: What we do with the inputs
  • Outputs: What we produce as a result of the activities
  • Outcomes: What our products will achieve for people or services (aims; these can vary over time e.g. short, medium or long-term and should be SMART)
  • Impacts: High-level, ultimate ambitions e.g. the quadruple aims of A healthier Wales
  • Barriers: What we may find difficult to influence or overcome (e.g. external factors)
  • Assumptions: What we hope is already in place (supportive conditions, etc.)

A logic model tries to establish sequential links between the above elements, in multi-row table or diagram form. Sometimes they are easier to populate right-to-left, instead of left-to-right (starting with inputs). For background information on logic models, refer to the following resources:

  • Logic models (Data Cymru): Using logic models to plan, map and identify the activities and inputs that lead to results, and to understand desired changes and who would be accountable for them.
  • Using logic models in evaluation (The Strategy Unit): This briefing has been prepared for NHS England, by the Strategy Unit, as part of a programme of training to support national and locally-based evaluation of the Vanguard programme and sites.

For a simple logic model template, see “Additional support resources” (below).

Evaluation plans
There is no magic formula for developing a universal evaluation plan. If evaluation was something of an afterthought (it happens!), a reflective and inclusive post-project review can recover some value by asking “What went well? What went less well? How would we do it differently next time?” A simple prospective evaluation plan might ask the following:

  • What do we want to know? The evaluation questions(s); the “things that matter”
  • How will we know it? The indications of success (or harm) we will use
  • How will we collect indicator data? The data source(s) and analysis method
  • When/ where will data be collected? Timeframes and tools e.g. point-of-care
  • Who will do this? Monitoring and evaluation roles and responsibilities

For a simple evaluation plan template, see “Additional support resources” (below); make sure to address each evaluation question on its own row. The Cluster Governance: A Guide to Good Practice offers examples of real-world cluster evaluations to learn from: audiology advanced practitioners; treatment unit; Mind in the Vale of Glamorgan evaluation/ therapies; and care home ANP innovation/ evaluation.

PCMW/ ACD monitoring and evaluation plan
The Primary Care Model for Wales (PCMW) and Accelerated Cluster Development (ACD) Programme implementation monitoring and evaluation plan sets out how these transformation ambitions will provide assurance of progress, shared learning, and support joining up of local and regional plans. It describes the step-wise introduction of several supporting tools and products:

  • Key indicator dashboard: a live tile on the Primary Care Information Portal for reporting metrics around PCMW, ACD and other primary care outcomes
  • Self-reflection tool: an annual online questionnaire asking clusters what went well, less well or could be done differently
  • Cluster Development Framework: sets out standards and maturity criteria expected for demonstrating implementation progress
  • Peer review process: describes how clusters and regional partnership boards will be involved in developmental appraisal once per IMTP cycle
  • National implementation progress report: an annual interim (monitoring) report that summarises progress and key learning to date
  • Contribution analysis: a method for approaching endpoint evaluation that is suited to understanding complexity.

See also ACD Toolkit  which details the PCMW/ACD monitoring and evaluation plan.

Additional support resources
The following resources provide further background on defining monitoring and evaluation requirements:

  • Cluster project planning: Cluster Governance: A Guide to Good Practice covers recording and monitoring, and evaluation (Appendix 12).
  • Resources to help you develop your cluster: Includes a section on evaluation (part of Cluster working in Wales)
  • Logic models in evaluation: Cluster Governance: A Guide to Good Practice covers logic model components, discussion of evaluation types and provides logic model and evaluation plan templates.
  • Introductory guide to evaluation (Data Cymru): This guide will support your understanding of what evaluation is and why it is important for your projects, programmes and policies; give you the 'basics', so you understand why and when you might undertake evaluation; provide you with direction as to the approaches and processes you might use to undertake effective evaluation; and provide pointers to further guidance and support.
  • The Magenta Book: Guidance for evaluation (HM Treasury): The Magenta Book is the recommended central government guidance on evaluation that sets out best practice for departments to follow; recommended by Data Cymru as the “go to” evaluation resource.
  • Monitoring and evaluation expertise is a scare resource; it may be available via local public health teams, or from an academic partner (who can add rigour and aid dissemination of learning, typically in return for data access).

Clusters can play a key role in developing novel approaches to address local challenges, identifying successful projects for upscaling/ mainstreaming, and adapting or implementing prior learning from across Wales.

Developing innovative ideas                                                                           
Innovation involves development of “new or improved health policies, systems, products and technologies, and services and delivery methods that improve people's health, with a special focus on the needs of vulnerable populations” (WHO, 2016). The following resources offer insights into getting started and pitching cluster ideas in supportive environments:

  • Innovation Programme for Health and Social Care (WG): Details TBC; draft logic model published for Innovation Strategy for Wales, Dec 2021 (here) aspires to deliver increased adoption of innovation; improved patient/ citizen outcomes; improved patient/ citizen experience; and improved resource efficiency.
  • Caring to change: how compassionate leadership can stimulate innovation in health care (The King’s Fund, 2017): This paper looks at compassion—which involves attending, understanding, empathising and helping—as a core cultural value of the NHS and how compassionate leadership results in a working environment that encourages people to find new and improved ways of doing things.
  • Open innovation in health: A guide to transforming healthcare through collaboration (NESTA, 2017): This guide explores examples of open innovation in the field of health from around the world. It analyses the ways that companies, governments, researchers and citizens are collaborating to improve the innovation process, from the way that problems are identified to how new products and services are created and then adopted by providers of healthcare.
  • Crafting an elevator pitch (Mind Tools): An elevator pitch is a brief, persuasive speech that you use to spark interest in what your organization does. You can also use it to create interest in a project, idea or product—or in yourself.
  • Creating a value proposition (Mind Tools): Communicating the benefits of your proposition simply and clearly.

Driver diagrams to support cluster innovation
Driver diagrams offer a tool to assist planning of improvement projects. They can:

  • Provide clarity and structure for clinical teams, whose focus is operational delivery
  • Be used to translate improvement goals/aims into a logical set of high level factors (primary drivers) that need to be influenced in order to achieve the agreed outcomes
  • Detail the specific projects/activities that are needed to act on the higher level factors.

This structured approach assists the allocation of tasks to individuals or groups and provides an estimate of the skills and capacity to deliver the agreed actions. This also encourages the prioritisation of objectives where there are multiple competing expectations.

Primary and community services are complex and it can be challenging to deliver innovation in these settings. Driver diagrams can be used to gain clinical engagement (by communicating the project in logical sequence and with defined tasks) and to clarify what can reasonably be expected within the objectives of a small cluster team. Health boards may also develop innovation teams with skills and capacity to enhance local teams for identified priorities.

Tasks that cannot be accommodated should be added to local risk registers to provide a clear analysis of the unaddressed potential for improvement.

Professional collaboratives should be encouraged to generate improvement proposals as independent groups and across system boundaries. Pan-cluster planning groups (PCPGs) should establish systems to receive and consider these submissions, ensuring that improvement efforts are addressed to the agreed local priorities. An evaluation should be integral to all proposals and learning should be shared. A schedule of current projects should be maintained to monitor progress and ensure that cycles of change are completed.

Example driver diagrams include:

  • IHC driver diagram examples (multiple), here
  • Safe and reliable patient care, here
  • Quality improvement work around COVID-19, here

Upscaling from pilot projects
Pilot projects serve to differentiate that which works from that which does not. Taking things that do work on a small scale to a larger scale (e.g. health board or all-Wales footprint) can be challenging; advice is contained with the following resources:

Learning from the Pacesetter Programme
Learning captured by the National Primary Care Pacesetter Programme critical appraisal (University of Birmingham, 2018) identified six transformation enablers. These enablers are recognised as key to successful transformation of health systems, both in the UK and internationally. Note that Pacesetters are superseded by the Strategic Programme for Primary Care Fund from April 2022:

  • Facilitating: External facilitation is made available to general practices to provide additional capacity and expertise in undertaking transformation.
  • Leading: Clinical and non-clinical leaders for the programme are identified within the practices and if relevant in local primary care networks and are given the time, support and space to reflect on the transformation process.
  • Learning: Learning and development in relation to new skills is available, and there is opportunity to learn from the implementation process through structured reflection on emerging evidence.
  • Engaging: Stakeholder engagement with patients, communities and wider clinical networks is embedded throughout with sufficient investment in associated infrastructure, capacity and skills.
  • Funding: Transitional funding to enable continuation of existing activities whilst new approaches are introduced and free up capacity for clinical and non-clinical leaders.
  • Evaluating: Robust evaluation to provide formative and summative insights against clear objectives and baselines.