Original Research | Published: 13 June 2024
open-url

Using systems mapping within the process evaluation of a randomised controlled trial of the ACE active ageing programme in England and Wales

https://doi.org/10.1136/bmjph-2023-000229

Request reuse permissionopen-url
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) licenseopen-url

Abstract

Background System mapping has mainly been used to develop theories and understanding of complex systems; to hypothesise how an intervention might work in a complex system or to inform intervention development. There are a few examples of the use of system mapping as part of process evaluation. In this paper, we describe an innovative approach to using system mapping as part of the process evaluation of a randomised controlled trial of the Active, Connected, Engaged (ACE) community-based active ageing programme.

Method Ten participatory workshops were held across three of the ACE sites (Cardiff, Stoke-on-Trent and Manchester, UK). These involved over 100 participants, volunteers and stakeholders (from National Health Service, statutory and voluntary sectors). Their aim was to gather area-specific information on participants’ barriers and facilitators to physical activity and the needs of peer volunteers and service providers; and create ‘baseline’ system maps before the launch of the programme in the three areas of ACE delivery.

Results System maps were produced showing the main outcome (physical activity) and the interactions between the key motivators and barriers described by older people, as well as ideas from stakeholders and volunteers about how these barriers can be addressed. Findings led to refinements to ACE intervention processes and the study’s logic model.

Conclusions System mapping helped to refine the ACE processes and fine-tune the logic model. The value of this approach will increase in the next phase when it will be used to explore any changes to the physical activity system including changes to stakeholders’ ways of working and collaborating to tackle barriers to activity following the completion of the ACE trial.

Trial registration number ISRCTN17660493.

What is already known on this topic

  • There are multiple motivators and barriers that influence older people’s participation in health-enhancing physical activity, but these are rarely considered as part of a wider physical activity system.

What this study adds

  • System mapping helped to capture the complex nature of the physical activity system and the connections between key motivators, barriers and potential intervention strategies. This contributed to the intervention’s process evaluation.

How this study might affect research, practice or policy

  • The results have already been fed directly into the design of the Active, Connected, Engaged randomised controlled trial, and the approach could be used as part of any process evaluation.

Introduction

In recent years, there has been an increasing interest in the application of systems thinking to exploring, understanding and acting on complex public health issues.1 The interest in systems thinking arises from the growing body of evidence acknowledging the multiple, systemic and complex causes of many public health issues, and that to address them, actions are required at both the individual and societal level.2 A complex system is best considered as a collection of elements (eg, subsystems, sectors), with interconnections between those elements.2 A systems approach to public health conceptualises poor health and health inequalities as outcomes of a multitude of interdependent elements within a connected whole.3 By developing an understanding of these interdependencies, we can develop interventions or approaches that are more sensitive to the complexities of human behaviour. This is consistent with guidance from the recent MRC framework for developing and evaluating complex interventions.1

System mapping has emerged as one of the first steps to exploring the nature of any complex system. The most prominent example of this approach in the UK was the Foresight report, published in 2007.4 This emphasised the complex nature of obesity and argued for it to be addressed using systems approaches, with accompanying system maps. These ‘maps’ are diagrams that show the factors that influence an outcome (in this case, obesity) and how they are—or might be—connected. Guidance from Public Health England now recommends system mapping as part of any approach to exploring and intervening in complex systems.5

A recent scoping review of systems approaches for increasing physical activity in populations found that system mapping has been applied to physical activity in three principal ways.6 First, system maps have been used as a theorising tool, identifying and comparing stakeholder understanding of a complex system. For example, in Derby UK, system maps were developed to help stakeholders explore the influences on physical activity in order to plan and modify an existing town-wide physical activity programme with many complex and interacting components.7 Second, system maps—predominantly using system dynamic modelling (SDM)—have been used to hypothesise and simulate how an intervention might impact on and interact in a complex system.8 For example, SDM was used to compare the effects of policies to increase bicycle commuting, based on participatory methods including interviews and workshops with policy, community and academic stakeholders.9 Third, system maps developed through consensus-based methods such as participatory system mapping or group model-building have been used to inform intervention development. For example, in the UK, participatory systems mapping was used with stakeholders to produce a theoretical framework and intervention plan for the Healthy New Town Programme.10

A further application of system mapping that has been used less frequently is as part of a process evaluation of an intervention or programme. The scoping review cited above6 found 155 examples of systems approaches, of which 11 were part of process evaluations. However, only one of these used a version of system mapping (using an ‘Action Mapping Tool’ to systematically record information related to local authority actions on obesity).11 In this paper, we describe an innovative approach to using system mapping to explore the wider factors that might influence the implementation or effectiveness of an intervention delivered in multiple sites. We will explore the use of system mapping as part of the ongoing process evaluation of a community-based active ageing intervention: the Active, Connected, Engaged (ACE) randomised controlled trial.12 In this paper, we will refer to the trial by the acronym ‘ACE’ although it is known as ‘ACTIF’ in Wales to avoid any confusion with a Welsh national programme called ACE that focuses on adverse childhood experiences.

Background to the ACE trial

The ACE trial aims to evaluate the effectiveness and cost-effectiveness of the ACE intervention, a community-based active ageing programme designed to improve physical function. ACE is a low-cost programme in which older volunteers (55 years+) support older people (65 years+) to improve their mobility by becoming more active within their communities. During the ACE programme, volunteers (n=150) meet with participants twice to get to know each other, find out about and discuss local community-based activities that the participant would like to join and identify and address any barriers to taking part. Then, over a 3-month period, the volunteer–participant pair attend at least three local activities chosen by the participant, together. Over the following 3 months, volunteers support the participant to continue attending these activities independently through regular phone calls, with further joint visits to activities scheduled if needed. Outcome measures include lower limb function; detailed analysis of objectively measured physical activity; muscle-strengthening exercise and a wide range of psychological and social outcomes.12 This is accompanied by an economic evaluation to test cost-effectiveness. ACE was based on the findings of an earlier feasibility study13–15 which led to the development of this definitive effectiveness and cost-effectiveness study, targeting diverse and deprived communities. These communities were identified by demographic data and deprivation index scores. The deprivation index is a measure of relative deprivation at a small local area level produced by the UK Government.16 It is based on seven different domains of deprivation including: income; employment; education; health and disability; crime; housing and living environment. This aims to capture a wide range of aspects of an individual’s living conditions. Based on these criteria, we selected four diverse areas in terms of deprivation including Stoke-on-Trent, Manchester and Bristol in England and Cardiff in Wales. This diversity increases the complexity of the trial and underscores the need for a comprehensive process evaluation to explore the workings of the intervention and appreciate the breadth of contextual factors influencing its effectiveness.

The process evaluation runs alongside the main trial and aims to explore the actual operation of the trial; to understand the mechanisms of change; and to test and, if necessary, refine the trial’s logic model.17 The process evaluation includes the following components (see online supplemental material file for the timing of each component):

  • Longitudinal, semistructured participant interviews to track the experiences of participants throughout the study.

  • Semistructured interviews to explore the experiences of peer volunteers and provider organisations.

  • Assessment of the quality of delivery of the ACE intervention by peer-volunteers, through audio-recordings of a purposive sample of participant-volunteer consultation meetings.

  • Quantitative data collection among participants and peer-volunteers to explore possible mechanisms of change of the ACE intervention.

  • System-level process evaluation using a system mapping protocol (the focus of this paper) before the launch of the ACE intervention and repeated after its completion.

System mapping

The ACE intervention has to operate within a complex system with many interacting factors that influence whether or not an individual may be physically active. These operate at a variety of levels, from individual factors such as mobility limitations; social factors such as social isolation and community norms; and environmental factors such as the existence of safe places to be active. Alongside these factors are a multitude of ‘stakeholders’: individuals and agencies who have an interest in helping older people to be active and improve their health and well-being. This part of the process evaluation sets out to explore the system around physical activity and volunteering and place the ACE intervention within this system. We developed the system maps based on primary participatory research with participants at the ACE delivery sites, as it was essential to explore locality-specific issues that might have been missed had we relied on the findings of existing reviews without appreciation for local characteristics. The system mapping component of the process evaluation had the following objectives:

  1. To gather area-specific information on participants’ barriers and facilitators to physical activity and the needs of peer volunteers and service providers prior to the start of the intervention. This can be seen as ‘formative evaluation’: using system mapping to propose refinements to the intervention’s design and delivery and the overall logic model for the intervention, which describes the assumptions and processes by which ACE aims to influence short-term, medium-term and long-term outcomes (see online supplemental material file).

  2. To create ‘baseline’ system maps before the intervention takes place in three ACE delivery sites. This process will be repeated after the completion of the intervention in each area in order to provide a system-level context for the longitudinal process and impact evaluations, helping to identify individual-level and systems-level factors that might influence the effectiveness of the ACE intervention.

Methods

10 system mapping workshops were held across three ACE delivery sites (Cardiff, Stoke-on-Trent and Manchester) to develop the system maps. Workshops were chosen instead of individual interviews in order to build on existing relationships among community groups and encourage conversation. Workshops were held in late 2021/early 2022 in three phases:

Phase 1: participants

These were attended by people meeting the eligibility criteria for taking part in the ACE study. They were community-dwelling, older adults aged 65 years and older, at risk of mobility disability due to reduced lower limb physical functioning.17 One face-to-face workshop was held at each site. Participants were recruited through community organisations (eg, health hubs, libraries, independent living services) and social groups (eg, ‘Men in Sheds’; lunch clubs; ‘knit and natter’ meetings). We aimed to recruit a socioeconomically and ethnically diverse sample across the study. There is a strong Chinese community in one of the study areas (Central Manchester) and a request was made for specific sessions with an interpreter. Two additional sessions were, therefore, held with interested representatives of the Chinese community in Manchester; one face-to-face and one online, as some participants were not comfortable meeting face-to-face in a large group due to the perceived risk of COVID-19. These sessions were separate from those outlined above due to the need for a language interpreter.

Most workshops were facilitated by the first author (NC) who is experienced in system mapping and qualitative research methods. The Manchester workshops with Chinese community members were facilitated by two ACE researchers (HH-H and AS) accompanied by an interpreter. Each workshop was approximately 2 hours long and held in an appropriate and convenient community location (eg, church hall; community centre; pub). Informed consent was obtained from all participants, and participants were given a £20 shopping voucher to compensate for their time. The facilitated discussion included the following general topics:

  • What does the term ‘physical activity’ mean to you?

  • What things encourage or help you to be active?

  • What things discourage or stop you being as active as you would like?

Participants were encouraged to think about personal level motivators and barriers, but also to consider system-level and contextual factors, such as lack of transport or access to leisure facilities. Participants were encouraged to write their thoughts concisely on post-it notes, that were then collected by the facilitator. These were stuck on a large sheet of paper and the group encouraged to discuss these factors and how they were connected. For example, if someone said that friends helped them to be motivated, then a line was drawn between ‘friends’ and ‘motivation’. The visual prompts from the emerging system map were then used to stimulate further participatory discussion, with care taken to elicit input from all participants. At the end of the session, the group was asked to comment on whether the draft system map summarised the barriers and enablers influencing their physical activity participation. The final output was thus a draft system map co-created by the participants and the researchers. The workshops were audio-recorded and subsequently transcribed by either a researcher or a professional university-approved transcriber. Researchers also took notes during the workshops.

Following the workshop, the digital version of the system map was drafted based on the content of the post-it notes maps created in the workshop, and thematic analysis of the transcripts. We employed an inductive coding approach18 and analysed the data through a critical realism epistemological perspective lens.19 Primary themes for motivations to be active (such as ‘independence’ or ‘being sociable’) were then expanded into secondary, more nuanced themes (such as the difference between ‘to look after my general health’ and ‘to prevent disease’.) A similar analytical process was undertaken for barriers, although here it was found to be helpful to group the barriers according to the issue they might affect. For example, the barrier ‘weather’ would be connected to the motivator ‘getting out and about’. In this way, the system map can be used to consider the relationships between all the issues that might influence participation—both positive and negative—and plan strategies accordingly. The themes were then transferred to a system map using Kumu system mapping software (www.kumu.io). On the maps a solid line shows where something may be supportive (such as ‘walking the dog’ helping people to ‘get out and about’) while a dashed line shows that an increase in one factor may be likely to lead to a decrease in another (such as ‘feeling lonely or depressed’ being likely to lead to a reduction in ‘keeping motivated’.) The draft system maps were checked by the researcher who was present at each fieldwork session and shared with the participants for any feedback.

Phase 2: volunteers

These were attended by people who had experience in volunteering in the community—often (but not exclusively) with older people. They were, therefore, people who might be eligible to be ACE volunteers (ie, aged 55+). One face-to-face workshop was held in Stoke and online workshops in Manchester and Cardiff (held online due to the COVID-19 restrictions at the time). Participants were recruited through charities (eg, Age Connects, FAN charity) and online advertisements (Volunteering Wales, Sport Cardiff, social media websites, etc).

The workshop lasted about 90 min and followed a semistructured discussion guide that focused on (a) exploring people’s experiences with volunteering to help older people (across a range of topics) and (b) seeking their reactions to the planned ACE approach of pairing active 55+ volunteers with inactive 65+ participants. The draft system map (produced by potential study volunteers) was used as prompt material, helping to stimulate discussion about the barriers that older people might face being active and how volunteers might plan strategies to help them. Any disagreements within the group were resolved through consensus. Following the workshop, the online system maps were updated to include the volunteers’ perspectives.

Phase 3: stakeholders

These were attended by people who represent agencies or organisations with an interest in the health and well-being of older people. These included physical activity organisations; older people’s groups (eg, Age Concern); and those offering physical activity or social groups or services. One online workshop was held at each site. Participants were recruited through charities (eg, Action for Elders, Age Connects), health and social services (eg, Public Health Wales, Stay Steady Clinic, Independent Living Services, Buzz Health and Well-Being Service Manchester, National Health Service (NHS)) and activity providers (eg, Re-engage, Move More - Sport Cardiff, MCRactive).

In these workshops, the draft system maps (combining data from both participants and volunteers) were used as prompt material to explore the stakeholders’ views, and specifically (a) how their organisations contribute to helping older people to be active and (b) how they might help with the ACE approach. Following the workshops, analysis of notes from these sessions was used to update the digital system maps developed with the online platform www.kumu.io to incorporate the stakeholders’ perspectives. In all cases, the system maps were constructed as far as possible using the workshops’ participants own words or phrases.

Patient and public involvement

The ACE study is based on extensive public involvement throughout the earlier feasibility studies.13–15 This shaped the design of the study, including the content of the intervention. This system mapping study also aimed to maximise public involvement by seeking the views of a diverse public including potential participants, volunteers and stakeholders. The results of this system mapping study were fed back to participants, and the results of the repeated systems mapping study will be disseminated widely. The design of the repeated system mapping approach was reviewed by the trials steering committee, which included patient and public involvement members.

Results

Workshop attendance

Workshops were attended by:

  • 54 participants (mean age 74; 78% female).

  • 17 volunteers (mean age 65; 76% female).

  • 31 stakeholders (from NHS, statutory and voluntary sectors; 42% female).

Details including ethnicity are in ‘online supplemental material, table A1–A3’. This shows that the majority of participants and volunteers were of white British origin (61%) with 31% Chinese, 4% black/black British/African and 4% other (including White Irish). This was due to differential response rates and not a deliberate sampling strategy.

System maps

The system maps drafted following discussions in each site are shown in figures 1–3. The maps show the main outcome (physical activity) and how it connects to the factors that participants said motivated them to be active; those that might prevent them from being active; the roles of stakeholders and the ideas of volunteers. Each will be described in turn.

Figure 1
Figure 1

System map for Stoke.

Figure 2
Figure 2

System map for Cardiff.

Figure 3
Figure 3

System map for Manchester.

Motivators

These were grouped into common themes (primary motivators). Across all groups, these included:

  • Getting out and about (including enjoying fresh air; seeing people; walking for transport; enjoying the countryside or surroundings).

  • Being sociable (including meeting friends and family; playing with grandchildren).

  • Improving/maintaining mental health (including keeping the mind active; tackling depression; feeling cheerful/alive).

  • Health benefits including

    • Specific benefits such as ‘helping to improve my chronic health condition’ and ‘preserving independence’.

    • More general ones such as keeping fit or feeling good; keeping agile/supple.

‘Enjoying the exercise’ was a motivating theme from the Cardiff and Stoke groups but was hardly mentioned in the Manchester groups, who focused primarily on benefits such as being sociable or ‘getting out and about’. Similarly, the participant groups in Manchester and Stoke highlighted the importance of maintaining independence, whereas this was not a dominant theme in the Cardiff group. In Cardiff, the group talked about ways to keep motivated (and things that reduced their motivation) while the Stoke group talked more about the feeling that they ‘ought’ to exercise. Both groups from the Chinese community in Manchester highlighted the relationship between mind and body as a motivator. A summary of the primary motivators for each site is in table 1.

Table 1
|
Primary motivators identified by participants in the three ACE sites

Barriers

Participants identified many potential barriers to being active. By far, the most common types of barriers discussed were psychological: lacking motivation; lacking confidence (to try something new or go somewhere new) or feeling nervous about not being able to do the activity. Physical barriers to activity included aches and pains; specific health issues or feeling tired. Environmental or contextual barriers included a lack of appropriate exercise sessions or facilities; weather and pollution and cost. Finally, a few participants said they did not have the time for physical activity, but this was not a common theme. This is consistent with the literature that shows lack of time is not a common barrier among middle-aged and older adults.20

The contribution of relevant stakeholders

Stakeholders discussed the types of activities they do to help older people to be active, either as part of an organisation, or in a voluntary capacity. These fell into the following broad categories:

  • Health service professionals providing services to older people, such as physiotherapists or podiatrists, or exercise referral services. These provided direct opportunities for the participants to be more active or addressed specific barriers or problems they described.

    Agencies providing physical activity opportunities such as exercise classes; falls prevention groups or led walks; or related sessions such as mindfulness or ‘knit and natter’. These helped participants to find appropriate places to be active.

  • Voluntary sector agencies that provide volunteering or befriending services. These help participants by providing social opportunities for activity and social support.

  • Providers of other related services such as community transport; community cafes. These provided general social opportunities.

The potential role of volunteers

Volunteers drew on their experience of volunteering with older people to highlight ways that would build on the motivational factors that participants described and tackle the barriers to being active. For example, they stressed the need to ask people what sort of activity they liked doing; start gently and work within people’s capabilities and make it as fun and enjoyable as possible.

Implications for the design and delivery of the ACE intervention

All three groups (participants, stakeholders and volunteers) discussed the proposed ACE intervention and reflected on how such an approach would be appropriate to their circumstances. They then used their experience to consider the wider contextual factors and to propose a number of suggestions to consider in the programme’s design and delivery. These are summarised below:

Suggestions to consider in the design and delivery of ACE

The content of the ACE intervention
  • The principles of ACE were universally supported by all groups. Many people reported experience of ‘partnering’ or ‘befriending’ services that had been positive.

  • Make sure volunteers are valued—they are a key piece of the puzzle.

  • See it as a partnership between the volunteers and the older people—both would probably benefit from more physical activity.

  • Volunteers need support to deal with practical issues such as finding the right activity opportunities, or transport to activity sessions.

  • Provide information on appropriate types of activity for people with health conditions.

Ways of volunteers working with participants
  • Start with just talking. Ask people what they want to do; what they like. Ask about what stops them being active.

  • Make sure you are welcoming.

  • It may be helpful to provide transport—at least at first.

  • Make sure people work within their capabilities.

  • Emphasise that people can ‘just try’ something—there is no commitment.

  • Volunteers can help participants overcome lack of confidence, for example, knowing if a class is right for someone or finding a better one.

  • Find opportunities to be active together.

  • Promote and encourage but don’t push too hard.

  • Try not to label people (eg, ‘the blind one’).

Pairing volunteers and participants
  • Careful pairing is critical, to make sure people get on.

  • Pair by beliefs; culture; sexuality; interests; politics; humour. Probably more important than age or gender.

  • Pair by activity preferences.

  • Pairing a blind person with a sighted person would be a ‘win-win’.

Types of activity suitable for ace participants
  • Consider adapted activities, for example, walking football.

  • Focus on whatever gets people out and about.

  • Try one thing then another.

  • Emphasise the importance of developing an active routine.

  • Maybe start indoors first to make it easier.

  • Remember that everyone is different!

  • Consider just having a walk with someone to start with.

Target social needs and loneliness
  • Recognise that people join due to loneliness—so make it a positive sociable experience.

  • Let people bring a friend.

  • Focus on the mental health benefits.

‘Technical’ issues for volunteers
  • There may be a need for specific training to understand specific health conditions and the role of exercise in managing them. For example, exercising with diabetes.

  • Some activities might be ‘no-go’ areas. Know when to stop.

Discussion

Findings from the workshops described the multiple influences on physical activity, both positive (motivators) and negative (barriers). These were largely consistent with the literature on physical activity among older people, which shows environmental factors and resources as the most commonly identified barriers, and social influences, reinforcement and receiving help from other people to manage changes in their lives as the most identified motivators.20 Workshop discussions showed how these act at multiple ‘levels’ from individual to societal and environmental, and how they interact in a complex series of connections. This is also consistent with the literature, where the socioecological framework has frequently been used to describe the range of influences on behaviour and the types of interventions across the system.21

The unique role of the system mapping approach taken in this evaluation has been to capture the complex nature of the physical activity system and explore and describe the ways that key issues are connected across the system. System mapping goes beyond the simple identification of barriers and enablers in isolation and helps to identify the complex relationships between them (and potentially identifying more sophisticated pathways to achieving the desired outcome, such as exploiting feedback loops in the system). This approach has been used in turn to help refine the ACE intervention. First, it identified a number of suggestions that had already been considered by the ACE team and helped to validate their approach. For example, it had always been planned to encourage participants to try a range of activities according to their likes and dislikes. Similarly, the need to match participants and volunteers carefully was shown in earlier pilot studies9 and was here highlighted as important by volunteers, who recounted examples where befriending schemes did not work due to personality clashes. As a result, a process was established in ACE whereby in the case of a personality clash the volunteer management organisation would manage a blame-free pairing change. Both participants and volunteers were made aware of this prior to being matched.

Some new suggestions were also identified through the system mapping that led to actions taken by the ACE team. These are summarised below:

Changes made to the ACE programme delivery as a result of the system mapping

  • Greater emphasis is placed on matching participants and volunteers according to common characteristics/interests.

  • Process agreed for changing matched pairs in the case of negative feedback.

  • Volunteer training programme was amended to include advice on seeking support for identified health issues.

  • Greater emphasis is placed on starting gently and building up activity levels slowly.

  • Clarification of the boundaries around volunteers (eg, they are not personal trainers or responsible for transport to activity venues).

  • Greater emphasis was given to the value of volunteers, with a focus on their importance to the programme and that they too are part of the research project.

  • Volunteers engaged as programme contributors and supported in feeling ownership of the programme by inviting them to identify what initiatives/activities are available in their area, what is missing and what needs updating in the resource developed by the research team.

For example, findings led to the adaptation of the volunteer training programme to (1) emphasise from whom they should seek support regarding a participant’s health issues and the implications for increasing activity, (2) to discuss the importance of a gentle start to getting out and about, supporting increasing confidence and awareness of how and when it may be possible to add in more physical activities and (3) to add further clarifications to what the volunteers are not expected to provide for participants (ie, medical advice, transport).

Some of these changes to ACE delivery were also applied to the logic model for the trial (see ‘online supplemental material’).

The system mapping approach taken here is one application of the principle of ‘cocreation’, in which communities are involved in helping to design approaches to improve social and health issues. While this is used increasingly in public health, there is a lack of a strong theoretical framework and a need for more consistent application of such frameworks.22 23 It may be that system mapping represents a particularly user-friendly way to involve communities as the approach is relatively easy to understand and user-friendly for diverse audiences.

Strengths and limitations of the approach

The ACE team has found the system mapping process and approach to have been a very useful to refine the intervention prior to the launch of the ACE trial for a number of reasons:

  1. It provided a number of considerations to input directly into the design and delivery of the ACE programme. The majority of the suggestions came from volunteers, but they drew on the content of the draft map to consider the issue as a whole, rather than focusing on one specific aspect. This in turn helps to modify and update the logic model for the trial (see ‘online supplemental material’).

  2. It provided a description and analysis of the physical activity system influencing older people’s physical activity, as part of the process evaluation. The approach allowed exploration of different groups’ experiences, and how they potentially interact. The process will be repeated after the completion of the delivery of the ACE intervention, aiming to invite the contributors of the first systems mapping process as well as actual ACE study participants and volunteers after follow-up assessments. This will enable exploration of the influencing factors in each of the programme areas, and the extent to which they have changed, and whether the system itself has been influenced by the intervention and how best to implement the ACE programme in other locations if the trial found it effective and cost-effective.

  3. The system maps provide a visual description of the physical activity system and have already been used as part of the ongoing communications about the ACE trial to illustrate the range of issues under consideration. The system mapping approach seems particularly well suited to issues where there are multiple ‘layers’ of influences or multiple stakeholders (in this case participants; stakeholders and volunteers) as it can display complex issues visually to allow detailed discussion and analysis.

  4. Informal feedback after the workshops showed that the participants found the process to be positive and said they enjoyed being consulted, drawing on their experiences and providing input to the ACE trial. It also facilitated discussion between stakeholders and potential participants, including some early signposting to existing programmes. While much of this might have been achieved by a more conventional focus group approach, the draft system maps did appear to help to stimulate more in-depth discussions and encourage people to see the whole system.

The main limitation of this approach is that the conclusions were based on relatively small numbers of people who volunteered to take part. Similar discussions among greater numbers of people—perhaps with multiple system mapping workshops—may have enhanced the results. Also, the results are based primarily on people of white British and Chinese ethnic origin, with only a small minority of people from other ethnic backgrounds. Involving people from more diverse backgrounds may well have produced different results. The findings would have been enhanced if we had the resources to seek more formal feedback on the usefulness of the system mapping approach from participants (which we hope to do in the next phase of the evaluation). Finally, it is acknowledged that the bias of the researchers involved may have influenced the results as they have a major role to play in deciding what is placed on the system maps.

Conclusions

Overall, employing the system mapping prior to the launch of the ACE programme has proven to be a useful component of the process evaluation for the ACE trial. It provided direct input to the intervention processes; helped to fine-tune the logic model and identified some new approaches that were incorporated into the intervention design. The value of this approach will be increased considerably when we reapply it postintervention to explore any changes to the physical activity system arising from the ACE intervention, including changes to stakeholders’ ways of working and collaborating to tackle barriers to activity. It is a relatively simple and low-cost tool for exploring complex systems that could be used as part of process evaluation in trials, as well as to develop or refine interventions.