The major challenges that society faces, such as noncommunicable diseases and climate change, require large-scale interventions that work in complex systems. These interventions are often at regional or national level scales and can rarely be evaluated using experimental methods as researchers typically can’t control who receives the intervention. Nevertheless, evaluation of these types of interventions is crucial for decision making.  

We propose an approach to develop systems-informed research propositions to evaluate large-scale policy or infrastructure interventions and reflect on our experiences of doing so around two NBS interventions in a newly published paper led by Dr Miriam Alvarado.

What are nature based solutions?

Nature based solutions (NBS) are defined as “solutions to societal challenges that are inspired and supported by nature” and include, for example, efforts to introduce street trees in urban spaces and restore wetlands in rural areas.

What are Causal Loop Diagrams?

Causal loop diagrams are visual tools that represent the relationships between components in a system using arrows to show cause-and-effect connections. These diagrams help people understand complex feedback loops and dynamics within a system.

Proposed process

We propose the following step-by-step process to develop research propositions which can be tested empirically when evaluating NBS interventions:

• Step 1. Represent the underlying system within which the intervention will be introduced. This can be carried out in several ways, for example making use of an existing CLD, developing a new CLD through reviewing the existing literature or collaborating with stakeholders.
• Step 2: Develop an understanding of the intervention. This can also be carried out through various approaches. For example, reviewing existing literature, reviewing project documents and or consultation with local stakeholders and experts.  
• Step 3: Identify links between the intervention (and its influences/impacts) and the underlying system to produce a combined CLD
• Step 4: Generate systems-informed research propositions from the CLD. We found that it was useful at this stage to produce simplified CLDs and explore the feedback loops.
• Step 5: check the systems-informed research propositions with subject area experts and stakeholders

Case studies

We applied the above process to two case studies focused on different types of NBS: (a) street trees in urban areas in Europe and (b) a large-scale wetland-restoration project in rural Denmark.

In both cases, we based our understanding of the underlying system (step 1) around mental health on a previously published CLD by Wittenborn et al., 2016. To carry out step 2, for street trees in urban areas (a), we carried out a literature review and for (b) a wetland restoration project, we reviewed project documents.

For example, below is a CLD representing an NBS intervention in a coastal wetland area intended to reduce flooding risk.  

Strengths

Our proposed approach is applicable even where a systems-based method was not used to design the intervention. It offers a practical process for creating systems-informed research propositions for various real-world policies and infrastructure projects. This method allows us to consider nonlinear and feedback aspects of systems thinking and can facilitate communication of complex ideas effectively across multidisciplinary project teams.

We received positive feedback from stakeholders, particularly street-tree experts who found the propositions aligned with expert knowledge and were valuable for explaining unexpected insights. They noted that the CLD and related propositions enhanced communication with municipal-level decision-makers by emphasizing the significance of resources for street-tree maintenance, a factor acknowledged by tree managers but often overlooked by city-level leaders.

Limitations

Incorporating stakeholders more extensively in the initial stages of the process would be valuable, provided there is enough time, resources, and access. While we employed a qualitative approach, incorporating a quantitative process is possible but may demand considerably greater resources. It is also important to consider the substantial time investment required for developing these propositions, which increases the overall amount of time required for evaluation.

Conclusion

While there have been many calls for a systems-informed approach to evaluation, there are limited examples or methodological guidance around how to carry this out. We propose this approach for evaluations considering predetermined interventions, over which researchers have no influence or control over the interventions being implemented. We suggest this may be useful for others in developing research propositions around other large-scale predetermined interventions.

Investing time and resources in developing systems-informed research propositions at an evaluation’s outset can improve our ability to generate policy-relevant insights.  This may support the introduction and maintenance of NBS that address a range of health and sustainability issues and reduce the risk of unintended consequences.

You can read the full paper called “Using causal loop diagrams to develop evaluative research propositions: opportunities and challenges in applications to nature-based solutions” Open Access at https://doi.org/10.1002/sdr.1756

This work was supported by the European Union’s Horizon 2020 research and innovation program under Grant No: 821016 (www.regreen-project.eu) and Grant No: 776866 (http://www.reconect.eu/).