When evaluating the impact of a biodiversity conservation intervention, a counterfactual is typically needed. Counterfactuals are possible alternative system trajectories in the absence of an intervention. Comparing observed outcomes against the chosen counterfactual allows the impact (change attributable to the intervention) to be determined. Because counterfactuals by definition never occur, they must be estimated. Sometimes, there may be many plausible counterfactuals, including various drivers of biodiversity change and defined on a range of spatial or temporal scales. Here, we posit that, by definition, conservation interventions always take place in social‐ecological systems (SES) (i.e., ecological systems integrated with human actors). Evaluating the impact of an intervention in an SES, therefore, means taking into account the counterfactuals assumed by different human actors. Use of different counterfactuals by different actors will give rise to perceived differences in the impacts of interventions, which may lead to disagreement about its success or the effectiveness of the underlying approach. Despite that there are biophysical biodiversity trends, it is often true that no single counterfactual is definitively the right one for conservation assessment, so multiple evaluations of intervention efficacy could be considered justifiable. Therefore, we propose calculating the sum of perceived differences, which captures the range of impact estimates associated with different actors in a given SES. The sum of perceived differences gives some indication of how closely actors in an SES agree on the impacts of an intervention. We applied the concept of perceived differences to a set of global, national, and regional case studies (e.g., global realization of Aichi Target 11 for marine protected areas, effect of biodiversity offsetting on vegetation condition in Australia, and influence of conservation measures on an endangered ungulate in Central Asia). We explored approaches for minimizing the sum, including a combination of negotiation and structured decision making, careful alignment of expectations on scope and measurement, and explicit recognition of any intractable differences between stakeholders.