Which term refers to evaluating the system and its components?

Why: To date, the majority of obesity prevention evaluation efforts have focused on methods that do not optimally recognize the interactions and relationships among the many factors that comprise a complex health issue such as obesity. As a result, important learnings may be missed. A systems approach to obesity prevention evaluation efforts can build on current evaluation methods and improve our understanding on the relationships and their impact among the Accelerating Progress in Obesity Prevention (IOM, 2012a) report strategies, outcomes, and stakeholders across a variety of settings.

What: Complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that form a whole greater than the sum of its parts (Holland, 2000). The explicit recognition by evaluators that obesity is complex will help to facilitate the increased use of systems approaches in evaluation efforts for obesity prevention.

How: From a systems perspective, evaluating progress of obesity prevention efforts should include support for better understanding of the interactions and relationships among individuals, groups, communities, stakeholders, and national efforts across a variety of settings and throughout the evaluation process framework (see Chapter 3).

Obesity is complex. Attempts to treat or prevent it with individual behavior-change strategies (as a simple problem) have generally not been effective. Consideration of a complex systems approach to identify the best choices for obesity prevention is a worthwhile, arguably essential, endeavor. The importance of considering a systems approach to obesity is explicitly identified in the Institute of Medicine (IOM) report Accelerating Progress in Obesity Prevention (APOP) (IOM 2012a). The report notes that “a systems perspective helps to reveal, and create, the potential for combined impacts (or synergies) that can further accelerate progress in preventing obesity” (IOM, 2012a, p. 7). From an evaluation perspective, systems science provides quantitative methodologies that enable evaluators to consider the dynamic relationships of factors at multiple levels of analysis, but it also includes qualitative approaches to actively engage members of the community in a participatory process. Evaluation deals with challenges of the real world—dealing with reality as it unfolds demands a complex adaptive systems perspective.

The complexity of obesity introduces notable challenges for evaluators. Simple or uncomplicated systems are likely to be homogenous, linear, static, and independent, with little feedback or connection to other levels or subsystems. On the other hand, complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that form a whole greater than the sum of its parts (Holland, 2000). Complex systems are mostly nonlinear, dynamic, interdependent, generate feedback, and have several connected levels. In a complex system, a given set of circumstances will not always yield the same result (Holmes et al., 2012).

A systems approach to obesity may help to make sense of the complexity (discussions related to systems and complexity tend to use a unique lexicon, and many of the terms used in this discussion are defined in Appendix B). By identifying leverage points and simple rules among a network of interconnecting nodes, evaluators can advance their understanding of how different components work together under various circumstances to influence movement in weight status. This enables evaluators to better identify where to look, what to measure, when to measure, how to recommend action, and why the proposed approach is important for decision makers to consider. A systems approach elevates the importance of understanding contextual factors and relationships to describe what is happening, why it is happening, and what may be the best action (how) to achieve progress in the intended direction.

To provide context in which a systems approach to evaluation may occur, the Committee considered some situations in which a systems approach could be used to take action. It should be kept in mind that systems bring together a set of elements into a meaningful set of relationships—connected and interdependent—that act together as a whole. In effect, systems work by uniting various elements in a meaningful relationship that acts as a whole. To evaluate these complex systems is to evaluate patterns of relationships, how they are sustained, how they self-organize, and how they emerge. Systems, therefore, can only be understood as an integrated whole; the result of both the sum and the relationships among its elements.

In the context of taking action to evaluate obesity prevention, action-oriented questions to ask include what happened, what will happen, and what is the best choice? All of these questions may apply to dealing with obesity as a system. However, as discussed below in more detail, the first two questions may be addressed by treating obesity as a predictable issue that may be solved through reductionist thinking and, as such, represent simple or at most complicated systems. On the other hand, the third question addresses obesity as a complex system that is not so predictable and, as such, cannot be understood through reductionist approaches (Finegood, 2011).

EVALUATING THE “WHAT HAPPENED?” QUESTION

To describe what happened, a basic reporting approach may be taken. This generally takes the form of surveillance and other monitoring techniques that describe the current state, which is attributed to activities conducted in the past. The “what happened” question context is representative of a static, independent, linear situation that potentially describes some combination of what has happened, when it might have happened, where it may have happened, and how much, how high, what level, or what score may have been attained.

EVALUATING THE “WHAT WILL HAPPEN?” QUESTION

To describe what will happen as a result of actions taken, evaluation approaches will generally use techniques that predict or estimate the likelihood of certain desired outcome(s) based on the observed association with another set of factors or the measured changes in proximal intervention targets that allow for the prediction of desired ultimate outcomes. To address this question, evaluators use several approaches, including randomized controlled trials and variations on experimental designs that attempt to control for many confounders (Mercer et al., 2007). These evaluation techniques also tend to address the problem in a reductionist manner, attempt to identify cause and effect, and often fall short in solving the complexity of the interactions among the many factors associated with obesity. On the other hand, this type of approach may be very useful for addressing, to some level of confidence, the potential impact of actions proposed or taken, how to prevent adverse outcomes, or how to provide an estimate of anticipated value of interventions.

EVALUATING THE “WHAT IS THE BEST CHOICE?” QUESTION

To address the question what is the best choice of action, considering all available options, complexity science is needed. Complexity science is an attempt to understand how things influence each other within the context of the whole. The Committee considers obesity to be a complex system insofar as it possesses characteristics consistent with complexity that include being heterogeneous, nonlinear, stochastic, interdependent, generating feedback, self-organizing, and involving emergence (Finegood, 2011). A systems approach to taking action demands respecting the characteristics of systems noted above and considering leverage points that may influence change within a complex system—attempting to identify the “best choice” for action. The remainder of this chapter will focus on addressing the third question: “What is the best choice?”

HOW WILL EVALUATION EFFORTS BENEFIT FROM A SYSTEMS APPROACH

Although relatively new, systems theory is already contributing to evaluations of obesity interventions (C3 Collaborating for Health, 2011). Evalations that do not consider systems dynamics or conditions will likely miss aspects of the intervention and its environment that influence the intervention's operation and success (Hargreaves, 2010). To optimize the development of practice-based evidence from such studies, researchers and practitioners will need to develop evaluation systems that enable continuous learning and the pooling of findings and best practices (Holmes et al., 2012). To move toward systems-informed evaluation, researchers will need to reconsider the types of evidence needed from a systems point of view. For example, questions around effective leadership to facilitate systems change and leveraging of networks to advance health improvements become increasingly important with increased complexity (Holmes et al., 2012). Systems science (as it applies to community-based prevention) can address how causal structures change over time, including the effect of changes in the type or number of interventions implemented, or how changes in social norms, community practices, and leadership may affect outcomes. Examining these causal structures can increase understanding about both intended and unintended consequences of interventions (IOM, 2012b; Meadows, 1999; Sterman, 2000; Ulrich, 2000). As one example of how a systems approach may be used to support evaluation, consider multiple uses of mapping the relationships among the many variables and factors of a complex system, including identification of leverage points, anticipation of unintended consequences and unexpected results, and comprehensive assessment of stakeholder views.

Mapping and the Use of Mapping

Systems maps (e.g., causal loop diagrams) help to visualize relationships. The process of developing a causal loop diagram is well suited for group work and facilitating communication among diverse stakeholders (Homer et al., 2006; Milstein and Homer, 2009). A systems map represents a dynamic hypothesis, an evidence-based or evidence-informed depiction of relationships, but the map itself cannot be tested (Homer et al., 2006; Milstein and Homer, 2009). Systems causal maps may be created with specific application to obesity and can be the result of a quantitative or qualitative approach to complex systems mapping. Excellent examples of complex systems maps for obesity are presented in Tackling Obesities: Future Choices—Building the Obesity Systems Map, a report from the United Kingdom Government's Foresight Program (Vandenbroeck et al., 2007),1 the APOP report (IOM, 2012a), or An Integrated Framework for Assessing the Value of Community-Based Prevention report (IOM, 2012b). An example of the latter is provided in Figure 9-1.

FIGURE 9-1

Example causal loop diagram for value of community-based prevention policies. SOURCE: IOM, 2012b.

Identification of Leverage Points

A map depicting the underlying theories of change and causal structures of a comprehensive obesity prevention policy system can increase the understanding of the multiple pathways of impact of obesity prevention policies and actions. Mapping these relationships helps to identify key leverage points in the system that may influence multiple other variables in the system. In turn these diagrams can help evaluators to identify and prioritize key variables and associated measures to be assessed. It guides evaluators on “where to look” for short-, intermediate-, and long-term effects of program and policy options, which may enhance the use and collection of relevant data (Diez Roux, 2011).

Anticipation and Identification of Unintended Consequences and Unexpected Results

The ideal, of course, would be to make only the best choices, but complex systems have emergent properties that make them unpredictable. Therefore, it is worthwhile to consider how to use systems approaches to identify and anticipate consequences of policies and actions taken that are not optimal or ideal but might be ameliorated. The use of obesity system causal maps can identify pathways that highlight actions and responses within the system that are more likely than others. For example, a map of the obesity prevention policy system can more explicitly elucidate the pathways from obesity prevention policies and programs to health outcomes as well as the underlying structures that reinforce or hinder change processes (IOM, 2012b, p. 146). This type of mapping can support the entire proposed framework (see Figure 3-1) for evaluating progress in obesity prevention. The feedback identified through mapping, positive and negative, reinforcing and weakening, anticipated and unexpected, provides opportunities for evaluators to explore the mechanisms that fuel or retard relationships and feedback in the system. This means paying attention to the relationships among actors in the system, among program components, and among levels of the system. One example of unintended consequences in efforts to prevent obesity and increase fruit and vegetable intake among school children is illustrated in Box 9-1.

BOX 9-1

yumPower School Challenge. In 2011, a not-for-profit health plan in Minnesota announced a multi-year community collaboration called “yumPower” (www.yumpower.com) to increase “better-for-you” food consumption across a variety (more...)

Comprehensive View of Elements and Stakeholders Involved or Affected by Change Efforts

To generate a useful systems model or map, multiple stakeholders and perspectives invested in the situation need to be engaged in the process (IOM, 2012a; Williams and Imam, 2006). Also known as participatory evaluation, systems-informed evaluators have a responsibility to “increase the voices of those in the margin” (Williams and Imam, 2006, p. 9) in all phases of development and evaluation of the intervention and establish common goals reflecting the various perspectives of the stakeholders. Their participation in building systems maps tends to unearth causal loops that experts and others from outside the community would not anticipate.

A SYSTEMS PERSPECTIVE ON THE CHALLENGE OF EVALUATING OBESITY PREVENTION

Complex systems are a configuration of interacting, interdependent parts, connected through a web of relationships, that represent a whole greater than the sum of the parts (Holland, 2000). Research supports the obesity-related influences that stem from genetics, neurobiology, physiology, psychology, behavior, family structure and influences, social context and social norms, environment, economics, markets, and public policy. However, the mechanisms of these relationships are not always well researched (Hammond, 2009). Because obesity is more likely to be the result of a multitude of factors interacting with each other over time than any single causal factor (Glass and McAtee, 2006) drives the field to recognize that obesity is complex and that progress in obesity prevention may gain significant benefits when evaluation strategies include a systems perspective.

An earlier IOM report, Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making (IOM, 2010), discusses the need to apply a systems perspective to obesity prevention. A systems approach to obesity prevention makes sense because multiple properties specific to obesity need to be considered in combination in order to address obesity prevention effectively (although this list of properties may not be all-inclusive). Properties to consider include the following:

• Obesity is related to social networks.

• Intervention outcomes for obesity prevention are affected by unanticipated, emergent situations and baseline assumptions that may not hold over time.

• Significant inter-individual variability exists in biological, physiological, and emotional responses to interventions.

• A significant knowledge gap exists related to weight accumulation.

• Resource allocation that addresses obesity prevention is related to other areas competing for the same resources.

With all of these properties to consider, using a systems perspective for evaluating obesity prevention efforts is challenging. Decisions around what systems approaches to consider may be informed by an understanding of which simple rules govern the behavior of the system and at what level of the system provides the most opportunity to make a difference. In addition, costs, scalability, and sustainability of interventions are important considerations (Hammond, 2009; Huang et al., 2009; Levy et al., 2011; Mabry et al., 2010; Nader et al., 2012). To discuss this in more detail, the next two sections address the concepts of “simple rules” and “system levels.”

Simple Rules

Complex systems, given their dynamic and interacting characteristics, are inherently unpredictable, and interacting with them leads to continually emerging novel behavior (Plsek and Greenhalgh, 2001). Nevertheless, it appears that this emergent property follows an inherent self-organization facilitated through simple, locally applied rules. Examples of simple rules identified to explain complex behavior in, for example, the context of health systems include those associated with large system transformation and those that attempt to explain how health systems create value for their stakeholders. Best et al. (2012) identified five simple rules that were likely to enhance the success of transformation initiatives: “1) blend designated leadership with distributed leadership, 2) establish feedback loops, 3) attend to history, 4) engage physicians, and 5) include patients and families” (p. 421). Kottke et al. (2012) identified five simple rules that generate value for the health system's stakeholders: “1) the stakeholders agree on a set of mutual, measurable goals for the system; 2) the extent to which the goals are being achieved is reported to the public; 3) resources are available to achieve the goals; 4) stakeholder incentives, imperatives, and sanctions are aligned with the agreed-upon health goals; and 5) leaders of all stakeholders endorse, promote, and honor the agreed-on health system goals.” Identifying the simple rules that govern the behavior of the system in question may point to a limited number of intervention opportunities that are likely to influence the outcomes and overall impact on the goal of obesity prevention.

Simple rules also relate to the concept of self-organization, that is, the capacity of a system to make its own structure more complex. For example, a researcher may consider the organization of a person's social network. People who have friends who are overweight are more likely to be overweight themselves (Valente et al., 2009). Furthermore, if their social system changes over time, the likelihood they may gain or lose weight is highly correlated with the weight of the friends to whom they are connected (Christakis and Fowler, 2007). As such, the relationships a person has may be considered a simple rule related to body weight, but caution must be exercised in not equating correlation with causation.

System Levels

In addition to simple rules, system levels also provide a means to make systems approaches more actionable. For example, when obesity interventions are only applied at the level of the individual and do not take into consideration what happens at the level of behavioral settings (e.g., worksites, schools, health care), sectors influencing (e.g., public health, education, business, government), or social norms and values, it is unlikely that the changes induced will be sustainable (IOM, 2012a). Malhi et al. (2009) simplified an original list described by Meadows (2008) into five system levels at which obesity interventions could be applied. These levels include (1) paradigm (a system's deepest belief), (2) goals (what the system is trying to achieve), (3) structure (enhancing connections across most of the system as a whole), (4) feedback and delays (to effect self-regulation, self-reinforcement, and adaptation of the system), and (5) structural elements (to affect subsystems, actors, and the physical structure of the system). The paradigm level is the most difficult at which to intervene, whereas the structural elements level is the least difficult. However, interventions at the more difficult levels tend to be more effective and impactful. In the context of childhood obesity, Malhi et al. (2009) found that most recommendations for interventions are made at the structural elements level. This is the level that includes a strong evidence base and is associated with evidence of cause and effect. Paradoxically, it is the level least likely to be effective in creating and sustaining change in a complex system. Table 9-1 describes the intervention levels and associates them with objectives for evaluation.

TABLE 9-1

Places to Intervene in a Complex System and the Associated Evaluation Objectives.

A clear example of interventions implemented at multiple levels of a system is the tobacco experience over the past several decades. The interrelated set of activities that eventually connected several levels of interventions represents an evolution toward a systems approach to address tobacco use in the United States. As outlined in Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making (IOM, 2010), the combined and coordinated set of activities implemented at the individual, community, population, system, and public education levels all contributed to overall tobacco use reductions in the country. Figure 9-2 presents this set of activities.

FIGURE 9-2

Evolution of tobacco control approaches toward system thinking. NOTE: ASSIST = American Stop Smoking Intervention Study; COMMIT = Community Intervention Trial for Smoking Cessation; ISIS = Initiative on the Study and Implementation of Systems; NCI = National (more...)

Simple rules and system levels can be used to identify opportunities to take action and may be regarded as a fundamentally different way of thinking about how to approach complex problems, such as obesity.

USING A SYSTEMS PERSPECTIVE TO ADDRESS POPULATION HEALTH: A REVIEW OF PRIOR IOM REPORTS

Previous IOM consensus committees have applied systems thinking to identify the types of evidence and research needed to guide decision making around obesity prevention; to identify promising strategies and actions; to anticipate the effects and unanticipated consequences of actions; and to explore different pathways through which interventions can lead to changes—for better or worse—in population behavior and health outcomes. This report builds from this knowledge base by applying a systems approach to evaluation design and implementation. The Committee built on three reports that present applicable discussion of system approaches to obesity prevention; they are briefly presented in Table 9-2 below.

TABLE 9-2

IOM Reports That Used Systems Perspectives.

EVALUATION TOOLS AND SYSTEMS SCIENCE

A diverse set of qualitative, quantitative, and mixed-methods evaluation and analytic tools are available to help to evaluate actions to address a complex problem such as obesity. The tools can be used to identify changes in patterns of system behavior over time and to advance an understanding of the factors and conditions underpinning these changes. The tools range from sophisticated computer models, such as systems dynamics modeling (Homer and Hirsch, 2006; Sterman, 2006), dynamic microsimulation modeling (Mitton et al., 2000), and agent-based modeling (Axelrod and Tesfatsion, 2006; Epstein, 2006), as well as qualitative approaches such as focus groups and Concept Mapping (Kane and Trochim, 2007), which require little technological support. Some examples of tools presented in Box 9-2 are discussed in more detail below.

BOX 9-2

Examples of Evaluation Tools/Methods for Complex Systems. SOURCE: Adapted from Hargreaves, 2010.

Simulated Virtual Worlds: A Quantitative Model

Simulated virtual worlds, an example of a quantitative method that is based on formal models, are widely used to advance hypotheses about how a system behaves over time. High-quality empirical research, especially well-designed experimental and quasi-experimental studies, is needed to inform systems models (IOM, 2010), which use data-driven assumptions to guide their predictions (Levy et al., 2011). The ReThink Health dynamics simulation model was designed to help planners to test different quantitatively estimated effects of scenarios by tracking changes in health status, utilization, costs, and equity following different intervention options (ReThink Health, 2012). To varying degrees, the proposed interventions address risk, care, capacity, cost, funding, and/or economic and health care trends (e.g., prevalence of uninsured individuals, local economic climate). These simulated scenarios are also useful for evaluators because they shed light on what types of impacts can be expected under different conditions in the short- and long-term. As a result, they point to where in the system evaluators should monitor and measure change, and at what intervals of time. In essence, they can help to identify the best choice among a variety of options.

Group Model Building: A Qualitative Model

Group model building (GMB), a qualitative method example, presents a participatory process for developing and analyzing a formal model. This iterative approach, conducted over several meetings, fosters collaboration and stakeholder involvement, and it can help to address issues of transparency by articulating underlying assumptions. It may be especially valuable when working with marginalized communities (Hovmand et al., 2012). GMB requires teamwork, distinct roles (e.g., facilitator, recorder), and facilitation to help to advance collaboration and manage conflict. Process maps can help teams to visualize and plan the overall sequence of GMB sessions.

SYSTEMS IN THE CONTEXT OF THE COMMITTEE'S PROPOSED EVALUATION FRAMEWORK

The framework, as presented in Chapter 3, outlines five components, including inputs, activities, outputs, intended outcomes, and impacts (see Figure 3-1). Each of these components can be viewed through a complexity lens. The various evaluation methods and tools discussed above can be applied to each component's set of activities shown in Figure 9-3 to support the evaluation and measurement effort. Some tools or methods can be applied to multiple sections or the entire framework at once, whereas others may be applied to a single step in the model. For example, a systems map can encompass sources of input, activities in the evaluation efforts, identified outputs and intended outcomes, and anticipated impacts.

FIGURE 9-3

Examples of how various systems approaches to evaluation may be applied to obesity prevention efforts in the context of the Committee's evaluation framework.

Examples of systems approaches to obesity prevention evaluation that may be applied to the framework are provided in Figure 9-3. The examples are presented as part of each step of the model as well as in the context of the entire model. The “Inputs ” box dialogue sessions may be one example of collecting insights that follow a systems approach methodology. The “Impacts ” box the need to collect School Health Policies and Practices Study and School Nutrition Dietary Assessment Study–like data on child care settings on a national and periodic basis represents another opportunity for cross-agency collaboration. In addition the ability to identify sociocultural and socioeconomic determinants, timing of exposure, and living and working conditions across populations illustrates another important multi-level relationship that follows a systems approach methodology (further described in Chapter 5 and applied in Chapter 6, 7, and 8). Looking at the framework as a whole, and recognizing that complex systems include, for example, characteristics such as nonlinearity, interdependency, feedback loops, and emergence (Finegood, 2011), one realizes that complex social networks serve as an example of a systems approach to evaluation that includes many of the systems characteristics noted and applies to the entire framework. Complex social networks, and their inherent feedback loops, have been shown to affect obesity. Using social network analysis, Christakis and Fowler (2007) were able to examine the spread of obesity over time among subjects of the Framingham Heart Study. These researchers showed that obesity was clearly associated with social relationships (e.g., friends, spouses, and siblings) and that the geographic distance among people (an example of physical environment) was less important than social environment. They were also able to show that obesity followed the friendships and concluded that obesity is, in fact, contagious.

Case Examples

Change in body weight is associated with an imbalance between energy content of food stuffs consumed and energy expended. Hence, to consider the potential of any obesity intervention, its impact on both energy intake and expenditure needs to be properly accounted for. Hall et al. (2011) provide an example of how predictions about the potential effect of an intervention may differ between more traditional approaches and systems approaches to obesity prevention. These researchers compared the results of a projected impact of taxing caloric sweetened beverages on obesity prevalence based on the extrapolation of a static weight loss prediction model to the results of a dynamic simulation model. The static model, presented in a report by the U.S. Department of Agriculture (USDA) (Smith et al., 2010), estimated the impact of a 20 percent tax to be an overall decrease in energy intake of about 40 kcal per day and a 1.8 kg body weight loss per year. Extrapolated over 5 years, this would result in almost 10 kg body weight loss. In comparison, the dynamic simulation model estimated that the overall impact of a decrease in energy intake as a result of the tax increase would reduce average body weight by about 1.8 kg over 5 years and about 1 kg of weight loss after 1 year (Hall et al., 2011). This result is in stark contrast to the estimates of the USDA analysis and highlights the importance of inter-individual variability of weight loss (caused by the same intervention), uncertainty of the assumptions made about energy expenditure, and the trajectory of human weight change. This example in the context of obesity prevention recognizes the importance of a careful consideration of the interactive and dynamic relationships among all factors and variables that influence the outcomes of interest over time.

Another case example outlines how a systems approach to obesity prevention evaluation may be used in a larger context to support decision making and dissemination of successful interventions. Figure 9-4 illustrates how a complex obesity systems map has been used to inform action (e.g., planning, decision making, and implementation) around obesity prevention. The case example starts with the explicit recognition that the workplace may be regarded as a complex adaptive system and that addressing obesity, considered complex in its own right, should be considered from a systems perspective. The Foresight obesity system causal map, created by the UK government (Butland et al., 2007), is used as the starting point to identify the best choice(s) for addressing obesity at the workplace. The workplace is mentioned in the context of physical activity (energy expenditure) in two of the seven main thematic clusters—the individual physical activity and physical activity environment clusters. More specifically, these clusters indicate the issues of overall level of occupational activity and dominance of sedentary employment. Hence, two places that can provide leverage to address obesity prevention successfully include the potential to increase person-level activity during the workday and to change the work environment by reducing the sedentary nature of the work tasks in which employees are engaged.

FIGURE 9-4

Case example of an obesity system casual map applied to a workplace setting to identify intervention opportunities and evaluate energy balance (physical activity) interventions. SOURCE: Vandenbroeck et al., 2007. Foresight: Tackling Obesities: Future (more...)

As a result of this evaluation of the Foresight obesity maps, program planners decided to invest resources in programmatic solutions. They designed and tested intervention concepts in a pilot project (Pronk et al., 2012). In the context of a multi-component, comprehensive health and well-being program, sit-stand devices were deployed in the work stations of employees with sedentary job types (call-center employees). Employees were engaged in a participatory manner, a supportive environment was established to optimize participation, and minimally invasive measurement methods were deployed to monitor performance. The results of the pilot study proved meaningful from the perspective of both the employees and management, and, as a result, the solution was deployed across all areas of the company that housed sedentary jobs (Koster, 2012).

SUMMARY

This chapter presents a systems approach to evaluate obesity prevention efforts in context and considers it from an end user's perspective by asking the question, “what is the best choice?” when taking action. It describes what a systems approach entails, what are its key components, how opportunities for action may be identified, where readers may find other discussions of systems applied to obesity, what evaluation tools and methods are available, and what benefits a systems approach may provide to evaluation efforts in this area. Finally, the chapter considers how systems may be applied to the Committee's proposed evaluation framework.

A systems approach may be useful to describe interrelationships among the many variables and elements that are involved in obesity and prevention efforts. It can help policy makers and implementers to identify the most important leverage points to achieve progress to determine “what is the best choice?” regarding what action to take and how best to invest resources.

Taking a systems approach to evaluating the APOP strategies will enable researchers and decision makers to think about evaluation efforts in new ways–identifying the types of data to collect and the methods for analyzing the data, identifying core indicators and measures that leverage networks and feedback loops among different sectors of society, identifying leaders, and engaging stakeholders in all aspects of evaluation. The result of these activities will improve the capacity and infrastructure of the collective evaluation system for obesity. And because obesity itself is an integral part of the larger systems of public and population health, it may be viewed as a major element in the larger complex system of the health of populations in communities, across states, and across the nation.

REFERENCES

• Axelrod R, Tesfatsion L. Appendix A. A guide for newcomers to agent-based modeling in the social sciences. Tesfatsion L, Judd KK, editors. Amsterdam, the Netherlands: North-Holland; 2006. pp. 1647–1659. (In Handbook of computational economics, vol. 2: Agent-based computational economics).

• Best A, Greenhalgh T, Lewis S, Saul JE, Carroll S, Bitz J. Large-system transformation in health care: A realist review. Milbank Quarterly. 2012;90(3):421–456. [PMC free article: PMC3479379] [PubMed: 22985277]

• Butland B, Jebb S, Kopelman P, McPherson P, Thomas S, Mardell J, Parry V. Foresight: Tackling obesities: Future choices–project report, 2nd edition. London, UK: Government Office for Science; 2007.

• Christakis NA, Fowler JH. The spread of obesity in a large social network over 32 years. New England Journal of Medicine. 2007;357(4):370–379. [PubMed: 17652652]

• Diez Roux AV. Complex systems thinking and current impasses in health disparities research. American Journal of Public Health. 2011;101(9):1627. [PMC free article: PMC3154209] [PubMed: 21778505]

• Epstein JM. Remarks on the foundations of agent-based generative social science. Tesfatsion L, Judd KK, editors. Amsterdam, the Netherlands: North-Holland; 2006. pp. 1585–1604. (In Handbook of computational economics, vol. 2: Agent-based computational economics).

• Finegood DT. The complex systems science of obesity. In: Cawley J, editor. In The Oxford handbook of the social science of obesity. New York: Oxford University Press; 2011. pp. 208–236.

• Glass TA, McAtee MJ. Behavioral science at the crossroads in public health: Extending horizons, envisioning the future. Social Science and Medicine. 2006;62(7):1650–1671. [PubMed: 16198467]

• Hall KD, Sacks G, Chandramohan D, Chow CC, Wang YC, Gortmaker SL, Swinburn BA. Quantification of the effect of energy imbalance on bodyweight. Lancet. 2011;378(9793):826–837. [PMC free article: PMC3880593] [PubMed: 21872751]

• Hargreaves M. Evaluating system change: A planning guide. Princeton, NJ: Mathematica Policy Research, Inc; 2010.

• HealthPartners. YumPower school challenge. Summary of key findings. Bloomington, MN: HealthPartners; 2012.

• Holland JH. Emergence: From chaos to order. Oxford, UK: Oxford University Press; 2000.

• Holmes BJ, Finegood DT, Riley BL, Best A. Systems thinking in dissemination and implementation research. In: Brownson RC, Colditz GA, Proctor EE, editors. In Dissemination and implementation research in health: Translating science to practice. New York: Oxford University Press; 2012. pp. 175–191.

• Homer JB, Hirsch GB. System dynamics modeling for public health: Background and opportunities. American Journal of Public Health. 2006;96(3):452–458. [PMC free article: PMC1470525] [PubMed: 16449591]

• Homer J, Milstein B, Dietz W, Buchner D, Majestic E. Obesity population dynamics: Exploring historical growth and plausible futures in the U.S; Paper presented at 24th International System Dynamics Conference; Nijmegen, Netherlands. 2006.

• Hovmand PS, Andersen DF, Rouwette E, Richardson GP, Rux K, Calhoun A. Group model-building “scripts” as a collaborative planning tool. Systems Research and Behavioral Science. 2012;29(2):179–193.

• IOM (Institute of Medicine). Bridging the evidence gap in obesity prevention: A framework to inform decision making. Washington, DC: The National Academies Press; 2010. [PubMed: 25032376]

• IOM. Accelerating progress in obesity prevention: Solving the weight of the nation. Washington, DC: The National Academies Press; 2012a. [PMC free article: PMC3648752] [PubMed: 22983849]

• IOM. An integrated framework for assessing the value of community-based prevention. Washington, DC: The National Academies Press; 2012b. [PubMed: 24901190]

• Isham GJ, Zimmerman DJ, Kindig DA, Hornseth GW. HealthPartners adopts community business model to deepen focus on nonclinical factors of health outcomes. Health Affairs. 2013;32(8):1446–1452. [PubMed: 23918490]

• Kane M, Trochim WMK. Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage Publication; 2007.

• Koster K. Employee Benefit News. Dec 1, 2012. Sit-stand workstations incorporate movement into work days of sedentary employees.

• Levy D, Mabry P, Wang Y, Gortmaker S, Huang TTK, Marsh T, Moodie M, Swinburn B. Simulation models of obesity: A review of the literature and implications for research and policy. Obesity Reviews. 2011;12(5):378–394. [PMC free article: PMC4495349] [PubMed: 20973910]

• Mabry PL, Marcus SE, Clark PI, Leischow SJ, Mendez D. Systems science: A revolution in public health policy research. American Journal of Public Health. 2010;100(7):1161–1163. [PMC free article: PMC2882409] [PubMed: 20530757]

• Malhi L, Karanfil O, Merth T, Acheson M, Palmer A, Finegood DT. Places to intervene to make complex food systems more healthy, green, fair, and affordable. Journal of Hunger and Environmental Nutrition. 2009;4(3–4):466–476. [PMC free article: PMC3489112] [PubMed: 23173029]

• Meadows D. Leverage points: Places to intervene in a system. Hartland, VT: The Sustainability Institute; 1999.

• Meadows DH. Thinking in systems: A primer. Wright D, editor. White River Junction, VT: Chelsea Green Publishing Company; 2008.

• Mercer SL, DeVinney BJ, Fine LJ, Green LW, Dougherty D. Study designs for effectiveness and translation research: Identifying trade-offs. American Journal of Preventive Medicine. 2007;33(2):139–154. [PubMed: 17673103]

• Mitton L, Sutherland H, Weeks M, editors. Microsimulation modelling for policy analysis: Challenges and innovations. Cambridge, UK: University Press; 2000.

• Nader PR, Huang TT, Gahagan S, Kumanyika S, Hammond RA, Christoffel KK. Next steps in obesity prevention: Altering early life systems to support healthy parents, infants, and toddlers. Childhood Obesity. 2012;8(3):195–204. [PubMed: 22799545]

• NCI (National Cancer Institute). Greater than the sum: Systems thinking in tobacco control. Bethesda, MD: Department of Health and Human Services, National Institutes of Health, National Cancer Institute; 2007. (Tobaco control monograph no. 18. NCI Tobacco Control Monograph Series. NIH Pub. No. 06-6085).

• Plsek PE, Greenhalgh T. Complexity science: The challenge of complexity in health care. British Medical Journal. 2001;323(7313):625–628. [PMC free article: PMC1121189] [PubMed: 11557716]

• Pronk NP. Physical activity promotion in business and industry: Evidence, context, and recommendations for a national plan. Journal of Physical Activity and Health. 2009;6(Suppl 2):S220–S235. [PubMed: 20120131]

• Smith TA, Lin BH, Lee JY. Taxing caloric sweetened beverages: Potential effects on beverage consumption, calorie intake, and obesity. Washington, DC: Economic Research Service, U.S. Department of Agriculture; 2010. (Economic Research Report No. 100).

• Sterman JD. Business dynamics. Systems thinking and modeling for a complex world. Boston, MA: McGraw-Hill; 2000.

• Ulrich W. Reflective practice in the civil society: The contribution of critically systemic thinking. Reflective Practice. 2000;1(2):247–268.

• Valente TW, Fujimoto K, Chou CP, Spruijt-Metz D. Adolescent affiliations and adiposity: A social network analysis of friendships and obesity. Journal of Adolescent Health. 2009;45(2):202–204. [PMC free article: PMC2747768] [PubMed: 19628148]

• Vandenbroeck P, Goossens J, Clemens M. Foresight tackling obesities: Future choices—building the obesity system map. London, UK: Government Office for Science; 2007.

• Williams B, Imam I, editors. Systems concepts in evaluation: An expert anthology. Los Angeles: EdgePress, American Evaluation Association; 2006.

What system evaluation?

What is system evaluation in sad?