How do you monitor and review effectiveness of risk treatments?
Consents involving hazard mitigation should be monitored to ensure adequate construction and maintenance and that measures are proving effective. Where hazard mitigation works are proven to be ineffective in managing the risk then the resource consent should be reviewed. It is important that information from development applications
that reveal the presence of hazards is recorded, even if the risk is avoided or mitigated through engineering works. This provides valuable information later on if assumptions made at the time prove to be wrong and the issue needs to be revisited. Section 35 of the RMA requires local authorities to monitor: the state of the environment; the efficiency and effectiveness of policies, rules or other methods; and a range of other matters set out in s35(2). This includes
gathering information, monitoring and keeping records for natural hazards. Local authorities also need to keep records of natural hazards as per s35(5)(j) of the Act. Hazard objectives in resource management plans should be written in such a way that enables progress towards them to be measured. There are a number of ways that councils can monitor the effectiveness of natural hazard provisions in their plans. Councils may need to revise the provisions in their plans when: Regional and district plan reviews are good opportunities to consider new information and data relating to natural hazards. A programme of consultation should accompany any changes to hazard information gained by the council. This ensures that communities
are aware of the hazards that they face. Plan effectiveness can be monitored by examining the: IntroductionThis tutorial concerns monitoring, an important process in most organisations. It is critical for the effectiveness of risk management and control assurance. The tutorial addresses several aspects of monitoring:
Monitoring business performance is an important task for all managers, but it is only addressed here insofar as it is an element of risk management or control assurance. Monitoring is an integral part of the ‘monitor and review’ stage of the risk management process (Figure 1). We distinguish between monitoring and reviewing: monitoring should be a continuous or more-or-less continuous process, whereas reviewing is a distinct, periodic and sometimes infrequent activity. (Control self-assessment is an important review process that is discussed here.) Figure 1: MonitoringMonitoring may be undertaken for a range of purposes, but there are several underlying principles that apply in most cases. The focus in this tutorial is on monitoring that contributes to better management of risk and improved effectiveness of controls. Monitoring is not just about compliance; its focus should be on encouraging the organisation to do better. After an organisation has made an important decision, it must ensure that decision remains worthwhile as both the internal and the external context evolves. This requires an understanding of the assumptions that have been made, and their current validity, based on monitoring the sources of uncertainty in the external and internal environment. Successful organisations anticipate changes in the external and internal environment, capitalise on opportunities and adjust past decisions. When developing an approach to monitoring, some of the most important questions are:
ResponsibilitiesRisk owners (line managers) are responsible for monitoring their business environment, ensuring the risk assessments for their risks are still valid and checking with control owners that the critical controls for their risks are in place and working effectively. Control owners are responsible for monitoring critical controls, and indicators of potential control weaknesses, to maintain the effectiveness of critical controls. Assurance providers, such as specialist functions at Line 2 of the three lines of assurance and internal audit at Line 3, may also monitor critical controls for effectiveness. Steps in the monitoring processTable 1 sets out the steps in defining and monitoring indicators. Detail is provided in the sections that follow. Table 1: Setting indicators to be monitored
Step 1: Define critical targetsDay-to-day monitoringThe simple bow tie structure in Figure 2 is useful for visualising the focus areas for day-to-day monitoring. Table 2 provides additional detail. The focus areas are not distinct and there are many overlaps. Figure 2: Focus for day-to-day monitoringTable 2: Identifying what is important
Critical controls are those whose effectiveness contributes materially to the achievement of the organisation’s strategic or business objectives, or that are required for policy, contractual or regulatory compliance. They are often associated with risks having high potential exposures. Less frequent monitoringOther focuses of monitoring in the context of risks and controls, not discussed in detail in this tutorial, include:
Step 2: Decide how to monitor changesMonitoring activities should generate a net benefit for the organisation. This means that the costs of monitoring should not be excessive. Ideally, use existing information systems for collecting, storing and analysing data for the main indicators that are of interest, rather than undertaking special-purpose data collection. Before initiating collection, storage and analysis activities for indicators that are not collected routinely, evaluate the relative benefits and costs of doing so. Some proposed indicators may be expensive to collect, and these may either be not collected at all, or collected only infrequently. For example, stakeholder, community and staff surveys may provide very important information to support business decisions, but they are expensive to conduct to a high standard, particularly if they require face-to-face interviews; hence many organisations do not conduct them often. Step 3: Select indicatorsLeading and lagging indicatorsLeading indicators look forward: they relate to changes or events that may happen in the future. They are explanatory and predictive in nature, supporting forecasts of future events and trends. Examples could include market trends associated with business inputs and products, the rate at which complaints are being received or the frequency of low impact safety near misses. Lagging indicators look backwards: they relate to changes or events that have happened already. They can clarify and confirm what has occurred and current circumstances, but they do not predict events or trends. Examples might include monthly sales and revenue, the nature and value of insurance claims or the rate of rejects coming off a production line. In many instances, leading indicators are related to the main inputs to business processes, while lagging indicators measure outputs of those processes. Many organisations use lagging indicators, based on measures of performance and outcomes. However, for an organisation to be agile and exploit changes, it is beneficial to identify changes early, so decisions can be made and responses can be implemented quickly. This requires us to use leading indicators if possible. SMART indicatorsThe acronym SMART was originally developed for goals and objectives, but it can also be used to describe the attributes of indicators for monitoring. Table 3 show how SMART might be interpreted for indicators used for monitoring. Table 3: SMART indicators
Examples of indicatorsTable 4 shows example of indicators. Table 4: Examples of indicators
Step 4: Track indicatorsIdeally, important indicators will be tracked as part of the organisation’s routine performance management system. Performance might be recorded in one or more individual systems, such as a production system, a sales management system or a financial system, or disparate measures might be collected in a more integrated way in a balanced scorecard. Trigger levels and alertsMonitoring of indicators should be linked to trigger levels for alerting managers and initiating actions (Figure 3). In practice, an indicator may be tracked on a control chart or an information system (Figure 4), ideally with automatic alerts when specified thresholds are breached.
Figure 3: Changes in the level of an indicatorFigure 4: Monitoring an indicator through timeWhen setting trigger levels, key questions for each of the most important indicators include:
Thresholds and accountabilities for monitoring indicators can be summarised in a form like Table 5. The table can be simplified if extremes are only important in one direction: for example, rising prices for business inputs like raw materials or energy may be critical, but low prices may be of far less interest except insofar as they offer an opportunity on which the organisation can capitalise. Table 5: Thresholds and accountabilities
TimingYou should tailor the frequency of monitoring to the potential rate of change of the indicator and its consequences (‘to the speed of risk’), or more frequently if required by regulators. Table 6 shows some examples. Table 6: Frequency of monitoring
Monitoring controlsAs set out in Table 5, for each significant risk and each critical control there should be clarity about:
Some of the responsibilities might be specified in position descriptions. Planning how, when and by whom controls are to be monitored is important: whether for an operational manager at Line 1 of the three lines of assurance; a more senior line manager at Line 2; a manager of a Line 2 assurance function such as health and safety or asset integrity; or a manager of a Line 3 independent audit function. At Line 1, operational managers should know if they are control owners. This should be recorded in the risk management information system, and often it is specified in a procedure or a job description. Monitoring may take the form of:
Step 5: Respond to alertsThe actions to be taken when indicators reach specified thresholds, the alert levels (T1) and action levels (T2) in Figure 3 and Table 5, should be planned, where it is practicable and sensible to do so. Matters that should be considered include:
ConclusionsMonitoring is an integral part of the ‘monitor and review’ stage of the risk management process. It is an important process in most organisations and critical for the effectiveness of risk management and control assurance. Monitoring the business environment and indicators associated with the causes of risks is important for risk owners, as it helps to ensure their perspectives of and assumptions about the risks for which they are responsible are still valid. Monitoring indicators of control performance and control management is important for control owners, as they are responsible for monitoring critical controls and maintaining their effectiveness. It is also important for assurance providers at Line 2 and Line 3 of the three lines of assurance. Effective monitoring should:
Wherever possible, monitoring should be integrated into the organisation’s day-to-day business processes and systems, so that:
|