We learn as one: Victoria's journey to collaborative lessons management

In November 2015, Victoria’s lessons management framework was released. The EM-LEARN framework established a model for lessons management, including a life cycle that defined cultural characteristics and lessons management process, based on research.

In 2014, the Victorian emergency management sector commenced an exploration of lessons management approaches. Emergency Management Victoria (EMV) conducted an environmental scan of local, national and international lessons management good practice. In addition, research was conducted through Charles Sturt University on what successful lessons management looks like in emergency management.¹

The environmental scan and research found:

• a strong culture of identifying themes, trends and lessons but not much success at ensuring lessons were learnt by creating lasting behaviour change
• no consistent model for capturing, analysing, sharing and implementing lessons leading to poorly defined roles, responsibilities and expectations
• that blame and shame, although diminished, was still prevalent in some parts of the sector
• a lack of visibility in the process of developing lessons, leading to a perception that personal observations and contributions were not influencing change
• there were many champions of learning practice in the field but there was a risk of losing momentum because of the perceived information ‘black holes’
• emergency management agencies (e.g. responder agencies, government departments and non- government partners) were working separately on lessons management, creating silos of knowledge and disconnected learning opportunities
• there was a limited understanding of principles and benefits of lessons

Overall, development of a learning culture was recognised as crucial to the success of lessons management. This required a strong foundation of robust collaboration and effective communication.

In November 2015, Victoria’s lessons management framework was released. The framework established a model for lessons management, including a life cycle that defined cultural characteristics and lessons management process (see Figure 1). The cultural characteristics include communication, accountability, just and fair, leadership and learning. The lessons management process includes two cycles that define how lessons will be identified and learnt based on the national model for lessons management analysis being observation, insight, lesson.² The release of the framework was a key communication and education tool as a first step to clarifying roles, responsibilities, processes, terminology and expectations.

EMV, together with the State Review Team (SRT), developed the framework over 12 months based on the environmental scan, research and extensive stakeholder engagement, including more than 70 meetings with 25 organisations. Through this process, a strong understanding of the sector’s shared strengths, challenges and future learning needs was identified. The framework was released as ‘approved for discussion’ to allow the communication and exploration of the lessons management concepts and lessons management model across the sector. The framework outlined five key project areas to support implementation: governance, communication, process, training and technology.

State Review Team

The SRT (previously the State Debrief Group) is Victoria’s key governance committee for lessons management, the development of the framework and championing lessons management over many years. The SRT began as a small group of people passionate about sharing and learning collaboratively, representing the traditional responder agencies. In 2018, it has evolved to consist of representatives from 17 emergency management organisations including non-government organisations, government departments, local government and agencies. The SRT focus has expanded beyond fire emergencies to a state-wide, coordinated approach. In particular, setting standards, capability and consistency for operational assurance and learning activities (monitoring, debriefing and reviewing) and lessons management processes to ensure continuous improvement. This includes all communities, all hazards, all year round and Class 1 and 2 emergencies, as defined by the Emergency Management Act 2013.³ The SRT is now the leadership group that provides strategic influence, direction and state-level oversight of operational assurance and learning activities to promote sector wide consistency, learning and continuous improvement in a coordinated and effective manner.

Figure 1: The EMV lessons management lifecycle is based on research and defines cultural characteristics and lessons management process.

Governance

The SRT was originally an informal network that would meet on an ad hoc basis to share, develop materials and work on progressing operational learning and improvement. Over a number of years the SRT has developed into a formal group including:

• meeting monthly with a focus on sharing agency progress and achievements
• a rolling chairperson that is held by representatives of the SRT
• a terms of reference that is reviewed as requireda formal governance structure where the SRT reports into the State Coordination Team (SCOT) which is chaired by the Emergency Management Commissioner.

The Operational Assurance and Learning Arrangements were approved by the SRT and SCOT in August 2017. The arrangements use the lessons management process in the framework to outline how community, business, industry, government and agencies provide learnings and experiences and how the SRT will analyse for trends, identify lessons, implement change and improvement and conduct monitoring and measuring to highlight lessons that have been learnt and that have resulted in behavioural and organisational change. The arrangements have been used  to  identify  Victoria’s first state-level, multi-agency lesson based on 27 observations, across greater than three events and six insights. There are actions being undertaken to support behaviour and organisational change based on the lesson and it is being distributed and implemented across the sector. The lesson has also been a focus during a real-time monitoring and evaluation deployment in late 2017 where further evidence was gathered to support the lesson.

Communication

In the early years, the SRT was sharing learnings and collaborating on the best ways to learn together as a sector and, in particular, it established the Post Season Review process. The Post Season Review Report was a key communication tool for sharing learnings from multi-agency debriefs at the conclusion of the fire danger period to inform continuous improvement activities prior to the next fire danger period. The document was released in 2011-12 and has since evolved to the Emergency Management Operational Review. In 2014-15, the Post Season Review was superseded by the Emergency Management Operational Review, which moved towards a broader focus on:

• year-round – broadened timeframe from the fire danger period to financial year
• multi-hazard – expanded beyond bushfire incidents
• all phases – expanded beyond response only activities
• multi-agency – expanded beyond responder agencies.

The Emergency Management Operational Review is now divided into three sections:

• Section 1 provides an overview and narrative outlining the broad spectrum of emergency management activities undertaken across the  sector  before, during and after emergencies experienced during the financial
• Section 2 includes a selection of case studies that were developed over the financial year and demonstrate the variety of incidents managed by Victorian emergency management
• Section 3 provides an update on the themes and insights identified during the financial year and highlights good practice, changes and improvements as part of an ongoing cycle of learning improvement.

The Emergency Management Operational Review is available online at www.emv.vic.gov.au

Process

The SRT is particularly focused on building consistency in operational debriefing, real-time monitoring and evaluation and reviewing processes. Resources are annually produced by the SRT and disseminated across the sector including guidelines, templates and supporting documentation. The Debriefing Guidelines has evolved from a focus on post-fire-danger period debriefing to establishing principles for planning, conducting and managing outputs from debriefs including direction on regional, community and interstate and international deployment debriefing.

The SRT was a key group of subject matter expertise that supported the Victoria Police Strategic Emergency Management Assurance Team (SEMAT) deployments. SEMAT was developed by Victoria Police (VicPol) out of the 2009 Black Saturday Royal Commission. It is a real-time assurance function that VicPol provided to the Emergency Management Commissioner or Chief Commissioner of Police to ensure effective control of response is established for each emergency. In 2014, responsibility for state-level response coordination in Class 1 and 2 emergencies transferred from VicPol to EMV.

The Country Fire Authority (CFA) and Department of Sustainability and Environment (now Department of Environment, Land, Water and Planning) also established a real-time performance monitoring (RTPM) capability after the alpine fires in 2005 to provide quality assurance and feedback of incident management structures. Over time, the program broadened to also apply to response structures at the regional level.

The Department of Health and Human Services (DHHS), the Australian Red Cross and local governments also established a real-time evaluation capability that focused on real time learning during relief coordination at state, regional and local government tiers. In 2015, responsibility for state-level relief and recovery coordination transferred from DHHS to EMV.

Due to the work of the SRT the real-time performance monitoring capability and real-time evaluation functions have developed to form a real-time monitoring and evaluation capability, which is applied during the readiness and response of Class 1 and 2 major emergencies and relief and early recovery phases of Class 1, 2 and 3 major emergencies, superseding real-time performance monitoring and real time evaluation functions. Since the real-time monitoring and evaluation capability has been formed in mid-2017, teams have been deployed at the incident and regional levels to inform real-time learning during a storm event and into the State Control Centre during a heat event.

Training

The SRT has always had a focus on building capability within organisations and across Victoria to support operational assurance and learning activities. In particular, the SRT has contributed to the development, delivery and participation in training and workshops, including:

• how to implement a culture that supports learning and improvement
• debriefing training in planning, conducting and managing outputs
• building real-time monitoring and evaluation capability
• developing skills in the lessons management process of observation, insight, lesson analysis
• assisting with organising and attending the AFAC Lessons Management Forum and workshops

The focus on building multi-agency capability including SRT co-facilitation of training and workshops has supported the implementation of lessons management across the state through empowering individuals to contribute to the continuous improvement of the sector. This can be seen in the diverse attendance at the sessions and local initiatives being undertaken.

The SRT is represented on the AFAC Knowledge, Innovation and Research Utilisation Network and is part of a National Lessons Management Working Group that builds consistency in lessons management across Australia.

Technology

The Observation Sharing Centre (OSC) was established by the CFA as an outcome of the Jones Inquiry (2011) into the arrangements made by the CFA for its volunteers. The inquiry highlighted the need for volunteers to be able to contribute experiences and innovative practices to the continuous improvement of the sector.⁴ The use of the OSC increased in momentum and received approximately 100-150 observations a year, including the submission of debrief reports that would be used to inform the Post Fire Danger Period Report.

EM-Share is an online platform that enables emergency management personnel to share observations and files from operational and non-operational activities, view insights and lessons, track user contributions and collaborate with others.

The OSC was a great start to sharing learnings but it was highlighted that an online platform to support lessons management required:

• greater transparency
• further configurability and analytical ability
• clear governance
• increased security, reliability and access
• a capacity to track change and improvement
• all lessons management resources and data to be in one location.

In October 2017, after two years of extensive work by the SRT, a lessons management IT system, EM-Share, was released. EM-Share enables users to share observations and files from operational and non-operational activities, view insights and lessons, track how user contributions facilitate the continuous improvement of the sector and collaborate with others about learning. Since its release, EM-Share has received over 1000 observations, with more than 60 insights identified and one state-level and multi-agency lesson.

Conclusion

Lessons management is about giving people the opportunity to share their experiences and learn from others to ensure continuous  improvement. People are empowered to actively contribute to continuous improvement to support the vision of safer and more resilient communities.

Through the collaboration of the SRT and their leadership role in lessons management, the sector is striving to continuously improve in real-time together rather than organisations or individuals learning independently and inefficiently. People are empowered to actively contribute to continuous improvement of the sector to support the vision of safer and more resilient communities. Lessons management provides the platform for holistic learning based on a diverse and comprehensive evidence base that wouldn’t necessarily be available to individuals or organisations.

The long-term vision is for continuous improvement activities to support improved behaviour change, organisational performance and  service-delivery planning for all communities, all hazards, all phases, all agencies and at all levels. Through the implementation of the framework and the collaboration of the SRT, lessons management in the sector will be integrated, evidence- based, continuous, consistent, transparent and holistic.

Victoria is starting to see a shift towards a learning and improvement culture. It is slowly moving away from recommendations and towards lessons; away from reports and towards case studies and away from action tracking and towards monitoring improvement.

Footnotes 

1. Jackson LM 2016, The influence of organisational culture on learning lessons: implementing a lessons management life cycle, Australia Journal of Emergency Management, vol. 31, no. 1, pp. 18-23.
2. Australian Institute for Disaster Resilience 2013, Australian Disaster Resilience Handbook 8: Lessons Management.
3. Emergency Management Act 2013 (Vic). At: http://www9.austlii.edu.au/cgi-in/viewdb/au/legis/vic/num_act/ema201373o2013236/
4. Jones DJ 2011, Report of inquiry into the effect of arrangements made by the Country Fire Authority on its volunteers. At: www.vfbv.com.au/index.php/news/inquiries/jonesinquiry