The State-of-The-Art of Enterprise Risk Management Maturity Models: A Review
Franciskus Antonius Alijoyo1, Ridwan Hendra2, Kevin Bastian Sirait3*
1Faculty of Economics, Universitas Katolik Parahyangan, Bandung, Indonesia. Email:
2Center for Risk Management & Sustainability, Bandung, Indonesia. Email:
3Center for Risk Management & Sustainability, Bandung, Indonesia. Email: [email protected]
Abstract. The risk management maturity model (RMMM) has the purpose of helping the firms in assessing their risk management practices and effectiveness in managing risks for various sectors, and yet there is a lack of studies in tracking the progress and status of RMMM at the enterprise level.
Following this issue, the purpose of this research is to list and synthesize the RMMM and map their similarities and differences to determine and identify the state-of-the-art of the existing enterprise RMMM. The synthesizing process of the enterprise RMMM uses the approach of literature review on the existing RMMM established after the introduction of the ISO 31000 standard in 2009. Moreover, the enterprise RMMM of interest are the models that emphasize practicality instead of theoretically.
The findings show that enterprise RMMM has its level of complexity and characteristics, indicating that not all maturity models are suitable for every firm, and the aspect of technology has an increase of importance within the current state-of-the-art of enterprise RMMM. Therefore, firms are recommended to select a RMMM that matches their risk management capabilities and effectiveness and capitalize on the advancement of technology to enhance their risk management maturity.
Keywords: Risk Management Maturity Models, Enterprise Risk Management, State-of-The-Art
1. Introduction
The risk management maturity model (RMMM) has the function to assess the firm's attributes and capability in terms of its risk management effectiveness and quality in managing risks for consistent implementation and continuous improvement of its risk management process [1]. However, every firm has different needs in implementing risk management due to the nature and the characteristic of the industry in which the firm operates. Moreover, the importance and the characteristic of risk management are different among firms and the parties that are associated with the firms [2]. Some of the firms are implementing extensive risk management practices to cope with the high degree of the risks they are faced due to the nature of their industries, while some of the firms are not [3]. Consequently, all firms have different maturity levels in terms of risk management effectiveness in coping with the risks.
Following this circumstance, some problems and damages arising from the firms' different maturity levels of risk management implementation and effectiveness. For instance, one of the major events that uncovered the problems and damages on the firms' risk maturity level is during the 2008 financial crisis. Even though the firms adopt the tools and the techniques of risk management, it is shown that some firms are vulnerable due to sudden changes in the market during the crisis [4,5]. In which, it reflects the difference among the firms in terms of their risk management maturity and the damages are worse if the firm has shallow risk management maturity.
At the enterprise level, the lack of understanding in risk management and inadequate level of maturity may jeopardize the firms' value. If the firms are not able to manage their risks effectively, the public perceptions of the firms are negatively affected and deemed to be incapable of maintaining the firm's value and the responsibility to its customers. Based on the global survey conducted by Aon [6] in 2019, the top two risks faced by the firms are the economic slowdown and damages to the firm's reputation. A firm with a low maturity level in risk management practices can create a ripple effect on all aspects of the firm, including its
business performance and reputation. As shown in the research conducted by Farrell and Gallagher [7], the risk management practices conducted by the firms are correlated with their firm's value, in which firms with mature risk management practices show higher firm value.
Thus, the appropriate RMMM is required to help the firms in identifying their maturity in risk management understanding and implementations in managing risks and aid them in establishing necessary steps to improve their maturity level.
In this regard, the needs of RMMM are growing to help the firms of all industries to manage their risks. According to Macgillivray et al. [8], maturity modeling experiences an increase in acceptance in the academic and industrial fields. Various RMMM is developed by scholars, universities, consulting firms, and government institutions. The function and the application of RMMM are diversified across disciplines, even though the primary purpose and the concepts among the existing RMMM are similar.
The existing maturity models are heavily oriented in software development and software engineering fields, and its impact on the field of information technology is also affecting the maturity models used in project management due to the software development is commonly handled in a project manner [9]. In which, these maturity models are based on the principles of product quality during the times of ``quality revolution'' [10]. At the enterprise level, the RMMM is not solely focused on a single product or a single project, but it focuses on all aspects of the firm, which also covers the firm's decision-making capabilities to its effect on the firm's business performance as a whole. Moreover, the maturity models that are used at the enterprise level must be able to provide the firms with structured steps to be sustainable and continuously improving itself in the long-term.
To address this issue, the purpose of this paper is to present the existing enterprise RMMM to uncover its state-of-the-art status and mapping the models' similarities and differences.
Furthermore, the review on the enterprise RMMM is also to determine if the existing models are still relevant to the current dynamics of risks faced by the firms at the enterprise level within the aspect of practicality. Therefore, the result of this study is to deliver a comprehensive analysis of the enterprise RMMM and the implications it brings to the risks practitioners and researchers.
As for the structure of this paper, it is presented as follows. The previous studies regarding enterprise risk management and its relevancy with the development of RMMM, along with its characteristics, are described in Section II. Section III presents the design, the analysis, and the synthesizing process of the literature review on the enterprise RMMM. The findings on similarities and differences of the selected enterprise RMMM, along with its key characteristics, are presented in Section IV. Lastly, the conclusions of the research along with its practical implications are presented in Section V.
2. Related Works
This section presents the basis of the theoretical framework in terms of the relevancy between the risk management practice at the enterprise level and its RMMM. Thereby, it consists of two sub-sections: (1) the enterprise risk management — focusing on the idea and the role of risk management principles at the enterprise environment and (2) risk management maturity model — focusing on the function and the characteristics of RMMM in measuring the firms' risk management maturity.
A. Enterprise risk management (ERM)
Within the context of an enterprise, a firm has a relationship with its stakeholders (such as, its customers and investors), suppliers, the government, and also with the other [2]. An enterprise firm does not solely focus on the internal aspect of the firm when dealing with risks.
Furthermore, the firms also have to take into account all the possibilities on how the other parties may be affected by the decisions or events that occur outside of the firm and even the
possible unfortunate events that arise internally. In which, the firms' capability in managing risks that comes internally or externally has its implications to the firm's value [7,11–13].
In layman's terms, enterprise risk management (ERM) is generally described as the firm's capability in identifying, analyzing, measuring, and producing the necessary actions to mitigate the adverse effect of the risks to the firm's objectives or minimizing the impact of the risks that have transformed into problems in the future [14,15]. As described by Oliva [2], a good risk management practice involves the harmonization between the firm's risk management activities and its awareness towards risks to enhance its operational performance, tactical effectiveness, and decision-making capabilities. Thereby, an adequate ERM practices depend on the firms' attitude towards risks and how the risks are managed to put the firms in an advantageous situation.
As for the principles of risk management, it is introduced by ISO [16] under the ISO 31000 standard. These risk management principles are summarized as follows:
Risk management creates and protects value to the firms.
Risk management becomes an integral part of the firm which includes the use of risk management as part of the decision-making process by using the best available information and used to address the uncertainty faced by the firm.
Risk management practices is systematic, structured and timely within a firm and it is transparent and inclusive.
The risk management is tailored made for the firm.
Risk management takes into account the human and cultural factor of the firm.
Risk management practices is dynamic, iterative, adapt-able to changes and continuously improves the firm’s risk management maturity
Following these principles, the risk management activities are defined as coordinated activities that are taken under the consideration of the risks faced by the firms [16]. It is also implied that the implementation of risk management is not the same among the firms, and risk management does not have the trait of `one-size-fits-all' due to the different external and internal contexts faced by the firms. The firms' external context is driven by the nature of the industries in which the firm operates (e.g., financial services, manufacturing, and agriculture).
In contrast, the firms' internal context is driven by their internal aspect (such as the origin of ownership, the platform of their digital-based business process, and their target market).
Furthermore, one of the most intriguing phenomena is the shift of orientation between the traditional and the latest version of ERM. Unlike the traditional risk management that is focussing on downside risks and managing risk in silos [7,8,17,18], the latest version of ERM emphasizes on the integration and consolidated framework that enable firms to pursue the risk-reward perspective and favoring the innovation and positive risk attitudes to exploit opportunities. As such, the latest version of ERM would give a balance view between managing downside risk and upside risk [18]. By definition, the downside risk is the risk due to bad things that could happen, whereas upside risk is the risk due to good things that do not happen [18]. If a downside risk occurs, it will damage the value, hence we fail to protect the firms' value. The downside risks are related to the circumstances where we face some threats and fail to mitigate them, either or both of its likelihood and impacts. On the other hand, If an upside risk occurs, it will not damage or produce any value, hence we fail to create a value to the firm. The upside risks are related to the circumstances where we face some opportunities and fail to exploit them, either or both of its likelihood and impacts.
In the case of implementing ERM, each of the firms has its perspective, priority, and needs in coping with the risks. Even though the underlying purpose and principles of ERM are the same, the degree and nature of the risks that are faced among the firms are not, practically
speaking. It shows that each firm has its unique and distinct risks that inherently exist within the industries in which the firms operate.
Moreover, the dynamic changes within the environment that the firms operate are also contributing to the expansions of uncertainties and risks -- for example, the influence of technological advancement to the firm's environment in the era of industry 4.0. Within the era of industry 4.0, the firms are not only coping with their traditional risks but also facing the risks arising from the integrated framework between the firms' physical (e.g., operational risk) and digital activities (e.g., cyber risk and information risk) [19]. The impact of the risks within the era that are dictated by the advancement of technology may even be more severe than the traditional risks due to the interconnected network, information, and devices. Thus, firms are introduced to these new risks because of the never-ending changes that are occurred globally and forced to cope with it.
In this regard, the firms can obtain the benefits of ERM through enhanced governance of risk management or even in its basic form [20]. The effectiveness of the ERM that are conducted by the firms are related with the firm's capability in managing the risks that may come internally or externally, and also the firm's capability to adapt to changes. In other words, the effectiveness of ERM depends on the firms' maturity.
B. Risk mangement maturity models (RMMM)
Due to the different level of maturity of risk management among the firms within many industries and the increase of various risks that happens because of the changes within the firms' industry (i.e., the advancement of technology, and the regulations at the national and international scale), the needs of risk management maturity models (RMMM) have become a primary tool for the firms to evaluate its risk management effectiveness and maturity, especially at the enterprise level. The purpose of RMMM is to improve the firm's risk management process, compare its risk management performance, and demonstrate their risk management capabilities to the parties involved with the firms [5,21]. In essence, firms can use the RMMM to identify the lacking aspects of their risk management practice and formulate appropriate objectives to improve their maturity level. Furthermore, it can also be used to compare firm's risk management maturity against its competitor or even against the best practice of risk management [22].
In measuring the firm's risk management maturity level, it often includes the aspect of the firm's organizational structure, culture, strategy, and business process [5]. The idea of RMMM is taken into account the firm as a whole in terms of its capability and effectiveness in dealing with risks rather than just focusing on a single aspect of a firm. Therefore, the firms are required to produce a strategic approach in producing structured risk management activities to implement proper risk management practices, especially the strategic approach produced by the firm's executives [22].
The structure of RMMM consists of two elements, namely, the maturity level and attributes. The element of attributes describes the firm's risk management implementation.
Meanwhile, the element of maturity level describes the firm's competencies and capabilities in practicing its risk management, which is usually represented using four or five maturity stages and ranging from ad-hoc to optimized risk management practices depending on the variation of the RMMM. Following these two elements, it is implied that the firm with mature risk management practices has an integrated risk management process, and the concepts and the implementation of risk management are embedded within the firm's business activities.
One of the earliest RMMM is the model proposed by Hillson [23] in 1997. Within this model, the aspect of culture is taken into account on measuring the firm's maturity in practicing risk management. It focuses on the firm's acceptance level on implementing the concepts and the principles risk management practices, and the firm's risk-awareness trait in
producing a risk-based decision-making mechanism. The model shows that the maturity of risk management of a firm is not solely pin-pointed at its techniques and approaches but also to the firm's attitude and awareness in managing risks, which also includes the firm's risk appetite, risk tolerance, and risk threshold.
To put it into perspective at the enterprise level and the current changes in the firms' environment, the firms are not only facing with the traditional risks (such as financial risk and operational risks) but also with the advancement of technology and the dynamics of the regulations at the national and international stage. As a result, there is a need for another attribute within the RMMM to uncover the firms' maturity level within the environment's dynamics changes.
Following this situation, Chapman [24] also proposed his version of RMMM. Even though the general idea is the same with Hillson [23], within this RMMM, it has the criteria of
``system'' which is explicitly designed to measure the firms' competence and maturity in formulating the firm's risk strategy and implementing risk management practice within the aspect of operational risk and the firm's business continuity management. This single attribute alone implies the importance of the firm's preparedness in regards to the potential events that may put the firm's sustainability at risk that may come internally or externally. In another viewpoint, this model is also considering the potential changes within the firm's environment or changes that happen globally that may introduce the firms to new risks and affects the firms negatively if not handled appropriately and effectively.
Based on the RMMM of Hillson [23] and Chapman [24], it is inferred that enterprise RMMM measures the firm's risk management maturity level in all aspects. Moreover, the risk management practices conducted by the firms must also create and protect their value, and through the implementation of risk management, it must also enable the firms to be sustainable in the long-term. In which, the results of the risk management process are directed to the firms internally (i.e., organizational structure, business activities, and its employee) and externally (i.e., stakeholders) [22]. Therefore, the RMMM does not only measure the firm's maturity in implementing the concepts and the principles of risk management within its business activities but also measuring the attitude and risk awareness that are embedded within the firm's organizational structure and business activities, and measuring the value and benefits of risk management to the firms.
3. Methodologies
Following the purpose of this research, the synthesizing processes in uncovering the similarities and the differences among the existing enterprise RMMM are using a literature review. As for acquiring the references of the enterprise risk maturity models, several services are used, such as ScienceDirect, CiteSeerX, and Google Scholars. In which the previous researches regarding enterprise RMMM are selected and analyzed to determine its relevancy with the context and the purpose of this paper.
The enterprise RMMM of interest are the models that are established after the introduction of the ISO 31000 standard by ISO [16] in 2009. Moreover, the focus point of the enterprise RMMM used in this research is the models that are predominantly oriented and emphasize on the practical aspect of measuring the firms' risk management maturity level. Thus, the enterprise RMMM that is considered as a proposed model or conceptual model is excluded in this research.
The research design is adapted from the approach used in the research conducted by Khoshgoftar and Osman [25] and Proença et al. [10]. Following this approach, the variables used for the analysis covers the perspective of the structure and the assessment criteria of the enterprise risk maturity models in order to uncover the similarities and differences among the models.
Under the variables of the enterprise RMMM structure, it is oriented on the composition of the models. In which, the key elements of the enterprise RMMM along with its maturity levels are selected to uncover the wide-range of spectrum in assessing firms' risk management maturity. Thus, the variables that are used to analyze the model are (1) the number of the model's key elements or criteria, (2) the number of the maturity levels, (3) lowest maturity level, and (4) highest maturity level.
As for the variables that are used to analyze the assessment approach of RMMM, it focuses on the aspect of the application of the model. The variables that are selected are focused on the implementation and the usage of the enterprise RMMM to the corresponding firms. Therefore, It uses the variables of (1) the availability of the assessment method, (2) Strong or weak points identification, (3) continuous improvement, (4) quantitative results, and (5) qualitative results.
4. Findings and Discussions
After filtering various models, it is found that there are six RMMM that fit the criteria on its implementation at the enterprise level after the introduction of ISO 31000 in 2009.Namely, the enterprise RMMM introduced by Audit Office Of New South Wales [26], CGMA [27], New Zealand Government [28], Association for Federal Enterprise Risk Management [29], OCEG [30], and Deloitte [31]. The synthesized results of the similarities and differences between these six models are presented in Table 1.
Among the six identified enterprise RMMM, each one of the models has similar key elements or criteria in assessing firms' risk management maturity, notably within the aspect of culture and risk management process. It shows a relation between the firm's attitude and awareness of risk with the firm's capabilities and effectiveness in embedding the concepts and principles of risk management into the firm's business activities. All of the identified RMMM have also incorporated the aspect of `technology' that considers the dynamics changes that may happen in the firms' environment. Moreover, the pattern of maturity levels among the models is also similar. In which it is described that the higher the firm’s risk management maturity, the concepts, and principles of risk management become more embedded and integrated into the firm’s business activities, and the firm becomes more aware of the risks that exist in its environment.
On the other hand, in terms of the differences between the models, it is found that the spectrum of assessing the firm's risk management maturity varies. Even though the general idea of the criteria in each of the model is the same, some of the models (i.e., New Zealand Government [28] and Deloitte [31]) have an emphasis on measuring the firm's effectiveness and capabilities in anticipating disruptive events that may negatively affect the firm which pointed to a specific event while the other models are only measuring the firm's risk management effectiveness in general circumstances. It is also uncovered that the existing RMMM has different complexity in implementing the models to measure the firm's risk management maturity, which may be unsuitable to a particular firm, especially for the firm that has a shallow risk management understanding and practices.
As for the state-of-the-art enterprise RMMM, it is found that the existing models are taken into consideration the effect of technological advancement to the firms' performance in managing risks. One of the attributes among the models is the implementation and the integration of technology to the firms' business activities and using its advantage to provide risk-based information that can be used by the firm to formulate effective risk-based strategies.
Furthermore, the current enterprise RMMM tend to assess the firm's maturity in terms of its capability to produce a stress-testing report to give the firm a general outlook on potential new risks.
Based on the differences and similarities of the identified models, the enterprise RMMM have their characteristics and complexity that may be unsuitable for a particular firm. In this regard, the maturity level at the enterprise level tends to appropriately and accurately measure
a firm's maturity based on the firm's nature and the environment in which the firm operates.
Following this distinction among the models and the nature of the firm, if a firm chooses an advanced risk maturity model without being a match with its capabilities, it could burden the firm rather than giving an accurate and practical approach to improving its maturity.
Table 1. The Synthesized Result of The Enterprise Risk Management Maturity Model Variabl
esa
Audit Office
Of New
South Wales [26]
CGMA [27]
New Zealand Governm ent [28]
Association for Federal
Enterprise Risk
Management [29]
OCEG [30]
Deloitte [31]
Panel A: The model's structure
A-1 5 8 4 4 4 3
A-2 5 4 5 5 5 5
A-3 Optimized Robust
ERM M5 Strategic Advantag
e
Risk Intelligent A-4 Initial
Just Getting Started
M1 Initial / ad-hoc Siloed Initial Panel B: The model's assessment
B-1 No Yes Yes No No No
B-2 Unspecified Yes Yes Unspecified Unspecifi
ed
Unspecifi ed
B-3 Yes Yes Yes Yes Yes Yes
B-4 Unspecified Yes Yes Unspecified Yes Yes
B-5 Unspecified No No Unspecified Yes Yes
Variables Description
A-1: Number of key elements.
A-2: Maturity levels.
A-3: Highest maturity A-4: Lowest maturity
B-1: Assessment method availability B-2: Strong or weak point identification B-3: Continuous improvement.
B-4: Qualitative results B-5: Quantitative results
a.The variables are adapted from Khoshgoftar and Osman [25] and Proença et al. [10].
5. Conclusion
This paper aims to uncover the similarities and differences among the enterprise RMMM and their state-of-the-art status. The enterprise RMMM of interest of this research is the models that are established after the introduction of the ISO 31000 in 2009 and emphasize on the practical aspect in measuring firms' risk management maturity.
The findings from the literature review suggest that the similarities among the model include the aspect of ―technology‖ in managing risks and using its benefits to provide the firms with the information that can enhance their risk-based strategies and decision-making capabilities. The current version of enterprise RMMM considers the effect of technological advancement to the firms and the potential new risk that may arise from it. In a sense, the
state-of-the-art enterprise RMMM responded to the changes that occurred happened globally.
Meanwhile, concerning the differences among the models, it is suggested that the level of complexity among the models is not the same and may not be suitable for a particular firm.
Thus, the firms are recommended to use enterprise RMMM that matches their level of risk management understanding and capabilities and its relevancy to their current aspect of the firm.
Following the findings of this research, it has its practical implication. The complexity among the enterprise RMMM is not the same and may not be suitable for every firm at the enterprise level. Therefore, the risk practitioners have to consider an appropriate enterprise RMMM that matches with the firm's nature along with the characteristics of the firm's environment and industry.
Although the findings fulfill the research's purpose, it has its limitations. First, there is a likelihood that there are significant or unique models excluded in the analysis due to the current scoping of the research which after the ISO 31000 standard introduction and with the emphasis on practicality rather than theoretically. Secondly, the number of selected models used in this research may be limited due to the proprietary rights of the RMMM that are owned by the firms exclusively and it is confidential for public use.
Following these two limitations, it is recommended to conduct further research to expand the scope of the enterprise RMMM. In which, the future research covers the expansion of the time-horizon to the times before the introduction of ISO 31000 standard and increasing the sample of enterprise RMMM in order to provide a more comprehensive analysis of the aspect that is suitable for generic and particular firms' maturity in implementing and understanding risk management. Last but not least, the future research might also consider to identify the need of an alternative RMMM which provides a balanced view to address and deal with downside and upside risks, while capturing the dynamic elements which affect the firms' sustainable development goals.
References
[1] Albliwi SA, Antony J, Arshed N. Critical literature review on maturity models for business process excellence. In: 2014 IEEE International Conference on Industrial Engineering and Engineering Management. Bandar Sunway: IEEE; 2014. p. 79–83.
[2] Oliva FL. A maturity model for enterprise risk management. Int J Prod Econ [Internet].
2016;173:66–79. Available from:
http://www.sciencedirect.com/science/article/pii/S0925527315005320
[3] Tegeltija M, Oehmen J, McMahon CA, Maier A, Kozin I, Škec S. Tailoring Risk Management in Design. In: Proceedings of the DESIGN 2018 15th International Design Conference. Dubrovnik: Design Society; 2018. p. 667–78.
[4] Strebel P, Lu H. Risk Management Starts At The Top. Bus Strateg Rev [Internet].
2010;21(1):18–23. Available from: https://doi.org/10.1111/j.1467-8616.2010.00639.x
[5] Mauelshagen C, Rocks S, Pollard S, Denyer D. Risk management pervasiveness and organisational maturity: a critical review. Int J Bus Contin Risk Manag. 2011;2(4):305–
23.
[6] Aon. Global Risk Management Survey 2019 - Executive Summary [Internet]. London:
Aon plc; 2019. Available from: https://aon.io/grms2019-sp-exec-rpt
[7] Farrell M, Gallagher R. The Valuation Implications of Enterprise Risk Management Maturity. J Risk Insur [Internet]. 2015 Sep 1;82(3):625–57. Available from:
https://doi.org/10.1111/jori.12035
[8] Macgillivray BH, Sharp J V., Strutt JE, Hamilton PD, Pollard SJT. Benchmarking Risk Management Within the International Water Utility Sector. Part I: Design of a Capability Maturity Methodology. J Risk Res [Internet]. 2007;10(1):85–104. Available from:
[9] Wendler R. The maturity of maturity model research: A systematic mapping study. Inf Softw Technol [Internet]. 2012;54(12):1317–39. Available from:
http://www.sciencedirect.com/science/article/pii/S0950584912001334
[10] Proença D, Estevens J, Vieira R, Borbinha J. Risk Management: A Maturity Model Based on ISO 31000. In: 2017 IEEE 19th Conference on Business Informatics (CBI).
Thessaloniki; 2017. p. 99–108.
[11] Froot KA, Scharfstein, DS, Stein JC. Risk Management: Coordinating Corporate Investment and Financing Policies. J Finance [Internet]. 1993;48(5):1629–58. Available from: https://doi.org/10.1111/j.1540-6261.1993.tb05123.x
[12] Hoyt RE, Liebenberg AP. The Value of Enterprise Risk Management. J Risk Insur [Internet]. 2011;78(4):795–822. Available from: https://doi.org/10.1111/j.1539- 6975.2011.01413.x
[13] Krause TA, Yiuman T. Risk management and firm value: recent theory and evidence. Int J Account Inf Manag [Internet]. 2016 Jan 1;24(1):56–81. Available from:
https://doi.org/10.1108/IJAIM-05-2015-0027
[14] FERMA. A Risk Management Standard. Brussels: Federation of European Risk Management Associations (FERMA); 2002.
[15] Fraser J, Simkins BJ. Enterprise Risk Management: Today’s Leading Research and Best Practice for Tomorrow’s Executives. New Jersey: John Wiley & Sons; 2010.
[16] ISO. Risk management — Principles and guidelines. ISO 31000:2009,International Organization for Standardization (ISO), Geneva, 2009.
[17] Lechner P, Gatzert N. Determinants and value of enterprise risk management: empirical evidence from Germany. Eur J Financ [Internet]. 2018;24(10):867–87. Available from:
https://doi.org/10.1080/1351847X.2017.1347100
[18] Alijoyo A. Enterprise Risk Management - Using ISO 31000. Bandung: Center for Risk Management Studies Indonesia; 2012.
[19] Bibby L, Dehe B. Defining and assessing industry 4.0 maturity levels – case of the defence sector. Prod Plan Control [Internet]. 2018 Sep 10;29(12):1030–43. Available from: https://doi.org/10.1080/09537287.2018.1503355
[20] Lundqvist SA, Vilhelmsson A. Enterprise Risk Management and Default Risk: Evidence from the Banking Industry. J Risk Insur [Internet]. 2018 Mar 1;85(1):127–57. Available from: https://doi.org/10.1111/jori.12151
[21] Strutt JE, Sharp J V., Terry E, Miles R. Capability maturity models for offshore organisational management. Environ Int [Internet]. 2006;32(8):1094–105. Available from: http://www.sciencedirect.com/science/article/pii/S0160412006000912
[22] Wieczorek-Kosmala M. Risk management practices from risk maturity models perspective. J East Eur Manag Stud [Internet]. 2014;19(2):133–59. Available from:
http://www.jstor.org/stable/24330969
[23] Hillson DA. Towards a Risk Maturity Model. Int J Proj Bus Risk Manag. 1997;1(1):35–
45.
[24] Chapman RJ. Simple Tools and Techniques for Enterprise Risk Management. 2nd ed.
Chichester: John Wiley & Sons; 2011.
[25] Khoshgoftar M, Osman O. Comparison of maturity models. In: 2009 2nd IEEE International Conference on Computer Science and Information Technology. Beijing:
IEEE; 2009. p. 297–301.
[26] Audit Office Of New South Wales. Risk Management Maturity Assessment Toolkit [Internet]. 2019 [cited 2020 Jun 25]. Available from: https://www.audit.nsw.gov.au/risk- assessment-tool
[27] CGMA. How to Evaluate Enterprise Risk Management Maturity [Internet]. London:
Chartered Global Management Accountant (CGMA); 2012. Available from:
https://www.cgma.org/resources/tools/evaluate-enterprise-risk-management.html
[28] New Zealand Government. Enterprise risk maturity [Internet]. 2020 [cited 2020 Jun 25].
Available from: https://www.digital.govt.nz/standards-and-guidance/governance/system- assurance/enterprise-risk-maturity/
[29] Association for Federal Enterprise Risk Management. TSA ERM Capability Maturity Model [Internet]. 2015 [cited 2020 Jun 25]. Available from:
https://www.aferm.org/resource/tsa-erm-capability-maturity/
[30] OCEG. A Maturity Model for Intergrated GRC. Phoenix: Open Compliance and Ethics Group (OCEG); 2016.
[31] Deloitte. Enterprise Risk Management — A ―risk-intelligent‖ approach. London: Deloitte LLP; 2015.
