Developing methods for assessing concept maps of process safety

Abstract

Process Safety is a key element of engineering that ensures the rapid human advancements can occur safely. It is important for the people who are already working in the process industries, and for those who will join them in the future. Process Safety education is a mandatory component and should be in the chemical engineering syllabus in every university around the world.

A concept map is a graphical representation of information that shows the relationships between concepts. Since its development, it has been used to represent ideas and concepts in a simple and holistic manner. This study investigates whether concept maps may be used to assess learning by individual students and by cohorts of students of the Process Safety domain. To achieve that objective, a method of assessing concept maps through a categorical scoring system is proposed by developing appropriate categorisations for the concepts, links and propositions.

The concept categories for the Process Safety domain were developed and validated via a novel process to remove human subjectivity. The process involved defining ten categories into which each of the concepts could be assigned. Several sets of concept maps were analysed independently by three assessors, requiring the assessors to assign every concept into one of the proposed categories. Analysis of the assessors’ responses was aided by presenting their responses in a three-way table. The use of the novel table allowed the assessors’ responses to be compared effectively. This comparison tool enabled identification of problematic categories for further refining.

The analysis of the distribution of the concepts, with the help of the proposed Link Quality Index, revealed more information about students’ understanding of the topic. This study analysed different types of connections between concepts to obtain more context and understanding of the concept maps, which represents students’ grasp of the Process Safety knowledge. The students generally appreciated the non-physical preventative measures, such as procedures and maintenance, but they did not recognise the importance of the education, training and values, such as responsibility towards Process Safety. This information is useful in helping to redesign the curriculum.

This study also proposes a method that may be applied beyond Process Safety domain; the classification of propositions into one of three proposed attributes, Professional Practice, Values, or Technical Knowledge. The application of this method to engineering students’ concept maps revealed that students show high awareness towards Technical Knowledge. Students also demonstrate the ability to use higher order thinking in explaining the relationship of the behaviour of the Professional Practices and Values. Upon applying this method to nursing students’ concept maps, on the topic of Oxygenation, it is found that nursing students had similar patterns in their awareness of technical attribute. However, nursing students generally were more oriented towards using lower order thinking in explaining relationship between concepts.

Overall, this study found that concept maps are a useful method to be applied in Process Safety domain; however, students need to be aware of the importance of having complete and clear propositions which are essential in indicating their understanding.