By Danial Sadeqi, eLearning Specialist, PASAI

When the COVID-19 pandemic brought international travel to a halt, we could no longer deliver in-person training to member SAIs. We responded by transitioning our programme delivery online to continue building Pacific SAIs’ capability. In doing so, we worked through several challenges, both in terms of infrastructure and online training delivery.

We recruited an eLearning Specialist to provide expertise in converting existing materials into an online format using instructional design principles and best practice. Our eLearning Specialist also worked with subject matter experts to design and develop online training and capability development solutions.

Throughout the development of our online courses, whether re-purposing existing materials or designing new content, we followed best practice in instructional design and workplace training to ensure high quality materials.

Our approach

We have maintained the learning and development standards which apply when designing in-person training. [1] That is, establishing learning roles, learning theories and methods used, and the models provided for applying learning (such as tasks and assignments). In addition, we draw on empirical evidence and best practice in the field of online learning to ensure our courses meet eLearning standards as well. [1, 2, 3, 4, 5, 6, 7, 8]

Evidence abounds in the importance of reflection [5, 9, 10, 11], and applying knowledge [4, 5, 10, 11] to facilitate learning. The “Kolb Cycle” [12] provides a four-stage learning cycle which not only captures different stages of learning (including reflection and applying knowledge) but also recognises four different learning styles. Therefore, we incorporate this model into our course design.

Our training objectives take inspiration from “Bloom’s Taxonomy” (revised in 2001 with verbs for the category names) [13]. We design training so participants will not only be able to remember course content, but also truly understand and apply it, and so on up the pyramid. Ideally they will be able to create new work that integrates their learnings.

To ensure quality we endeavoured to create courses which:

• have a professional design and layout [1, 2]

• address learners’ current needs [14, 15]

• provide an engaging learning experience [16]

• induce frequent learner interaction [2, 17, 18]

• increase flexibility [19] and

• are segmented to allow gradual accumulation of knowledge. [20]

While the principles above informed all steps of our learning design, our aim was to set realistic goals considering our resources, timelines, budget, and other commitments. Moreover, curriculum development is a continuous process which takes a significant amount of time before even getting close to the desired outcome. [2, 15] Therefore, we aimed to get our online courses up and running as fast as we could to avoid any disruptions to our training. Then we would be in a place to move towards gradual improvement of our courses.

Once we established a project plan for converting upcoming courses into an online format, we procured a Learning Platform to host these courses, manage all eLearning content, and track and administer learners’ progress. Choosing a webinar tool was the least challenging part of the process since Zoom had already established itself as the dominant platform of choice.

The Chair of our Governing Board officially launched our Learning Platform in November 2020. The first online training which followed included a webinar and a course component on the Learning Platform. In the webinar, the facilitator delivered a Powerpoint presentation with a Q&A section. The course on the Learning Platform included a forum discussion, workshop slides, supplementary materials and a survey.

Our course design

We then set out to change the design of our online courses to maximise learning. Given that a detailed account of all the work which went into our course design and the research that informed it is well beyond the scope of this blog, I will go over three main design components.

We started off by adding a ‘Pre-course’ section which aimed to improve learning by activating existing knowledge and preparing the learners for the workshop ahead. Activating learners’ existing knowledge benefits them by acting as a building foundation for the training that follows [21]. We did this by adding:

• a survey which elicits learners’ current knowledge of the subject matter and their expectations from the course and

• a task which requires learners to upload recent work on the topic and answer guiding questions that elicit their familiarity with the topic.

The analysis of the above helps facilitators’ make the content more relevant to the learners and improve learning outcomes. [2, 14, 15, 16]

We added a practical, hands-on element based on the learners’ working context to give them an opportunity to apply their learnings and crystalise their knowledge [11, 12] in an authentic context. [22, 23] For instance, in our ‘Report Writing’ course, we ask learners to:

1. select an extract from a recently written audit-focused report and assess it against a set of given questions all of which are based on the main principles taught during the course

2. edit the extract with tracked changes based on what they learnt in the course and

3. submit their edited extract with comments on why they changed things.

This task given after the last webinar provided the learners with an opportunity to activate what they had learnt during the course in an authentic context. The facilitators provided detailed feedback on all submissions.

Our commitment to innovation and continuous improvement

We offer a post-training evaluation, and respond to our learners’ feedback by continuously improving our training offerings and Learning Platform.

User queries and trouble-shooting proved to be the most challenging aspect of online delivery. Given that this was the first time many participants and facilitators were doing an online course, we supported our learners with queries regarding registering for, logging in and navigating the Learning Platform.

We have streamlined the registration process and made significant improvements to the Learning Platform to enhance user experience. To achieve this, we re-designed our user interface, simplified the platform navigation and made the registration process easier for participants.

These changes resulted in a decrease in queries and an increase in user satisfaction ratings, in particular for ease of use and navigation. According to Allen, [2] “more responsive customer service, increased throughput, reduced accidents and errors,” and “better-engineered designs” help the success of eLearning.

Next steps

We are working to mainstream gender and inclusion into our programme of work. This is reflected in our improvements to our Learning Platform to ensure it meets accessibility standards and provides an inclusive learning experience for all our learners.

References

[1] de Freitas, S. (2014). Education in computer generated environments. Routledge.

[2] Allen, M. (2016). Michael Allen’s guide to e-Learning (second edition): Building interactive, fun and effective learning programs for any company. Wiley.

[3] Jia, J. (2012). Educational stages and interactive learning from kindergarten to workplace training. Information Science Reference.

[4] Ifenthaler, D. (Ed.). (2018a). Digital workplace learning: Bridging formal and informal learning with digital technologies. Springer.

[5] Clark, R. C. & Mayer, R. E. (Eds.) (2016). E-Learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning (4th ed). John Wiley & Sons, Incorporated.

[6] Hung, D., & Khine, M. S. (Eds.). (2006). Engaged learning with emerging technologies. Springer.

[7] Mayer, R. E. (2011). Instruction based on visualizations. In R. E. Mayer & P. A. Alexander (Eds.), Handbook of research on learning and instruction (pp. 427–445). New York, NY: Routledge.

[8] Mayer, R. E. (2014). Research-based principles for designing multimedia instruction. In V. A. Benassi, C. E. Overson, & C. M. Hakala (Eds.), Applying science of learning in education: Infusing psychological science into the curriculum. Retrieved from the Society for the Teaching of Psychology web site: http://teachpsych.org/ebooks/asle2014/index.php

[9] Weedon, E. M., & Cowan, J. (2002). Commenting electronically on students’ reflective learning journals. In Rust, C. (ED.), Improving student learning, theory and practice using learning technology (pp. 99-110). Oxford, UK: Oxford Centre for Staff & Learning Development.

[10] Moon, J. A. (2004). A handbook of reflective and experiential learning. London: Routledge Falmer.

[11] Cowan, J. A. (2006). On becoming an innovative university teacher (2nd ed.). Buckingham, UK: Open University Press.

[12] Kolb, D. A. (1984). Experiential learning. Englewood Cliffs, NJ: Prentice-Hall.

[13] Anderson, L.W. (Ed.), Krathwohl, D.R. (Ed.), Airasian, P.W., Cruikshank, K.A., Mayer, R.E., Pintrich, P.R., Raths, J., & Wittrock, M.C. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s Taxonomy of Educational Objectives (Complete edition). New York: Longman.

[14] Wallace, D. (2019). Know thy learners. Talent Development, 73(3), 46-50.

[15] FAO. 2021. E-learning methodologies and good practices: A guide for designing and delivering e-learning solutions from the FAO elearning Academy, second edition. Rome. https://doi.org/10.4060/i2516e

[16] Hung, D., Tan, S. C., & Koh, T. S. (2006). Engaged learning: Making learning an authentic experience. In D. Hung & M.S. Khine (Eds.), Engaged learning with emerging technologies (pp. 29-48).

[17] Harteveld, C. (2011). Triadic game design: Balancing reality, meaning and play. Springer.

[18] Harteveld, C. (2012) Making sense of virtual risks: A quasi-experimental investigation into game-based training. Doctoral Thesis. Delft University.

[19] Hashey, A. I., & Stahl, S. (2014). Making online learning accessible for students with disabilities. Teaching.

[20] Dziuban, C. Howlon, C., Moskal, P., Johnson, C., Parker, L., & Campbell, M. (2018). Adaptive learning: A stabilizing influence across disciplines and universities. Online Learning, 22(3), 7-39.

[21] Merrill, M. D. (2002). First principles of instruction. Education Technology Research and Development, 50 (3), 43-59. https://doi.org/10.1007/BF02505024

[22] Jonassen, D. H., & Strobel, J. (2006). Modeling for meaningful learning. In D. Hung & M.S. Khine (Eds.), Engaged learning with emerging technologies (pp. 1-27). Springer.

[23] Jonassen, D. H., Howland, J., Moore, J., & Marra, R. M. (2003). Learning to solve problems with technology: A constructivist perspective, 2nd. Prentice-Hall.

The Pacific Association of Supreme Audit Institutions (PASAI) is the official association of supreme audit institutions (SAIs) in the Pacific region, promoting transparent, accountable, effective and efficient use of public sector resources in the Pacific. It contributes to that goal by helping its member SAIs improve the quality of public sector auditing in the Pacific to recognised high standards. Our blog includes topics that may help auditors think about some implications to service delivery because of the global coronavirus pandemic (COVID-19).