Learning reflections in Year 2
Reflection 1 (September 25th 2016)
I made the decision to defer Year 2 in 2015, mainly due to work commitments relating to an EU funded project QualiBuild for which I was responsible for course development and piloting. When planning for return in 2016, I was fortunate to receive permission to change my research proposal to a study which allowed me to carry forward the work of the QualiBuild project. By choosing a subject area for my research which is much closer to my own practice, I am much more invested in the outcomes of the study and the value of what I am hoping to develop.
The main learning theory underpinning my research proposal is learning styles and the use of multi-modal eLearning to appeal to different learning preferences. This is an area that I have been interested in for some time. I have witnessed over many years the preference of learners of construction skills for highly visual materials and learning by doing. I am also keen to explore the potential of eLearning with this cohort as it is an area where there is a dearth of studies. While I believe that eLearning could complement more traditional face-to-face delivery, I can also see that there are some obvious barriers to engagement, particularly computer literacy levels and lack of exposure to digital learning environments.
The first tutorial of 2016 in Aungier Street was a little bit strange as the main group were well formed and bonded from their Year 1 experience. Fortunately, there were a couple of familiar faces present who had also deferred their Year 2. It is definitely a different dynamic though from what I was used to in Year 1, where group projects and collaborative modules had been very effective in establishing good relationships.
The main message that I took from the first tutorial back was the importance of allocating time sufficiently across the three main activities of the final year project: 1) literature review/journal paper; 2) artefact development; and 3) ePortfolio. As such, my focus in the initial stages was on project management and time management. It became apparent from discussions within the group and with my supervisor that early ethics submission would be a key factor affecting progress on the work.
I am delighted to have Dr Claire McDonnell as my supervisor as I am familiar with her from Year 1. She has been very supportive of my change of research proposal and the value of working on something so close to my own practice.
The main learning theory underpinning my research proposal is learning styles and the use of multi-modal eLearning to appeal to different learning preferences. This is an area that I have been interested in for some time. I have witnessed over many years the preference of learners of construction skills for highly visual materials and learning by doing. I am also keen to explore the potential of eLearning with this cohort as it is an area where there is a dearth of studies. While I believe that eLearning could complement more traditional face-to-face delivery, I can also see that there are some obvious barriers to engagement, particularly computer literacy levels and lack of exposure to digital learning environments.
The first tutorial of 2016 in Aungier Street was a little bit strange as the main group were well formed and bonded from their Year 1 experience. Fortunately, there were a couple of familiar faces present who had also deferred their Year 2. It is definitely a different dynamic though from what I was used to in Year 1, where group projects and collaborative modules had been very effective in establishing good relationships.
The main message that I took from the first tutorial back was the importance of allocating time sufficiently across the three main activities of the final year project: 1) literature review/journal paper; 2) artefact development; and 3) ePortfolio. As such, my focus in the initial stages was on project management and time management. It became apparent from discussions within the group and with my supervisor that early ethics submission would be a key factor affecting progress on the work.
I am delighted to have Dr Claire McDonnell as my supervisor as I am familiar with her from Year 1. She has been very supportive of my change of research proposal and the value of working on something so close to my own practice.
Reflection 2 (October 15th 2016)
My focus at this stage is on ethics approval. I am fortunate in that I have support within my own organisation for my project, including management involved with the research committee. However, my preconception that ethics submission is purely a form-filling exercise was misplaced. I soon realised the significant upfront work required such as revisions to my research proposal and preparation of focus group outlines/questions and survey questionnaire. While this has been challenging from a time perspective, it is very useful in helping me focus on the details of my research and how I intended to conduct it. Following ethics approval submission, I am much clearer on the project plan and implementation.
One of the earliest tasks we have been given is to identify a target journal for our paper. This is also very useful for focus as it required an exploration of the various styles and formats required by different journals. I chose to target the European Journal of Training & Development, based on the training focus of my project and the EU project context. By choosing a particular journal, I was even clearer on how I would conduct my research and write it up.
For my second meeting with supervisor Claire, there was more detailed discussion on the direction of my literature review. She encouraged me to consider relevant learning theory beyond learning styles and to consider the challenges to this theory that have emerged. Most importantly, Claire has pointed me in the direction of cognitive load theory and its relationship with multi-modal/multimedia presentation of learning material. Specifically, an article by Richard E. Mayer on research-based design principles for multimedia instruction has been particularly useful.
This has guided and informed plans for my artefact development. I am intrigued by the area of memory and information processing and how it can affect the quality of learning. My reading now has moved towards authors such as Mayer, Moreno and Sweller who have conducted significant research into cognitive load theory, transient learning effect and multimedia instruction.
I am also finally using a reference management software. I settled on Refworks as it is used by the library at ITB and I have access to an account. I am finding this really useful for tracking my reading.
One of the earliest tasks we have been given is to identify a target journal for our paper. This is also very useful for focus as it required an exploration of the various styles and formats required by different journals. I chose to target the European Journal of Training & Development, based on the training focus of my project and the EU project context. By choosing a particular journal, I was even clearer on how I would conduct my research and write it up.
For my second meeting with supervisor Claire, there was more detailed discussion on the direction of my literature review. She encouraged me to consider relevant learning theory beyond learning styles and to consider the challenges to this theory that have emerged. Most importantly, Claire has pointed me in the direction of cognitive load theory and its relationship with multi-modal/multimedia presentation of learning material. Specifically, an article by Richard E. Mayer on research-based design principles for multimedia instruction has been particularly useful.
This has guided and informed plans for my artefact development. I am intrigued by the area of memory and information processing and how it can affect the quality of learning. My reading now has moved towards authors such as Mayer, Moreno and Sweller who have conducted significant research into cognitive load theory, transient learning effect and multimedia instruction.
I am also finally using a reference management software. I settled on Refworks as it is used by the library at ITB and I have access to an account. I am finding this really useful for tracking my reading.
Reflection 3 (November 6th 2016)
The main focus in recent days has been on preparation for my first focus group to be held at ITB. I have to accommodate colleagues who will be present on the day and also a couple who will be joining by teleconference. I found a useful article online by Anita Gibbs of Oxford University. It gives a good overview of focus group research, its limitations and practical organisation. Another article by Richard E. Krueger included particularly good guidance on designing questions.
The main objective of my first focus group is to get input from instructors on the potential for eLearning resources to complement existing QualiBuild training and the optimum format that they should adopt. This will help inform my own ideas and approach to the artefact development.
I have also been using Google Forms to develop my online survey questionnaire. This is to be conducted with trainers who were up-skilled under the QualiBuild project to deliver training to construction workers. The questions will follow three main themes: 1) the potential benefit of having online learning supports; 2) the optimum format for these resources and the topics that they should cover; and 3) the potential barriers/enablers to engagement with the resources.
The questions for both the focus group and survey questionnaire are closely related to my main research question and sub-questions. The data collected should be really useful to me as I refine the design of my eLearning artefacts.
My literature review has now become similarly focussed. I have specifically targeted research into cognitive load theory in multimedia instruction, specific studies into learning styles and eLearning with construction students and factors affecting engagement with online learning. Finding papers specific to construction workers has been challenging. Many of the studies I have found have been conducted on third level construction students rather than blue collar workers. However, they are still somewhat relevant as many of the third level students have progressed from trade qualifications to these programmes and have similar learner characteristics.
References:
Gibbs, A. 1997. Focus groups. Social Research Update, Winter. Retrieved October 10th 2016 from: http://isites.harvard.edu/fs/docs/icb.topic549691.files/Focus_Groups.pdf
Krueger, R. E. (2002). Designing and Conducting Focus Group Interviews. Retrieved October 26th 2016 from: http://www.eiu.edu/ihec/Krueger-FocusGroupInterviews.pdf
The main objective of my first focus group is to get input from instructors on the potential for eLearning resources to complement existing QualiBuild training and the optimum format that they should adopt. This will help inform my own ideas and approach to the artefact development.
I have also been using Google Forms to develop my online survey questionnaire. This is to be conducted with trainers who were up-skilled under the QualiBuild project to deliver training to construction workers. The questions will follow three main themes: 1) the potential benefit of having online learning supports; 2) the optimum format for these resources and the topics that they should cover; and 3) the potential barriers/enablers to engagement with the resources.
The questions for both the focus group and survey questionnaire are closely related to my main research question and sub-questions. The data collected should be really useful to me as I refine the design of my eLearning artefacts.
My literature review has now become similarly focussed. I have specifically targeted research into cognitive load theory in multimedia instruction, specific studies into learning styles and eLearning with construction students and factors affecting engagement with online learning. Finding papers specific to construction workers has been challenging. Many of the studies I have found have been conducted on third level construction students rather than blue collar workers. However, they are still somewhat relevant as many of the third level students have progressed from trade qualifications to these programmes and have similar learner characteristics.
References:
Gibbs, A. 1997. Focus groups. Social Research Update, Winter. Retrieved October 10th 2016 from: http://isites.harvard.edu/fs/docs/icb.topic549691.files/Focus_Groups.pdf
Krueger, R. E. (2002). Designing and Conducting Focus Group Interviews. Retrieved October 26th 2016 from: http://www.eiu.edu/ihec/Krueger-FocusGroupInterviews.pdf
Reflection 4 (December 10th 2016)
I held my first focus group on November 14th at ITB. I had three colleagues in attendance and another two joined by teleconference. Thankfully the technology for the recording of the session worked well. The session ran really smoothly as I had spent the time well in planning and preparation. I just needed to keep people on topic and make sure that everyone had an opportunity to contribute. There was some very useful discussion which I was able to summarise by taking notes during the session. I will do some more detailed worked on this at a later date when I transcribe the recording.
Artefact development has been in full swing over the last few weeks. I have been using a number of different technologies to develop a series of interactive videos across 4 topics. This combines some existing animated videos from the QualiBuild project, which detail some of the issues with energy efficiency that the ‘O’ Brien’ family are experiencing in their home, with some 3D simulations created on SketchUp software. Everything is pulled together on PowerPoint with introductions, instruction and videos embedded to create a video with audio narration. This is then uploaded to Youtube for hosting before being embedded on H5P software where interactions are added.
The use of the different technologies has been time consuming as there are many different stages and layers. However, I am determined to use open source or readily available technologies to develop the resources so that there are minimum barriers to other tutors adding to the work in the future. I had considered Articulate Storyline, which I had used on a mini-project in Year 1, but was put off by the prohibitive cost of the software.
I have adopted the SAM model of instructional design (Allen, 2012), which is an agile process suited to evaluation by other trainers. I design a storyboard for each artefact which I forward to my colleagues via Moodle for feedback. Once finalised, I then develop a draft interactive video for testing and feedback. The biggest drawback with this approach has been the reliance on timely feedback from my colleagues, particularly during busy periods in the semester.
The WIP presentations were held in Aungier Street on December 6th (see video below). It was interesting to see that there were quite a few of the other projects that were either targeted at construction students and/or focusing on similar learning theories to my own work. I was also reassured to see that work on my project was relatively well advanced.
Reference:
Allen, M. (2012). Leaving ADDIE for SAM. ASTD Press.
Artefact development has been in full swing over the last few weeks. I have been using a number of different technologies to develop a series of interactive videos across 4 topics. This combines some existing animated videos from the QualiBuild project, which detail some of the issues with energy efficiency that the ‘O’ Brien’ family are experiencing in their home, with some 3D simulations created on SketchUp software. Everything is pulled together on PowerPoint with introductions, instruction and videos embedded to create a video with audio narration. This is then uploaded to Youtube for hosting before being embedded on H5P software where interactions are added.
The use of the different technologies has been time consuming as there are many different stages and layers. However, I am determined to use open source or readily available technologies to develop the resources so that there are minimum barriers to other tutors adding to the work in the future. I had considered Articulate Storyline, which I had used on a mini-project in Year 1, but was put off by the prohibitive cost of the software.
I have adopted the SAM model of instructional design (Allen, 2012), which is an agile process suited to evaluation by other trainers. I design a storyboard for each artefact which I forward to my colleagues via Moodle for feedback. Once finalised, I then develop a draft interactive video for testing and feedback. The biggest drawback with this approach has been the reliance on timely feedback from my colleagues, particularly during busy periods in the semester.
The WIP presentations were held in Aungier Street on December 6th (see video below). It was interesting to see that there were quite a few of the other projects that were either targeted at construction students and/or focusing on similar learning theories to my own work. I was also reassured to see that work on my project was relatively well advanced.
Reference:
Allen, M. (2012). Leaving ADDIE for SAM. ASTD Press.
Reflection 5 (January 30th 2017)
Work is continuing apace on development of videos. I have been fortunate to have such constructive feedback from my colleagues in the agile design process. Interestingly, I have had a very low response to the storyboards that I have forwarded in advance of video iterations. On the other hand, when I release a video for testing the response rate is very high. I believe that this is consistent with the fact that the trainers involved are all from similar backgrounds to the target learners for the project and, therefore, much prefer engaging with the active learning environment of the videos.
Feedback has been positive with plenty of comments and suggestions on improving the artefacts. What has been most prominent and interesting in these is the high number directly relating to Mayer’s research-based principles of multimedia instruction. The following are a sample of responses consistent with Mayer’s principles:
Reference:
Mayer, R. E. (2014). Research-Based Principles for Designing Multimedia Instruction. Applying Science of Learning in Education - Infusing Psychological Science into the Curriculum. Division 2 American Psychological Association. Retrieved online October 12th 2016 from: http://hilt.harvard.edu/files/hilt/files/background_reading.pdf
Feedback has been positive with plenty of comments and suggestions on improving the artefacts. What has been most prominent and interesting in these is the high number directly relating to Mayer’s research-based principles of multimedia instruction. The following are a sample of responses consistent with Mayer’s principles:
- Remove extraneous material (Coherence)
- Use less text/words (Redundancy)
- I think you need to embed some information on the subject matter using appropriately sized text at say, the top left or right of the video. Just like the News Channels do with logos and evolving news. Use colour for highlighting (Signalling Principle)
- Use arrows and labels to highlight heat loss pathways (Spatial Contiguity)
- A teaching video requires sequential or step-by step textual/graphical information to support the narrative (Temporal Contiguity)
- I suppose if the video could be broken up into two separate smaller videos it would be better, bite sized is best (Segmenting)
- Some terminology in questions not introduced in video (Pre-Training)
- Use intonation in narration, Try a more conversational tone (Modality, Personalisation and Voice)
- Suggest to maybe use a Character on the screen (Embodiment)
Reference:
Mayer, R. E. (2014). Research-Based Principles for Designing Multimedia Instruction. Applying Science of Learning in Education - Infusing Psychological Science into the Curriculum. Division 2 American Psychological Association. Retrieved online October 12th 2016 from: http://hilt.harvard.edu/files/hilt/files/background_reading.pdf
Reflection 6 (February 25th 2017)
As the development of the interactive videos has continued, a pattern has emerged where the duration is increasing with each topic covered. The first iteration of Video 1 was just over 5 minutes but subsequent videos have been 8, 11 and almost 13 minutes. This has been highlighted in feedback from colleagues. Most believe that the videos should be shorter and snappier, however, they agree that the content is all relevant and good and quality. There have been some suggestions to split Videos 3 and 4 in two, resulting in a total of six videos. The problem that I face at this stage is that the time required to create an additional two videos, regardless that most of the content is available, makes it unfeasible.
Following discussions with my supervisor, it is likely that I will have to settle for discarding content and reducing duration so that all videos are between 4 – 8 minutes, considered to be optimum for artefacts of this type. I am a little disappointed to be leaving out developed content but I understand that I have a lot of work remaining on the project which will need my attention.
I have been organising my second and final focus group for the week following Paddy’s weekend. The purpose of this focus group is to reflect on the artefacts developed and to evaluate their usefulness. This means that I will need to have all revisions completed in order to circulate final versions of videos two weeks in advance.
Following discussions with my supervisor, it is likely that I will have to settle for discarding content and reducing duration so that all videos are between 4 – 8 minutes, considered to be optimum for artefacts of this type. I am a little disappointed to be leaving out developed content but I understand that I have a lot of work remaining on the project which will need my attention.
I have been organising my second and final focus group for the week following Paddy’s weekend. The purpose of this focus group is to reflect on the artefacts developed and to evaluate their usefulness. This means that I will need to have all revisions completed in order to circulate final versions of videos two weeks in advance.
Reflection 7 (March 28th 2017)
I settled on a compromise for reducing the size of the interactive videos whereby I reduced Video 3 and split Video 4 in two. I also got access to a good quality microphone and made a big effort to make my narration more ‘conversational’. I took on board comments relating to use of characters and was able to integrate segments from existing QualiBuild videos. Adding speech bubbles to characters gave me the flexibility to use the characters for additional instruction.
I ended up with 5 videos in total and I am pleased with the final versions. On reflection, I had probably become a little over reliant on feedback from my colleagues and was not as self-critical as I could have been with initial work. When I looked back over the earlier iterations I was disappointed to see times when I had fallen into the trap of presenting information in traditional PowerPoint slide format. This was something that I had been very keen to avoid from the outset, hoping instead to develop content that was visually stimulating and dynamic. Bearing in mind the limitations of the software that I was using, I am reasonably happy with the end result.
I have just completed the final focus group with my colleagues, another 40 minute session. There was some really positive discussion around the possibilities for using the artefacts and future development of resources. I am guessing that transcribing a 40 minute duration will be very time consuming but I am happy to have so much useful data to work with.
I have also closed my survey questionnaire, to which I had 31 responses from a possible 57, really happy with that. I used Google forms, which I found excellent, and this has generated a series of colourful pie and bar charts with the data illustrating the responses very clearly. Again, similar to the focus groups, I think that I am left with plenty of relevant data to work with.
I ended up with 5 videos in total and I am pleased with the final versions. On reflection, I had probably become a little over reliant on feedback from my colleagues and was not as self-critical as I could have been with initial work. When I looked back over the earlier iterations I was disappointed to see times when I had fallen into the trap of presenting information in traditional PowerPoint slide format. This was something that I had been very keen to avoid from the outset, hoping instead to develop content that was visually stimulating and dynamic. Bearing in mind the limitations of the software that I was using, I am reasonably happy with the end result.
I have just completed the final focus group with my colleagues, another 40 minute session. There was some really positive discussion around the possibilities for using the artefacts and future development of resources. I am guessing that transcribing a 40 minute duration will be very time consuming but I am happy to have so much useful data to work with.
I have also closed my survey questionnaire, to which I had 31 responses from a possible 57, really happy with that. I used Google forms, which I found excellent, and this has generated a series of colourful pie and bar charts with the data illustrating the responses very clearly. Again, similar to the focus groups, I think that I am left with plenty of relevant data to work with.
Reflection 8 (April 23rd 2017)
I have been transcribing both of the focus group sessions over the Easter period. This has been painstakingly slow for me but it has allowed me to become very familiar with the data and identify some really interesting themes which are relevant to my research sub-questions. Between this and the results of the survey questionnaire, I believe I have the basis of some really interesting discussion points to include in my paper. I am still a little disappointed that I did not have the opportunity to test the artefacts with students. However, I have been very fortunate to have had the engagement of trainers with so much experience who are so passionate about improving practice.
I am know looking to move my attention to data analysis and writing up my research. This will also mean picking back up on reading, something that has taken a backseat while I focused on completing the artefacts.
I am know looking to move my attention to data analysis and writing up my research. This will also mean picking back up on reading, something that has taken a backseat while I focused on completing the artefacts.
reflection 9 (May 24th 2017)
I was very pleased that an FES course scheduled for May 10th at ITB went ahead with 13 students. This gave me a first opportunity to test my artefacts with a student group. I pulled together a basic website using Weebly to give access to the interactive videos to the students. Initial indications are that 7-8 of the students engaged with them, which is pretty pleasing considering that they are not a mandatory element of the course. I will ask the group to complete an online questionnaire at the end of the course to give me some feedback.
The main focus of my current work is the journal paper. I have drafted my structured abstract and introduction, as per the guidelines for my target journal paper. I have also used a thematic analysis (Aronson, 1994) to present the results of my data analysis of the two focus groups and online questionnaire. This was a very useful method for aligning my findings with the research question and sub-questions.
At the second work in progress presentation at DIT, I was in a position to present the development of my artefact from prototype to final version (see below).
Reference:
Aronson, J. (1994) A Pragmatic View of Thematic Analysis. The Qualitative Report.