Reflections on conducting a research project
Getting Started:
My Msc project was based on the EU funded QualiBuild project, which I worked on extensively at ITB from 2013-2016. By choosing a subject area for my research which was close to my own practice, I was very much invested in the outcomes of the study.
The main learning theory underpinning my research proposal was learning styles and the value of using multi-modal e-learning to appeal to different learning preferences. The project allowed me to study an area that I have been interested in for some time, having witnessed over many years the preference of learners of construction skills for highly visual materials and learning by doing. While I set out believing that e-learning could complement more traditional face-to-face delivery, I could also foresee some obvious barriers to engagement, particularly around computer literacy levels and lack of exposure to digital learning environments.
The focus at the early stage of my project was on ethics approval. My preconception that ethics submission is purely a form-filling exercise was misplaced. I came to understand the significant upfront work required, such as revisions to my research proposal and preparation of focus group outlines/questions and survey questionnaire. While this was challenging from a time perspective, it was also very useful in helping me focus on the detail of my research and how I intended to conduct it. Following ethics approval submission, I was much clearer on the project plan and implementation.
One of the earliest tasks I had was to identify a target journal for my paper. By choosing a particular journal and familiarising myself with its format, I was even clearer on how I would conduct my research and write up my paper.
Research Methods:
For my two focus groups, I had considerable difficulty accommodating the availability of my colleagues. I had to settle for having some present and others joining by teleconference. The logistics were difficult, meaning I had to plan the sessions carefully. Thankfully the technology for the recording of the sessions worked well and all ran smoothly. The biggest challenge was to keep people on topic and make sure that everyone had an opportunity to contribute.
I left transcribing of both focus group sessions until the Easter period. In hindsight, this was probably a mistake. If I had completed this soon after the first session (in November) I would have learned more about the management of focus groups and the importance of clear enunciation which would have informed the second. I found the process of transcribing the sessions painstakingly slow. However, it did result in me becoming very familiar with the data which made it possible to identify some really interesting themes relevant to my research sub-questions.
I used Google Forms to develop my online survey questionnaire. This was conducted with trainers who were up-skilled under the QualiBuild project to deliver training to construction workers. My survey questionnaire had 31 responses from a possible 57, which I was very happy with. I used Google forms, which I found excellent, and this generated a series of colourful pie and bar charts which illustrated the responses very clearly. Again, similar to the focus groups, I felt that I was left with plenty of relevant data to work with.
Artefact Development:
The use of a number of different technologies for my artefact development was very time consuming as it created many different stages and layers in the development process. However, I was determined to use open source or readily available software to develop the resources with a view to minimising barriers for other tutors adding to the work in the future. This did mean that I had to compromise somewhat on my original vision for the artefacts.
I adopted the SAM model of instructional design (Allen, 2012), an agile process suited to evaluation by other trainers. I posted storyboards and prototype videos on Moodle for my colleagues to review. The biggest drawback with this approach was the reliance on timely feedback from my colleagues, particularly during busy periods in the college semester. On reflection, I probably became a little over reliant on the feedback and was not as self-critical as I should have been with some of the initial drafts.
As the development of the interactive videos progressed, a pattern emerged where the duration was increasing with each topic covered. The first iteration of Video 1 was just over 5 minutes while subsequent videos were 8, 11 and almost 13 minutes. This was highlighted in feedback from colleagues and my supervisor. Most felt that the videos should be shorter and snappier, with suggestions to split videos in two, consistent with existing research into multimedia instruction.
This was challenging at that stage as time left for development was limited. I settled on a compromise of reducing one video and splitting another in two. I was reluctant to be leaving out developed content but I needed to be realistic with the amount of work still to be completed on the project.
After putting so much effort into their development, I was particularly disappointed that I did not have the opportunity to test the artefacts with students as part of the study. However, towards the end of the project, I did have access to a group of students who trialled the resources. I am looking forward to getting their feedback.
Journal Paper:
My supervisor encouraged me to consider relevant studies challenging learning styles theory and other theories relevant to multimedia instruction. Most importantly, this led me in the direction of cognitive load theory and its relationship with multi-modal/multimedia presentation of learning material. Specifically, an article by Richard E. Mayer on research-based design principles for multimedia instruction was particularly useful (2014). This guided and informed my research project considerably.
I specifically targeted research into cognitive load theory in multimedia instruction, learning styles and studies of e-learning applied with construction students. Finding papers specific to construction workers was challenging. Many of the studies I found had been conducted with third level construction students rather than blue collar workers. However, they were still somewhat relevant as many of these third level students had progressed from trade qualifications to the programmes and shared similar learner characteristics.
I adopted a thematic analysis (Aronson, 1994) to present the results of the two focus groups and the survey questionnaire. I was not familiar with this approach and found it to be a very useful method for aligning my findings with the research question and sub-questions.
Final Thoughts:
The project certainly reinforced for me the importance of good planning and time management. Even on a relatively small scale study such as this, I was still involving almost forty participants between focus groups and questionnaire.
I found that this study challenged some of my preconceptions about learning styles and the potential of e-learning. Conducting my literature review gave me a much broader knowledge of learning theory and educational research that challenges some aspects of my own approach to my practice.
References:
Allen, M. (2012). Leaving ADDIE for SAM. ASTD Press.
Aronson, J. (1994). A Pragmatic View of Thematic Analysis. The Qualitative Report.
Mayer, R. E. (2014). Research-Based Principles for Designing Multimedia Instruction. Applying Science of Learning in Education - Infusing Psychological Science into the Curriculum. Division 2 American Psychological Association. Retrieved online October 12th 2016 from: http://hilt.harvard.edu/files/hilt/files/background_reading.pdf