Reflections on Conducting Qualitative Research During a Pandemic
This blog was written by Samantha McAleese and three youth participants – Sahil, Erin, and Kate (pseudonyms used to maintain anonymity).
What comes to mind when you hear the word algorithm? Can you explain how machine learning works? Do you have any privacy or data collection concerns regarding the increased reliance on artificial intelligence? These are just a few of the questions that we asked young people in our recent qualitative research project Algorithmic Awareness: Conversations with Young Canadians about Artificial Intelligence and Privacy. From November 2020 to January 2021, we facilitated eight focus groups with 22 youth ages 13 to 17, where we combined game-based learning with discussion and reflection to gain insight into how young Canadians understand the relationship between artificial intelligence, algorithms, and privacy.
MediaSmarts is no stranger to interactive research projects, but things looked a bit different this time around. The COVID-19 pandemic presented many challenges to researchers in 2020. We adapted our initial plan to play the #ForYou card game in person with young participants to adhere to public health guidelines. This pandemic pivot included posting recruitment materials online, meeting via Zoom, and playing the card game on Roll20. Despite some challenges⸺a lack of non-verbal communication (when participants chose to keep their cameras off), connectivity issues, and difficulty collaborating during gameplay⸺we feel like the project was a success. In fact, conducting the focus groups online presented some benefits, including connecting with young people outside of Ottawa, eliminating travel time, and communicating and recording additional information using the chat function throughout the focus group sessions. Knowing that virtual research will become a regular practice at MediaSmarts moving forward, we followed up with a few youth to ask them about their experiences participating in this project.
What motivated you to participate in this research project?
Sahil (15): “I am very interested in artificial intelligence and computer algorithms and wanted to learn more about how they affect me in my life.”
Erin (17): “I have participated in MediaSmarts projects before, and I always enjoy how organized they are. I also enjoy seeing the results/report afterwards to see how my contributions have been used. The incentive is also a nice bonus.”
Kate (16): “I participated in this research project because I was motivated to learn more about algorithms and artificial intelligence. AI is not something that I hear a lot about, so I was driven to educate myself on what AI can accomplish and how it affects me in my life as a young Canadian.”
Tell us about your experience participating in the focus group? Did you like participating in an online focus group? Can you think about the pros and cons of doing research like this online?
Sahil (15): “I enjoyed participating in a virtual focus group as I was able to learn from the comfort of my home. I believe that a benefit of doing research like this online is the ability to utilize online resources, such as online articles and software. One downside would be the lack of human interactivity beyond being on a Zoom call.”
Erin (17): “I really enjoyed participating in a virtual focus group. Participating from my home is much easier than travelling downtown. It is a little easier to share in person because there is room to make a connection with those interviewing you. However, virtual is great too because it gives those who are on the more shy side the chance to use the chat function. I think it is easier for young people to participate on online platforms because they do not need to travel to get places, as well they might feel more comfortable typing their responses or using a chat function.”
Kate (16): “I found the virtual focus group to be very enjoyable and a great way to learn. Despite the fact that the conversation could not flow as it normally would in person, I found that I was able to share my thoughts and opinions easily and was able to listen to other's ideas just as well. Everybody had the chance to contribute to the conversation. I would say that I preferred the virtual focus group because I think I had more confidence in my contributions, while in person I may not have. One of the good things about doing an online focus group is that we were able to form a diverse group. All of us that participated were from different regions of the country, which would not have been possible had the focus group been in person. Another pro for the virtual meeting was that the #ForYou card game was accessible to all of us and was able to be played on any computer.”
What did you think about the #ForYou card game? Do you think this is a good teaching/learning tool? Would other youth like playing this game?
Sahil (15): “I really enjoyed the #ForYou card game as a learning tool because it added another level of interactivity into the research project. I believe that other youths would enjoy playing this game in conjunction to understanding the logic behind it.” (Sahil, 15)
Erin (17): “I really enjoyed playing the card game. I really like how MediaSmarts gathers information from their participants. I recall another survey I did with MediaSmarts where I shared my ideas through art and creativity. Having a game involved was a lot better than just being asked interview questions over and over again. I think youth would certainly like playing this game; it was fun and organized. I recall the monitor of the game was very helpful and explained everything very thoroughly, and helped us along the way. I think this is a great idea because youth are so often used to learning through presentations or on paper, and this is a great resource to switch things up.”
Kate (16): “I really liked the card game that we played during the focus group. It helped put things into perspective about how different aspects of algorithms and AI maximize interactions between consumers and online programs. The game is a great tool to really see how your choices online affect the way media is generated for you. Other people my age would like this game and could gain a lot from playing it.”
Have you reflected on the content/discussions from the focus group since participating? What did you learn that surprised you? Have you thought more about algorithms, artificial intelligence, or online privacy since participating?
Sahil (15): “During the focus group, I learned more about how algorithms affect me online, through tracking my data, recommending different things that would suit me, etc. I’ve started to think more about online privacy and have started to become aware of privacy policies and how companies use my data.”
Erin (17): “I have certainly reflected on the discussions from the focus group. Prior to the focus group, I did not know much about artificial intelligence. However, after I learned more about how the information is used, I have certainly been more cautious. It is somewhat scary knowing that some youth are unaware of how artificial intelligence is used. I used to shop online for fun without the purpose of buying anything, but just to browse. However, now that I have learned that actions like these can control algorithms, I have been using them less and less. I have been using more private browsing tabs in order to reduce the ads that will come up in my other feeds like Facebook as an example.”
Kate (16): “Since the focus group, every time I see an advertisement or a video on social media, I think about how an algorithm has probably picked that specific one for me based on what information it knows about me. I was surprised to see things like your gender and race being factors for algorithms in choosing which advertisements will be seen by you.”
Is there anything else you'd like to share with us about your experience participating in this research project?
Sahil (15): “I really enjoyed this research project, and I believe that by learning these things, people can become more aware about how their data is used and how to protect themselves.”
Erin (17): “Samantha, Kara and their entire team at MediaSmarts are all wonderful. I have participated in a variety of research projects, and MediaSmarts has always exceeded my expectations. They are incredibly patient and always explain everything very thoroughly. As well, I have always been sent the final project/report, and I am always so impressed by how the information is being used and how it is displayed. Overall, I am so grateful I have been able to participate in some of MediaSmarts’ research projects, and I am looking forward to working with them in the future.”
Kate (16): “I really appreciated this focus group research project because it caused me to reflect on my online footprint and think more in-depth about things that I like, share, or post on social media.”
We are pleased to hear participants confirm what we found beneficial in moving this research project online, and we will take lessons learned forward with us in subsequent projects. We are grateful to the youth who shared their insights, experiences, and strategies for encouraging young Canadian’s to learn more about their privacy rights in a way that empowers and motivates them to use AI to their advantage.
This research was made possible by the financial contributions from the Office of the Privacy Commissioner of Canada’s contributions program.