Computational Methods for Assessing and Improving Quality of Study Group Formation
Ana Tudor
EECS Department, University of California, Berkeley
Technical Report No. UCB/EECS-2022-163
May 20, 2022
http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-163.pdf
The student peer group has been established as one of the most important influences on student development. As such, ensuring students have access to high quality classroom peer groups, referred to as study groups, is beneficial to student learning. However, both in instructor-assigned and self-formed groups, students may encounter less than positive experiences. Notably, students from underrepresented communities often face challenges in finding social support for their education when compared with those from majority groups. Several algorithmic systems have been developed to allow instructors to form study groups informed by student preferences and needs. For existing systems, addressing student feedback can help ensure that students of all demographics receive acceptable peer group support.
This focus of this project concerns incorporating student feedback to improve algorithmic study group formation. The project considers three aspects of this problem: devising a survey to collect student feedback, analyzing impact based on feedback data, and investigating a computational method to improve groups based on student feedback. This work is applied in the context of an existing Scalable, Inclusive Matching of Groups (SIM-G) system, which inclusively generates preference-based study groups for student. SIM-G currently operates in large introductory classrooms at UC Berkeley.
First we focus on developing and validating a survey which assesses the quality of study groups. This is based on a construct of group quality which includes reliability and availability of the peer group, effectiveness in aiding course learning, and student psychological safety. The survey is demonstrated to provide a valid and reliably consistent measure of group quality, with suggested deployment after slight revision.
Next, the project conducts analysis of the impact of study group formation in large EECS classrooms at UC Berkeley, based on datasets generated from group matching in courses using the SIM-G system. It is found that study groups matched by SIM-G have roughly equitable outcomes across demographics, and that students from under-represented demographics preferentially choose software-matched groups over self-formed groups. The analysis also reveals opportunities for improvement in providing study groups in classrooms, namely in facilitating student meetup and communication. Finally, positive performance in assessment grades is correlated with a combination of measures of student comfort in the group, and frequency of group interaction.
Finally, the project explores a method for computationally forming and improving study groups, via a Reinforcement Learning model. A system is outlined for regressing on study group quality based on preference and demographic features of each student. The project also attempts the incorporation of a clustering model towards group formation, ultimately rejecting it as appropriate for this application. Ultimately, this modeling is used to approach iterative improvement of study groups within a Reinforcement Learning Actor-Critic framework, demonstrating its potential feasibility given a more appropriate group formation model.
Advisors: Gireeja Ranade
BibTeX citation:
@mastersthesis{Tudor:EECS-2022-163,
Author= {Tudor, Ana},
Title= {Computational Methods for Assessing and Improving Quality of Study Group Formation},
School= {EECS Department, University of California, Berkeley},
Year= {2022},
Month= {May},
Url= {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-163.html},
Number= {UCB/EECS-2022-163},
Abstract= {The student peer group has been established as one of the most important influences on student development. As such, ensuring students have access to high quality classroom peer groups, referred to as study groups, is beneficial to student learning. However, both in instructor-assigned and self-formed groups, students may encounter less than positive experiences. Notably, students from underrepresented communities often face challenges in finding social support for their education when compared with those from majority groups. Several algorithmic systems have been developed to allow instructors to form study groups informed by student preferences and needs. For existing systems, addressing student feedback can help ensure that students of all demographics receive acceptable peer group support.
This focus of this project concerns incorporating student feedback to improve algorithmic study group formation. The project considers three aspects of this problem: devising a survey to collect student feedback, analyzing impact based on feedback data, and investigating a computational method to improve groups based on student feedback. This work is applied in the context of an existing Scalable, Inclusive Matching of Groups (SIM-G) system, which inclusively generates preference-based study groups for student. SIM-G currently operates in large introductory classrooms at UC Berkeley.
First we focus on developing and validating a survey which assesses the quality of study groups. This is based on a construct of group quality which includes reliability and availability of the peer group, effectiveness in aiding course learning, and student psychological safety. The survey is demonstrated to provide a valid and reliably consistent measure of group quality, with suggested deployment after slight revision.
Next, the project conducts analysis of the impact of study group formation in large EECS classrooms at UC Berkeley, based on datasets generated from group matching in courses using the SIM-G system. It is found that study groups matched by SIM-G have roughly equitable outcomes across demographics, and that students from under-represented demographics preferentially choose software-matched groups over self-formed groups. The analysis also reveals opportunities for improvement in providing study groups in classrooms, namely in facilitating student meetup and communication. Finally, positive performance in assessment grades is correlated with a combination of measures of student comfort in the group, and frequency of group interaction.
Finally, the project explores a method for computationally forming and improving study groups, via a Reinforcement Learning model. A system is outlined for regressing on study group quality based on preference and demographic features of each student. The project also attempts the incorporation of a clustering model towards group formation, ultimately rejecting it as appropriate for this application. Ultimately, this modeling is used to approach iterative improvement of study groups within a Reinforcement Learning Actor-Critic framework, demonstrating its potential feasibility given a more appropriate group formation model.},
}
EndNote citation:
%0 Thesis %A Tudor, Ana %T Computational Methods for Assessing and Improving Quality of Study Group Formation %I EECS Department, University of California, Berkeley %D 2022 %8 May 20 %@ UCB/EECS-2022-163 %U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2022/EECS-2022-163.html %F Tudor:EECS-2022-163