Courses we are considering

Executive Summary

of Collaborative Learning Study from CS101 Summers 2002 & 2003

By Mike Battig

Overview

During the Summer of 2002 I taught CS101 with a Control Group of 9 students. The Control Group completed all labs on an individual basis in JeanMarie 142. During the Summer of 2003 I taught CS101 with an Experimental Group of 6 students who completed all labs on a group basis using the Collaborative Learning Lab (Davis) in Bergeron. The detailed results of the surveys are given at the end of this document. Through CS 101, we were studying the impact on learning that collaboration presents. CS 101 is a 4-credit course that in a traditional semester requires 3 hours per week of lecture and a 2-hour closed lab session. During Summer Semester, the CS Department has offered the course during an intensive 3-week session, which has resulted in higher Summer Term enrollments. Traditionally, the laboratory work has been completed on an individual basis. More details regarding this course are available online: http://academics.smcvt.edu/compsci_courses/CS101/

Outcomes & Discussion

In addition to asking students about their impressions of the course, instructor, and learning experience, I asked students to give me their names on the survey. I told them that the surveys would not be viewed until after final grades were given to the Registrar. After the final grades were submitted, I requested the GPAs for the students on the roster. Therefore, I could attempt to make some inferences about the surveys based on past academic performance of the students in this study. The results are shown below.

Control Group		Experimental Group
GPA	Course	GPA	Course
2.68	3.04	2.48	2.93

The first observation worth noting is that the GPAs are generally lower than what we typically find at SMC. This may indicate that Summer School students are more likely to have academic problems. During the Experimental course (Summer 2003), one student informed me that s/he “needed an A” in my course in order to graduate from SMC since the only obstacle for graduation was an insufficient GPA. Also note that over one-fourth (26.6%) of the students inflate their self-reported GPA. The second observation is that the Control Group is stronger academically than the Experimental Group. The importance of this observation is that throughout both groups more talented students gave higher responses to all questions (i.e., more positive results) than less talented students. The final observation concerning this data is perhaps the most important: the number of students in each group is relatively small. Therefore, if one wished to present evidence with confidence, the use of traditional statistics would be limited (given the small sample sizes, we could consider the use of some non-parametric statistics).

Many of the survey questions used with both groups were the same or similar to those used at SMC in our SUMMA evaluations. In general, there are no glaring differences between the results from either group. For example, question 6 (“The instructor seems to care about my learning”) received very similar responses from both groups. Questions 15 and 22 are of particular interest since they deal directly with the issues related to working with classmates in learning the material. The experimental group gave slightly lower marks in learning from their classmates (i.e., they felt interactions with classmates was less beneficial). However, the small number of data points in this study is relevant. Within the experimental group, the sample was biased by negative responses by two students (one of which was the highest academic performer). The remaining survey question of interest is number 29 (2002) / 36 (2003) where students are asked to provide percentages to “influences” on their learning. The experimental group felt they were slightly less influenced by classmates.

Given that I can’t find any significant learning differences from my limited study, I have wondered why others on campus have had such great successes in the Davis Lab (specifically the Biology Dept.). In my discussions with Biology faculty, I have found some differences in our courses that perhaps are significant. First of all, the Biology courses are comprised of upper class Biology majors, whereas the CS101 sections studied here were comprised of all non-majors (some students indicated on their surveys that CS101 was required for their major, however, none of the subjects were CS majors). Second, the collaborative nature of the Biology assignments is indeed the state of the practice in biological lab work in general. Whereas, in computing, the state of the practice is almost exclusively individualistic. For example, programmers always work solo (there is an experimental paradigm called “Extreme Programming” where developers work in pairs, but it is still very new and has not been validated as an approach). Thus, the work the students were being asked to do in the CS101 lab is something that is traditionally done solo whereas the Biology work is something that is traditionally done in a team context with a “divide and conquer” approach. I should mention here that steps were taken in the experiment to promote a “divide and conquer” team approach. Students in CS101 were divided into teams and encouraged with each lab with potential approaches for dividing the work so as to optimize the Davis Lab environment. Nonetheless, students in my limited study did not report any significant benefit from this approach.

Another issue that is worth accounting for in this study is that several students in the experimental group disclosed to me during lab times that they were feeling overwhelmed with the workload of keeping up with my course and Dianne Lynch’s Journalism course, JO 102 (another EdTech funded course). This is a troubling revelation for several reasons. On the first day of my class, I urged all students to be certain to “clear their calendars” for the three weeks of CS101. CS101 is a four-credit course that meets for three hours of lecture each morning and two hours of lab each afternoon. Homework, reading, and lab completion is accomplished outside of class/lab each day. Therefore, it is unrealistic that students can do things like hold down a job (which another student confided he was doing) or take another course. Dianne’s course (as her report indicates) was demanding in terms of time and effort. The question of why these students were taking both courses is simple (as they explained it to me). They determined that the combination of an intensive 3 week course and an “online” course would be the easiest path for completing two courses necessary for graduation (two of the experimental students needed 2 or 3 more courses to graduate, another student needed to raise his GPA). Therefore, the nature of some of our summer school student’s motivations for taking courses should be accounted for in these reports and future EdTech experiments that involve summer school.

For each question, check 5 for Strongly Agree….or check 1 for Strongly Disagree.	5	4	3	2	1
1. The contents of the assignments contribute to my understanding of the subject	3	4	2
2. The instructor’s presentation often caused me to think in depth about this subject	1	4	3	1
3. The methods used to evaluate my work (tests, projects, etc.) are reasonable	4	3	1	1
4. Adequate opportunities are provided by the instructor for me to ask questions	3	5		2
5. The instructor seems to be well prepared	3	5	1
6. The instructor seems to care about my learning	3	3	3
7. During the term, I looked forward to attending this class		4	4		1
8. Compared with other courses on this level carrying an equal amount of credit, the effort I put into this course is as much as in other courses	1	5	3
9. The instructor demonstrates a personal commitment to high standards of professional competence	2	5	2
10. In this course, I am learning much	3	5		1
11. The out-of-class assignments are challenging (i.e., labs)	6	2	1
12. Overall, I rate this instructor a good teacher.	4	3	2
13. Examinations cover material or skills emphasized in the course	2	4	2	1
14. The textbooks contribute to my understanding of the material	1	5	2	1
15. Interactions with other classmates contribute to my understanding of the material	1	5	3
16. Interactions with the instructor outside of class contribute to my understanding of the material	1	4	3		1
17. This course is practical and useful given the goals and content area of the course	2	6			1
18. The laboratory experiences meet my learning needs for this course	3	4	2
19. The instructor explains or illustrates laboratory techniques clearly	1	3	5
20. The laboratory contributes to my understanding of the subject	4	2	3
21. The laboratory web pages explain the procedures to be followed during lab time	2	5	1	1
22. Interacting with other students contributes to my understanding of the lab work	5	4
23. Interacting with the instructor contributes to my understanding of the lab work	4	4	1

Circle the appropriate response for the following questions:

24. My perception of the teaching method used in this course is: (3)1-Lecture 2-Discussion

(1)3-Demonstration (4)4-Combination 5-Other

25. This course is: 1-In my major 2-General requirement 3-An elective 4-Required cognate 5-Other

(2) (3) (3) (1)

26. My class is: 1-Freshman 2-Sophomore 3-Junior 4-Senior 5-Graduate

(4) (5)

27. My grade point average to date is (round off): (4.0-3.5) (3.4-3.0) (2.9-2.5) (2.4-2.0) (under 2.0)

2 out of 9 students inflated their GPA.

28. The grade I presently have in this class is: A B C D F

all 9 students correctly reported their course grades

29. Regarding the lab work, I would say that my learning was influenced according the following percentages (be certain that it adds up to 100%):

Textbooks/web materials:__________ 32.7

Instructor: ___________ 42.8

Classmates: ___________ 24.4

Total: 100%

30. Regarding the lectures & exams, I would say that my learning was influenced according the following percentages (be certain that it adds up to 100%):

Textbooks/web materials:__________ 30.6

Instructor: ___________ 60.0

Classmates: ___________ 9.4