//Logo Image
Authors: Che-Chang Yang, Yeh-Liang Hsu (2009-04-13)recommend: Yeh-Liang Hsu (2009-04-21).
Note: This paper was presented in the 2009 ASME Asian-Pacific Engineering Education Congress (APEEC 2009).

Improving student participation in oral presentation in a project-based engineering design course

Abstract

A new pedagogical approach of adopting peer evaluation to incorporate oral presentations in a project-based engineering design course is proposed in this paper. Students are allowed to grade the oral presentations of other students. The effect of oral presentations with/without the intervention of peer evaluation was compared. The grading scores by the students had positive correlation to those by the professor. The results of the questionnaire from the students also showed positive agreement in improving student participation toward the class and therefore enhance the learning outcome.

Keyword: project-based learning, Learning-by-doing, peer evaluation, oral presentation

1.     Introduction

Engineering design is a practical problem-solving profession and a critical element in engineering education. In addition to gaining domain knowledge and the ability to use engineering tools, the training to deal with real engineering problems are important learning process for engineering students [Atman et al., 2007]. Therefore, in many engineering design courses, design projects are commonly adopted as a means for students to “experience design”. Students can learn more from working on projects than sitting in lectures, i.e., learning-by-doing, and therefore the pedagogical approaches such as project-based learning or problem-based learning have been developed in many engineering courses [Byqstad et al., 2009]. Project-based learning also provides opportunities for interdisciplinary learning, and has the potential to enhance student participation, motivation, and learning effectiveness [Jou et al., 2008].

One problem we have to deal with in a large class (about 110 students each year) is that the professor and TA do not have enough time to coach each design team. Therefore an adapted “guided design procedure” is built into the design course [Hsu et al., 2003]. Guided design is a structured way of having students work through case studies [Wales et al., 1974]. The guided design procedure advances step by step through a specific problem-solving or design procedure. In this adapted guided design procedure, the professor and TA do not have to supervise each team closely. However, the students received the comments the professor gives to the written reports of each team on each projects. It is crucial that the professor focuses efforts on grading the reports, giving useful comments to the students. In the standard guided design procedure, it is the design process that is being taught, and not a particular answer. Therefore it is important to stress that the feedback sheet does not represent the solution but only shows what the professionals did. But in our student projects, it is important to assess correct use of the design tools just learned and therefore is included in the professor's feedback.

Oral presentation can be used to assess the student performances in design projects [Khandaker et al., 2008]. It is a good training for students to clearly show their design concepts. Students can also communicate design information and experiences or ideas from each other over the presentation session. However, arranging oral presentation is difficult and ineffective in a large engineering design course. The practical problems encountered in our experience in the past few years are as follows:

Ÿ    There are 30 teams in a class of around 110 students. It takes too much time to go through the 4 project presentations assigned in a semester.

Ÿ    Students do not pay attention to other teams’ presentations. This discourages the interaction and feedback from the students.

Ÿ    The students usually prepare their oral presentations only for the professor who is considered the intended and only audience.

Ÿ    Students feel the grading by the professor alone is subjective.

In a problem-solving process there might be multiple potential solutions or alternatives for a design. Emerging pedagogical approaches such as cooperative learning or peer assessment/evaluation are well suited in engineering design courses [Atman et al., 2007]. Peer evaluation has been documented as important elements in project teamwork [Carr et al., 2005]. It was reporte        d that students welcomed structured feedback from peer-evaluation for a project and the finding by William et al. [William et al., 2007] also suggested professor should structure peer-feedback during a project with peer-evaluation at the end of a project.

This paper proposed a new scheme to incorporate peer evaluation into oral presentations in a large engineering design course. Students are allowed to grade the oral presentations of other students. This approach is proposed to improve student participation toward project oral presentation in class and therefore enhance their learning outcome for the design projects. Section 2 of this paper describes the design of the new scheme for the oral presentation. The grading scores by the students and questionnaires results were analyzed in Section 3. Finally Section 4 concludes the paper.

2.     Design of peer evaluation for oral presentation

In the Mechanical Design course taught by the authors, 4 projects are organized to guide the design process to complete the design of a shooting robot. Each project represents a progress checkpoint for the design process. Around 110 students per year were grouped into 30 teams, with 3 to 4 members per team. At least one oral presentation is required for each student in the semester. Details about the new scheme are as follows:

Ÿ    Each oral presentation by each team is limited to 5 minutes. Presentation has to stop after 5 minutes. In addition to saving class time for 30 team presentations, this requirement also intends to train the students to present their work, ideas and design more clearly and concisely.

Ÿ    Students are asked to grade the oral presentations (including the presentation of their own team). The students’ gradings have 50% impact on the final scores of the oral presentation, and the other 50% is given by the professor. The professor gave written comments to each team on their work and oral presentation. The students can also provide their written opinions and responses to the presentations.

Ÿ    The presentation scores range from 6 to 10 points, and the distribution of the scores of all the teams is pre-defined in the scoring sheet. Typically in a 15-team session, a student (and the professor) is allowed to give 1 6-point, 3 7-point, 5 8-point, 4 9-point and 2 10-point.

We had several questions in mind when we first implemented this new scheme:

Ÿ    Is 5 minutes for an oral presentation enough?

Ÿ    Does this scheme enhance the student participation?

Ÿ    Will students grade the oral presentation groundlessly, carelessly, or even randomly?

Ÿ    Do the students have enough ability to do professional judgment?

In the first semester when we tested this new scheme, all teams are divided into two sessions (Session I and Session II) for oral presentations. Peer evaluation is conducted only in Session I. The grading scores by the students and the questionnaires results were analyzed and compared between Session I and Session II trying to answer these 3 questions.

3.     Analysis and Discussion

(1)   Is 5 minutes for an oral presentation enough?

The oral presentation is limited to 5 minutes. The students have to present their work in the short period of time. The number of their presentation slides ranges from 7 to 22 pages. For most teams, they can complete the oral presentation as scheduled. The average time the teams spent is 4:40.

(2)   Does this scheme enhance the student participation?

At the end of the semester after the 4 oral presentations, all students either in Session I or Session II were asked to fill out the questionnaires which had not been announced in advance. The students answered those questions by indicating the level of agreement in the following statements where 2=strongly agree, 1=agree, 0=neither disagree nor agree, -1=disagree, -2= strongly disagree.

Table 1 shows the results of the questionnaires answered by the students in both Sessions I and Session II. The questions in the questionnaire can be divided into 4 groups. Group 1 (Question 1~4) asked students for a self-evaluation of class participation. The numbers reveal that the students in Session I have higher participation in this self-evaluation. Group 2 (Question 5~10) asked students about their attitude towards the oral presentation. Students in Session I paid more attention to the interaction with the professor and students during the oral presentation. Group 3 (Question 11~17) asked students about their feeling towards the grading. Students in Session I also indicated stronger agreement with whether the grading is subjective or not (Question 17). Group 4 (Question 18~24) asked students whether they learn from doing oral presentation. The results from Question 20~24 reveal that the student participation was enhanced for students in Session I.

Table 1 The results of the questionnaires

No.

Question

Session I

Session II

1

How much time did you usually spend on each design project, except the class?

-0.12

0.00

2

How did you participate in the design project?

1.08

0.91

3

Your attendance?

1.34

0.97

 

How many times were you present at the oral presentation?

1

2

3

4

1

2

3

4

6%

0%

4%

90%

7%

0%

8%

85%

5

I will try to satisfy the professor’s requirement when preparing oral presentations.

1.34

0.88

6

I will consider the responses from the classmates when preparing oral presentations.

1.00

0.50

7

I am interested in the other’s oral presentations, and I listen to their presentation carefully.

0.95

0.90

8

The other classmates were interested in the oral presentation of your team.

0.60

0.31

9

I can learn the strengths from listening to the oral presentations of the other teams.

1.44

0.90

10

I can learn the skills in oral presentation from other classmates when listening to the others’ presentation.

1.10

0.78

11

I care the grades of my team after project presentation.

0.77

1.06

12

I care the comments given by the professor after project presentation

0.88

1.02

13

I think it fair to have a pre-decided distribution of grading for all the presentations.

0.15

0.40

14

I think it’s fair to allow the students doing the grading.

0.45

0.56

15

I think the grading of the oral presentation emphasized more on the design content.

0.65

0.46

16

I think the grading of the oral presentation emphasized more on the skill of oral presentation.

0.65

0.46

17

I think the grading for oral presentation is objective.

0.94

0.16

18

The comments given by the professor helped me understand the pros and cons of my team.

0.91

0.77

19

The scores of my team for the oral presentation met my expectation.

0.53

0.32

20

I’ll be more serious about the design project because of the oral presentation.

1.26

0.62

21

I will understand the professional knowledge because of the oral presentation.

1.14

0.72

22

The oral presentations helped enhance my ability of expression and communication.

0.94

0.78

23

I know more how to cooperate with the other people due to oral presentations.

0.90

0.74

24

I would take the oral presentation rather than exams.

1.07

0.77

(3)   Is peer evaluation fair?

The students co-grade the oral presentations in the Session I. We might doubt whether the students grade the presentation carelessly and unconcernedly to result in less meaningful scores. Therefore, their grading scores were compared with the grading numbers simulated by a computer program. The simulated numbers are generated randomly but follow the grading rules as introduced. The average of coefficients of variation was calculated and the results were shown in Table 2. The numbers show that the scores graded by the students have much lower average of coefficients of variation, which means that scores given by students are far from random.

Table 2 The reliability comparison of the scores given by random generation and students’ grading

Scores samples

Average of coefficients of variation (%)

Random

20.6

Session I

13.1

(4)   Do the students have enough ability to do professional judgment?

Here we assumed the professor’s judgment is “professional” and compared the difference between the professor’s and the students’ grading. Table 3 shows the results from the four oral presentations (P1 to P4). The average of the correlation of coefficient between the professor’s and the students’ grades in the four oral presentations is 0.57. This indicates a large positive correlation between the professor’s and the students’ judgments.

Table 3 The correlation between the professor’s and the students’ grades

 

P1

P2

P3

P4

Average

Correlation of coefficient

0.68

0.50

0.56

0.55

0.57

4.     Conclusion

This paper proposed a new pedagogical approach of adopting peer evaluation to incorporate oral presentations in a large engineering design course. Students are allowed to grade the oral presentations of other students. The effect of oral presentations with/without the intervention of peer evaluation was compared. From our data, we can reach the following conclusions:

Ÿ    5 minutes seems to be enough for an oral presentation.

Ÿ    This new scheme indeed enhances student participation in oral presentations.

Ÿ    Students (at least most of them) will not grade the oral presentation randomly.

Ÿ    Students do have the ability to do professional judgment.

After this study, the authors felt confident in this new scheme and implemented it in the following years. Now it has become a standard practice for the oral presentations in the mechanical design course.

Reference

Atman, C. J., Adams, R. S., Cardella, M.E., Turns, J., Mosborq, S., Sallem, J., 2007. “Engineering design processed: A comparison of student and expert practitioners,” Journal of Engineering Education, vol. 96, no. 4, p. 359-379.

Byqstad, B., Kroqstie, B. R., Gronli, T. M., 2009 “Learning from achievement: Scaffolding student projects in software engineering,” International Journal of Networking and Virtual Organization, vol. 6, no. 2, p. 109-122.

Jou, M., Wu, M. J., Wu, D. W., 2008. “Development of online inquiry environments to support project-based learning of robotics,” Proc. of 1st World Summit on the Knowledge Society, WSKS 2008, p. 341-353.

Hsu, Y. L., Yo, C. Y., 2003. “The problem-solving approach for the fundamental hands-on practice courses in mechanical engineering education,” Journal of the Chinese Society of Mechanical Engineers, vol. 24, no. 5, p. 517-524.

Wales, C.E., R.A. Stagers, and T.R. Long, Guided Engineering Design, West Publishing Company, St. Paul, MN, 1974.

Khandaker, M., Orono, P., Ekwaro-Osire, P., 2008. “Undergraduate engineering team project: Is there any correlation between presentation and participation?” Proceedings of ASEE Annual Conference and Exposition 2008.

Carr, S. D., Herman, E. D., Keldsen, S. Z., Miller, J. G., Wakefield P. A. The team learning assistant workbook, McGraw Hill, N.Y., 2005.

William, B. C., He, B. B., Elqer, D. F., Schemacher, B. E., 2007. “Peer evaluation as a motivator for improved team performance in Bio/Ag engineering design classes,” Engineering Education, vol. 23, no. 4, p. 698-704.