Developing a learning gains framework

TU/e has developed a learning gains framework for engineering education
4TU Delft
4TU Eindhoven
4TU Twente
4TU Wageningen

Developing a learning gains framework for engineering education

At Eindhoven University of Technology, students participate in different courses to obtain their degree in engineering education. But what do they actually learn during their time at university? And how do they perceive their own learning? To answer these questions, TU/e developed a ‘learning gains framework’ for engineering education. Leaning on the CDIO framework as a basis, Martina van Uum and Birgit Pepin were inspired by other frameworks, such as the mathematical proficiency framework of the National Research Council, and a general learning gains framework (Vermunt et al., 2018). The framework developed by Van Uum and Pepin consists of five strands that are interwoven: 1) Disciplinary conceptual and procedural knowledge, 2) General cognitive learning, 3) Attitude, thought and learning, 4) Teamwork and communication, 5) The enterprise (see the figure below).

Figure: Learning gains framework for engineering education (amended from National Research Council, 2001)

Validation

To validate the framework for the TU/e context, the researchers interviewed 13 students, in groups of two or individually, from different faculties of Eindhoven University of Technology at the end of their second year, and let them talk freely about their perceived learning gains, without mentioning these strands. Moreover, they used a particular strategy, ‘pie chart drawing’, to get an indication about students’ perceived importance of their self-reported learning gains (in addition to the course/s the learning gain/s were acquired). In the data analysis Van Uum and Pepin compared the learning gains of the students and grouped them into categories.

Matching outcomes

The results showed that these categories of students’ perceived learning gains matched the developed framework. The students talked about gains in their theoretical knowledge and how they applied this knowledge - this matches the first strand. Ann (anonymized name) explained: “For example, sustainability. People are often not sure what that really is. And then I explain it. And use the three things I have learnt. That economics, ecology and social aspects are together sustainable. That it is not just throwing solar panels somewhere.”

The students also discussed how they learnt how to design products, conduct investigations and critically review their own work and that of others- this matches the second strand. Charlotte explained that she learnt to be critical: “Being critical about evidence. And how you are going to prove something. Just the question: Is it right what was written down? You think that some things are very trivial. And then you prove it, and it turns out to be wrong. Or there is an exception to which it does not apply.”

Moreover, the students said that they learnt how to take responsibility for directing their own learning, and that they gained understanding about ethics and taking into account the social context for innovations. These elements match with the third strand. Emma explained she had learnt to take different cultures into account:

“When you make food for the Netherlands and you take it into India, it will probably not work. Because they have another way of living.”

The fourth strand contains perceived learning gains about teamwork and communication, both of which were addressed by the students. Mandy explained: “I think you learn working in teams mostly by experience. When I was together with my project group for the first time, we decided to talk about ourselves first. There was one person, for example, who said: I am chaotic. I am mostly late. So we can say something about it now, e.g.: Pay attention to that. That worked really well.”

Finally, the students addressed gains via experiences with companies that were part of the fifth strand. James explained: “You are working with a stakeholder that has certain interests. You can do a lot of research, but in a company they want to know, not what is behind it, but what can I do with it.”

Future research

Van Uum and Pepin have concluded that the developed framework is valid, because the perceived learning gains identified and quantified by the students (in the interviews and drawings) matched the five strands. A next step is to further develop the framework by testing it in different contexts (e.g. different universities) and different parts of the university (e.g. innovation Space), and different years of study.

More information

If you are interested to know more about this study, please contact:

Martina van Uum  &   Birgit Pepin
m.uum@tue.nl           b.e.u.pepin@tue.nl 

Developing a learning gains framework

Developing a learning gains framework for engineering education

At Eindhoven University of Technology, students participate in different courses to obtain their degree in engineering education. But what do they actually learn during their time at university? And how do they perceive their own learning? To answer these questions, TU/e developed a ‘learning gains framework’ for engineering education. Leaning on the CDIO framework as a basis, Martina van Uum and Birgit Pepin were inspired by other frameworks, such as the mathematical proficiency framework of the National Research Council, and a general learning gains framework (Vermunt et al., 2018). The framework developed by Van Uum and Pepin consists of five strands that are interwoven: 1) Disciplinary conceptual and procedural knowledge, 2) General cognitive learning, 3) Attitude, thought and learning, 4) Teamwork and communication, 5) The enterprise (see the figure below).

Figure: Learning gains framework for engineering education (amended from National Research Council, 2001)

Validation

To validate the framework for the TU/e context, the researchers interviewed 13 students, in groups of two or individually, from different faculties of Eindhoven University of Technology at the end of their second year, and let them talk freely about their perceived learning gains, without mentioning these strands. Moreover, they used a particular strategy, ‘pie chart drawing’, to get an indication about students’ perceived importance of their self-reported learning gains (in addition to the course/s the learning gain/s were acquired). In the data analysis Van Uum and Pepin compared the learning gains of the students and grouped them into categories.

Matching outcomes

The results showed that these categories of students’ perceived learning gains matched the developed framework. The students talked about gains in their theoretical knowledge and how they applied this knowledge - this matches the first strand. Ann (anonymized name) explained: “For example, sustainability. People are often not sure what that really is. And then I explain it. And use the three things I have learnt. That economics, ecology and social aspects are together sustainable. That it is not just throwing solar panels somewhere.”

The students also discussed how they learnt how to design products, conduct investigations and critically review their own work and that of others- this matches the second strand. Charlotte explained that she learnt to be critical: “Being critical about evidence. And how you are going to prove something. Just the question: Is it right what was written down? You think that some things are very trivial. And then you prove it, and it turns out to be wrong. Or there is an exception to which it does not apply.”

Moreover, the students said that they learnt how to take responsibility for directing their own learning, and that they gained understanding about ethics and taking into account the social context for innovations. These elements match with the third strand. Emma explained she had learnt to take different cultures into account:

“When you make food for the Netherlands and you take it into India, it will probably not work. Because they have another way of living.”

The fourth strand contains perceived learning gains about teamwork and communication, both of which were addressed by the students. Mandy explained: “I think you learn working in teams mostly by experience. When I was together with my project group for the first time, we decided to talk about ourselves first. There was one person, for example, who said: I am chaotic. I am mostly late. So we can say something about it now, e.g.: Pay attention to that. That worked really well.”

Finally, the students addressed gains via experiences with companies that were part of the fifth strand. James explained: “You are working with a stakeholder that has certain interests. You can do a lot of research, but in a company they want to know, not what is behind it, but what can I do with it.”

Future research

Van Uum and Pepin have concluded that the developed framework is valid, because the perceived learning gains identified and quantified by the students (in the interviews and drawings) matched the five strands. A next step is to further develop the framework by testing it in different contexts (e.g. different universities) and different parts of the university (e.g. innovation Space), and different years of study.

More information

If you are interested to know more about this study, please contact:

Martina van Uum  &   Birgit Pepin
m.uum@tue.nl           b.e.u.pepin@tue.nl