Promoting Learning of Instructional Design via Overlay Design Tools

Andy Carle

EECS Department
University of California, Berkeley
Technical Report No. UCB/EECS-2012-246
December 14, 2012

http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-246.pdf

Design is a notoriously difficult profession to practice, and it is even more difficult to learn. Traditionally, learning of design skills has been situated in the context of apprenticeships or formal design studios. Unfortunately, these methods are inaccessible to practicing professionals due to constraints on time and location. And, indeed, professional designers must continuously update their knowledge as paradigm shifts in design practice threaten to make their skills obsolete. An ideal resolution to this problem is to situate the learning of design skills within the professional practice of design. This dissertation studies an approach to this mode of situated learning, focusing on integrating learning mechanisms into practical design tools. These tools provide scaffolding for novices as they construct an understanding of best practices in design while engaging in real design work.

I begin by introducing Virtual Design Apprenticeship (VDA), a learning model – built on a solid foundation of education principles and theories – that promotes learning of design skills via overlay design tools. In VDA, when an individual needs to learn a new design skill or paradigm she is provided accessible, concrete examples that have been annotated with design rationale. These annotations make expert thinking visible and allow the novice to immediately use, and gradually understand, new best practices. By combining abstract rationale with concrete design instances, annotated artifacts become more useful than either could be alone. I also present a set of design principles that guide the creation of VDA design tools – user interfaces built to mediate an individual’s interactions with annotated examples.

While VDA is applicable to most design fields, I narrow the scope of consideration to one particular domain of design by focusing in-depth on the instructional design difficulties that university-level faculty members face and how the VDA approach can address them. I engaged with instructors and a curriculum design research group in a six-year period of contextual inquiry. Findings from this study influenced my formulation of the VDA framework and the design of PACT, a design tool that leverages the learning principle of making thinking visible to assist novices as they transition from concrete to abstract reasoning about curriculum design. I discuss the iterative design and implementation of PACT in detail, highlighting the ways in which it embodies the VDA design principles for promoting learning of instructional design via overlay design tools.

Finally, I present two evaluations of the PACT system. First, a formative study that examines the utility of PACT for expert curriculum designers and curriculum research groups, using a sample annotation process – and reflection on the outcomes of that process – to demonstrate that my approach is feasible and useful for those groups. Second, a summative user study of the utility of PACT for novice learner-centered curriculum designers. I demonstrate PACT’s significant impact on how novice designers learn from expert-generated examples, how they perceive the credibility of those examples, and the quality of curriculum designs those novices can produce. These findings show that the VDA approach to learning works and that the PACT overlay curriculum design tool is a successful realization of VDA’s design principles.

Advisor: John F. Canny


BibTeX citation:

@phdthesis{Carle:EECS-2012-246,
    Author = {Carle, Andy},
    Title = {Promoting Learning of Instructional Design via Overlay Design Tools},
    School = {EECS Department, University of California, Berkeley},
    Year = {2012},
    Month = {Dec},
    URL = {http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-246.html},
    Number = {UCB/EECS-2012-246},
    Abstract = {<p>Design is a notoriously difficult profession to practice, and it is even more difficult to learn.  Traditionally, learning of design skills has been situated in the context of apprenticeships or formal design studios.  Unfortunately, these methods are inaccessible to practicing professionals due to constraints on time and location.  And, indeed, professional designers must continuously update their knowledge as paradigm shifts in design practice threaten to make their skills obsolete.  An ideal resolution to this problem is to situate the learning of design skills within the professional practice of design.  This dissertation studies an approach to this mode of situated learning, focusing on integrating learning mechanisms into practical design tools.  These tools provide scaffolding for novices as they construct an understanding of best practices in design while engaging in real design work.</p>
<p>I begin by introducing Virtual Design Apprenticeship (VDA), a learning model – built on a solid foundation of education principles and theories – that promotes learning of design skills via overlay design tools.  In VDA, when an individual needs to learn a new design skill or paradigm she is provided accessible, concrete examples that have been annotated with design rationale.  These annotations make expert thinking visible and allow the novice to immediately use, and gradually understand, new best practices.  By combining abstract rationale with concrete design instances, annotated artifacts become more useful than either could be alone.  I also present a set of design principles that guide the creation of VDA design tools – user interfaces built to mediate an individual’s interactions with annotated examples.</p>
<p>While VDA is applicable to most design fields, I narrow the scope of consideration to one particular domain of design by focusing in-depth on the instructional design difficulties that university-level faculty members face and how the VDA approach can address them.  I engaged with instructors and a curriculum design research group in a six-year period of contextual inquiry.  Findings from this study influenced my formulation of the VDA framework and the design of PACT, a design tool that leverages the learning principle of making thinking visible to assist novices as they transition from concrete to abstract reasoning about curriculum design. I discuss the iterative design and implementation of PACT in detail, highlighting the ways in which it embodies the VDA design principles for promoting learning of instructional design via overlay design tools.</p>
<p>Finally, I present two evaluations of the PACT system.  First, a formative study that examines the utility of PACT for expert curriculum designers and curriculum research groups, using a sample annotation process – and reflection on the outcomes of that process – to demonstrate that my approach is feasible and useful for those groups. Second, a summative user study of the utility of PACT for novice learner-centered curriculum designers.  I demonstrate PACT’s significant impact on how novice designers learn from expert-generated examples, how they perceive the credibility of those examples, and the quality of curriculum designs those novices can produce.  These findings show that the VDA approach to learning works and that the PACT overlay curriculum design tool is a successful realization of VDA’s design principles.</p>}
}

EndNote citation:

%0 Thesis
%A Carle, Andy
%T Promoting Learning of Instructional Design via Overlay Design Tools
%I EECS Department, University of California, Berkeley
%D 2012
%8 December 14
%@ UCB/EECS-2012-246
%U http://www2.eecs.berkeley.edu/Pubs/TechRpts/2012/EECS-2012-246.html
%F Carle:EECS-2012-246