This paper proposes an approach to investigate associations between design concepts and design outcomes. In the proposed approach, three deliverables (concept sketches, proof-of-concept (POC) prototypes, and final products) are evaluated using metrics called creative product semantic scale (CPSS). CPSS scores are analyzed using two methods: correlation analysis and classification tree analysis. Correlation analysis is used to investigate associations between the concept sketches or the POC prototypes and the final products. Classification tree analysis (together with leave-one-out cross-validation (LOOCV) and permutation test) is used to investigate relationships between CPSS scores and the retention of design concepts (i.e., concepts that are embodied in the final products). The proposed approach is illustrated using deliverables in a project-based design class.
Issue Section:
Design Theory and Methodology
References
1.
Leary
, W. E.
, 2002
, “The Inquiring Minds Behind 200 Years of Inventions
,” The New York Times, Oct. 22, 2002.2.
3.
Åstebro
, T.
, 1998
, “Basic Statistics on the Success Rate and Profits for Independent Inventors
,” Entrepreneurship Theory Pract.
, 23
(2
), pp. 41
–48
.4.
Ward
, A. C.
, Liker
, J. K.
, Cristiano
, J. J.
, and Sobek
, II, D. K.
, 1995
, “The Second Toyota Paradox: How Delaying Decisions Can Make Better Cars Faster
,” Sloan Manage. Rev.
, 36
(3
), pp. 43
–61
.5.
Fredman
, G.
, 2002
, The IDEO Difference
, Hemispheres
, Pace Communications, Greensboro, NC, pp. 52
–57
.6.
Zuber
, C. D.
, Alterescu
, V.
, and Chow
, M. P.
, 2005
, “Fail Often to Succeed Sooner: Adventures in Innovation
,” Perm. J.
, 9
(4
), pp. 44
–49
.7.
National Academy of Engineering
, 2004
, The Engineer of 2020: Visions of Engineering in the New Century
, National Academies
, Washington, DC
.8.
National Academy of Engineering
, 2005
, Educating the Engineer of 2020: Adapting Engineering Education to the New Century
, National Academies
, Washington, DC
.9.
Dym
, C. L.
, Agogino
, A. M.
, Frey
, D. D.
, Eris
, O.
, and Leifer
, L. J.
, 2005
, “Engineering Design Thinking, Teaching, and Learning
,” J. Eng. Educ.
, 94
(1
), pp. 103
–120
.10.1002/j.2168-9830.2005.tb00832.x10.
Baillie
, C.
, 2002
, “Enhancing Creativity in Engineering Students
,” Eng. Sci. Educ. J.
, 11
(5
), pp. 185
–192
.10.1049/esej:2002050311.
Liu
, Z.
, and Schönwetter
, D. J.
, 2004
, “Teaching Creativity in Engineering
,” Int. J. Eng. Educ.
, 20
(5
), pp. 801
–808
.12.
Amabile
, T. M.
, 1982
, “Social Psychology of Creativity: A Consensual Assessment Technique
,” J. Pers. Soc. Psychol.
, 43
(5
), pp. 997
–1013
.10.1037/0022-3514.43.5.99713.
Campbell
, D. T.
, 1960
, “Blind Variation and Selective Retention in Creative Though as in Other Knowledge Process
,” Psychol. Rev.
, 67
(6
), pp. 380
–400
.10.1037/h004037314.
Mednick
, S. A.
, 1962
, “The Association Basis of the Creative Process
,” Psychol. Rev.
, 69
(3
), pp. 220
–232
.10.1037/h004885015.
Rothenberg
, A.
, 1979
, The Emerging Goddess: The Creative Process in Art, Science, and Other Fields
, University of Chicago
, Chicago, IL
.16.
Finke
, R. A.
, Ward
, T. B.
, and Smith
, S. M.
, 1992
, Creative Cognition: Theory, Research, and Applications
, MIT
, Cambridge, MA
.17.
Finke
, R. A.
, 1995
, “Creative Realism
,” Creative Cognition Approach
, S. M.
Smith
, T. B.
Ward
, and R. A.
Finke
, eds., MIT
, Cambridge, MA
.18.
Ward
, T. B.
, Smith
, S. M.
, and Vaid
, J.
, 1997
, Creative Thought: An Investigation of Conceptual Structures and Processes
, American Psychological Association
, Washington, DC
.10.1037/10227-00019.
Ward
, T. B.
, Smith
, S. M.
, and Vaid
, J.
, 1997
, “Conceptual Structures and Processes in Creative Thought
,” Creative Thought: An Investigation of Conceptual Structures and Processes
, T. B.
Ward
, S. M.
Smith
, and J.
Vaid
, eds., American Psychological Association
, Washington, DC
.20.
Domino
, G.
, 1970
, “Identification of Potentially Creative Persons From the Adjective Check List
,” J. Consult. Clin. Psychol.
, 35
(1
), pp. 48
–51
.10.1037/h002962421.
Gough
, H. A.
, 1979
, “A Creative Personality Scale for the Adjective Check List
,” J. Pers. Soc. Psychol.
, 37
(8
), pp. 1398
–1407
.10.1037/0022-3514.37.8.139822.
Strack
, S.
, and Lorr
, M.
, 1990
, “Item Factor Structure of the Personality Adjective Check List
,” J. Pers. Assess.
, 55
(1–2
), pp. 86
–94
.10.1207/s15327752jpa5501&2_923.
Kaduson
, H. G.
, 1991
, “Concurrent Validity of the Creative Personality Scale of the Adjective Check List
,” Psychol. Rep.
, 69
(2
), pp. 601
–602
.10.2466/pr0.1991.69.2.60124.
Torrance
, E. P.
, 1962
, Guiding Creative Talent
, Prentice-Hall
, Englewood Cliffs, NJ
.10.1037/13134-00025.
Jansson
, D. G.
, and Smith
, S. M.
, 1991
, “Design Fixation
,” Des. Stud.
, 12
(1
), pp. 3
–11
.10.1016/0142-694X(91)90003-F26.
Purcell
, A. T.
, and Gero
, J. S.
, 1996
, “Design and Other Types of Fixation
,” Des. Stud.
, 17
(4
), pp. 363
–383
.10.1016/S0142-694X(96)00023-327.
Kokotovich
, V.
, and Purcell
, T.
, 2000
, “Mental Synthesis and Creativity in Design: An Experimental Examination
,” Des. Stud.
, 21
(5
), pp. 437
–449
.10.1016/S0142-694X(00)00017-X28.
Hey
, J.
, Linsey
, J.
, Agogino
, A.
, and Wood
, K.
, 2008
, “Analogies and Metaphors in Creative Design
,” Int. J. Eng. Educ.
, 24
(2
), pp. 283
–294
.29.
Linsey
, J.
, Wood
, K.
, and Markman
, A.
, 2008
, “Modality and Representation in Analogy
,” Artif. Intell. Eng. Des. Anal. Manuf.
, 22
(2
), pp. 85
–100
.10.1017/S089006040800006130.
Tseng
, I.
, Moss
, J.
, Cagan
, J.
, and Kotovsky
, K.
, 2008
, “The Role of Timing and Analogical Similarity in the Stimulation of Idea Generation in Design
,” Des. Stud.
, 29
(3
), pp. 203
–221
.10.1016/j.destud.2008.01.00331.
Ball
, L. J.
, and Christensen
, B. T.
, 2009
, “Analogical Reasoning and Mental Simulation in Design: Two Strategies Linked to Uncertainty Resolution
,” Des. Stud.
, 30
(2
), pp. 169
–186
.10.1016/j.destud.2008.12.00532.
Nagai
, Y.
, Taura
, T.
, and Mukai
, F.
, 2009
, “Concept Blending and Dissimilarity: Factors for Creative Concept Generation Process
,” Des. Stud.
, 30
(6
), pp. 648
–675
.10.1016/j.destud.2009.05.00433.
Linsey
, J.
, Tseng
, I.
, Fu
, K.
, Cagan
, J.
, Wood
, K.
, and Schunn
, C.
, 2010
, “A Study of Design Fixation, its Mitigation and Perception in Engineering Design Faculty
,” ASME J. Mech. Des.
, 132
(4
), p. 041003
.10.1115/1.400111034.
Linsey
, J. S.
, Clauss
, E. F.
, Kurtoglu
, T.
, Murphy
, J. T.
, Wood
, K. L.
, and Markman
, A. B.
, 2011
, “An Experimental Study of Group Idea Generation Techniques: Understanding the Roles of Idea Representation and Viewing Methods
,” ASME J. Mech. Des.
, 133
(3
), p. 031008
.10.1115/1.400349835.
Linsey
, J.
, Markman
, A.
, and Wood
, K.
, 2012
, “Design by Analogy: A Study of the WordTree Method for Problem Re-Representation
,” ASME J. Mech. Des.
, 134
(4
), p. 041009
.10.1115/1.400614536.
Schütze
, M.
, Sachse
, P.
, and Römer
, A.
, 2003
, “Support Value of Sketching in the Design Process
,” Res. Eng. Des.
, 14
(2
), pp. 89
–97
.37.
Dong
, A.
, 2004
, “A Document Analysis Method for Characterizing Design Team Performance
,” ASME J. Mech. Des.
, 126
(3
), pp. 378
–385
.10.1115/1.171181838.
Ruder
, J. A.
, and Sobek
, II, D. K.
, 2007
, “Experiment on a System-Level Design Tool
,” J. Eng. Des.
, 18
(4
), pp. 327
–342
.10.1080/0954482060100892839.
Song
, S.
, and Agogino
, A. M.
, 2004
, “Insights on Designers' Sketching Activities in New Product Design Teams
,” ASME
Paper No. DETC2004-57474.10.1115/DETC2004-5747440.
Yang
, M. C.
, 2009
, “Observations on Concept Generation and Sketching in Engineering Design
,” Res. Eng. Des.
, 20
(1
), pp. 1
–11
.10.1007/s00163-008-0055-041.
Wilde
, D. J.
, 2009
, Teamology: The Construction and Organization of Effective Teams
, Springer
, NY
.42.
Shah
, J. J.
, Kulkarni
, S. V.
, and Vargas-Hernandez
, N.
, 2000
, “Evaluation of Idea Generation Methods for Conceptual Design: Effectiveness Metrics and Design of Experiments
,” ASME J. Mech. Des.
, 122
(4
), pp. 377
–384
.10.1115/1.131559243.
Shah
, J. J.
, Vargas-Hernandez
, N.
, and Smith
, S. M.
, 2003
, “Metrics for Measuring Ideation Effectiveness
,” Des. Stud.
, 24
(2
), pp. 111
–134
.10.1016/S0142-694X(02)00034-044.
Chusilp
, P.
, and Jin
, Y.
, 2006
, “Impact of Mental Iteration on Concept Generation
,” ASME J. Mech. Des.
, 128
(1
), pp. 14
–25
.10.1115/1.211870745.
O'Quin
, K.
, and Besemer
, S. P.
, 1989
, “The Development, Reliability, and Validity of the Revised Creative Product Semantic Scale
,” Creativity Res. J.
, 2
(4
), pp. 267
–278
.10.1080/1040041890953432346.
Besemer
, S. P.
, and Treffinger
, D. J.
, 1981
, “Analysis of Creative Products: Review and Synthesis
,” J. Creative Behav.
, 15
(3
), pp. 158
–178
.10.1002/j.2162-6057.1981.tb00287.x47.
Besemer
, S. P.
, and O'Quin
, K.
, 1986
, “Analyzing Creative Products: Refinement and Test of a Judging Instrument
,” J. Creative Behav.
, 20
(2
), pp. 115
–126
.10.1002/j.2162-6057.1986.tb00426.x48.
Besemer
, S. P.
, and O'Quin
, K.
, 1987
, “Creative Product Analysis: Testing a Model by Developing a Judging Instrument
,” Frontiers of Creativity Research
, S. G.
Isaksen
, ed., Bearly Limited
, Buffalo, NY
, pp. 341
–357
.49.
Dahl
, D. W.
, Chattopadhyay
, A.
, and Gorn
, G. J.
, 2001
, “The Importance of Visualisation in Concept Design
,” Des. Stud.
, 22
(1
), pp. 5
–26
.10.1016/S0142-694X(99)00028-950.
Kim
, M. H.
, Kim
, Y. S.
, Lee
, H. S.
, and Park
, J. A.
, 2007
, “An Underlying Cognitive Aspect of Design Creativity: Limited Commitment Mode Control Strategy
,” Des. Stud.
, 28
(6
), pp. 585
–604
.10.1016/j.destud.2007.04.00651.
Takai
, S.
, 2011
, “A Pre-Post Assessment of Creativity Methods in an Experimental Course
,” Int. J. Eng. Educ.
, 27
(2
), pp. 380
–388
.52.
White
, A.
, and Smith
, B. L.
, 2001
, “Assessing Advertising Creativity Using the Creative Product Semantic Scale
,” J. Advertising Res.
, 41
(6
), pp. 27
–34
.53.
White
, A.
, Shen
, F.
, and Smith
, B. L.
, 2002
, “Judging Advertising Creativity Using the Creative Product Semantic Scale
,” J. Creative Behav.
, 36
(4
), pp. 241
–253
.10.1002/j.2162-6057.2002.tb01067.x54.
Smith
, B. L.
, 1993
, “Interpersonal Behaviors that Damage the Productivity of Creative Problem Solving Groups
,” J. Creative Behav.
, 27
(3
), pp. 171
–187
.10.1002/j.2162-6057.1993.tb00705.x55.
Bytheway
, C. W.
, 2007
, FAST Creativity & Innovation: Rapidly Improving Processes, Product Development and Solving Complex Problems
, J. Ross Pub.
, Fort Lauderdale, FL
.56.
Olivero
, M.
, 1990
, “Get Crazy! How to Have a Breakthrough Idea
,” Working Woman
, 213
(1
), pp. 145
–147
, 222.57.
Osborn
, A. F.
, 1963
, Applied Imagination: Principles and Procedures of Creative Problem-Solving
, Scribner
, NY
.58.
Pahl
, G.
, and Beitz
, W.
, 1996
, Engineering Design—A Systematic Approach
, Springer Limited
, London, UK
.59.
Ojala
, M.
, and Garriga
, G. C.
, 2010
, “Permutation Tests for Studying Classifier Performance
,” J. Mach. Learn. Res.
, 11
(6
), pp. 1833
–1863
.60.
Shrout
, P. E.
, and Fleiss
, J. L.
, 1979
, “Intraclass Correlations: Uses in Assessing Rater Reliability
,” Psychol. Bull.
, 86
(2
), pp. 420
–428
.10.1037/0033-2909.86.2.42061.
Tseng
, I.
, Cagan
, J.
, Kotovsky
, K.
, and Wood
, W.
, 2012
, “Form Function Fidelity
,” ASME J. Mech. Des.
, 135
(1
), p. 011006
.10.1115/1.400793362.
Hallgren
, K. A.
, 2012
, “Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial
,” Tutorials Quant. Methods Psychol.
, 8
(1
), pp. 23
–34
.63.
Cicchetti
, D. V.
, 1994
, “Guidelines, Criteria, and Rules of Thumb for Evaluating Normed and Standardized Assessment Instruments in Psychology
,” Psychol. Assess.
, 6
(4
), pp. 284
–290
.10.1037/1040-3590.6.4.28464.
Multon
, K.
, 2010
, “Interrater Reliability
,” Encyclopedia of Research Design
, N.
Salkind
, ed., SAGE Publications, Inc.
, Thousand Oaks, CA
, pp. 627
–629
.65.
Takai
, S.
, and Ishii
, K.
, 2010
, “A Use of Subjective Clustering to Support Affinity Diagram Results in Customer Needs Analysis
,” Concurrent Eng.: Res. Appl.
, 18
(2
), pp. 101
–109
.10.1177/1063293X10372792Copyright © 2015 by ASME
You do not currently have access to this content.
