3 – L1 – Matrix Evaluation

Written by Muqi Wulan

You may have gone through the following design processes: 1) understand customer needs, 2) develop product design specification (PDS), and 3) generate design concepts. When many design alternatives have been created at the concept stage, the designer will confront with how to select the best one from these alternatives for further investigation, testing or development. Evaluating alternatives is frequent in design activities. Decisions can be made by guesswork, by intuition, by experience or by arbitrary choice. However rational methods are required in this process (Cross 2008: 151). Pugh concept selection method, invented by Stuart Pugh, is a commonly used, structural technique to narrow the choice of design concepts.

The Pugh method, through a concept selection chart or matrix, compares each concept relative to a reference or datum concept, and determines if the concept is better, poorer than, or the same as the reference one(Pugh 1991: 74) (Pugh 1996: 167) (Clausing 1994: 153) (Dieter and Schmidt 2009: 277). The steps in the concept selection method are shown as follows (Figure 1).

figure-1-pugh-concept-selection-matrix
Figure 1 – Pugh concept selection matrix (Lecture on Pugh Method 2011)

Step 1: Select the criteria by which the concepts will be evaluated.
The list of criteria is developed from the customer needs and engineering characteristics. All team members should contribute in making the list. The list of the 15 to 20 criteria is created based on Quality Function Deployment (QFD) with other important factors which are not covered by the QFD. Then these criteria is organised in the order of priority and entered into the matrix. Figure 2 shows the list of the criteria identified in the example of a bicycle splashguard.

figure-2-criteria-for-design-concepts-on-a-bicycle-splashguard
Figure 2 – Criteria for design concepts on a bicycle splashguard (Lecture on Pugh Method 2011)

Step 2: Clarify the design concepts and choose the reference concept.
All the members of the development team are brought together to reach a common understanding on each possible design concept. Then they decide the concepts to be evaluated. One concept is picked as a datum, or a reference concept with which all other concepts are compared. In the case of bicycle splashguard, there are seven design concepts are proposed. The concept VII is chosen as the reference. Other six concepts will be evaluated against this one.

figure-3-design-concepts-on-a-bicycle-splashguard
Figure 3 – Design concepts on a bicycle splashguard (Lecture on Pugh Method 2011)

Step 3: Build the concept comparison matrix.
Each design concept is compared with the datum in terms of each criterion. A three level scale is used. For each comparison, if the concept is evaluated better (+), worse (-) than, or the same (s) is as the datum, the appropriate symbol is input into the cell of the matrix. The example is depicted in Figure 4.

figure-4-concept-comparison-matrix-on-a-bicycle-splashguard
Figure 4 – Concept comparison matrix on a bicycle splashguard (Lecture on Pugh Method 2011)

Step 4: Score the design concepts.
Once the comparison matrix is completed, the sums of the “+” and “-” ratings are calculated for each concept, as listed in the rows of “TOTAL +” and “TOTAL –”. Therefore the overall score on each design concept is determined based on the above values. It is inserted in the row of “OVERALL SCORE”, as shown in the example of the bicycle splashguard (Figure 5). Since the criteria have different weights on meeting customer needs, these weights are taken into account in the “WEIGHTED OVERALL SCORE”. From the result of these weighted score, it is easy to find that the concepts II, IV and V are the best design of the top three.

figure-5-concept-selection-matrix-for-a-bicycle-splashguard
Figure 5 – Concept selection matrix for a bicycle splashguard (Lecture on Pugh Method 2011)

Step 5: Rerun concept selection matrix.
The task in this step is to establish a new datum, usually the concept that received the highest score in the first round, and rerun the matrix again using the concepts in the top three. Eliminate the lowest scored concept from the second round. In Figure 6, the concept II is set as the datum to compare with the concept IV and the concept V. From the result, the concept II is the most promising design for the bike splashguard. The purpose of this round is not to verify the selection in the first round is valid but to inspire further creativity by comparing the left design concepts.

figure-6-the-matrix-in-the-second-round-concept-selection-for-a-bicycle-splashguard
Figure 6 – The matrix in the second round concept selection for a bicycle splashguard (Lecture on Pugh Method 2011)

Step 6: Examine the selected concept for improvement opportunities.
After the best design concept is identified, each criterion that performed worse than the datum should be examined. It will lead to design improvement on the selected concept.

Summary
Pugh concept selection method uses a structured concept selection matrix, and provides comparative means to evaluate design concepts. It gives insights into strong and weak areas of the concepts. The method is implemented in procedure by the design team, usually in the iterative way of examination and deliberation.
Step 1: Select the criteria by which the concepts will be evaluated.
Step 2: Clarify the design concepts and choose the reference concept.
Step 3: Build the concept comparison matrix.
Step 4: Score the design concepts.
Step 5: Rerun concept selection matrix.
Step 6: Examine the selected concept for improvement opportunities.

Back to Concept Design
Back to MAE Design Model

Selected References
Clausing, D. (1994) Total quality development. New York: ASME Press
Lecture on Pugh Method (2011) Concept evaluation and selection [online] available from    <http://www.me.berkeley.edu/ME110/presentations/ConceptSelection.pdf> [27July 2011]
Pugh, S. (1991) Total design: integrated methods for successful product engineering. Harlow: Pearson Education
Pugh, S. (1996) ‘Concept selection: a method that works’. in Creating innovative products using total design. ed. By Clausing, D. and Andrade, R. Massachusetts: Addison-Wesley