Observing, Analyzing, and Modeling Design Team Problem Decompositions in Facility Design
This page contains the following sections:
Funding Source: This project is supported by National Science Foundation grants CMMI-1435074 and CMMI-1435449.
Background and Project Objectives
This research project is studying how teams of engineers decompose complex system design problems.
When faced with the problem of designing a complex system, a design team must make many decisions. Because many design problems are too difficult to solve all at once, design teams often decompose these problem into more manageable subproblems. The way that a problem is decomposed may affect the quality of the solution that can be constructed, especially when time and resources are limited.
Investigating and understanding the impact of decomposition on design solution quality will support the design of better design processes. Most design is carried out by teams of bounded rational human designers. Although many aspects of engineering design teams have been studied, we have a limited understanding of how teams solve design problems and how their decomposition strategies affect solution quality.
Our research is generating knowledge about how and how well teams of human engineers solve facility design problems.
In particular, we are identifying the decompositions used by design teams to solve facility design problems and establishing the relationships between the decomposition of the design problem and the quality of the design that is generated using that decomposition.
To do this, we are observing teams of human designers as they decompose and solve facility design problems, analyzing their decisions, and identifying their decomposition strategies.
We are considering two instances of facility design problems. In the first, teams of public health professionals are designing a mass antibiotic distribution facility (also known as a point-of-dispensing). In the second, teams of professional engineers are redesigning a manufacturing facility to make it more efficient.
The current research is part of a larger effort that contains two main parts. First, we address the question: how do teams of human designers decompose a complex problem into subproblems? Preliminary results suggest that teams informally decompose problems into subproblems and address these subproblems in various orders, reflecting different design strategies such as top-down and bottom-up. To confirm or revise these findings, we will identify decompositions used by design teams through two empirical studies of design teams developing solutions to realistic, challenging facility design problems.
Second, we will address the question: how does the decomposition of a complex system design problem affect the quality of the generated solution? The proposed research will draw upon the empirical results to develop mathematical models that simulate critical aspects of the behavior of teams who are solving complex design problems. These models will simulate the observed decompositions to investigate their impact on solution quality.
This figure shows the overall approach, highlighting the distinct methods required to address the research questions. Observations of design teams and analysis of their activities will lead to structured descriptions of their patterns of decision making, including the decompositions that are the essence of their design processes. Decompositions will be modeled as separations, which will be simulated using searches and other algorithms that model the bounded rational designers’ heuristics. These models will enable studies that investigate the impact of selected decompositions on the quality and characteristics of design solutions.
This paper presents a method for assessing the quality of a progressive design process by measuring the profitability of the product that the process generates. The proposed approach uses separations, a type of problem decomposition, to model progressive design processes. The subproblems in the separations correspond roughly to phases in the progressive design processes. The proposed method simulates the choices of a bounded rational designer for each subproblem using different search algorithms. This paper presents a simple two-variable problem to help describe the approach and then applies the approach to assess motor design processes. Different types and versions of these search processes are considered to determine if the results are robust to the decision-making model. The results indicate that well-designed progressive design processes are the best way to generate profitable product designs. Methods for assessing the quality of engineering design processes can be used to guide improvements to engineering design processes and generate more valuable products.
When faced with the problem of designing a complex system, a design team must make many decisions. Because many design problems are too difficult to solve all at once, the design problem is decomposed into more manageable subproblems. The way in which a problem is decomposed may affect the quality of the solution that can be constructed, especially when time and resources are limited. This paper describes the results of a study designed to (a) understand how design teams decompose complex design problems into sets of related subproblems and (b) assess the impact of these decompositions on the quality of the solutions that the design teams generate. In this preliminary study, we observed four teams of professional engineers who redesigned a manufacturing facility, and we analyzed their decision-making processes and the facility layouts that they generated. As we expected, teams used a variety of decomposition approaches. Some teams focused initially on designing manufacturing cells, while others began by laying out a high-level flow through the facility. These two strategies appeared to lead to different kinds of final facility designs (cellular and functional, respectively).
This paper focuses specifically on understanding whether and how decomposition strategies are discussed by design teams and examining whether decomposition processes are explicit or implicit (Ho, 2001). Based on our analysis of the results, we have identified when these teams discussed decomposition options and selected decomposition strategies. The results presented here are based on the same data that were analyzed by Gralla and Herrmann (2014), who considered the decompositions that were used and concluded that problem decomposition does indeed influence the character and quality of design solutions and, in particular, that top-down and bottom-up decompositions lead to different types of design solutions. The present study re-analyzed the data to determine whether and when teams discussed decomposition and whether and when an explicit decomposition was used.
In practice, when faced with a complex optimization problem, teams of human decision-makers often separate it into subproblems and then solve each subproblem instead of tackling the complete problem. It would be useful to know the conditions in which separating the problem is the superior approach and how the subproblems should be assigned to members of the teams. This paper describes a mathematical model of a search that represents a bounded rational decision-maker (agent) solving a generic optimization problem. The agent's search can be modeled as a discrete-time Markov chain, which allows one to calculate the probability distribution of the value of the solution that the agent will find. We compared the distributions generated by the model to the distribution of results from searches of solutions to traveling salesman problems. Using this model, we evaluated the performance of two- and three-agent teams who used different solution approaches to solve generic optimization problems. In the all-at-once approach, the agents collaborate to search the entire set of solutions in a sequential manner: the next agent begins where the previous agent stopped. In the separation approach, the agents separate the problem into two subproblems: (1) find the best set of solutions, and (2) find the best solution in that set. The results show that teams found better solutions using separation when high-value solutions are less likely. Using the all-at-once approach yielded better results when the values were uniformly distributed. The optimal assignment of subproblems to teams also depended upon the distribution of values in the solution space.
This research investigates how complex systems engineering design problems are decomposed and how that decomposition affects the quality of the solution. The problem we study is that of designing a point of dispensing (POD), which is a temporary facility for rapidly dispensing antibiotics or other healthcare. When designing a POD, a design team must make many decisions to determine the staffing and layout of the POD in a way that is feasible for the given location, uses available staff, and has sufficient capacity. Because the POD design problem is too difficult to solve all at once, it must be decomposed into more manageable subproblems. The decomposition affects the quality of the solution that is generated. To investigate how the problem is decomposed and its effect on the solutions, we observed teams of experienced public health emergency preparedness planners who designed a POD for a new high school, and we analyzed their decision-making processes and the designs that they generated. We used qualitative and quantitative techniques to identify the subproblems that were solved.
Because of the complexity inherent in the design of a complex system, many design teams decompose large problems into more manageable subproblems. The strategies employed in decomposing a design problem may directly impact the quality of the solution that they produce, particularly when constraints are placed on time and resources. This paper describes the preliminary results of a study designed to (a) understand how design teams decompose complex design problems into sets of related subproblems and (b) assess the impact of these decompositions on the quality of the solutions that the design teams generate. In this study, we observed the strategies of ten teams of professional engineers as they re-designed a manufacturing facility, and we analyzed their decision-making processes. We then developed metrics and used expert advice to assess the quality of the facility layouts that they generated. Although the teams informally decomposed the problem, we were able to identify the subproblems that they considered as they solved the problem, and we looked for patterns among those decompositions. We also measured the quality of each team's design solution in order to link decomposition strategy to solution quality.
Many design problems are too difficult to solve all at once; therefore, design teams often decompose these problems into more manageable subproblems. While there has been much interest in engineering design teams, no standard method has been developed to understand how teams solve design problems. This paper describes a method for analyzing a team's design activities and identifying the subproblems that they considered. This method uses both qualitative and quantitative techniques; in particular, it uses association rule learning to group variables into subproblems. We used the method on data from ten teams who redesigned a manufacturing facility. This approach provides researchers with a clear structure for using observational data to identify the problem decomposition patterns of human designers.
Designers work in teams to design complex systems. They separate the design problem into subproblems and solve the smaller, more manageable subproblems. Because this affects the overall quality of their design, it is important to understand how teams decompose system design problems, which will ultimately enable future research on how to design better design processes. We studied teams of experts solving two different facility design problems. We developed a novel approach that combines qualitative and quantitative techniques. It records a team's discussion, identifies the design variables using qualitative coding techniques, and groups these variables into subproblems. A subproblem is a set of variables that are considered together. We evaluated four clustering algorithms that group the coded variables into subproblems. This paper discusses the data collection, the clustering algorithms, and the evaluation techniques. The the algorithms generated similar but not identical clusters, and no algorithm's clusters consistently out-performed the others on quantitative measures of cluster quality. The clusters do provide insights into the subproblems that the design team solved.
Design problems are inherently intricate and require multiple dependent decisions. Because of these characteristics, design teams generally choose to decompose the main problem into manageable subproblems. This thesis describes the results of a study designed to (a) explore clustering algorithms as a new and repeatable way to identify subproblems in recorded design team discussions, (b) assess the quality of the identified subproblems, and (c) examine any relationships between the subproblems and final design or team experience level. We observed five teams of public health professionals and four teams of undergraduate students and applied four clustering algorithms to identify the team's subproblems and achieve the aforementioned research goals. The use of clustering algorithms to identify subproblems has not been documented before, and clustering presents a repeatable and objective method for determining a team's subproblems. The results from these algorithms as well as metrics noting the each result's quality were captured for all teams. We learned that each clustering algorithm has strengths and weaknesses depending on how the team discussed the problem, but the algorithms always accurately identify at least some of the discussed subproblems. Studying these identified subproblems reveals a team's design process and provides insight into their final design choices.
Understanding how humans decompose design problems will yield insights that can be applied to develop better support for human designers. However, there are few established methods for identifying the decompositions that human designers use. This paper discusses a method for identifying subproblems by analyzing when design variables were discussed concurrently by human designers. Four clustering techniques for grouping design variables were tested on a range of synthetic datasets designed to resemble data collected from design teams, and the accuracy of the clusters created by each algorithm was evaluated. A spectral clustering method was accurate for most problems and generally performed better than hierarchical (with Euclidean distance metric), Markov, or association rule clustering methods. The method's success should enable researchers to gain new insights into how human designers decompose complex design problems.
When available, de-identified data for use by other researchers will be found here.
Last updated on May 9, 2018, by Jeffrey W. Herrmann.