In this blog post, we formally define or describe the basic terminology that is commonly used in Design of Experiments. Some of the terms we have already been using in prior posts, but they will also be presented here for completeness. This is Part I of a two part blog covering DOE Terminology.
Response
The response(s) in an experiment are the outcomes that we measure that align with the objectives of the experiment. For example, if we were trying to produce a formulation that meets a specific viscosity specification, then the actual measured viscosity would the natural response.
Different types of responses exist. A “Location Response” is the most common and refers to a variable measurement such as hardness, strength, a dimension, etc. The location response allows us to understand how we might ensure we hit a certain variable response on average. A “Variance Response” is a type of response that is calculated from multiple repetitions. The purpose of the variance response is to model variation in the response as a function of significant factors. Often this occurs when the purpose of the study is to reduce variation. Finally, a proportion response may be used when the actual outcome is binary (e.g. pass/fail). The response could be the proportion of trials that caused a failure. Proportion responses are difficult due to the large sample sizes that are needed to detect changes in events that are relatively unlikely (e.g. low scrap rates). It is recommended to find a location response that can serve a proxy for a proportion response. For example, instead of using leak/no-leak as a response, the actual leak rate could be measured providing a variable result.
To review, suppose we are producing mirrors and are trying to optimize the process. Following are three types of responses.
Factors
Factors are variables that we manipulate in the study because we believe they may have an impact on the response(s) in the study.
In most experiments, the factors are independent of each other (an exception is Mixture experiments) and can be controlled at least during the study. Factors may be quantitative (e.g. temperature) or qualitative (e.g. resin type).
In the last blog and this one, we have discussed why statistically based DOE provides several advantages over more simplistic approaches such “one-factor-at-a-time” experimentation.
k
The letter k is used to designate the number of factors in an experiment. In the graphic below, we see three factors that may be adjusted, so k = 3.
Level/Setting
Each factor may be set at 2 or more different levels during the study.
As we discussed in an earlier post, screening studies the factors have few (e.g. 2) levels since they are much more efficient. In optimization studies, we typically have more than 2 levels so that non-linearities may be accounted for in the predictive model.
Treatment
A treatment (or trial) is a set of conditions that determine where each factor is set. Each treatment combination is set up and the response is measured. The graphic below the levels are simply coded as “-“ (low level) and “+” (high level)
Factorial Design
A factorial design (or full factorial design) is an experimental design where all possible (unique) combinations of treatments are run. Fortunately, when lots of factors are present in the study, we do not need to run the Full Factorial design.
One of the simplest experiments is a 2-factor, 2-level study that is illustrated below. Please note that this type of study is generally not recommended for reasons that will be discussed in a future blog post. Here, we have two factors (X1 and X2) and they are set at low “-“ and high “+” levels. The four possible combinations are shown in the matrix and also shown geometrically as the four corners of a square.
You can imagine that the center of square is the origin of an x1, x2 coordinate system. A “-“ can be thought of a value of -1 and a “+” is +1. The corners of the square form the bounds of the square that is called the design space. This bounds the values of x1, x2 that the model will be used for prediction. We generally do not use our model to predict responses for factor combinations that live outside of the design space. In total there are 2k = 22 = 4 treatments.
Below, we extend the situation to 3 factors (still each factor is at only 2 levels)
Now the design space is a cube and the 3rd factor provides the depth dimension in the geometric interpretation. In total there are 2k = 23 = 8 treatments. This can be extended for more than 3 factors, although it becomes challenging geometrically.
Main Effects
Main Effects are the quantitative impact that changing a factor has on the response. For 2-level studies, main effects are easy to compute. For each factor, the main effect is the average response observed when the factor is at the high level minus the average response when the factor is at the low level. It essentially describes the change in average response as the factor goes from a low level to a high level.
Main effects and interaction effects must be tested for statistical significance before they are incorporated into a predictive model. The methods for doing this will be covered in an upcoming blog post.
The basic terminology defined in this article will be used going forward as DOE is explored in greater detail.