#### Presentation Title

Accounting for Type 2 Error in the Judgment of Significance of Effects in a Two-level Factorial Design

#### Faculty Mentor

Jessica Jaynes

#### Start Date

23-11-2019 8:45 AM

#### End Date

23-11-2019 9:30 AM

#### Location

224

#### Session

poster 2

#### Type of Presentation

Poster

#### Subject Area

physical_mathematical_sciences

#### Abstract

Experimenters making judgments about the significance of effects will have to account for the potential of making two kinds of decision-making errors, α and β risk corresponding to the probability of a type I error (false positive) or a type II error (false negative), respectively. The interest of this study was to design a model that detailed the relationship between the two error types allowing for more risk-tolerant conclusions to be made by giving clear comparisons between α and β. In practice, a 5% risk of a type I error is commonly set as the comparative standard used to determine significance, while type II error is only considered indirectly through power analysis. However, preliminary screening experiments, such as two-level factorial designs, would benefit from lessening the chance of such false negatives. To relate the two error types, the concept of minimum effect size of interest (MESI) has been proposed. In our study, MESI was used to define non-central t-distributions, from which experimenters could find t-values based on desired β risk tolerance, rather than the typical α. As an example, we considered sample data from a previously published study that might benefit from specific β consideration. This data was from an unreplicated factorial design that measured the responses (as percentages) of four factor’s abilities, set at high and low levels, to remove tetracycline from wastewater. A regression model is one way to quantify the effects of factors and was used to calculate main and interaction effects for the four factors. These effect values were converted to t-values for significance testing. By plotting the non-central t-distribution corresponding to several choices of MESI along with α risk, the two risk probabilities associated with a t-statistic based judgment of significance for each of the factors could be seen and used to make precise determinations.

Accounting for Type 2 Error in the Judgment of Significance of Effects in a Two-level Factorial Design

224

Experimenters making judgments about the significance of effects will have to account for the potential of making two kinds of decision-making errors, α and β risk corresponding to the probability of a type I error (false positive) or a type II error (false negative), respectively. The interest of this study was to design a model that detailed the relationship between the two error types allowing for more risk-tolerant conclusions to be made by giving clear comparisons between α and β. In practice, a 5% risk of a type I error is commonly set as the comparative standard used to determine significance, while type II error is only considered indirectly through power analysis. However, preliminary screening experiments, such as two-level factorial designs, would benefit from lessening the chance of such false negatives. To relate the two error types, the concept of minimum effect size of interest (MESI) has been proposed. In our study, MESI was used to define non-central t-distributions, from which experimenters could find t-values based on desired β risk tolerance, rather than the typical α. As an example, we considered sample data from a previously published study that might benefit from specific β consideration. This data was from an unreplicated factorial design that measured the responses (as percentages) of four factor’s abilities, set at high and low levels, to remove tetracycline from wastewater. A regression model is one way to quantify the effects of factors and was used to calculate main and interaction effects for the four factors. These effect values were converted to t-values for significance testing. By plotting the non-central t-distribution corresponding to several choices of MESI along with α risk, the two risk probabilities associated with a t-statistic based judgment of significance for each of the factors could be seen and used to make precise determinations.