ETX1100/ETX5900 Business Statistics

Week 6: Multiple regression and model evaluation

12 January 2027

Today’s journey

  1. Extend regression to several predictors.
  2. Interpret partial slopes while holding other variables constant.
  3. Test coefficients and evaluate overall model fit.
  4. Represent categorical predictors with dummy variables.
  5. Refine, diagnose, predict, and communicate a model.

From one predictor to several

Retrieval check

  • What does a simple-regression slope mean?
  • What does R^2 measure?
  • Why inspect residuals?
  • Why can an omitted variable distort a simple-regression relationship?

Multiple linear regression

Population model:

Y_i=\beta_0+\beta_1X_{1i}+\cdots+\beta_kX_{ki}+\varepsilon_i.

Fitted model:

\hat Y_i=\hat\beta_0+\hat\beta_1X_{1i}+\cdots+\hat\beta_kX_{ki}.

Multiple regression estimates each predictor’s association with Y conditional on the others in the model.

Partial slopes

\hat\beta_j is the predicted mean change in Y for a one-unit increase in X_j, holding all other included predictors constant.

This is not automatically a causal effect. “Holding constant” is a model comparison, not necessarily an intervention.

Watch me: simple versus multiple slopes

In real-estate data, the simple slope for area mixes area with correlated features such as bedrooms and bathrooms.

Add those predictors and compare:

  • the area coefficient;
  • its standard error; and
  • the meaning of the conditional comparison.

A changed coefficient is evidence that model context matters.

Now you try: interpret partial slopes

For

\widehat{Price}=86.8+47.4Beds+216.5Baths+14.5Cars+0.831Area,

interpret the coefficients of Baths and Area, including units and the variables held constant. Explain why neither wording should use “causes”.

Building the model in Excel

Prepare the data

  • One row per observation.
  • One column per variable.
  • Response in one contiguous column.
  • Predictors in adjacent columns for Excel’s Regression tool.
  • Clear headers and consistent units.
  • Missing values resolved explicitly—not silently converted to zero.

Watch me: fit a real-estate model

Open week6-realestate-data.xlsx.

  1. Choose Price as the Input Y Range.
  2. Select a contiguous block of numerical predictors for Input X Range.
  3. Tick Labels, residuals, and residual plots.
  4. Place output on a new worksheet.
  5. Map every output column back to the model equation.

Reading Excel regression output

Three regions answer different questions:

  • Regression Statistics: fit and typical error.
  • ANOVA: whether the predictor set has overall linear explanatory value.
  • Coefficients: estimated partial slopes, uncertainty, tests, and intervals.

Never interpret a p-value before confirming which row and hypothesis it belongs to.

Now you try: reconstruct the model

From your Excel output:

  1. write the fitted equation;
  2. interpret two slopes;
  3. identify any meaningless intercept interpretation;
  4. predict price for one observed property; and
  5. calculate its residual.

Inference and model evaluation

Test an individual coefficient

For predictor X_j:

H_0:\beta_j=0,\qquad H_1:\beta_j\ne0.

Excel’s coefficient p-value is two-sided. A small p-value indicates evidence of a linear association with Y after accounting for other included predictors.

Watch me: coefficient evidence

For each coefficient:

  1. identify estimate and standard error;
  2. verify t=\hat\beta_j/SE(\hat\beta_j);
  3. compare p-value with \alpha;
  4. inspect the confidence interval; and
  5. state the conditional business conclusion.

Now you try: decide what contributes

At \alpha=0.05:

  1. identify predictors with evidence of contribution;
  2. identify candidates for removal;
  3. explain why p-value alone is not enough to delete a variable; and
  4. name one variable that business logic might require even if its p-value is large.

R^2 and adjusted R^2

  • R^2 cannot decrease when predictors are added.
  • Adjusted R^2 penalises additional predictors that add little explanatory value.
  • Compare models fitted to the same response and observations.
  • A high value does not establish correct specification, causality, or good out-of-sample prediction.

Now you try: compare two models

Fit a full and a reduced real-estate model.

Compare:

  1. adjusted R^2;
  2. standard error of regression;
  3. coefficient stability;
  4. residual behaviour; and
  5. interpretability.

Recommend one model and justify the trade-off.

Overall F test

H_0:\beta_1=\beta_2=\cdots=\beta_k=0

H_1:\text{at least one slope is non-zero}.

Excel reports the p-value as Significance F. This evaluates the predictor set collectively; it does not say every predictor is significant.

Watch me: separate model and coefficient claims

A small Significance F supports an overall linear relationship between Y and the predictor set.

Then inspect individual coefficient tests to learn which predictors contribute conditionally. A significant model can contain non-significant individual predictors.

Now you try: interpret the ANOVA block

Using your model output:

  1. state the F-test hypotheses;
  2. report Significance F;
  3. make the decision at 5%;
  4. write the contextual conclusion; and
  5. explain why it does not prove every slope is non-zero.

Categorical predictors

Dummy variables

For a categorical predictor with j categories, use j-1 dummy variables.

  • Each dummy takes 0 or 1.
  • The omitted category is the base category.
  • Dummy coefficients compare their category with the base, holding other predictors constant.
  • Including all j dummies with an intercept creates perfect redundancy.

Watch me: create training-method dummies

Open week6-underwriting-data.xlsx.

With courseware app as the base category, create:

  • DClassroom = IF(Method="Classroom",1,0);
  • DOnline = IF(Method="Online",1,0).

Check that courseware rows have zero for both dummies.

Now you try: verify the coding

  1. Complete both dummy columns.
  2. Check at least two observations from each training method.
  3. Explain why no DCourseware column enters the model.
  4. Predict the dummy values for a new online trainee.

Interpret dummy coefficients

For

\widehat{Exam}=-63.981+1.126Proficiency-22.289DClassroom+8.088DOnline,

  • classroom is predicted 22.289 points below courseware at equal proficiency;
  • online is predicted 8.088 points above courseware at equal proficiency.

The base category is essential to the interpretation.

Now you try: underwriting model

Fit the underwriting model and:

  1. interpret the proficiency slope;
  2. interpret both method coefficients;
  3. predict the score for proficiency 100 under each method;
  4. test each coefficient at 5%; and
  5. interpret adjusted R^2 and Significance F.

Refinement and diagnostics

Model reduction is an argument

A defensible reduction process considers:

  • the business question;
  • coefficient evidence and intervals;
  • adjusted R^2 and predictive error;
  • coefficient stability after refitting;
  • hierarchy and important controls; and
  • residual diagnostics.

Do not repeatedly delete the largest p-value without explanation.

Residual diagnostics

Look for:

  • curvature: missing nonlinear structure;
  • funnel shape: non-constant variance;
  • clusters: missing groups;
  • extreme residuals: poor fit for particular observations; and
  • influential points: observations that may strongly affect coefficients.

Watch me: refit and compare

Remove one defensible candidate from the full real-estate model, refit, and compare:

  • remaining coefficient estimates;
  • adjusted R^2;
  • standard error;
  • residual pattern; and
  • clarity of interpretation.

Record the reason for the change before viewing whether fit “improves”.

Now you try: model recommendation

Recommend either the full or reduced model.

Your brief must include:

  1. fitted equation;
  2. two coefficient interpretations;
  3. overall and individual evidence;
  4. adjusted R^2 and residual assessment;
  5. one prediction; and
  6. two limitations.

Common traps

  • Interpreting a partial slope as a simple bivariate association.
  • Forgetting “holding other included predictors constant”.
  • Treating the base category as missing data.
  • Interpreting Significance F as proof every predictor matters.
  • Comparing R^2 across different responses or samples.
  • Deleting variables mechanically by p-value.
  • Claiming causal effects from an observational regression.

Closing check

Can you:

  • build a multiple regression in Excel?
  • interpret numerical and dummy-variable slopes conditionally?
  • test individual coefficients and the overall model?
  • compare models using adjusted R^2, error, and diagnostics?
  • make predictions with correct dummy coding?
  • recommend a model with evidence and limitations?