ETX1100/ETX5900 Business Statistics

Week 5: Relationships and simple regression

5 January 2027

Today’s journey

  1. Revisit relationships between categorical variables.
  2. Visualise and quantify numerical relationships.
  3. Distinguish association from causation.
  4. Build and interpret a simple linear regression in Excel.
  5. Predict responsibly and diagnose limitations.

Retrieval check

  • How would you assess association between two categorical variables?
  • What does independence mean in probability language?
  • What does a small p-value say—and not say?
  • Which variable types can be placed on a scatterplot?

Match the method to the variables

Explanatory variable Outcome variable Useful tools
categorical categorical contingency table, conditional probability
numerical numerical scatterplot, covariance, correlation, regression
categorical numerical group summaries, boxplots

Method selection begins with the business question and variable types.

Categorical association

Two categorical variables are associated when the outcome distribution changes across explanatory-variable groups.

Compare conditional probabilities such as:

P(\text{tried}\mid\text{high income})

with the corresponding probabilities in other income groups or with the marginal probability.

Watch me: categorical relationship

Using a contingency table, ask:

  1. What is the outcome?
  2. What groups are being compared?
  3. Are row or column percentages appropriate?
  4. Do conditional probabilities differ meaningfully?
  5. Is causal language justified?

Now you try: explain the association

A loyalty-app sample finds 72% high satisfaction among users and 61% among non-users.

  1. State the observed association.
  2. Explain why the two percentages are conditional.
  3. Give two plausible confounders.
  4. Rewrite “the app increases satisfaction” as a defensible conclusion.

Numerical relationships

Scatterplots first

A scatterplot reveals:

  • direction: positive or negative;
  • form: linear or nonlinear;
  • strength: tight or diffuse pattern; and
  • unusual observations: outliers or influential points.

Put the explanatory variable X on the horizontal axis and response Y on the vertical axis.

Covariance

Sample covariance measures joint movement:

s_{XY}=\frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{n-1}.

  • Positive: variables tend to move in the same direction.
  • Negative: variables tend to move in opposite directions.
  • Magnitude depends on measurement units, limiting comparison.

Correlation

Correlation standardises covariance:

r=\frac{s_{XY}}{s_Xs_Y},\qquad -1\le r\le1.

  • Sign gives direction.
  • Magnitude gives strength of linear association.
  • Correlation is unit-free and symmetric in X and Y.

Excel: =CORREL(x_range,y_range).

Watch me: income and spending

Open week5-regression-data.xlsx, sheet IncomeSpend.

  1. Insert a scatterplot of Spend against Income.
  2. Label axes with units.
  3. Add a linear trendline.
  4. Calculate =CORREL(Income,Spend).
  5. Describe direction, form, strength, and unusual points.

Now you try: age and market price

On HouseSize:

  1. plot market price against house age;
  2. add a linear trendline;
  3. calculate correlation;
  4. interpret its sign and magnitude; and
  5. identify whether the plot supports a linear summary.

Correlation cautions

  • r\approx0 does not rule out a nonlinear relationship.
  • One influential observation can alter r greatly.
  • Restricted ranges can weaken an observed correlation.
  • Combining groups can create a misleading aggregate pattern.
  • Correlation does not establish causation.

Now you try: correlation critique

For each statement, identify the problem:

  1. “r=0, so the variables are unrelated.”
  2. “Sales and advertising are correlated, so advertising caused every sale.”
  3. “The correlation is 0.8 kilograms.”
  4. “Removing the unusual store is fine because it improves r.”

Simple linear regression

Population and fitted models

Population model:

Y_i=\beta_0+\beta_1X_i+\varepsilon_i.

Fitted sample model:

\hat Y_i=\hat\beta_0+\hat\beta_1X_i.

Residual:

e_i=Y_i-\hat Y_i.

Least squares

Least squares chooses the line that minimises:

SSE=\sum_{i=1}^{n}e_i^2.

Squaring prevents positive and negative residuals cancelling and penalises larger errors more strongly.

The fitted line describes the mean response predicted at each X.

Interpreting coefficients

  • Intercept \hat\beta_0: predicted Y when X=0.
  • Slope \hat\beta_1: predicted mean change in Y for a one-unit increase in X.

Include both variables, units, direction, and “on average”. Question whether X=0 is observed or meaningful.

Watch me: regression in Excel

For Spend as Y and Income as X:

  1. Data → Data Analysis → Regression;
  2. select the labelled Y and X ranges;
  3. tick Labels and request residual output;
  4. locate coefficients, R^2, standard errors, and p-values; and
  5. write the fitted equation.

Now you try: interpret IncomeSpend

From the Excel output:

  1. report and interpret the intercept;
  2. report and interpret the slope;
  3. predict the spending change for an AUD 10,000 income increase;
  4. state whether the intercept is practically meaningful; and
  5. compare the slope sign with the correlation sign.

Coefficient test

To test for a linear relationship:

H_0:\beta_1=0,\qquad H_1:\beta_1\ne0.

Excel reports the two-sided p-value for the slope. A small p-value supports a population linear association, subject to model assumptions and study design.

Coefficient of determination

R^2=\frac{\text{variation explained by the model}}{\text{total variation in }Y}.

Interpret R^2 as the percentage of sample variation in the response explained by its linear relationship with X.

It is not the percentage of observations predicted correctly.

Watch me: predict from the line

For the fitted model

\widehat{Spend}=8254.575+0.296(Income),

substitute an income within the observed range. Keep units consistent and distinguish:

  • predicted mean spending; and
  • an individual’s actual spending, which also contains residual variation.

Now you try: HouseSize regression

Regress market price (AUD thousands) on house age.

  1. Write and interpret the fitted equation.
  2. Interpret the slope in dollars per year.
  3. Predict the change associated with five additional years of age.
  4. Interpret R^2 and the slope p-value.
  5. Compare fitted and observed values for one house.

Validation and responsible prediction

Residual plots

A useful linear model has residuals that are:

  • centred around zero;
  • randomly scattered rather than curved;
  • roughly constant in vertical spread; and
  • not dominated by a few influential cases.

A pattern in residuals is information the model failed to capture.

Watch me: inspect residuals

Create predicted values and residuals for the age model.

  1. Plot residuals against fitted values.
  2. Look for curvature or a funnel shape.
  3. Identify unusually large residuals.
  4. Return to the original data before deciding whether any point is erroneous.

Now you try: model audit

Audit the IncomeSpend model:

  1. inspect its residual plot;
  2. identify one strength and one limitation;
  3. assess whether the line is suitable for prediction;
  4. name one omitted variable; and
  5. state whether a causal interpretation is justified.

Extrapolation

Prediction outside the observed X range is extrapolation.

  • The relationship may change beyond the data.
  • A mathematically valid calculation can be substantively implausible.
  • Report whether a requested X is inside the sample range.
  • Prefer collecting relevant data to extending the line blindly.

Now you try: prediction guardrails

Before predicting spending for a household income of AUD 500,000:

  1. check the observed income range;
  2. label the request interpolation or extrapolation;
  3. explain the main risk;
  4. recommend an alternative if the request is unsupported.

Integrated application

Watch me: a defensible regression brief

A concise brief contains:

  1. business question and variable roles;
  2. scatterplot and correlation;
  3. fitted equation with coefficient interpretations;
  4. R^2, slope test, and prediction;
  5. residual and range checks; and
  6. a recommendation with limitations.

Now you try: choose the better predictor

Using HouseSize, compare separate simple regressions of market price on:

  • house age; and
  • house size.

Recommend the more useful single predictor using plots, correlation, R^2, slope evidence, residuals, and business reasoning. Explain why this is not yet a multiple-regression conclusion.

Common traps

  • Reversing X and Y.
  • Reporting correlation without examining the scatterplot.
  • Interpreting the intercept when X=0 is meaningless.
  • Omitting units or “on average” from a slope interpretation.
  • Claiming R^2 measures causality or accuracy.
  • Extrapolating without checking the observed range.
  • Treating association as causal evidence.

Closing check

Can you:

  • select tools for categorical and numerical relationships?
  • interpret a scatterplot and correlation together?
  • build a simple regression in Excel?
  • interpret intercept, slope, R^2, and slope p-value?
  • make an in-range prediction and calculate a residual?
  • communicate model limitations and avoid causal overreach?