Report

STAT 8561 | Fall 2026
Due on December 13, 2026
Course-grade weight: 20% (see Grade Distribution)
Points: 100
MS students: 1-4 students per group
PhD students: Complete the project individually

Use a real dataset to explain an outcome with multiple linear regression, and compare its predictive performance with one machine learning regression algorithm of your choice. Begin with the project overview.

Research question and data

Choose a continuous response and at least two scientifically meaningful candidate predictors. State what one row represents, the response’s units, the population or setting of interest, and the question your analysis will answer. Choose a dataset large enough to support the proposed models and a meaningful evaluation on held-out observations.

  • Link to and cite the original data source. Describe how the observations were collected, the sample size, and the variable definitions.
  • Explain exclusions, missing values, unusual observations, and any transformations. Give the number of observations retained for analysis.
  • Distinguish variables available when a prediction would be made from information recorded afterward. Use predictors that would actually be available in the intended application.
  • Explain whether observations are independent. If rows share a subject, site, or time sequence, use a splitting strategy that respects that structure and state the implications for inference.

Before choosing models, write one question about association or explanation and one about prediction. For example, an adjusted association between size and price is a different question from predicting the price of a new property.

Linear regression analysis

Use the ideas from least squares estimation, OLS inference, model comparison, and multiple regression and categorical predictors.

  1. Specify the model. Write the mean model, define its variables and units, and explain why the predictors belong in the analysis. State reference categories and justify any interactions or transformed terms.
  2. Explain the coefficients. Interpret at least two scientifically relevant coefficients or contrasts in context, stating the predictor change, response units, and other variables held fixed. Explain the intercept’s baseline and whether it is meaningful for the observed data. With an interaction, make clear which coefficient interpretation depends on the other variable’s value.
  3. Report uncertainty. Give estimates and 95% confidence intervals for the coefficients or contrasts you interpret. State the assumptions supporting the inference. Distinguish an interval for a model parameter from a prediction interval for an individual outcome. Do not equate a small p-value with a large practical effect.
  4. Compare nested models. Compare one reduced and one full linear model on the same training observations and response scale. State the restriction, such as an added coefficient or group of coefficients being zero. Use an appropriate partial F test when its assumptions are defensible; otherwise explain the limitation of that test. Explain whether a retained coefficient’s meaning or estimate changes when the adjustment set changes.
  5. Check the model. Use residual-versus-fitted and normal Q-Q plots, and examine influential observations. Discuss nonlinearity, unequal variance, unusual observations, and the plausibility of independence. Justify any remedial action; do not remove an observation solely to improve a result. See observation diagnostics and checking the fitted functional form.

Use the training data for these modeling decisions. If you select terms or transformations after examining the data, report that process and treat ordinary confidence intervals and p-values as exploratory: standard fixed-model formulas do not account for data-driven selection. Treat findings from observational data as associations unless the study design justifies a causal interpretation.

Machine learning comparison

Choose one method suited to predicting a continuous response, for example a regression tree, random forest, gradient boosting, or support vector regression. Explain how it produces predictions, why it is appropriate for your data, and which important tuning parameters you choose. Cite the method and software documentation.

Existing implementations are appropriate. Identify the functions, software versions, preprocessing, and settings needed to reproduce the fit. If you report a feature-importance measure or an effect plot, explain what it measures; it is not automatically an adjusted regression coefficient or a causal effect.

A fair evaluation

  1. Reserve a test set first. State the split and its rationale. For independent observations, an 80% training / 20% test split is a reasonable starting point. Use a group-aware or chronological split when appropriate. Conduct model-building exploration on the training data.
  2. Use the same comparison. Both methods predict the same response for the same test observations and use the same available candidate predictors. Explain any model-specific encoding, transformations, or training-based variable selection.
  3. Keep learning within training data. Estimate imputation, scaling, encoding rules, and other learned preprocessing using training data. If you use cross-validation to select a model or tune parameters, repeat all learned preprocessing inside each training fold. Apply the fitted transformations to validation or test observations.
  4. Tune without the test outcomes. Choose linear-model terms and machine learning settings using training data, with cross-validation or a separate validation subset as appropriate. Report the candidate settings, validation design, and selection metric. Finalize the models before evaluating them on the test set; do not revise them after seeing test performance.
  5. Report the comparison. Evaluate a training-mean baseline, the final linear regression, and the machine learning model on the same test set. Report RMSE and MAE in the original response units. Explain how any response transformation is reversed and how any correction is estimated using training data.

For \(m\) test observations, calculate

\[ \operatorname{RMSE}=\sqrt{\frac{1}{m}\sum_{i=1}^{m}(y_i-\hat y_i)^2}, \qquad \operatorname{MAE}=\frac{1}{m}\sum_{i=1}^{m}|y_i-\hat y_i|. \]

The baseline predicts the training-set mean for every test observation. Smaller RMSE and MAE indicate smaller prediction errors. Training \(R^2\) alone does not establish which model predicts new observations better. See model assessment and prediction and the tidymodels resampling guide.

Use this format for the main results table; fill it with your own results:

Model Test RMSE Test MAE
Training-mean baseline Your result Your result
Multiple linear regression Your result Your result
Chosen machine learning method Your result Your result

Include an observed-versus-predicted plot for both fitted models and discuss where errors are largest. Differences on one test set are estimates, not proof of universal superiority; discuss how the sample size, split, and application limit the comparison.

For implementation guidance, consult the scikit-learn guide to avoiding data leakage and the official ensemble-methods documentation when relevant to your chosen method.

Explain the findings

Your conclusion should answer the research question in ordinary language and address all of the following:

  • Substantive findings: Which adjusted relationships are supported, how large are they in meaningful units, and how uncertain are they?
  • Model comparison: Did the machine learning method improve test prediction, and is the difference useful in this application? Explain why a flexible model might or might not improve performance.
  • Interpretation: What does the linear model make easy to explain? What additional patterns, if any, does the machine learning analysis suggest? Distinguish coefficient estimates from prediction-based importance measures.
  • Recommendation and limitations: Which model would you use for the stated purpose, and why? Discuss assumptions, data quality, the scope of generalization, and one useful next step.

Report format and reproducibility

Write 8-10 pages of main text, counted using the rule below. Use Quarto or R Markdown to produce a PDF with readable 11- or 12-point text, standard margins, and numbered sections. For a Python or Jupyter Notebook workflow, follow the prior-discussion requirement in the submission note.

NotePage count: 8-10 pages of main text

The finished report must contain at least 8 and at most 10 pages of main text.

  • Count: the abstract, Sections 1-6 (Introduction through Discussion and conclusion), and all figures, tables, captions, equations, and code examples within those sections.
  • Do not count: a separate title-only cover page, References, Author contributions, and appendices or supplementary material. Put these end sections after the main text; start References on a new page.
  • A first page containing the title, author names, and abstract counts as a main-text page. Only a separate page containing the title and author information alone is excluded; a separate cover page is optional.

The 9-page plan below is a suggested allocation, not a different requirement. The short template PDF shows headings and prompts; its number of pages is not the required report length. The full submitted PDF may exceed 10 pages because the excluded material comes afterward.

Writing model: The American Statistician

Write the report in the style of an applied article in The American Statistician (TAS): introduce a statistical question, explain the analysis to a statistically literate reader, and use evidence to support a clear conclusion. Connect estimates, uncertainty, diagnostic plots, and prediction results to the application. Organize the main text as a coherent article; put lengthy software output and implementation details in supporting material.

The structure below is a course adaptation of this publication style. Use it with this project’s 8-10 page limit and existing rubric. A journal submission or an exact reproduction of the publisher’s typesetting is not required. Keep the course’s PDF, source-file, font-size, and margin requirements above.

Read these references before drafting:

Suggested report structure

Use a specific title and list all authors and the course. A useful starting point is a 150-200 word abstract followed by 3-5 keywords. These are course writing suggestions. The abstract should state the question, data, methods, main quantitative finding, and conclusion; write it after completing the analysis.

The following is an illustrative 9-page plan, including the abstract and main-text figures and tables. Adjust the allocation within the required 8-10 counted pages; apply the page-count rule above.

Section What the reader should learn Pages
Abstract The question, approach, main result, and takeaway. 0.5
1. Introduction Why the application matters; the association and prediction questions; relevant background. 1
2. Data and study design Source, units, cleaning, training-data exploration, and the train/test design. Include a motivating plot. 1.5
3. Linear regression analysis Model and assumptions; interpreted estimates and confidence intervals; nested-model comparison; diagnostics and limitations. 2
4. Machine learning method One chosen method, its rationale, preprocessing, tuning design, and final settings. 1
5. Results and predictive comparison Baseline, regression, and ML test RMSE/MAE; observed-versus-predicted plots; practical meaning of the differences. 2
6. Discussion and conclusion Answer the research question, recommend a model for the stated purpose, and explain uncertainty and limitations. 1

After the main text, include References, an Author contributions statement, and any Supplementary material. Cite data, methods, software, and adapted code. Use consistent author-year citations and complete references, with DOI or source links where available.

Number figures and tables, give informative captions with units and a legend where needed, and discuss each in the text. Present selected results as readable tables and plots. Keep full code, secondary diagnostics, and long output in the appendix or supporting files; include everything needed to reproduce the reported numbers.

Preview and download the template

Expand the three-page PDF preview below, open a larger view, or download the PDF. This PDF is rendered directly from the downloadable template below and shows the layout, section headings, and writing prompts. It is a short outline, not an example of the required length; your completed report must contain 8-10 counted pages of main text under the page-count rule.

You may refer to published articles in The American Statistician and the examples linked above for statistical exposition, figures, tables, and interpretation. Use them as models for organizing and presenting your own work.

Open PDF preview (3 pages)Close PDF preview

Use the page and zoom controls to read the template. Click the heading again to collapse the preview.

Download the STAT 8561 report starter (ZIP). Unzip it to obtain Project_Report.qmd and README.md. The Quarto file contains the section headings and writing prompts above. Replace every prompt with your own analysis, add executable code and citations, and render Project_Report.pdf. The starter follows the course’s TAS-inspired structure and submission format.

Set the seed to 8561 for randomized splitting, resampling, and model fitting; in Python, set the relevant random_state or generator seed to 8561. Save the split identifiers and record the software environment. Prefer ggplot2 for R figures. Use relative paths and provide instructions that work from a fresh session.

Assessment criteria

The project totals 100 points and is worth 20% of the course grade (Grade Distribution). Each weight below is a percentage of the project grade: 25% equals 25 points. A score of 90/100 contributes 18 percentage points to the course grade.

Criterion Weight Full-credit evidence (points)
Question and data 10% Clear question, observational unit, and outcome units (3); cited source and variable definitions (3); justified cleaning, missing-data handling, and dependence assessment (4).
Linear model and interpretation 25% Correct mean model and predictor coding (5); correct interpretations of at least two coefficients or contrasts and the intercept, including units and adjustment (12); correct 95% confidence intervals and uncertainty discussion (8).
Model comparison and diagnostics 20% Valid nested-model restriction and test, with assumptions addressed (6); explanation of retained coefficients across models (4); informative residual, Q-Q, and influence checks with justified responses to problems (10).
Machine learning method 10% Appropriate regression method and accurate explanation (4); documented implementation, preprocessing, and final settings (6).
Fair predictive evaluation 15% Appropriate common test set and no data leakage (6); documented training-only validation and tuning (4); correct baseline, test RMSE/MAE, and prediction plots for both models (5).
Findings and critical discussion 10% Contextual conclusions about effect size and uncertainty (4); reasoned prediction comparison and model recommendation (3); meaningful limitations and next step (3).
Communication and reproducibility 10% Organized report and readable, discussed figures and tables (3); runnable source that reproduces results with documented data, seed, and environment (5); citations and each member’s contribution (2).
Total 100% 100 points

Subitem points appear in parentheses. Partial credit reflects accuracy, completeness, and justification; omissions earn zero. Full credit does not require machine learning to outperform linear regression.

Submission

NoteReport submission checklist

MS students work in groups of 1-4 students. PhD students must complete the project individually. Upload the following to the iCollege project submission area by December 13, 2026. The submission folder will be announced on iCollege:

  • Project_Report.pdf.
  • Its matching Project_Report.qmd or Project_Report.Rmd source (the default format; see the Python/Jupyter Notebook note below).

If the source depends on supporting files, include them with the source in a ZIP. Provide any required scripts, data that can be redistributed or precise data-access instructions, split identifiers, and software versions and rendering instructions.

Put all author names and STAT 8561 on the report. The PDF must be generated from the submitted source file; HTML alone does not satisfy the submission requirement. Check that the submitted files reproduce the figures, tables, and model-comparison results from beginning to end.

NotePython and Jupyter Notebook submissions

If you would like to use Python or submit a Jupyter Notebook (.ipynb), discuss your plan with the instructor in advance and agree on the submission format. Python/Jupyter Notebook submissions are subject to that discussion; they are not an automatic alternative to the default .qmd/.Rmd requirement.

Any agreed alternative must still include a PDF report, the reproducible source (including the notebook if applicable), and the supporting files needed to regenerate the analysis. The same report requirements and grading criteria apply.

Related: Project Instruction.