Skip to content
All projects

Project 08 // Machine learning · 2026

Melbourne Housing Price Prediction

Does the way you test a model change the answer?

Error bars: 13.1% under random and time splits, 18 to 38% when a whole region is held out.
Random-fold error (18–38% on an unseen region)
13.1%
Error under random CV
13.1%
Error on an unseen region
18–38%

In short

The question

How accurate is a house-price model, and does the answer depend on how the model is tested?

What I built

  1. 01
    Melbourne sales CSV
  2. 02
    pandas features
    • pandas
  3. 03
    scikit-learn models
    • scikit-learn
    • Gradient boosting
  4. 04
    Three test designsRandom CV, a time split, and a held-out region
    • Cross-validation design
  5. 05
    Error analysisSales missing BuildingArea, and sales predicted far off
    • Error analysis

What it found

Average error when BuildingArea is…
  • Present12.5%
  • Missing13.8%

7.9% of sales are predicted more than 30% away from the sale price.

  • Gradient boosting was best: 13.1% average error under random CV (R² on log price 0.887), and the same 13.1% when predicting future sales.
  • On a region the model has never seen, error rises to 18–38%, and to 37.9% for Western Metropolitan, nearly three times worse.
  • Random cross-validation answers “how well does this work on more houses like these?”, not “how well does it work somewhere new?”.
  • Python
  • scikit-learn
  • pandas

Full case study // tap a section to open it

01Problem & context

How accurate is a house-price model, and does the answer depend on how the model is tested?

A price model that looks accurate in testing can still fail when it is used somewhere new. Knowing how a model was tested tells you where its accuracy actually holds.

A modeling study built on a Machine Learning course dataset: 13,580 Melbourne home sales from January 2016 to September 2017, with log price as the target.

The same four models are tested three different ways. The point is that the evaluation design, not just the model, decides how much you can trust the accuracy number.

02My role & approach

Extended a Machine Learning course exploration into a full modeling study, from feature preparation to three evaluation designs and an error analysis. I chose the questions and designed the evaluations, and used AI tools to double-check code for edge cases.

  1. 01

    Prepared 13,580 sales (Jan 2016 – Sep 2017) with log price as the target and rooms, type, distance, land and building area, location, sale method, and region as features.

  2. 02

    Compared a median baseline, ridge regression, random forest, and gradient boosting.

  3. 03

    Tested each model three ways: random 5-fold cross-validation, training on past sales to predict the last 25%, and holding out one region at a time.

  4. 04

    Broke down the best model’s error by property type and by missing data.

Problems I hit and how I handled them

  • Handling heavy missing data honestly: BuildingArea is missing for 47.5% of sales and YearBuilt for 39.6%, so the model gets median values plus missing-value flags.
  • Designing three different tests (random folds, past-to-future, and leave-one-region-out) so the results answer different real questions.
03Tech stack
  • pandas

    Feature preparation, log transforms, and median imputation with missing-value flags.

  • scikit-learn

    Ridge regression, random forest, gradient boosting, and the three cross-validation designs.

  • matplotlib

    The comparison charts.

Skills shown

  • scikit-learn
  • Cross-validation design
  • Gradient boosting
  • Missing-data handling
  • Error analysis
  • pandas
04More results
  • 7.9% of sales are predicted more than 30% away from the sale price, and sales missing BuildingArea have higher error (13.8% versus 12.5%).
05What I learned & would improve

Lessons

  • How you split the data decides what “accurate” means: 13.1% error on random folds became 18–38% on a region the model had never seen.
  • Sales missing building area have somewhat higher error (13.8% versus 12.5%); I did not test whether that gap is real.

What I would do next

  • Add location features that transfer between regions, such as distance to transit and schools.
  • Try one model per region, or a hierarchical model that shares strength across regions.

Walkthrough

What each dashboard and diagram shows, and why it matters. Click an image to enlarge it.

  1. 1

    Random folds versus the future

    Training on past sales and predicting the last 25% barely changes the error over this 20-month window.

    • Gradient boosting: 13.1% average error under random 5-fold CV and 13.1% on future sales.
    • The median baseline is off by about 42–44%; ridge regression by about 21–23%.
  2. 2

    A region the model has never seen

    Holding out one region at a time shows where the accuracy breaks.

    • Error rises to 18–38% depending on the region.
    • Western Metropolitan reaches 37.9%, nearly three times the random-CV error.
    • Random CV measures “more houses like these”, not “somewhere new”.
Next projectCan a Model Nowcast Census Vacancy from Mail Data?
Esc

↑ ↓ to moveEnter to open