Sunday, July 16, 2023

Boston Housing dataset ethical/data quality problems

I'm not sure if I've worked with the Boston Housing dataset previously -- maybe? -- but I've at least heard of it, I'm pretty sure. I was surprised to see this message when attempting to import it today as part of working through "TensorFlow 2.0: Quick Start Guide":
ImportError: 
`load_boston` has been removed from scikit-learn since version 1.2.

The Boston housing prices dataset has an ethical problem: as
investigated in [1], the authors of this dataset engineered a
non-invertible variable "B" assuming that racial self-segregation had a
positive impact on house prices [2]. Furthermore the goal of the
research that led to the creation of this dataset was to study the
impact of air quality but it did not give adequate demonstration of the
validity of this assumption.

The scikit-learn maintainers therefore strongly discourage the use of
this dataset unless the purpose of the code is to study and educate
about ethical issues in data science and machine learning.

In this special case, you can fetch the dataset from the original
source::

    import pandas as pd
    import numpy as np

    data_url = "http://lib.stat.cmu.edu/datasets/boston"
    raw_df = pd.read_csv(data_url, sep="\s+", skiprows=22, header=None)
    data = np.hstack([raw_df.values[::2, :], raw_df.values[1::2, :2]])
    target = raw_df.values[1::2, 2]

Alternative datasets include the California housing dataset and the
Ames housing dataset. You can load the datasets as follows::

    from sklearn.datasets import fetch_california_housing
    housing = fetch_california_housing()

for the California housing dataset and::

    from sklearn.datasets import fetch_openml
    housing = fetch_openml(name="house_prices", as_frame=True)

for the Ames housing dataset.

[1] M Carlisle.
"Racist data destruction?"


[2] Harrison Jr, David, and Daniel L. Rubinfeld.
"Hedonic housing prices and the demand for clean air."
Journal of environmental economics and management 5.1 (1978): 81-102.

It was interesting reading M Carlisle's investigation and the scikit-learn discussion around it. I also found Fairlearn's discussion enlightening. (I One other surprise for me in this is the existence of an Ames, Iowa, housing dataset. Article describing it: Ames, Iowa: Alternative to the Boston Housing Data as an End of Semester Regression Project [PDF] (2011) by the original creator of the dataset, Dean DeCock.