{
"cells": [
{
"cell_type": "markdown",
"id": "53826d67-3b91-4bf2-bd5d-1a8654d0cbf7",
"metadata": {},
"source": [
"# House Prices\n",
"\n",
"This guide runs through building, training and evaluation of models for estimating house prices in California. The models in this example are using quad pooling operations. In the guide we will be using [California Housing](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_california_housing.html) dataset. The dataset is comprised of 20640 data points with 8 features each, including geographical coordinates."
]
},
{
"cell_type": "markdown",
"id": "71601828-2e25-4da8-9f0d-c37a468b59da",
"metadata": {},
"source": [
"\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "33030ca2-dad2-48ce-b08c-9c963afabfa5",
"metadata": {},
"outputs": [],
"source": [
"!pip install torch-geopooling matplotlib osmnx geopandas torchmetrics scikit-learn -qqq"
]
},
{
"cell_type": "markdown",
"id": "5b9981b3-c6b3-4915-800a-006c49eb6ac0",
"metadata": {},
"source": [
"## Defining the Dataset\n",
"\n",
"In order to use the data for training a neural network, we need to define a dataset class, that should inherit from the standard [torch.utils.data.Dataset](https://pytorch.org/docs/stable/data.html#torch.utils.data.Dataset), and implement `__len__` and `__getitem__`.\n",
"\n",
"Since the dataset is relatively small, we could simply download it into memory and then parse CSV file to turn it into the Pandas `DataFrame`."
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "6b0964c3-c7c1-4b4d-93fe-84bb324abd8d",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" MedInc \n",
" HouseAge \n",
" AveRooms \n",
" AveBedrms \n",
" Population \n",
" AveOccup \n",
" Latitude \n",
" Longitude \n",
" MedHouseVal \n",
" \n",
" \n",
" \n",
" \n",
" 0 \n",
" 8.3252 \n",
" 41.0 \n",
" 6.984127 \n",
" 1.023810 \n",
" 322.0 \n",
" 2.555556 \n",
" 37.88 \n",
" -122.23 \n",
" 4.526 \n",
" \n",
" \n",
" 1 \n",
" 8.3014 \n",
" 21.0 \n",
" 6.238137 \n",
" 0.971880 \n",
" 2401.0 \n",
" 2.109842 \n",
" 37.86 \n",
" -122.22 \n",
" 3.585 \n",
" \n",
" \n",
" 2 \n",
" 7.2574 \n",
" 52.0 \n",
" 8.288136 \n",
" 1.073446 \n",
" 496.0 \n",
" 2.802260 \n",
" 37.85 \n",
" -122.24 \n",
" 3.521 \n",
" \n",
" \n",
" 3 \n",
" 5.6431 \n",
" 52.0 \n",
" 5.817352 \n",
" 1.073059 \n",
" 558.0 \n",
" 2.547945 \n",
" 37.85 \n",
" -122.25 \n",
" 3.413 \n",
" \n",
" \n",
" 4 \n",
" 3.8462 \n",
" 52.0 \n",
" 6.281853 \n",
" 1.081081 \n",
" 565.0 \n",
" 2.181467 \n",
" 37.85 \n",
" -122.25 \n",
" 3.422 \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from matplotlib import pyplot as plot\n",
"from matplotlib import colors\n",
"from matplotlib import cm\n",
"from shapely import Polygon, LineString\n",
"\n",
"\n",
"def render_coastlines(ax):\n",
" for coastline in coastlines.itertuples():\n",
" if isinstance(coastline.geometry, Polygon):\n",
" ax.plot(*coastline.geometry.exterior.xy, color=\"black\", lw=1)\n",
" if isinstance(coastline.geometry, LineString):\n",
" ax.plot(*coastline.geometry.xy, color=\"black\", lw=1)\n",
"\n",
"fig, ax = plot.subplots(ncols=2)\n",
"fig.set_dpi(300)\n",
"fig.set_size_inches(15, 8)\n",
"\n",
"\n",
"cmap = plot.get_cmap(\"RdBu_r\")\n",
"norm = colors.Normalize(\n",
" vmin=dataset[\"MedHouseVal\"].min(),\n",
" vmax=dataset[\"MedHouseVal\"].max(),\n",
")\n",
"\n",
"ax[0].set_xlabel(\"Longitude, deg\")\n",
"ax[0].set_ylabel(\"Latitude, deg\")\n",
"ax[0].scatter(dataset[\"Longitude\"], dataset[\"Latitude\"], s=1, color=\"blue\", alpha=0.1)\n",
"ax[0].set_title(\"Locations of California houses\")\n",
"\n",
"ax[1].set_xlabel(\"Longitude, deg\")\n",
"ax[1].set_ylabel(\"Latitude, deg\")\n",
"ax[1].scatter(\n",
" dataset[\"Longitude\"], dataset[\"Latitude\"],\n",
" s=20, color=cmap(norm(dataset[\"MedHouseVal\"])),\n",
" alpha=0.1\n",
")\n",
"ax[1].set_title(\"Prices of houses in California\")\n",
"\n",
"fig.colorbar(cm.ScalarMappable(norm=norm, cmap=cmap), ax=ax[1], label=\"House Value, $\")\n",
"\n",
"render_coastlines(ax[0])\n",
"render_coastlines(ax[1])"
]
},
{
"cell_type": "markdown",
"id": "1bdf3438-e771-4938-89e5-03da301dc4e1",
"metadata": {},
"source": [
"On the next step we need to split dataset into training and test sub-sets, in this example we will use 80% of data for training and 20% of data for testing (evaluation) of a model."
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "bf9491c3-6571-4406-a179-8908c05654fe",
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import torch\n",
"from torch.utils.data import Dataset, random_split\n",
"\n",
"\n",
"class HousePricesDataset(Dataset):\n",
" def __init__(self, df):\n",
" self.df = df\n",
" self.max_value = df[\"MedHouseVal\"].max()\n",
"\n",
" def __len__(self):\n",
" return len(self.df)\n",
"\n",
" def __getitem__(self, idx):\n",
" # Take necessary row from the dataset and convert it to torch.Tensor.\n",
" x = self.df.iloc[idx][[\"Longitude\", \"Latitude\"]].to_numpy(dtype=np.float64)\n",
" y = self.df.iloc[idx][[\"MedHouseVal\"]].to_numpy(dtype=np.float64)# / self.max_value\n",
" return torch.DoubleTensor(x), torch.DoubleTensor(y)\n",
"\n",
"\n",
"train_size = int(len(dataset) * 0.8)\n",
"test_size = len(dataset) - train_size\n",
"\n",
"houses_prices_dataset = HousePricesDataset(dataset)\n",
"train_set, test_set = random_split(houses_prices_dataset, [train_size, test_size])"
]
},
{
"cell_type": "markdown",
"id": "a925901f-5012-44c0-8bcc-9d0670d8ae35",
"metadata": {},
"source": [
"## Defining Models\n",
"\n",
"The model we are going to use will be compare regular quad pooling layer and adaptive quad pooling layer. Essentially, those modules build [quadtree](https://en.wikipedia.org/wiki/Quadtree) for spatial decomposition of the points. We assume that houses within similar geographical coordinates cost similar money.\n",
"\n",
"Adaptive module allows to dynamically shape the decomposition based on density of points: regions with higher density are finer grained than regions with fewer points. Regular pooling module on the other hand, tiles the target polygon into quads of the same size.\n",
"\n",
"Below we define two models `QuadNN` and `AdaptiveQuadNN` that use quad pooling operations: regular and adaptive."
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "076c613d-34c2-40d6-9535-c6ba9434c10e",
"metadata": {},
"outputs": [],
"source": [
"from torch import nn\n",
"from torch_geopooling.nn import AdaptiveQuadPool2d\n",
"from torch_geopooling.nn import QuadPool2d\n",
"\n",
"\n",
"class QuadNN(nn.Module):\n",
"\n",
" def __init__(self, bbox):\n",
" super().__init__()\n",
" n, s, e, w = bbox\n",
" self.pool = QuadPool2d(\n",
" feature_dim=1,\n",
" polygon=Polygon([(w, s), (e, s), (e, n), (w, n)]),\n",
" exterior=(-180, -90, 360, 180),\n",
" max_depth=10\n",
" )\n",
" self.linear = nn.Linear(1, 1, dtype=torch.float64)\n",
"\n",
" def forward(self, x):\n",
" x = torch.clamp(self.pool(x), 0.0)\n",
" x = self.linear(x)\n",
" return x\n",
"\n",
"\n",
"class AdaptiveQuadNN(nn.Module):\n",
"\n",
" def __init__(self):\n",
" super().__init__()\n",
" self.pool = AdaptiveQuadPool2d(\n",
" feature_dim=1,\n",
" exterior=(-180, -90, 360, 180),\n",
" capacity=10,\n",
" max_depth=12\n",
" )\n",
" self.linear = nn.Linear(1, 1, dtype=torch.float64)\n",
"\n",
" def forward(self, x):\n",
" x = self.pool(x)\n",
" x = self.linear(x)\n",
" return x"
]
},
{
"cell_type": "markdown",
"id": "5f931d6b-9f94-4a24-a82a-873081b5af33",
"metadata": {},
"source": [
"## Training Models\n",
"\n",
"Further we use a standard training loop and Stochastic Gradient Descent optimization algorithm to train models. Note, that adaptive pooling operations are using sparse weight tensor and therefore only a limited set of optimization algorithms could be used for training."
]
},
{
"cell_type": "code",
"execution_count": 7,
"id": "e84a5472-22b9-4269-8d09-603d6b474ff1",
"metadata": {},
"outputs": [],
"source": [
"from torch.utils.data import DataLoader\n",
"from torch.optim import SGD\n",
"\n",
"\n",
"def train(model, lr=0.1, epochs=10):\n",
" loss_fn = nn.HuberLoss()\n",
" optimizer = SGD(model.parameters(), lr=lr)\n",
"\n",
" for epoch in range(epochs):\n",
" for batch in DataLoader(train_set, batch_size=2048):\n",
" x, y = batch\n",
" optimizer.zero_grad()\n",
" y_pred = model(x)\n",
" \n",
" loss = loss_fn(y, y_pred)\n",
" loss.backward()\n",
" optimizer.step()\n",
" return model"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "b13858bf-78f4-4a48-a9d1-8c53f756d3b3",
"metadata": {
"scrolled": true
},
"outputs": [],
"source": [
"qnn = train(QuadNN(bounding_box), lr=1.0, epochs=50)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "6fcc4c0c-6f69-4aad-bfd2-ce6f8f982fa0",
"metadata": {},
"outputs": [],
"source": [
"aqnn = train(AdaptiveQuadNN(), lr=1.0, epochs=50)"
]
},
{
"cell_type": "markdown",
"id": "af95d213-5dd9-41fd-b767-0f2119e00de2",
"metadata": {},
"source": [
"## Evaluating Models\n",
"\n",
"On the final step, we evaluate models' performance and compute mean absolute error and weighted mean absolute percentage error. As you could see the from the table below, both models require improvements.\n",
"\n",
"Additionally, `torch_geopooling` library provides `TileWKT` transformer that allows to inspect the model and plot quads that where learned by the model. Below are presented learned decomposition for regular and adaptive pooling operations."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "ee39c2c9-473d-4fd5-b47c-36a127ed90b2",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" QuadNN \n",
" AdaptiveQuanNN \n",
" \n",
" \n",
" \n",
" \n",
" MAE \n",
" 0.62189245 \n",
" 0.8252109 \n",
" \n",
" \n",
" wMAPE \n",
" 0.30851611 \n",
" 0.5629974 \n",
" \n",
" \n",
"
\n",
"
"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"from torch_geopooling.transforms import TileWKT\n",
"from shapely import from_wkt\n",
"\n",
"\n",
"def render_tiles(ax, model, tiles, bbox):\n",
" centers = []\n",
" exterior = (-180, -90, 360, 180)\n",
" transformer = TileWKT(exterior=exterior, internal=False)\n",
"\n",
" # Calculate centers of terminal nodes in a quadtree and then use these\n",
" # coordinates to compute associated prices for each quad in the final tree.\n",
" for tile in transformer(tiles):\n",
" poly = from_wkt(tile)\n",
" center = poly.representative_point()\n",
" centers.append((center.x, center.y))\n",
"\n",
" with torch.no_grad():\n",
" values = model(torch.DoubleTensor(centers)).detach().numpy()\n",
"\n",
" # Use different colors to represent housing prices: warm colors of the\n",
" # spectre are associated with higher prices, while cooler colors of the\n",
" # spectre depic lower prices.\n",
" cmap = plot.get_cmap(\"RdBu_r\")\n",
" norm = colors.Normalize(vmin=np.min(values), vmax=np.max(values))\n",
"\n",
" for i, tile in enumerate(transformer(tiles)):\n",
" poly = from_wkt(tile)\n",
" ax.fill(*poly.exterior.xy, color=cmap(norm(values[i])), lw=0.5, edgecolor=\"black\", alpha=0.8)\n",
"\n",
" n, s, e, w = bbox\n",
" ax.set_xlim(w, e)\n",
" ax.set_ylim(s, n)\n",
" ax.set_xlabel(\"Longitude, deg\")\n",
" ax.set_ylabel(\"Latitude, deg\")\n",
"\n",
"\n",
"fig, ax = plot.subplots(ncols=2)\n",
"fig.set_dpi(300)\n",
"fig.set_size_inches(15, 8)\n",
"\n",
"ax[0].set_title(\"Decomposition of QuadPool2d module\")\n",
"ax[1].set_title(\"Decomposition of AdaptiveQuadPool2d module\")\n",
"\n",
"\n",
"render_tiles(ax[0], qnn, qnn.pool.tiles, bounding_box)\n",
"render_tiles(ax[1], aqnn, aqnn.pool.weight.coalesce().indices().t()[:,:-1], bounding_box)\n",
"\n",
"render_coastlines(ax[0])\n",
"render_coastlines(ax[1])"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.0"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
\n",
"