Linear Regression using DNN

Linear regression of California housing dataset using deep neural networks
keras
teserflow
AI
python
Author

Aakash Basnet

Published

March 4, 2024

Linear regression of California housing dataset using deep neural networks

from sklearn.datasets import fetch_california_housing
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

housing = fetch_california_housing()
housing

{'data': array([[   8.3252    ,   41.        ,    6.98412698, ...,    2.55555556,
           37.88      , -122.23      ],
        [   8.3014    ,   21.        ,    6.23813708, ...,    2.10984183,
           37.86      , -122.22      ],
        [   7.2574    ,   52.        ,    8.28813559, ...,    2.80225989,
           37.85      , -122.24      ],
        ...,
        [   1.7       ,   17.        ,    5.20554273, ...,    2.3256351 ,
           39.43      , -121.22      ],
        [   1.8672    ,   18.        ,    5.32951289, ...,    2.12320917,
           39.43      , -121.32      ],
        [   2.3886    ,   16.        ,    5.25471698, ...,    2.61698113,
           39.37      , -121.24      ]]),
 'target': array([4.526, 3.585, 3.521, ..., 0.923, 0.847, 0.894]),
 'frame': None,
 'target_names': ['MedHouseVal'],
 'feature_names': ['MedInc',
  'HouseAge',
  'AveRooms',
  'AveBedrms',
  'Population',
  'AveOccup',
  'Latitude',
  'Longitude'],
 'DESCR': '.. _california_housing_dataset:\n\nCalifornia Housing dataset\n--------------------------\n\n**Data Set Characteristics:**\n\n:Number of Instances: 20640\n\n:Number of Attributes: 8 numeric, predictive attributes and the target\n\n:Attribute Information:\n    - MedInc        median income in block group\n    - HouseAge      median house age in block group\n    - AveRooms      average number of rooms per household\n    - AveBedrms     average number of bedrooms per household\n    - Population    block group population\n    - AveOccup      average number of household members\n    - Latitude      block group latitude\n    - Longitude     block group longitude\n\n:Missing Attribute Values: None\n\nThis dataset was obtained from the StatLib repository.\nhttps://www.dcc.fc.up.pt/~ltorgo/Regression/cal_housing.html\n\nThe target variable is the median house value for California districts,\nexpressed in hundreds of thousands of dollars ($100,000).\n\nThis dataset was derived from the 1990 U.S. census, using one row per census\nblock group. A block group is the smallest geographical unit for which the U.S.\nCensus Bureau publishes sample data (a block group typically has a population\nof 600 to 3,000 people).\n\nA household is a group of people residing within a home. Since the average\nnumber of rooms and bedrooms in this dataset are provided per household, these\ncolumns may take surprisingly large values for block groups with few households\nand many empty houses, such as vacation resorts.\n\nIt can be downloaded/loaded using the\n:func:`sklearn.datasets.fetch_california_housing` function.\n\n.. topic:: References\n\n    - Pace, R. Kelley and Ronald Barry, Sparse Spatial Autoregressions,\n      Statistics and Probability Letters, 33 (1997) 291-297\n'}
x_train_data, x_test, y_train_data, y_test = train_test_split(housing.data, housing.target)
x_train, x_valid, y_train, y_valid = train_test_split(x_train_data, y_train_data)

scaler = StandardScaler()
x_train = scaler.fit_transform(x_train)
x_valid = scaler.fit_transform(x_valid)
x_test = scaler.fit_transform(x_test)

Training the model using Sequencital API

import keras


input_layer = keras.layers.Dense(30, activation='relu', input_shape = x_train.shape[1:])
output_layer = keras.layers.Dense(1)

model = keras.models.Sequential([input_layer, output_layer])
model.compile(loss='mean_squared_error', optimizer='sgd')
model.summary()
Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 30)                270       
                                                                 
 dense_1 (Dense)             (None, 1)                 31        
                                                                 
=================================================================
Total params: 301 (1.18 KB)
Trainable params: 301 (1.18 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

from keras.utils import plot_model
plot_model(
    model,
    show_shapes=True,
    show_dtype=False,
    show_layer_names=False,
    rankdir="LR",
    expand_nested=False,
    show_layer_activations= True,
    dpi=96,
)


history = model.fit(x_train, y_train, epochs=10, validation_data = (x_valid, y_valid)) 
Epoch 1/10
363/363 [==============================] - 0s 534us/step - loss: 0.7398 - val_loss: 0.4427
Epoch 2/10
363/363 [==============================] - 0s 496us/step - loss: 0.5281 - val_loss: 0.4122
Epoch 3/10
363/363 [==============================] - 0s 346us/step - loss: 0.4744 - val_loss: 0.5118
Epoch 4/10
363/363 [==============================] - 0s 337us/step - loss: 0.4596 - val_loss: 0.4985
Epoch 5/10
363/363 [==============================] - 0s 345us/step - loss: 0.4481 - val_loss: 0.5587
Epoch 6/10
363/363 [==============================] - 0s 370us/step - loss: 0.4368 - val_loss: 0.7226
Epoch 7/10
363/363 [==============================] - 0s 364us/step - loss: 0.4358 - val_loss: 0.7338
Epoch 8/10
363/363 [==============================] - 0s 356us/step - loss: 0.4266 - val_loss: 0.9216
Epoch 9/10
363/363 [==============================] - 0s 365us/step - loss: 0.4444 - val_loss: 0.8432
Epoch 10/10
363/363 [==============================] - 0s 376us/step - loss: 0.4182 - val_loss: 1.0728
model.evaluate(x_test,y_test)
162/162 [==============================] - 0s 241us/step - loss: 0.3810
0.3810201585292816
x_1 = x_test[:3]
y_pred = model.predict(x_1)
print(y_pred)
1/1 [==============================] - 0s 38ms/step
[[3.08389  ]
 [1.2611595]
 [0.9729465]]

Building complex models ushing Functional API (non sequential)

input_ = keras.layers.Input(shape = x_train.shape[1:])
hidden_layer_1 = keras.layers.Dense(30, activation = 'relu')(input_)
hidden_layer_2 = keras.layers.Dense(30, activation='relu') (hidden_layer_1)
concat_layer = keras.layers.Concatenate()([input_, hidden_layer_2])
output_layer = keras.layers.Dense(1)(concat_layer)

model2 = keras.Model(inputs = [input_], outputs = [output_layer])
model2.summary()
Model: "model"
__________________________________________________________________________________________________
 Layer (type)                Output Shape                 Param #   Connected to                  
==================================================================================================
 input_1 (InputLayer)        [(None, 8)]                  0         []                            
                                                                                                  
 dense_2 (Dense)             (None, 30)                   270       ['input_1[0][0]']             
                                                                                                  
 dense_3 (Dense)             (None, 30)                   930       ['dense_2[0][0]']             
                                                                                                  
 concatenate (Concatenate)   (None, 38)                   0         ['input_1[0][0]',             
                                                                     'dense_3[0][0]']             
                                                                                                  
 dense_4 (Dense)             (None, 1)                    39        ['concatenate[0][0]']         
                                                                                                  
==================================================================================================
Total params: 1239 (4.84 KB)
Trainable params: 1239 (4.84 KB)
Non-trainable params: 0 (0.00 Byte)
__________________________________________________________________________________________________
plot_model(
    model2,
    show_shapes=True,
    show_dtype=True,
    show_layer_names=True,
    rankdir="TB",
    expand_nested=False,
    show_layer_activations= True,
    dpi=96,
)

Back to top