Interactive Project: Tune Polynomial Regression to Predict House Prices
Companion Exercises for Practical Machine Learning with R
The interactive R environment and the required packages must be loaded in your browser. This typically takes 1–2 minutes. You can monitor the progress in the bottom-right corner.
After packages are loaded, it still takes some time because the website must run a pre-tuning procedure. Please be patient!!!
When everything is loaded, you get a “Ready to roll” message below. Thank you for your patience!
Libraries already loaded in the background:
- tidyverse
- tidymodels
Step 1 - Generating Training and Testing Data:
As before, you use the housing dataset and split the data into training (DataTrain) and testing data (DataTest). Complete the ______ in the code below to assign 80% of the data to the training data and 20% to the testing data
Here is a hint:
The prop= argument determines the proportion of training data observations. The remaining observations become testing data.
Here is the Solution:
Split8020=initial_split(prop=0.8, #<1>
strata=Price, breaks=5, #<2>
data=DataHousing)
print(Split8020)
- 1
-
prop=0.8determines that 80% of the data are assigned to the training data. All other observations are assigned to the testing data. - 2
-
strata=Priceensures that the mean price in training and testing data observations is similar. Thebreaks=5argument ensures that this is true for all quintiles.
Step 2 - Create a Recipe:
Here, you will create a recipe and store it in the R object RecipePoly.
The recipe creautes the variables for \(Sqft^2\), \(Sqft^3\) and so on, depending on which degree is chosen.
Since you plan to tune the argument degree=, you have to add the placeholder tune(). Remember that you cannot set degree to a specific numerical value because you want to tune the hyper-parameter degree later on.
Note that the data frame DataTrain has already been loaded in the background:
Here is a hint:
You have to assign the placeholder tune() to the argument degree=. Later, when you tune the workflow, it will be replaced by the values you choose to try out.
Here is the Solution:
RecipePoly=recipe(Price~Sqft, data=DataTrain) |>
step_poly(Sqft, degree = tune(), options = list(raw = TRUE))
print(RecipePoly)
RecipePoly=recipe(Price~Sqft, data=DataTrain) |>
step_poly(Sqft, degree = tune(), options = list(raw = TRUE))
print(RecipePoly)Step 3 - Create a Model Design:
Next, you will create the model design and store it into the R object ModelDesignPoly. It is exactly the same that you use for a linear regression.
Here is a hint:
- The
tidymodelscommand for a linear regression islinear_reg() - Your ananlysis is a regression not a classification
Here is the Solution:
ModelDesignPoly= linear_reg() |>
set_engine("lm") |>
set_mode("regression")
print(ModelDesignPoly)
ModelDesignPoly= linear_reg() |>
set_engine("lm") |>
set_mode("regression")
print(ModelDesignPoly)Step 4 - Add the RecipePoly and the Model Design to a Workflow:
The code block below adds the R object Recipe and the model design object ModelDesignPoly to a workflow model named TuneWFModelPoly. The R code has been executed already.
You can see in the printout above that the workflow is not finalized because degree has not been set in the model design. The hyper-parameter degree is set to tune() instead. Later, in Step 7, tune() will be replaced in a trial and error process with different values for degree.
Step 5 - Create a Hyper-Parameter Grid:
When tuning is executed in Step 7, values reaching from 1 – 10 for the hyper-parameter neighbors shall be tried out. You need to provide these values in a data frame column that is named the same as the hyper-parameter. Below, replace the ______ to define a column degree in the data frameParGrid. The column should contain values from 1 – 10:
Here is a hint:
Use the c() command to create a vector containing the numbers \(1 - 10\) either separated by a comma, or you can use a colon (:) to indicate from/to.
Here is the Solution:
ParGrid=data.frame(neighbors=c(1,2,3,4,5,6,7,8,9,10))
ParGrid
ParGrid=data.frame(neighbors=c(1,2,3,4,5,6,7,8,9,10))
ParGridStep 6 - Creating Resamples for Cross-Validation:
The values you have created above for degree of the Polynomial Regression (hyper-parameter degree) will be evaluated later using five folds (resamples).
Each fold contains the complete training data, but different sections are used for training and assessment in each fold. Please create five folds below by substituting ______ accordingly.
The folds will be saved in the R object FoldsForTuning:
Here is a hint:
The argument v= determines the number of folds to be used for tuning.
Here is the Solution:
set.seed(123)
FoldsForTuning=vfold_cv(DataTrain, v=5, strata=Price)
print(FoldsForTuning)
set.seed(123)
FoldsForTuning=vfold_cv(DataTrain, v=5, strata=Price)
print(FoldsForTuning)Step 7 - Tune the Workflow and Train All Models:
Now it is time to run the tuning procedure using the tune_grid() command. Be patient because it will take some time to fully execute. Since we have to try out 10 parameters and use five folds for each model, the tune_grid() command has to fit 50 models \((10\cdot 5=50)\).
Please substitute ______ with a list of metrics to be calculated. Use the metrics rmse (Root Mean Squared Error), mae (Mean Absolute Error) and rsq (r Squared \(r^2\)).
After the tuning is finished, all results are stored in the R object TuneResults, and they can be evaluated by different metrics commands.
For example the command autoplot() provides a diagrammatic overview of the results for the three metrics.
Here is a hint:
The command metric_set() takes a comma-separated list of metrics to be evaluated.
Consider that you want to evaluate the metrics rmse, mae, and rsq.
Here is the Solution:
TuneResults=tune_grid(TuneWFModelPoly, resamples=FoldsForTuning,
grid=ParGrid, metrics=metric_set(rmse, mae, rsq))
autoplot(TuneResults)
TuneResults=tune_grid(TuneWFModelPoly, resamples=FoldsForTuning,
grid=ParGrid, metrics=metric_set(rmse, mae, rsq))
autoplot(TuneResults)The three graphs that you created in the exercise above show the values for each rmse, mae, and rsq for all tried out hyper-parameters (degree).
To calculate these values the five folds that were used for validation are averaged.
It looks like \(r^2\) has the clearst fluctutions. It seems reasonable to assume that degree=6 provides the best results. Let us confirm this in the Step 8.
Step 8 - Extract the Best Hyper-Parameter(s):
All assessment results for the specified metrics are stored in the tuning object TuneResults. Choose the metric rsq by substituting ______ accordingly. Afterward, when you execute the code the select_best() command extracts the best hyper-parameter (value for degree) for the metric you specified:
Here is a hint:
Substitute ______ with rsq and make sure to not delete the quotes.
Here is the Solution:
BestHyperPar=select_best(TuneResults, metric="rsq")
print(BestHyperPar)
BestHyperPar=select_best(TuneResults, metric="rsq")
print(BestHyperPar)As you will see in the printout after executing the R code, the best value for the hyper-parameter degree based on rsq is \(degree=6\). It is saved in the data frame BestHyperPar.
You can try other metrics in the code block above and see if the result changes.
Step 9 - Finalize and Train the Best Workflow Model:
The code block below is executed already. The finalize_workflow() command used the value from BestHyperPar to substitute the tune() placeholder in the R object TuneWFModelKNN (created in Steps 2 – 4). The hyper-parameter is now set to \(degree=6\) .
At the end of the printout above, you can see that a Degree 6 Polynominal Regression is used and waht the related coefficients are.
Step 10: Assess Prediction Quality Based on the Testing Data:
Since WFModelBest is a fitted model, you can use it for predictions. In this last step, you use the augment() command to predict house prices (Price). The augment() command will then add the prediction results as column .pred to the testing data.
The metric() command compares the predictions in column .pred to the true values to calcualte common metrics for a regression. But before, you have to substitute ______ with the variable name (column name) for the outcome variable.
Here is a hint:
The (column) name for the outcome variable is Price and the predictions are stored in the column .pred.
Here is the Solution:
DataTestWithPredBestModel=augment(WFModelBest, DataTest)
metrics(DataTestWithPredBestModel, truth=Price, estimate=.pred)
DataTestWithPredBestModel=augment(WFModelBest, DataTest)
metrics(DataTestWithPredBestModel, truth=Price, estimate=.pred)