Freight Delivery Time Prediction using Stepwise Regression in ML
Upgrade Your Skills, Upgrade Your Career - Learn more
Logistics companies must accurately predict freight delivery times to optimise routing, meet customer expectations, and reduce costs associated with delays. In this freight delivery time prediction ML project, we will estimate the actual delivery duration of freight shipments—from pickup to drop‑off —measured in hours, using features such as distance, vehicle type, loading time, traffic congestion indicators, and weather conditions. Therefore, by applying stepwise regression, we’ll identify the most significant predictors and build a compact linear model that balances interpretability with predictive accuracy.
Libraries Required
import pandas as pd # Data manipulation import numpy as np # Numerical operations import statsmodels.api as sm # Statistical modeling (OLS) from sklearn.model_selection import train_test_split # Data splitting from sklearn.metrics import r2_score, mean_squared_error # Evaluation import matplotlib.pyplot as plt # Visualization
Dataset
Step-by-Step Code Implementation
Data Loading & Initial Inspection
We import the freight delivery dataset from the Kaggle competition, which includes key features such as Distance_km, Load_Time_min, and categorical fields (Vehicle_Type, Traffic_Level, Weather_Condition), along with the target Delivery_Hours.
# Block 1: Load dataset
# Kaggle Competition: Delivery Time Prediction :contentReference[oaicite:1]{index=1}
df = pd.read_csv("train.csv")
# Quick inspection
print(df.head())
print(df.info())
print(df.describe())
Data Preprocessing
Categorical variables are one‑hot encoded to convert them into numeric dummy columns. We drop any rows with missing entries to ensure data integrity. The feature matrix X excludes the response column Delivery_Hours, which is stored in y. We perform an 80/20 split into training and test subsets.
# Block 2: Encode categoricals & clean
# Assume columns include: 'Distance_km', 'Vehicle_Type', 'Load_Time_min',
# 'Traffic_Level' (Low/Medium/High), 'Weather_Condition', and target 'Delivery_Hours'
df_enc = pd.get_dummies(df,
columns=["Vehicle_Type", "Traffic_Level", "Weather_Condition"],
drop_first=True)
# Drop any rows with missing values
df_enc = df_enc.dropna()
# Define predictors and target
X = df_enc.drop("Delivery_Hours", axis=1)
y = df_enc["Delivery_Hours"]
# Split into train/test sets (80/20)
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
Stepwise Regression Function
The stepwise_selection function iteratively adds the excluded predictor with the lowest p‑value under 0.01 (forward step) and removes the included predictor with the highest p‑value above 0.05 (backward step). Hence, the process continues until no variables meet the criteria for addition or removal.
# Block 3: Forward–backward stepwise selection
def stepwise_selection(X, y,
initial_list=[],
threshold_in=0.01,
threshold_out=0.05,
verbose=True):
included = list(initial_list)
while True:
changed = False
# Forward step: test adding each excluded predictor
excluded = list(set(X.columns) - set(included))
new_pvals = pd.Series(index=excluded, dtype=float)
for col in excluded:
model = sm.OLS(y, sm.add_constant(X[included + [col]])).fit()
new_pvals[col] = model.pvalues[col]
best_pval = new_pvals.min()
if best_pval < threshold_in:
best_var = new_pvals.idxmin()
included.append(best_var)
changed = True
if verbose:
print(f"Add {best_var:25} p-value {best_pval:.4f}")
# Backward step: test removing each included predictor
model = sm.OLS(y, sm.add_constant(X[included])).fit()
pvals = model.pvalues.iloc[1:] # exclude intercept
worst_pval = pvals.max()
if worst_pval > threshold_out:
worst_var = pvals.idxmax()
included.remove(worst_var)
changed = True
if verbose:
print(f"Drop {worst_var:25} p-value {worst_pval:.4f}")
if not changed:
break
return included
Model Building & Evaluation
Using the selected features, we fit an Ordinary Least Squares regression via statsmodels. Hence, the .summary() output provides detailed coefficient estimates, p‑values, R², adjusted R², and diagnostic statistics, highlighting the significance and impact of each predictor.
We generate predictions on the held‑out test set and compute R² (variance explained) and RMSE (root‑mean‑square error) to quantify the model’s predictive accuracy on unseen data.
# Block 4: Select features via stepwise regression
selected_features = stepwise_selection(X_train, y_train)
# Fit final OLS model
X_train_sel = sm.add_constant(X_train[selected_features])
model = sm.OLS(y_train, X_train_sel).fit()
print(model.summary())
# Predict on test set
X_test_sel = sm.add_constant(X_test[selected_features])
y_pred = model.predict(X_test_sel)
# Compute performance metrics
print("Test R²:", r2_score(y_test, y_pred))
print("Test RMSE:", np.sqrt(mean_squared_error(y_test, y_pred)))
Residual Diagnostics
Thus, plotting residuals versus predicted values checks for non‑random patterns or heteroscedasticity, validating the assumptions of linear regression.
# Block 5: Residual plot
residuals = y_test - y_pred
plt.scatter(y_pred, residuals)
plt.axhline(0, linestyle="--")
plt.xlabel("Predicted Delivery Hours")
plt.ylabel("Residuals")
plt.title("Residuals vs. Predicted Delivery Time")
plt.show()
Summary
Therefore, by applying stepwise regression to freight delivery data, we distil the most influential factors—such as distance, loading time, traffic level, and vehicle type—while pruning redundant variables. Hence, the resulting linear model achieves a strong balance between interpretability (through precise coefficient estimates and p‑values) and predictive performance (high test‑set R², low RMSE). As a result, Logistics planners can leverage these insights to forecast delivery durations more accurately, optimise route schedules, and improve overall operational efficiency.
