Home » Support Vector Regression

Support Vector Regression

by IndiaSuccessStories
0 comment

Support Vector Regression


Support Vector Regression (SVR) is a type of regression analysis that uses Support Vector Machines (SVM) to predict continuous variables. It’s particularly useful when dealing with datasets that have non-linear relationships between features and targets.

Here's an explanation of how Support Vector Regression works:

  1. Kernel Trick


Similar to Support Vector Machines for classification, SVR also uses a kernel trick to transform the input features into higher-dimensional space. This transformation allows SVR to capture complex relationships between the features and the target variable.


  1. Margin


SVR aims to find a hyperplane in the higher-dimensional space that has the maximum margin and still includes as many data points as possible within a certain margin of error (epsilon). Unlike in classification where the hyperplane separates different classes, in regression, the hyperplane is fitted to best approximate the target variable.


  1. Epsilon-Insensitive Loss Function


SVR introduces an epsilon-insensitive loss function, which allows some errors to be ignored within a certain margin (epsilon). Data points within this margin are considered to have been correctly predicted and do not contribute to the loss.


  1. Regularization Parameter (C)


Similar to SVM, SVR has a regularization parameter (C) that controls the trade-off between maximizing the margin and minimizing the error. A smaller C value allows for a wider margin but may lead to more training errors, while a larger C value reduces training errors but may lead to overfitting.


  1. Kernel Functions


SVR supports different kernel functions such as linear, polynomial, radial basis function (RBF), and sigmoid. These kernel functions allow SVR to handle non-linear relationships between the features and the target variable.


  1. Prediction


Once the SVR model is trained, it can be used to make predictions on new data points by mapping them to the higher-dimensional space and finding their corresponding values on the hyperplane.


  1. Hyperparamter Tuning


SVR involves tuning hyperparameters such as the choice of kernel function, epsilon, and regularization parameter (C) to optimize the model's performance. This tuning process is usually done using techniques like cross-validation.


In summary, Support Vector Regression is a powerful regression technique that leverages the concepts of SVM to predict continuous variables. It's effective for handling non-linear relationships and can be fine-tuned using various kernel functions and hyperparameters to achieve optimal performance on different types of datasets.

Let's walk through an example of Support Vector Regression (SVR) using a synthetic dataset to predict the price of houses based on their size (in square feet). We'll use the SVR implementation from scikit-learn library in Python.

Here's how to do it step by step:

  1. Generate Synthetic Data


import numpy as np

import pandas as pd


# Generate synthetic data


n_samples = 1000


size_sqft = np.random.randint(800, 3000, size=n_samples)  # Random square footage (800 to 3000 sqft)

price = 50000 + 100 * size_sqft + np.random.normal(0, 10000, size=n_samples)  # Generate price with noise


# Create DataFrame

data = pd.DataFrame({'Size_sqft': size_sqft, 'Price': price})


  1. Explore the Data





  1. Data Preprocessing


X = data[['Size_sqft']]  # Features

y = data['Price']        # Target variable


  1. Split Data into Train and Test Sets


from sklearn.model_selection import train_test_split


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)


  1. Create and Fit the SVR Model


from sklearn.svm import SVR


model = SVR(kernel='linear', C=100)

model.fit(X_train, y_train)


  1. Make Predictions


y_pred = model.predict(X_test)


  1. Evaluate the Model


from sklearn.metrics import mean_squared_error, r2_score


mse = mean_squared_error(y_test, y_pred)

r2 = r2_score(y_test, y_pred)


print("Mean Squared Error:", mse)

print("R-squared:", r2)


  1. Visualize Results


import matplotlib.pyplot as plt


plt.scatter(X_test, y_test, color='blue')  # Plot test data

plt.plot(X_test, y_pred, color='red')      # Plot regression line

plt.xlabel("Size (sqft)")


plt.title("Support Vector Regression: House Size vs. Price")


You may also like

Leave a Comment

Indian Success Stories Logo

Indian Success Stories is committed to inspiring the world’s visionary leaders who are driven to make a difference with their ground-breaking concepts, ventures, and viewpoints. Join together with us to match your business with a community that is unstoppable and working to improve everyone’s future.

Edtior's Picks

Latest Articles

Copyright © 2024 Indian Success Stories. All rights reserved.