0

One question to consider when constructing classifiers is whether or not the method is location-scale invariant. This property holds if subjecting any subset of features in the training data to a linear trans- formation cannot change the prediction. For example, a classifier is location-scale invariant if changing the units of a feature from feet to inches, or from degrees fahrenheit to celsius, does not change the predictions (as long as the transformation is consistently applied to all classes and test data).

This is a crucial question, since if a classifier is not location-scale invariant, we need to decide how to make the units of each feature comparable. One way of doing this is to standardize quantitative features, typically by subtracting the mean then dividing by the standard deviation, so that each feature has mean zero and standard deviation 1. Doing this makes them essentially unitless. However, for some applications this may not be a suitable approach to this problem.

my quesstion is that KNN classifier, assuming Euclidean distance is used, location-scale invariant? Why or why not?

Lyu Keting
  • 1
  • 1
  • KNN is assumes a distance metric - this could be Euclidean, Metropolis or some other metric. The metric could be a transform that would mean that we have overlap where as in the original basis we could create a separation plane. However, this is usually the reason of making a transformation (look at kernel trick) as a case where we transform to a different dimension to get a separable plane. – Chinny84 Mar 14 '17 at 17:28
  • But coming back to your point we usually scale variables to reduce the magnitude of the variable being a weighted factor. Since $\sqrt{x^2+\delta^2} \approx x$ so how can you compare features? Just try looking at the data. – Chinny84 Mar 14 '17 at 17:28

0 Answers0