Connect your moderator Slack workspace to receive post notifications:
Sign in with Slack

Nearest neighbor decision boundary

Dear TAs,

Could you explain in detail how to sketch the decision boundaries for the k-nearest neighbor classifier?

Is this correct?
https://stats.stackexchange.com/questions/370531/knn-decision-boundary

Thanks in advance :)

This comment was deleted..

Good question!

For 1 nearest neighbor (1-NN or NN) you can follow the procedure below. For K>1, visualizing the decision boundary by hand becomes quite hard, and is usually done computationally with a grid as in the link you mention. See also here: https://stackoverflow.com/questions/45075638/graph-k-nn-decision-boundaries-in-matplotlib

For (1-)NN:

  1. Construct (dashed) lines between the closest pairs of points in different classes.
  2. Draw a simple (nondashed) decision boundary for each closest pair (only looking at the points in the pair).
    Step 1 + Step 2 illustrated:
    rsz_step1step2.jpg
  3. End the simple decision boundaries where they intersect.
    rsz_step3.jpg

Source: http://web.mit.edu/6.034/wwwbob/recitation6-notesfall11.pdf

As a note, the decision boundary for K=1 is equivalent to the Voronoi diagram, which you can find many implementations to draw online. K>1 is more involved and the easiest and non-formal way would be:
1) dividing the region into a grid
2) checking the decision for each point
3) finding the neighbor of points where the decision changes
Best,
Semih

Wow! Thank you so much!! :D

Page 1 of 1

Add comment

Post as Anonymous Dont send out notification