The notion of Euclidean distance, which works well in the two-dimensional and three-dimensional worlds studied by Euclid, has some properties in higher dimensions that are contrary to our (maybe just *my*) geometric intuition which is also an extrapolation from two and three dimensions.

Consider a $4\times 4$ square with vertices at $(\pm 2, \pm 2)$. Draw four
unit-radius circles centered at $(\pm 1, \pm 1)$. These "fill" the square,
with each circle touching the sides of the square at two points, and each
circle touching its two neighbors. For example, the circle centered at
$(1,1)$ touches the sides of the square at $(2,1)$ and $(1,2)$, and
its neighboring circles at $(1,0)$ and $(0,1)$. Next, draw a small
circle *centered at the origin* that touches all four circles. Since
the line segment whose endpoints are the centers of
two osculating circles passes through the point of osculation, it is
easily verified that the small circle has radius $r_2 = \sqrt{2}-1$
and that it touches touches the four larger circles at $(\pm r_2/\sqrt{2}, \pm r_2/\sqrt{2})$.
Note that the small circle is "completely surrounded" by the four
larger circles and thus is also completely inside the square. Note also
that the point $(r_2,0)$ lies on the small circle. Notice also that from the origin, one cannot "see" the point $(2,0,0)$ on the edge of the square because the line of sight passes through the point of osculation $(1,0,0)$ of the two circles centered at $(1,1)$ and $(1,-1)$.
Ditto for the lines of sight to the other points where the axes pass through
the edges of the square.

Next, consider a $4\times 4 \times 4$ cube with vertices at
$(\pm 2, \pm 2, \pm 2)$. We fill it with $8$ osculating
unit-radius spheres centered at $(\pm 1, \pm 1, \pm 1)$, and
then put a smaller osculating sphere centered at the origin.
Note that the small sphere has radius $r_3 = \sqrt{3}-1 < 1$
and the point $(r_3,0,0)$ lies on the surface of the small sphere.
But notice also that in three dimensions, one *can* "see" the point
$(2,0,0)$ from the origin; there are no bigger bigger spheres
blocking the view as happens in two dimensions. These clear lines of sight
from the origin to the points where the axes pass through the surface of the cube occur in all larger dimensions as well.

Generalizing, we can consider a $n$-dimensional hypercube of side
$4$ and fill it with $2^n$ osculating unit-radius hyperspheres
centered at $(\pm 1, \pm 1, \ldots, \pm 1)$ and then
put a "smaller" osculating sphere of radius
$$r_n = \sqrt{n}-1\tag{1}$$ at the origin. The point $(r_n,0,0, \ldots, 0)$
lies on this "smaller" sphere.
But, notice from $(1)$ that when $n = 4$, $r_n = 1$ and so the
"smaller" sphere has unit radius and thus really does not deserve
the soubriquet of "smaller" for $n\geq 4$. Indeed, it would be better
if we called it the "larger sphere" or just "central sphere". As noted in the last paragraph, there is a clear line of sight from the origin to
the points where the axes pass through the surface of the hypercube.
Worse yet, when $n > 9$, we have from $(1)$ that $r_n >2$, and thus the point
$(r_n, 0, 0, \ldots, 0)$ on the central sphere
*lies outside the hypercube of side $4$
even though it is "completely surrounded" by the unit-radius hyperspheres
that "fill" the hypercube (in the sense of packing it).* The central
sphere "bulges" outside the hypercube in high-dimensional space.
I find this very counter-intuitive because my mental translations of
the notion of Euclidean distance to higher dimensions, using
the geometric intuition that I have developed
from the 2-space and 3-space that I am familiar with, do not
describe the reality of high-dimensional space.

My answer to the OP's question "Besides, what is 'high dimensions'?"
is $n \geq 9$.

because the conditions under which it has a known distribution are overly restrictive (essentially that variables are not correlated to one another). – user603 – 2014-05-18T19:17:09.860

5

Closely related: Euclidean distance is usually not good for sparse data? as pointed out by facuq.

– cardinal – 2014-05-19T02:22:49.2075

This is likely too basic for you; I wrote a series of blog posts on the subject of the Euclidean metric in higher dimensions and how that impacts searching vector spaces for nearest matches. http://blogs.msdn.com/b/ericlippert/archive/tags/high+dimensional+spaces/

– Eric Lippert – 2014-05-19T03:02:06.2171@HorstGrünbusch see answers below for some references. Variance of distances becomes small compare to the average. So at some point, you run into trouble choosing thresholds, weights, ordering; and you may even get numerical precision problems, too. But if your data is sparse, it likely is of much lower

intrinsicdimensionality. – Anony-Mousse – 2014-05-19T08:17:58.9403"high dimensions" seems to be a misleading term - some answers are treating 9-12 as "high dimensions", but in other areas high dimensionality would mean thousands or a million dimensions (say, measuring angles between bag-of-words vectors where each dimension is the frequency of some word in a dictionary), and 100 dimensions would be called low, not high. – Peteris – 2014-05-20T07:58:42.740

2This question could really do with some context.

Not good for what?– Szabolcs – 2014-05-20T21:04:23.2171@Peteris Dilip's answer points out that Euclidian distance in what you impy are a small number of dimensions is non-intuitive. I may be wrong, but I think it only gets more confusing as dimensions increases beyond nine. – Sycorax – 2014-05-21T03:32:39.427