text: 195,56,1395500630614106114,2818695390,Sasho Nikolov,"New paper up, with amazing U of T student Deepanshu Kush and Haohua Tang (who was an undergrad while working on this): Near Neighbor Search via Efficient Average Distortion Embeddings <LINK>. 🧵 [1/9] The goal in approximate near neighbor search (ANN): preprocess n points P (in some metric space) into a small data structure s.t. queries can be answered quickly: given a new point q, if P has some x* that’s close to q, return a point x in P that’s approximately close to q. [2/9] We want to answer a query in less time than it takes to scan P, while keeping space and preprocessing time low. Approximation is necessary for this, but how much? It depends on how we measure distance. With Euclidean or Manhattan distance, constant approx is possible. [3/9] What about other distances? You can approximate your favorite metric by a subset of Euclidean/Manhattan space, i.e., find a low distortion embedding. You can get lots of non-trivial results like this, but you are stuck with bad approximation poly(d) even for ell_p^d spaces [4/9] In papers with Andoni, Naor, @ilyaraz2, and Waingarten we showed that you can get very good approximation poly(log d) for *any* distance given by a norm, much better than embeddings (@QuantaMagazine article: <LINK>) [5/9] Big caveat: the data structures require exponential preprocessing. This is what the new results improve: for ell_p and other spaces, we give data structures with the same approx as ANNRW but efficient preprocessing, via a new general method that could work for all norms. [6/9] The idea is to use an embedding into Euclidean space with a much weaker guarantee: low *average* distortion, rather than low distortion. I.e. the embedding should not expand distances, but it can contract them, as long as the average distance between pts in P stays large. [7/9] Turns out that such embeddings are enough to get good random hash functions, and from them good data structures. We also construct low avg distortion embeddings for ell_p and other spaces. Finally, a fascinating open problem: [8/9] Find an efficiently computable embedding of (the 1/2-snowflake) of any d-dimensional norm into Euclidean space with avg distortion sqrt(log d). Naor proved (<LINK>) these exist, but did not give an explicit embedding, let alone an algorithm to compute it. [9/9]",https://arxiv.org/abs/2105.04712,"A recent series of papers by Andoni, Naor, Nikolov, Razenshteyn, and Waingarten (STOC 2018, FOCS 2018) has given approximate near neighbour search (NNS) data structures for a wide class of distance metrics, including all norms. In particular, these data structures achieve approximation on the order of $p$ for $\ell_p^d$ norms with space complexity nearly linear in the dataset size $n$ and polynomial in the dimension $d$, and query time sub-linear in $n$ and polynomial in $d$. The main shortcoming is the exponential in $d$ pre-processing time required for their construction. In this paper, we describe a more direct framework for constructing NNS data structures for general norms. More specifically, we show via an algorithmic reduction that an efficient NNS data structure for a given metric is implied by an efficient average distortion embedding of it into $\ell_1$ or into Euclidean space. In particular, the resulting data structures require only polynomial pre-processing time, as long as the embedding can be computed in polynomial time. As a concrete instantiation of this framework, we give an NNS data structure for $\ell_p$ with efficient pre-processing that matches the approximation factor, space and query complexity of the aforementioned data structure of Andoni et al. On the way, we resolve a question of Naor (Analysis and Geometry in Metric Spaces, 2014) and provide an explicit, efficiently computable embedding of $\ell_p$, for $p \ge 2$, into $\ell_2$ with (quadratic) average distortion on the order of $p$. We expect our approach to pave the way for constructing efficient NNS data structures for all norms. ",Near Neighbor Search via Efficient Average Distortion Embeddings,9,"['New paper up, with amazing U of T student Deepanshu Kush and Haohua Tang (who was an undergrad while working on this): Near Neighbor Search via Efficient Average Distortion Embeddings <LINK>. 🧵 [1/9]', 'The goal in approximate near neighbor search (ANN): preprocess n points P (in some metric space) into a small data structure s.t. queries can be answered quickly: given a new point q, if P has some x* that’s close to q, return a point x in P that’s approximately close to q. [2/9]', 'We want to answer a query in less time than it takes to scan P, while keeping space and preprocessing time low. Approximation is necessary for this, but how much? It depends on how we measure distance. With Euclidean or Manhattan distance, constant approx is possible. [3/9]', 'What about other distances? You can approximate your favorite metric by a subset of Euclidean/Manhattan space, i.e., find a low distortion embedding. You can get lots of non-trivial results like this, but you are stuck with bad approximation poly(d) even for ell_p^d spaces [4/9]', 'In papers with Andoni, Naor, @ilyaraz2, and Waingarten we showed that you can get very good approximation poly(log d) for *any* distance given by a norm, much better than embeddings (@QuantaMagazine article: https://t.co/KYqJqt3sjP) [5/9]', 'Big caveat: the data structures require exponential preprocessing. This is what the new results improve: for ell_p and other spaces, we give data structures with the same approx as ANNRW but efficient preprocessing, via a new general method that could work for all norms. [6/9]', 'The idea is to use an embedding into Euclidean space with a much weaker guarantee: low *average* distortion, rather than low distortion. I.e. the embedding should not expand distances, but it can contract them, as long as the average distance between pts in P stays large. [7/9]', 'Turns out that such embeddings are enough to get good random hash functions, and from them good data structures. We also construct low avg distortion embeddings for ell_p and other spaces. Finally, a fascinating open problem: [8/9]', 'Find an efficiently computable embedding of (the 1/2-snowflake) of any d-dimensional norm into Euclidean space with avg distortion sqrt(log d). Naor proved (https://t.co/p63oPk5NV9) these exist, but did not give an explicit embedding, let alone an algorithm to compute it. [9/9]']",21,05,2308