In the field of machine learning, embeddings serve as a way to represent data efficiently, which proves highly valuable for tasks such as identifying similar items. These models take the input data and transform it into a “latent space,” often a Euclidean vector space, where crucial information is preserved. This space organizes similar items in close proximity and distant ones farther away, simplifying intricate object comparisons, e.g. based on Euclidean distance.
A manifold is a mathematical concept that can exhibit curvature but appears locally flat. Manifold learning streamlines the analysis of high-dimensional data, particularly in settings that don’t adhere to traditional Euclidean geometry. It achieves this by projecting data onto lower-dimensional subspaces, such as e.g. 2D planes, thereby revealing concealed patterns. This simplification of data for analysis remains effective, even in non-Euclidean spaces where conventional Euclidean distance measurements don’t typically apply.
Autoencoders and Transformer networks serve as tools for nonlinear dimensionality reduction and manifold learning, because images from the same object class, even when they are distorted or rotated, effectively forming a manifold.
Image source of “Swiss role”: http://ravitejav.weebly.com/uploads/2/4/7/2/24725306/mvu.pdf , In the initial form points are closer together as in the unrolled form.