A Unified Framework for Subspace Clustering: Unification of Graph Neural Networks, Attention, and Coding Rate Reduction

Subspace clustering refers to the task of segmenting data points that lie on a union of subspaces. Current state-of-the-art methods for this task leverage the self-expressive property of subspaces, where each data point is represented as a linear combination of other points within the same subspace. Recent efforts have extended this approach to data on nonlinear manifolds by using neural networks or graph convolution techniques to learn embeddings from the original data. These methods often include a regularization term, defined by a self-expressive loss, to align the embedded data with a union of linear subspaces. However, they still fail to explicitly capture the relationship between the affinity matrix and the segmentation process. To address this limitation, we propose a novel framework, Graph Representation learning with Attention and Coding rate rEduction (GRACE). GRACE jointly learns data representations and the graph structure of data points while linking them to the segmentation process. It employs a graph neural network to generate effective data representations and incorporates a graph learning module with an attention mechanism to infer the graph structure. Additionally, it includes a pseudo-labeling module to effectively integrate the segmentation process with both graph learning and representation learning. Furthermore, we introduce a new objective function for unsupervised graph and representation learning, based on maximizing coding rate reduction, to enhance the discriminative power of the learned representations. Extensive evaluations on real-world datasets demonstrate that GRACE achieves competitive or superior performance compared to state-of-the-art subspace clustering methods.