ExMeshCNN: An Explainable Convolutional Neural Network Architecture for 3D Shape Analysis

초록

Triangular meshes have been actively used in computer graphics to represent 3D shapes. However, due to their non-uniform and irregular nature, learning such data with a Deep Neural Network is not straightforward. Transforming mesh data to simpler structures (e.g., voxel grids, point clouds, or multi-view 2D images) leads to other issues including spatial information loss and scalability. Traditional descriptors for mesh data simply extract hand-crafted features, which might not be effective in various environments. Several deep architectures that directly consume mesh data have been proposed, but their input features are still heuristic and unable to fully capture both geodesic and geometric characteristics of a mesh. In addition, their model architectures are not designed to be capable of providing visual explanations of their decision making. In this paper, we propose ExMeshCNN, a novel and explainable CNN structure for learning 3D meshes. In the first layer, we implement a descriptor layer composed of two types of learnable descriptors where each focuses on geodesic and geometric characteristics of a mesh, respectively. Then a series of convolution layers follow the descriptor layer to learn local features, where the convolution operations are carefully designed to be performed in a per-face manner. The final layer consists simply of the Global Average Pooling operation and the softmax output. In this way, ExMeshCNN learns mesh data in a completely end-to-end manner while retaining spatial information, where each layer is capable of computing face-level activations and gradients. Owing to these promising properties, existing visual attribution methods for model interpretability, such as LRP and Grad-CAM, can be easily applied to ExMeshCNN to highlight the salient surfaces of a 3D mesh for the corresponding prediction. Experimental results show that ExMeshCNN not only exhibits state-of-the-art or comparable performances in the 3D mesh classification and segmentation with the smallest number of parameters, but also provides the visual explanations of why it makes a specific prediction in the 3D space.

키워드