Publications
You can also find my articles on my Google Scholar.
2024
- Synchronous Diffusion for Unsupervised Smooth Non-Rigid 3D Shape MatchingDongliang Cao, Zorah Laehner, and Florian BernardIn ECCV , 2024
Most recent unsupervised non-rigid 3D shape matching methods are based on the functional map framework due to its efficiency and superior performance. Nevertheless, respective methods struggle to obtain spatially smooth pointwise correspondences due to the lack of proper regularisation. In this work, inspired by the success of message passing on graphs, we propose a \emphsynchronous diffusion process which we use as regularisation to achieve smoothness in non-rigid 3D shape matching problems. The intuition of synchronous diffusion is that diffusing the same input function on two different shapes results in consistent outputs. Using different challenging datasets, we demonstrate that our novel regularisation can substantially improve the state-of-the-art in shape matching, especially in the presence of topological noise.
@inproceedings{cao2024synchronous, author = {Cao, Dongliang and Laehner, Zorah and Bernard, Florian}, title = {Synchronous Diffusion for Unsupervised Smooth Non-Rigid 3D Shape Matching}, booktitle = {ECCV}, year = {2024} } - DiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape MatchingIn ECCV , 2024
In this work we propose to combine the advantages of learning-based and combinatorial formalisms for 3D shape matching. While learning-based methods lead to state-of-the-art matching performance, they do not ensure geometric consistency, so that obtained matchings are locally non-smooth. On the contrary, axiomatic, optimisation-based methods allow to take geometric consistency into account by explicitly constraining the space of valid matchings. However, existing axiomatic formalisms do not scale to practically relevant problem sizes, and require user input for the initialisation of non-convex optimisation problems. We work towards closing this gap by proposing a novel combinatorial solver that combines a unique set of favourable properties: our approach (i) is initialisation free, (ii) is massively parallelisable and powered by a quasi-Newton method, (iii) provides optimality gaps, and (iv) delivers improved matching quality with decreased runtime and globally optimal results for many instances.
@inproceedings{roetzer2024disco, author = {Roetzer, Paul and Abbas, Ahmed and Cao, Dongliang and Bernard, Florian and Swoboda, Paul}, title = {DiscoMatch: Fast Discrete Optimisation for Geometrically Consistent 3D Shape Matching}, booktitle = {ECCV}, year = {2024} } - A Universal and Flexible Framework for Unsupervised Statistical Shape Model LearningNafie El Amrani, Dongliang Cao, and Florian BernardIn MICCAI , 2024
We introduce a novel unsupervised deep learning framework for constructing statistical shape models (SSMs). Although unsupervised learning-based 3D shape matching methods have made a major leap forward in recent years, the correspondence quality of existing methods does not meet the demanding requirements necessary for the construction of SSMs of complex anatomical structures. We address this shortcoming by proposing a novel \emphdeformation coherency loss to effectively enforce smooth and high-quality correspondences during neural network training. We demonstrate that our framework outperforms existing methods in creating high-quality SSMs by conducting extensive experiments on five challenging datasets with varying anatomical complexities. Our proposed method sets the new state of the art in unsupervised SSM learning, offering a universal solution that is both flexible and reliable. Our source code is publicly available at https://github.com/NafieAmrani/FUSS.
@inproceedings{elamrani2024fuss, author = {El Amrani, Nafie and Cao, Dongliang and Bernard, Florian}, title = {A Universal and Flexible Framework for Unsupervised Statistical Shape Model Learning}, booktitle = {MICCAI}, year = {2024} } - Spectral Meets Spatial: Harmonising 3D Shape Matching and InterpolationIn CVPR , 2024
Although 3D shape matching and interpolation are highly interrelated, they are often studied separately and applied sequentially to relate different 3D shapes, thus resulting in sub-optimal performance. In this work we present a unified framework to predict both point-wise correspondences and shape interpolation between 3D shapes. To this end, we combine the deep functional map framework with classical surface deformation models to map shapes in both spectral and spatial domains. On the one hand, by incorporating spatial maps, our method obtains more accurate and smooth point-wise correspondences compared to previous functional map methods for shape matching. On the other hand, by introducing spectral maps, our method gets rid of commonly used but computationally expensive geodesic distance constraints that are only valid for near-isometric shape deformations. Furthermore, we propose a novel test-time adaptation scheme to capture both pose-dominant and shape-dominant deformations. Using different challenging datasets, we demonstrate that our method outperforms previous state-of-the-art methods for both shape matching and interpolation, even compared to supervised approaches.
@inproceedings{cao2024spectral, author = {Cao, Dongliang and Eisenberger, Marvin and El Amrani, Nafie and Cremers, Daniel and Bernard, Florian}, title = {Spectral Meets Spatial: Harmonising 3D Shape Matching and Interpolation}, booktitle = {CVPR}, year = {2024} } - Unsupervised 3D Structure Inference from Category-Specific Image CollectionsWeikang Wang, Dongliang Cao, and Florian BernardIn CVPR , 2024
Understanding 3D object structure from image collections of general object categories remains a long-standing challenge in computer vision. Due to the high relevance of image keypoints (e.g. for graph matching, controlling generative models, scene understanding, etc.), in this work we specifically focus on inferring 3D structure in terms of sparse keypoints. Existing 3D keypoint inference approaches rely on strong priors, such as spatio-temporal consistency, multi-view images of the same object, 3D shape priors (e.g. templates, skeleton), or supervisory signals e.g. in the form of 2D keypoint annotations. In contrast, we propose the first unsupervised 3D keypoint inference approach that can be trained for general object categories solely from an inhomogeneous image collection (containing different instances of objects from the same category). Our experiments show that our method not only improves upon unsupervised 2D keypoint inference, but more importantly, it also produces reasonable 3D structure for various object categories, both qualitatively and quantitatively.
@inproceedings{wang2024unsupervised, author = {Wang, Weikang and Cao, Dongliang and Bernard, Florian}, title = {Unsupervised 3D Structure Inference from Category-Specific Image Collections}, booktitle = {CVPR}, year = {2024} } - Revisiting Map Relations for Unsupervised Non-Rigid Shape MatchingDongliang Cao, Paul Roetzer, and Florian BernardIn 3DV , 2024
We propose a novel unsupervised learning approach for non-rigid 3D shape matching. Our approach improves upon recent state-of-the art deep functional map methods and can be applied to a broad range of different challenging scenarios. Previous deep functional map methods mainly focus on feature extraction and aim exclusively at obtaining more expressive features for functional map computation. However, the importance of the functional map computation itself is often neglected and the relationship between the functional map and point-wise map is underexplored. In this paper, we systematically investigate the coupling relationship between the functional map from the functional map solver and the point-wise map based on feature similarity. To this end, we propose a self-adaptive functional map solver to adjust the functional map regularisation for different shape matching scenarios, together with a vertex-wise contrastive loss to obtain more discriminative features. Using different challenging datasets (including non-isometry, topological noise and partiality), we demonstrate that our method substantially outperforms previous state-of-the-art methods.
@inproceedings{cao2024revisiting, author = {Cao, Dongliang and Roetzer, Paul and Bernard, Florian}, title = {Revisiting Map Relations for Unsupervised Non-Rigid Shape Matching}, booktitle = {3DV}, year = {2024} }
2023
- DefCor-Net: Physics-aware ultrasound deformation correctionZhongliang Jiang, Yue Zhou, Dongliang Cao, and Nassir NavabMedical Image Analysis, 2023
The recovery of morphologically accurate anatomical images from deformed ones is challenging in ultrasound (US) image acquisition, but crucial to accurate and consistent diagnosis, particularly in the emerging field of computer-assisted diagnosis. This article presents a novel anatomy-aware deformation correction approach based on a coarse-to-fine, multi-scale deep neural network (DefCor-Net). To achieve pixel-wise performance, DefCor-Net incorporates biomedical knowledge by estimating pixel-wise stiffness online using a U-shaped feature extractor. The deformation field is then computed using polynomial regression by integrating the measured force applied by the US probe. Based on real-time estimation of pixel-by-pixel tissue properties, the learning-based approach enables the potential for anatomy-aware deformation correction. To demonstrate the effectiveness of the proposed DefCor-Net, images recorded at multiple locations on forearms and upper arms of six volunteers are used to train and validate DefCor-Net. The results demonstrate that DefCor-Net can significantly improve the accuracy of deformation correction to recover the original geometry (Dice Coefficient: from 14.3±20.9 to 82.6±12.1 when the force is 6N).
@article{jiang2023defcor, author = {Jiang, Zhongliang and Zhou, Yue and Cao, Dongliang and Navab, Nassir}, title = {DefCor-Net: Physics-aware ultrasound deformation correction}, journal = {Medical Image Analysis}, year = {2023} } - Unsupervised learning of robust spectral shape matchingDongliang Cao, Paul Roetzer, and Florian BernardACM Transactions on Graphics (ToG), 2023
We propose a novel learning-based approach for robust 3D shape matching. Our method builds upon deep functional maps and can be trained in a fully unsupervised manner. Previous deep functional map methods mainly focus on predicting optimised functional maps alone, and then rely on off-the-shelf post-processing to obtain accurate point-wise maps during inference. However, this two-stage procedure for obtaining point-wise maps often yields sub-optimal performance. In contrast, building upon recent insights about the relation between functional maps and point-wise maps, we propose a novel unsupervised loss to couple the functional maps and point-wise maps, and thereby directly obtain point-wise maps without any post-processing. Our approach obtains accurate correspondences not only for near-isometric shapes, but also for more challenging non-isometric shapes and partial shapes, as well as shapes with different discretisation or topological noise. Using a total of nine diverse datasets, we extensively evaluate the performance and demonstrate that our method substantially outperforms previous stateof-the-art methods, even compared to recent supervised methods.
@article{cao2023unsupervised, author = {Cao, Dongliang and Roetzer, Paul and Bernard, Florian}, title = {Unsupervised learning of robust spectral shape matching}, journal = {ACM Transactions on Graphics (ToG)}, year = {2023} } - Self-Supervised Learning for Multimodal Non-Rigid 3D Shape MatchingDongliang Cao, and Florian BernardIn CVPR (Highlight, 2.5% submissions) , 2023
The matching of 3D shapes has been extensively studied for shapes represented as surface meshes, as well as for shapes represented as point clouds. While point clouds are a common representation of raw real-world 3D data (e.g. from laser scanners), meshes encode rich and expressive topological information, but their creation typically requires some form of (often manual) curation. In turn, methods that purely rely on point clouds are unable to meet the matching quality of mesh-based methods that utilise the additional topological structure. In this work we close this gap by introducing a self-supervised multimodal learning strategy that combines mesh-based functional map regularisation with a contrastive loss that couples mesh and point cloud data. Our shape matching approach allows to obtain intramodal correspondences for triangle meshes, complete point clouds, and partially observed point clouds, as well as correspondences across these data modalities. We demonstrate that our method achieves state-of-the-art results on several challenging benchmark datasets even in comparison to recent supervised methods, and that our method reaches previously unseen cross-dataset generalisation ability.
@inproceedings{cao2023self, author = {Cao, Dongliang and Bernard, Florian}, title = {Self-Supervised Learning for Multimodal Non-Rigid 3D Shape Matching}, booktitle = {CVPR}, year = {2023}, }
2022
- Unsupervised Deep Multi-Shape MatchingDongliang Cao, and Florian BernardIn ECCV , 2022
3D shape matching is a long-standing problem in computer vision and computer graphics. While deep neural networks were shown to lead to state-of-the-art results in shape matching, existing learning-based approaches are limited in the context of multi-shape matching: (i) either they focus on matching pairs of shapes only and thus suffer from cycle-inconsistent multi-matchings, or (ii) they require an explicit template shape to address the matching of a collection of shapes. In this paper, we present a novel approach for deep multi-shape matching that ensures cycle-consistent multi-matchings while not depending on an explicit template shape. To this end, we utilise a shape-to-universe multi-matching representation that we combine with powerful functional map regularisation, so that our multi-shape matching neural network can be trained in a fully unsupervised manner. While the functional map regularisation is only considered during training time, functional maps are not computed for predicting correspondences, thereby allowing for fast inference. We demonstrate that our method achieves state-of-the-art results on several challenging benchmark datasets, and, most remarkably, that our unsupervised method even outperforms recent supervised methods.
@inproceedings{cao2022unsupervised, author = {Cao, Dongliang and Bernard, Florian}, title = {Unsupervised Deep Multi-Shape Matching}, booktitle = {ECCV}, year = {2022} }
