Joint Distribution Matters: Deep Brownian Distance Covariance

for Few-Shot Classification

Jiangtao Xie1,*, Fei Long1,*, Jiaming Lv1, Qilong Wang2, Peihua Li1,‡

1Dalian University of Technology, 2Tianjin University

*Equal contribution. ‡The corresponding author.

CVPR 2022 (Oral)

Code: https://github.com/Fei-Long121/DeepBDC Paper: arXiv Video: Youtube (in English), B Site (in Chinese)

Abstract

Few-shot classification is a challenging problem as only very few training examples are given for each new task. One of the effective research lines to address this challenge focuses on learning deep representations driven by a similarity measure between a query image and few support images of one class. Statistically, this amounts to measure the dependency of image features, viewed as random vectors in a high-dimensional embedding space. Previous methods either only use marginal distributions without considering joint distributions, suffering from limited representation capability, or are computationally expensive though harnessing joint distributions. In this paper, we propose a deep Brownian Distance Covariance (DeepBDC) method for few-shot classification. The central idea of DeepBDC is to learn image representations by measuring the discrepancy between joint characteristic functions of embedded features and product of the marginals. As the BDC metric is decoupled, we formulate it as a highly modular and efficient layer. Furthermore, we instantiate DeepBDC in two different few-shot classification frameworks. We make experiments on six standard few-shot image benchmarks, covering general object recognition, fine-grained categorization and cross-domain classification. Extensive evaluations show our DeepBDC significantly outperforms its counterparts, while establishing new state-of-the-art results.

Citation

@inproceedings{DeepBDC-CVPR2022,
title={Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification},
author={Jiangtao Xie and Fei Long and Jiaming Lv and Qilong Wang and Peihua Li},
booktitle={CVPR},
year={2022}}

arXiv, Github

 

Highlights

The proposed DeepBDC is distingushed from previous, metric-based few-shot learning (FSL) methods with statistical modeling, and the differences are summarized in the Table below. Our contributions are :

  • For the first time, we introduce Brownian distance covariance, a fundamental but overlooked dependency modeling technqiue, into deep learning.
  • DeepBDC considers joint distributions with no density modeling, highly modular and efficient, fit for meta-training paradigm and pretraining+linear classifier;
  • It pushes state-of-the-art results for few-shot classification, showing great potential of BDC and suggesting its future applications in other computer vision tasks.

To quantify the dependency between random vectors X and Y , moments based FSL methods [20, 33, 44] only model marginal distributions, suffering from limited representatin capability; though achieving state-of-the-art performance by considering joint distributions, DeepEMD [47] is computationally expensive. Our DeepBDC measures discrepancy between joint characteristic function and product of the marginals, which can be efficiently computed in closed-form, and model non-linear relations and fully characterizes independence.

 

Results

Comparison with state-of-the-art FSL methods for 5-way task on (a) general and (b) fine-grained datasets. The best results are in bold black and second-best ones are in red. † Reproduced with our setting.

Comparison with state-of-the-art FSL methods for 5-way task in cross-domain scenarios. We perform meta-training on the whole miniImageNet, while testing on the meta-test set of (a) CUB, (b) Aircraft and (c) Cars, respectively.