Jiaming Lv*, Haoyuan Yang*, Peihua Li. Wasserstein Distance (WD) Rivals Kullback-Leibler Divergence for Knowledge Distillation. Advances in Neural Information Processing Systems (NeurIPS), 2024. *Equal contribution. [Project][code][pdf][poster][video]

(We extend beyond the classical Kullback-Leibler divergence-based method of knowledge distillation.)

Qilong Wang, Zhaolin Zhang, Mingze Gao, Jiangtao Xie, Pengfei Zhu, Peihua Li, Wangmeng Zuo, Qinghua Hu. Towards a Deeper Understanding of Global Covariance Pooling in Deep Learning: An Optimization Perspective. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2023.

(Analyze second-order pooling from optimization view).

Jiangtao Xie*, Fei Long*, Jiaming Lv, Qilong Wang, Peihua Li. Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classification. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2022. *Equal contribution. (Oral presentation, acceptance rate ~5%) [Project][code][pdf][video]

(For the first time, we introduce Brownian distance covariance (BDC), a powerful yet overlooked similarity/distance measure, into deep learning.)

Qilong Wang, Mingze Gao, Zhaolin Zhang, Jiangtao Xie, Peihua Li, Qinghua Hu. DropCov: A Simple yet Effective Method for Improving Deep Architectures. Advances in Neural Information Processing Systems (NeurIPS), 2022. [Code][pdf]

(The power in Matrix Power Normalization (MPN) tradeoffs generalization and accuracy of 2nd-order pooling.)

Zilin Gao, Qilong Wang, Bingbing Zhang, Qinghua Hu and Peihua Li. Temporal-adaptive Covariance Pooling Networks for Video Recognition. Advances in Neural Information Processing Systems (NeurIPS), 2021. [Code][pdf]

(Action recogniton using temporal attention-based 2nd-pooling.)

Qilong Wang, Jiangtao Xie, Wangmeng Zuo, Lei Zhang and Peihua Li. Deep CNNs Meet Global Covariance Pooling: Better Representation and Generalization. IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), 2021. [Project Page][pdf][bibtex]

(Methodology of Matrix Power Normalized Covariance Pooling Networks.)

Qilong Wang, Li Zhang, Banggu Wu, Dongwei Ren, Peihua Li, Wangmeng Zuo, Qinghua Hu. What Deep CNNs Benefit from Global Covariance Pooling: An Optimization Perspective. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2020. [Code][pdf]

(Matrix Power Normalized Covariance Pooling (MPN-COV) makes optimization landscape more smooth and gradient prediction more stable than global average pooling.)

Zilin Gao, Jiangtao Xie, Qilong Wang and Peihua Li. Global Second-order Pooling Convolutional Networks. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2019. [Code in PyTorch][pdf][bibtex]

(Second-order attention for CNN.)

Qilong Wang, Peihua Li, Qinghua Hu, Pengfei Zhu, Wangmeng Zuo. Deep Global Generalized Gaussian Networks. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), 2019.[pdf][bibtex]

(Generalized Gaussian CNN.)

Peihua Li, Jiangtao Xie, Qilong Wang and Zilin Gao. Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization. IEEE Int. Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 947-955, 2018. [Code in PyTorch][Code in TensorFlow][Code in MatConvNet][pdf][bibtex]

(A fast algorithm for Matrix Power (1/2) Normalized COVariance pooling (MPN-COV).)

Peihua Li, Jiangtao Xie, Qilong Wang and Wangmeng Zuo. Is Second-order Information Helpful for Large-scale Visual Recognition? IEEE Int. Conf. on Computer Vision (ICCV),  pp. 2070-2078, 2017. [Code in MatConvNet][pdf][bibtex]

(Disclose statistical and geometrical insights in Matrix Power Normalized COVariance pooling (MPN-COV), which outperforms gobal average pooling in large-scale recogniton)

Qilong Wang*, Zilin Gao*, Jiangtao Xie, Wangmeng Zuo and Peihua Li. Global Gated Mixture of Second-order Pooling for Improving Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems (NIPS), 2018. *Equal contribution. [Code in PyTorch] [pdf][bibtex]

(Mixture of covariance pooling at the end of CNN)

Qilong Wang, Peihua Li, Lei Zhang. G2DeNet: Global Gaussian Distribution Embedding Network and Its Application to Visual Recognition. Int. Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 2730-2739, 2017. (Oral presentation, acceptance rate 2.7%) [pdf][code][bibtex]

(Plugging Gaussian distribution into deep CNN)