Group GitHub:

• LibAUC: A Deep Learning Library for X-Risk Optimization. Total Downloads: 87k+
  
  
• DisCO: Discriminative Constrained Optimization for Large Reasoning Models. Improves GRPO, DAPO, Dr. GRPO, etc in multiple aspects with significant improvements.
  
  
• DFT: Discriminative Fine-tuning of LLMs without reward models and human preference data. Improves SFT and is Competitive SF-->PO.
  
  
• DRRho-CLIP: Model steering for CLIP training. Beats OpenAI CLIP using 8 GPUs with 2 times less data size (192M vs 400M) and 10 times less sample complexity (1.28b vs 12.8b).
  
  
• SogCLR: Stochastic Optimization for Global Contrastive Learning without Large Mini-batches. Achieves the same performance as SimCLR with 32 times less batch size (256 vs 8192).
  
  

Surveys/Tutorials

• Algorithmic Foundations of Empirical X-risk Minimization
  Tianbao Yang
  
  
• AUC Maximization in the Era of Big Data and AI: A Survey (In ACM Computing Surveys)
  Tianbao Yang, Yiming Ying
  
  
• CVPR'22 Tutorial: Deep AUC Maximization: From Algorithms to Practice
  Tianbao Yang, Yiming Ying, Harikrishna Narasimhan, Mingrui Liu
  
  

Preprints

• (New! ): Myna: Masking-Based Contrastive Learning of Musical Representations.
  Ori Yonay, Tracy Hammond, Tianbao Yang
  
  
• (New! ): A Retention-Centric Framework for Continual Learning with Guaranteed Model Developmental Safety.
  Gang Li, Wendi Yu, Yao Yao, Wei Tong, Yingbin Liang, Qihang Lin, Tianbao Yang
  
  
• (New! ): Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraints.
  Ming Yang, Gang Li, Qihang Lin, Tianbao Yang
  
  
• (New! ): FastCLIP: A Suite of Optimization Techniques to Accelerate CLIP Training with Limited Resources.
  Xiyuan Wei, Fanjiang Ye, Ori Yonay, Xingyu Chen, Baixi Sun, Dingwen Tao, Tianbao Yang
  
  
• (New! ): Provable Optimization for Adversarial Fair Self-supervised Contrastive Learning.
  Qi Qi, Quanqi Hu, Qihang Lin, Tianbao Yang
  
  
• (New! ): Deterministic and Stochastic Accelerated Gradient Method for Convex Semi-Infinite Optimization.
  Yao Yao, Qihang Lin, Tianbao Yang
  
  

Conference/Journal

• (New! ): Model Steering: Learning with a Reference Model Improves Generalization Bounds and Scaling Laws.
  Xiyuan Wei, Ming Lin, Fanjiang Ye, Fengguang Song, Liangliang Cao, My T. Thai, Tianbao Yang
  Accepted to ICML 2025 (spotlight, 2.6%)
  
  
• (New! ): Discriminative Finetuning of Generative Large Language Models without Reward Models and Preference Data.
  Siqi Guo, Ilgee Hong, Vicente Balmaseda, Chonglong Yu, Liang Qiu, Xin Liu, Haoming Jiang, Tuo Zhao, Tianbao Yang
  Accepted to ICML 2025
  
  
• (New! ): Discovering Global False Negatives On the Fly for Self-supervised Contrastive Learning.
  Vicente Balmaseda, Bokun Wang, Ching-Long Lin, Tianbao Yang
  Accepted to ICML 2025
  
  
• (New! ): ALEXR: Optimal Single-Loop Algorithms for Convex Finite-Sum Coupled Compositional Stochastic Optimization
  Bokun Wang, Tianbao Yang
  Accepted to ICML 2025
  
  
• (New! ): Gradient Aligned Regression via Pairwise Losses.
  Dixian Zhu, Tianbao Yang, Livnat Jerby
  Accepted to ICML 2025
  
  
• (New! ): On Discriminative Probabilistic Modeling for Self-Supervised Representation Learning .
  Bokun Wang*, Yunwen Lei, Yiming Ying, Tianbao Yang
  In ICLR 2025
  
  
• (New! ): Communication-Efficient Federated Group Distributionally Robust Optimization.
  Zhishuai Guo*, Tianbao Yang
  In NeurIPS 2024
  
  
• (New! ): Single-loop Stochastic Algorithms for Difference of Max-Structured Weakly Convex Functions.
  Quanqi Hu*, Qi Qi, Zhaosong Lu, Tianbao Yang
  In NeurIPS 2024
  
  
• (New! ): Adaptive Preference Scaling for Reinforcement Learning with Human Feedback.
  Ilgee Hong, Zichong Li, Alexander Bukharin, Yixiao Li, Haoming Jiang, Tianbao Yang, Tuo Zhao
  In NeurIPS 2024
  
  
• (New! ): To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO.
  Zihao Qiu*, Siqi Guo*, Mao Xu, Tuo Zhao, Lijun Zhang, Tianbao Yang
  In ICML 2024
  
  
• (New! ): Stability and Generalization of Stochastic Compositional Gradient Descent Algorithms
  Ming Yang*, Xiyuan Wei*, Tianbao Yang, Yiming Ying
  In ICML 2024
  
  
• (New! ): Efficient and Effective Implicit Dynamic Graph Neural Network.
  Yongjian Zhong, Hieu Vu, Tianbao Yang, Bijaya Adhikari
  In KDD 2024
  
  
• (New! ): Everthing Pertubed All at Once:Enabling Differentiable Graph Attacks.
  Haoran Liu, Bokun Wang, Jianling Wang, Xiangjue Dong, Tianbao Yang, James Caverlee
  In WWW 2024
  
  
• Non-Smooth Weakly-Convex Finite-sum Coupled Compositional Optimization.
  Quanqi Hu*, Dixian Zhu*, Tianbao Yang
  In NeurIPS 2023
  
  
• Maximization of Average Precision for Deep Learning with Adversarial Ranking Robustness.
  Gang Li*, Wei Tong, Tianbao Yang
  In NeurIPS 2023 (Spotlight)
  
  
• Stochastic Approximation Approaches to Group Distributionally Robust Optimization.
  Lijun Zhang, Peng Zhao, Tianbao Yang, Zhi-Hua Zhou
  In NeurIPS 2023
  
  
• Federated Compositional Deep AUC Maximization.
  Xinwen Zhang*, Yihan Zhang*, Tianbao Yang, Richard Souvenir, Hongchang Gao
  In NeurIPS 2023
  
  
• SpatialRank: Urban Event Ranking with NDCG Optimization on Spatiotemporal Data.
  Bang An*, Xun Zhou, Yongjian Zhong*, Tianbao Yang
  In NeurIPS 2023
  
  
• LibAUC: A Deep Learning Library for X-risk Optimization.
  Zhuoning Yuan*, Dixian Zhu*, Zi-Hao Qiu*, Gang Li*, Xuanhui Wang, Tianbao Yang
  In KDD 2023
  
  
• Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization
  Zi-Hao Qiu*, Quanqi Hu*, Zhuoning Yuan*, Denny Zhou, Lijun Zhang, Tianbao Yang
  In ICML 2023
  
  
• Generalization Analysis for Contrastive Representation Learning
  Yunwen Lei, Tianbao Yang, Yiming Ying, Ding-xuan Zhou
  In ICML 2023
  
  
• FeDXL: Provable Federated Learning for Deep X-Risk Optimization
  Zhishuai Guo*, Rong Jin, Jiebo Luo, Tianbao Yang
  In ICML 2023
  
  
• Learning Unnormalized Statistical Models via Compositional Optimization
  Wei Jiang*, Jiayu Qin, Lingyu Wu, Changyou Chen, Tianbao Yang, Lijun Zhang
  In ICML 2023
  
  
• Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization
  Quanqi Hu*, Zi-Hao Qiu, Zhishuai Guo, Lijun Zhang, Tianbao Yang
  In ICML 2023
  
  
• Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity
  Dixian Zhu*, Yiming Ying, Tianbao Yang
  In ICML 2023
  
  
• Provable Multi-instance Deep AUC Maximization with Stochastic Pooling
  Dixian Zhu*, Bokun Wang*, Zhi Chen, Yaxing Wang, Milan Sonka, Xiaodong Wu, Tianbao Yang
  In ICML 2023
  
  
• Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints
  Yao Yao*, Qihang Lin, Tianbao Yang
  In AISTATS 2023
  
  
• Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization
  Quanqi Hu*, Yongjian Zhong*, Tianbao Yang
  NeurIPS 2022
  
  
• Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization
  Wei Jiang*, Gang Li*, Yibo Wang, Linjun Zhang, Tianbao Yang
  NeurIPS 2022
  
  
• Large-scale Optimization of Partial AUC in a Range of False Positive Rates
  Yao Yao*, Qihang Lin, Tianbao Yang
  NeurIPS 2022
  
  
• Smoothed Online Convex Optimization Based on Discounted-Normal-Predictor
  Lijun Zhang, Wei Jiang, Jinfeng Yi, Tianbao Yang
  NeurIPS 2022
  
  
• When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee
  Dixian Zhu*, Gang Li*, Bokun Wang, Xiaodong Wu, Tianbao Yang
  In ICML 2022
  
  
• Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applications
  Bokun Wang*, Tianbao Yang
  In ICML 2022
  
  
• Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance
  Zhuoning* Yuan, Yuexin Wu, Zihao Qiu, Xianzhi Du, Lijun Zhang, Denny Zhou, Tianbao Yang
  In ICML 2022
  
  
• Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence
  Zi-hao Qiu*, Quanqi Hu*, Yongjian Zhong, Lijun Zhang, Tianbao Yang
  In ICML 2022
  
  
• Optimal Algorithms for Stochastic Multi-Level Compositional Optimization
  Wei Jiang*, Bokun Wang*, Yibo Wang, Lijun Zhang, Tianbao Yang
  In ICML 2022
  
  
• GraphFM: Improving Large-Scale GNN Training via Feature Momentumn
  Haiyang Yu*, Limei Wang*, Bokun Wang*, Tianbao Yang, Shuiwang Ji
  In ICML 2022
  
  
• A Simple yet Universal Strategy for Online Convex Optimization
  Lijun Zhang, Guanghui Wang, Jinfeng Yi, Tianbao Yang
  In ICML 2022 as long talk (118/5630=2%)
  
  
• Compositional Training for End-to-End Deep AUC Maximization.
  Zhuoning Yuan*, Zhishuai Guo, Nitesh Chawla, Tianbao Yang.
  In ICLR 2022 as Spotlight (176/3391=5%)
  
  
• Momentum Accelerates the Convergence of Stochastic AUPRC Maximization.
  Guanghui Wang, Ming Yang, Lijun Zhang, Tianbao Yang
  In AISTATS 2022
  
  
• An Online Method for Distributionally Deep Robust Optimization.
  Qi Qi*, Zhishuai Guo*, Yi Xu, Rong Jin, Tianbao Yang
  In NeurIPS 2021
  
  
• Stochastic Optimization of Area Under Precision-Recall Curve for Deep Learning with Provable Convergence.
  Qi Qi*, Youzhi Luo*, Zhao Xu*, Shuiwang Ji, Tianbao Yang
  In NeurIPS 2021
  
  
• Revisiting Smoothed Online Learning.
  Lijun Zhang, Wei Jiang, Shiyin Lu, Tianbao Yang
  In NeurIPS 2021
  
  
• Online Convex Optimization with Continuous Switching Constraint.
  Guanghui Wang, Yuanyu Wan, Tianbao Yang, Lijun Zhang
  In NeurIPS 2021
  
  
• Simple Stochastic and Online Gradient Descent Algorithms for Pairiwise Learning.
  Zhenhuan Yang, Yunwen Lei, Puyu Wang, Tianbao Yang, Yiming Ying
  In NeurIPS 2021
  
  
• Robust Deep AUC Maximization: A New Surrogate Loss and Empirical Studies on Medical Image Classification.
  Zhuoning Yuan*, Yan Yan, Milan Sonka, Tianbao Yang
  In ICCV 2021
  
  
• Federated Deep AUC Maximization for Heterogeneous Data with a Constant Communication Complexity.
  Zhuoning Yuan*, Zhishuai Guo*, Yi Xu, Yiming Ying, Tianbao Yang
  In ICML 2021
  
  
• Stability and Generalization of Stochastic Gradient Methods for Minimax Problems.
  Yunwen Lei, Zhenhuan Yang, Tianbao Yang, Yiming Ying
  In ICML 2021 (long talk)
  
  
• Improved Schemes for Episodic Memory based Lifelong Learning Algorithm.
  Yunhui Guo*, Mingrui Liu*, Tianbao Yang, and Tajana Rosing
  In NeurIPS 2020 (Spotlight acceptance rate: 4%)
  
  
• Decentralized Parallel Algorithm for Training Generative Adversarial Nets.
  Mingrui Liu*, Youssef Mroueh, Wei Zhang, Xiaodong Cui, Jerret Ross, Tianbao Yang, Payel Das
  In NeurIPS 2020
  
  
• Optimal Epoch Stochastic Gradient Descent Ascent Methods for Min-Max Optimization.
  Yan Yan*, Yi Xu, Qihang Lin, Wei Liu, Tianbao Yang
  In NeurIPS 2020
  
  
• Accelerating Deep Learning with Millions of Classes.
  Zhuoning Yuan*, Zhishuai Guo*, Xiaotian Yu, Xiaoyu Wang, Tianbao Yang
  In ECCV 2020
  
  
• A Simple and Effective Framework for Pairwise Deep Metric Learning.
  Qi Qi*, Yan Yan*, Xiaoyu Wang, Tianbao Yang
  In ECCV 2020
  
  
• Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks.
  Zhishuai Guo*, Mingrui Liu*, Zhuoning Yuan*, Li Shen, Wei Liu, Tianbao Yang
  In ICML 2020
  
  
• Stochastic Optimization for Non-convex Inf-Projection Problems.
  Yan Yan*, Yi Xu*, Lijun Zhang, Xiaoyu Wang, Tianbao Yang
  In ICML 2020
  
  
• Proximally Constrained Methods for Weakly Convex Optimization with Weakly Convex Constraints.
  Runchao Ma, Qihang Lin, Tianbao Yang
  In ICML 2020
  
  
• Minimizing Dynamic Regret and Adaptive Regret Simultaneously.
  Lijun Zhang, Shiyin Lu, Tianbao Yang
  In AISTATS 2020
  
  
• Towards Better Understanding of Adaptive Gradient Algorithms in Generative Adversarial Nets.
  Mingrui Liu*, Youssef Mroueh, Jerret Ross, Wei Zhang, Xiaodong Cui, Payel Das, Tianbao Yang
  In ICLR 2020
  
  
• Stochastic AUC Maximization with Deep Neural Networks.
  Mingrui Liu*, Zhuoning Yuan*, Yiming Ying, Tianbao Yang
  In ICLR 2020
  
  
• Adversarial Localized Energy Networks for Structured Prediction.
  Pingbo Pan, Ping Liu, Yan Yan, Tianbao Yang, Yi Yang
  In AAAI 2020
  
  
• Deep Unsupervised Binary Coding Networks for Multivariate Time Series Retrieval.
  Dixian Zhu, Dongjin Song, Yuncong Chen, Cristian Lumezanu, Wei Cheng, Bo Zong, Jingchao Ni, Takehiko Mizoguchi, Tianbao Yang, Haifeng Chen
  In AAAI 2020
  
  
• Why Does Stagewise Training Accelerate Convergence of Testing Error Over SGD?.
  Zhuoning Yuan*, Yan Yan*, Rong Jin, Tianbao Yang
  In NeurIPS 2019
  
  
• Non-asymptotic Analysis of Stochastic Methods for Non-Smooth Non-Convex Regularized Problems.
  Yi Xu*, Rong Jin, Tianbao Yang
  In NeurIPS 2019
  
  
• Katalyst: Boosting Convex Katayusha for Non-Convex Problems with a Large Condition Number.
  Zaiyi Chen*, Yi Xu*, Haoyuan Hu, Tianbao Yang
  In ICML 2019
  
  
• Stochastic Optimization for DC Functions and Non-smooth Non-convex Regularizers with Non-asymptotic Convergence.
  Yi Xu*, Qi Qi*, Qihang Lin, Rong Jin, Tianbao Yang
  In ICML 2019
  
  
• On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Optimization.
  Yi Xu*, Zhuoning Yuan*, Sen Yang, Rong Jin, Tianbao Yang
  In IJCAI 2019
  
  
• Learning with Non-Convex Truncated Losses by SGD.
  Yi Xu*, Shenghuo Zhu, Sen Yang, Chi Zhang, Rong Jin, Tianbao Yang
  In UAI 2019
  
  
• EIGEN: Ecologically-Inspired GENetic Approach for Neural Network Structure Searching from Scratch.
  Jian Ren, Zhe Li*, Jianchao Yang, Ning Xu, Tianbao Yang, David Foran
  In CVPR 2019
  
  
• A Robust Zero-Sum Game Framework for Pool-based Active Learning.
  Dixian Zhu*, Zhe Li, Xiaoyu Wang, Boqing Gong, Tianbao Yang
  In AISTATS 2019
  
  
• Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions.[Arxiv Version]
  Zaiyi Chen*, Zhuoning Yuan*, Jinfeng Yi, Bowen Zhou, Enhong Chen, Tianbao Yang
  In ICLR 2019
  
  
• Faster Online Learning of Optimal Threshold for Consistent F-measure Optimization.
  Xiaoxuan Zhang*, Mingrui Liu*, Xun Zhou, Tianbao Yang
  In NeurIPS 2018
  
  
• Adaptive Negative Curvature Descent with Applications in Non-convex Optimization.
  Mingrui Liu*, Zhe Li*, Xiaoyu Wang, Jinfeng Yi, Tianbao Yang
  In NeurIPS 2018
  
  
• First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time.
  Yi Xu*, Rong Jin, Tianbao Yang
  In NeurIPS 2018
  
  
• Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions.
  Mingrui Liu*, Xiaoxuan Zhang*, Lijun Zhang, Rong Jin, Tianbao Yang
  In NeurIPS 2018
  
  
• How Local is the Local Diversity?
  Yandong Li, Liqiang Wang, Tianbao Yang, Boqing Gong
  To Appear in ECCV 2018
  
  
• Improving Sequential Determinantal Point Processes for Supervised Video Summarization
  Aidean Sharghi, Ali Borji, Chengtao Li, Tianbao Yang, Boqing Gong
  To Appear in ECCV 2018
  
  
• Fast Stochastic AUC Maximization with O(1/n)-Convergence Rate [Supplement]
  Mingrui Liu*, Xiaoxuan Zhang*, Zaiyi Chen, Xiaoyu Wang, Tianbao Yang
  In ICML 2018
  
  
• SADAGRAD: Strongly Adaptive Stochastic Gradient Methods [Supplement]
  Zaiyi Chen*, Yi Xu*, Enhong Chen, Tianbao Yang
  In ICML 2018
  
  
• Level-Set Methods for Finite-Sum Constrained Convex Optimization [Supplement]
  Qihang Lin, Runchao Ma, Tianbao Yang
  In ICML 2018
  
  
• Strongly Adaptive Regret Implies Optimally Dynamic Regret
  Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
  In ICML 2018
  
  
• A Deep Learning Approach to Traffic Accident Prediction on Heterogeneous Spatio-Temporal Data
  Zhuoning Yuan*, Xun Zhou, Tianbao Yang
  In KDD 2018
  
  
• A Unified Analysis of Stochastic Momentum Methods for Deep Learning [arxiv version]
  Yan Yan*, Tianbao Yang, Zhe Li, Qihang Lin, Yi Yang
  In IJCAI 2018
  
  
• A Generic Approach for Accelerating Stochastic Zeroth-Order Convex Optimization
  Xiaotian Yu*, Irwin King, Michael R. Lyu, Tianbao Yang
  In IJCAI 2018
  
  
• A Simple Analysis for Exp-concave Empirical Minimization with Arbitrary Convex Regularizer.
  Tianbao Yang, Zhe Li, Lijun Zhang
  In AISTATS 2018, Oral Presentation
  
  
• Improved dynamic regret for non-degeneracy functions. [Supplement]
  Lijun Zhang, Tianbao Yang, Jinfeng Yi, Rong Jin, Zhi-Hua Zhou
  In NIPS 2017
  
  
• Adaptive Accelerated Gradient Converging Methods under Holderian Error Bound Condition. [Supplement]
  Mingrui Liu*, Tianbao Yang
  In NIPS 2017
  
  
• Adaptive SVRG Methods under Error Bound Conditions with Unknown Growth Parameter. [ Supplement ]
  Yi Xu*, Qihang Lin, Tianbao Yang
  In NIPS 2017
  
  
• ADMM without a Fixed Penalty Parameter: Faster Convergence with New Adaptive Penalization. [ Supplement ]
  Yi Xu*, Mingrui Liu*, Qihang Lin, Tianbao Yang
  In NIPS 2017
  
  
• Empirical Risk Minimization for Stochastic Convex Optimization: O(1/n)- and O(1/n^2)-type of Risk Bounds.
  with Lijun Zhang, Rong Jin
  In COLT 2017
  
  
• A Richer Theory of Convex Constrained Optimization with Reduced Projections and Improved Rates. [ with Supplement ]
  Tianbao Yang, Qihang Lin, Lijun Zhang
  In ICML 2017
  
  
• Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence. [Supplement][ arXiv version ]
  Yi Xu*, Qihang Lin, Tianbao Yang
  In ICML 2017
  
  
• SVD-free Convex-Concave Approaches for Nuclear Norm Regularization.
  Yichi Xiao*, Zhe Li*, Tianbao Yang, Lijun Zhang
  In IJCAI 2017
  
  
• Efficient Non-oblivious Randomized Reduction for Risk Minimization with Improved Excess Risk Guarantee [arXiv version]
  Yi Xu*, Haiqing Yang, Lijun Zhang, Tianbao Yang
  In AAAI 2017
  
  
• A Framework of Online Learning with Imbalanced Streaming Data
  Yan Yan*, Tianbao Yang, Yi Yang, Jianhui Chen
  In AAAI 2017
  
  
• A Two-stage Approach for Learning a Sparse Model with Sharp Excess Risk Analysis
  Zhe Li*, Tianbao Yang, Lijun Zhang, Rong Jin
  In AAAI 2017
  
  
• Improved Dropout for Shallow and Deep Learning.
  Zhe Li*, Boqing Gong, Tianbao Yang
  In NIPS 2016
  
  
• Homotopy Smoothing for Non-Smooth Problems with Lower Complexity than O(1/ε).[arXiv version] [spotlight video]
  Yi Xu*, Yan Yan*, Qihang Lin, Tianbao Yang
  In NIPS 2016
  
  
• Sparse Learning for Large-scale and High-dimensional Data: A Randomized Convex-concave Optimization Approach
  Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
  In ALT, 2016
  
  
• Online Asymmetric Active Learning with Imbalanced Data
  Xiaoxuan Zhang*, Tianbao Yang, Padmini Srinivasan
  In KDD, 2016
  
  
• Optimal Stochastic Strongly Convex Optimization with a Logarithmic Number of Projections
  with Jianhui Chen, Qihang Lin, Lijun Zhang, Yi Chang
  In UAI as Plenary Presentation, 2016
  
  
• Tracking Slowly Moving Clairvoyant: Optimal Dynamic Regret of Online Learning with True and Noisy Gradient
  Tianbao Yang, Lijun Zhang, Rong Jin, Jinfeng Yi
  In ICML, 2016
  
  
• Online Stochastic Linear Optimization under One-bit Feedback
  Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
  In ICML, 2016
  
  
• Learning Attributes Equals Multi-Source Domain Generalization
  Chuang Guan*, Tianbao Yang, Boqing Gong
  In CVPR as Spotlight, 2016
  
  
• Stochastic Optimization for Kernel PCA
  Lijun Zhang, Tianbao Yang, Rong Jin and Zhi-Hua Zhou
  In AAAI, 2016
  
  
• Fast and Accurate Refined Nystrom based Kernel SVM
  Zhe Li*, Tianbao Yang, Lijun Zhang and Rong Jin.
  In AAAI, 2016
  
  
• Big Data Analytics: Optimization and Randomization
  Tianbao Yang, Qihang Lin, Rong Jin
  Tutorial In KDD, 2015
  
  
• An Efficient Semi-Supervised Clustering Algorithm with Sequential Constraints
  Jinfeng Yi, Lijun Zhang, Tianbao Yang, Wei Liu and Jun Wang
  In KDD, 2015
  
  
• Theory of Dual-sparse Regularized Randomized Reduction
  Tianbao Yang, Lijun Zhang, Rong Jin, Shenghuo Zhu
  In ICML, 2015
  
  
• An Explicit Sampling Dependent Spectral Error Bound for Column Subset Selection
  Tianbao Yang, Lijun Zhang, Rong Jin, Shenghuo Zhu
  In ICML, 2015
  
  
• Hyper-class Augmented and Regularized Deep Learning for Fine-grained Image Classification
  Saining Xie*, Tianbao Yang, Xiaoyu Wang, Yuanqing Lin. (*student author)
  In CVPR, 2015
  
  
• A Simple Homotopy Algorithm for Compressive Sensing
  with Lijun Zhang, Rong Jin, Zhi-Hua Zhou
  In AISTATS, 2015
  
  
• Online Bandit Learning with Non-convex Losses
  with Lijun Zhang, Rong Jin, Zhi-Hua Zhou
  In AAAI, 2015
  
  
• Extracting Certainty from Uncertainty: Transductive Pairwise Classification from Pairwise Similarities [Supplement]
  with Rong Jin
  In NIPS, 2014
  
  
• Efficient Low-Rank Stochastic Gradient Descent Methods for Solving Semi-definite Programming
  with Jianhui Chen, Shenghuo Zhu
  In AISTATS, 2014
  
  
• Trading Computation for Communication: Distributed Stochastic Dual Coordinate Ascent [supplement]
  Tianbao Yang
  In NIPS, 2013
  (This work proposes the local updates and model averaging for distributed training of SVM and establishes the first non-asymptotic convergence, which directly motivates the development of Federated Learning Paradigm. This was acknowledged in this paper that coines the term "Federated Optimization")
  
  
• Stochastic Convex Optimization with Multiple Objectives
  with Mehrdad Mahdavi and Rong Jin
  In NIPS, 2013
  
  
• Recoverying Optimal Solution by Dual Random Projection
  with Lijun Zhang, Mehrdad Mahdavi, Rong Jin and Shenghuo Zhu
  In COLT, 2013.
  
  
• O(logT) Projections for Stochastic Optimization of Smooth and Strongly Convex Functions
  with Lijun Zhang, Rong Jin and Xiaofei He
  In ICML, 2013.
  
  
• Robust Ensemble Clustering by Matrix Completion
  Jinfeng Yi, Tianbao Yang, Rong Jin, Anil Jain
  In ICDM, 2012.
  
  
• Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning
  Jinfeng Yi, Rong Jin, Anil Jain, Shaili Jain, Tianbao Yang
  In NIPS, 2012.
  
  
• Nystrom Method vs Random Fourier Features: A Theoretical and Empirical Comparison [supplement]
  Tianbao Yang, Yu-Feng Li, Mehrdad Mahdavi, Rong Jin, Zhi-Hua Zhou
  In NIPS, 2012.
  
  
• Stochastic Gradient Descent with Only One Projection [Supplement] [Note]
  with Mehrdad Mahdavi, Rong Jin, Shenghuo Zhu and Jinfeng Yi
  In NIPS, 2012.
  
  
• A Simple Algorithm for Semi-supervised Learning with Improved Generalization Error Bound [Supplement]
  with Ming Ji, Binbin Lin, Rong Jin, Jiawei Han
  In ICML 2012
  
  
• Multiple Kernel Learning from Noisy Labels by Stochastic Programming [bibtex]
  Tianbao Yang, Mehrdad Mahdavi, Rong Jin, Lijun Zhang, Yang Zhou
  In ICML 2012
  
  
• Online Optimization with Gradual Variations [bibtex]
  with Chao-Kai Chiang, Chia-Jung Lee, Mehrdad Mahdavi, Chi-Jen Lu, Rong Jin, Shenghuo Zhu
  In COLT 2012. Best Student Paper Award.
  This paper is a merged paper of two COLT submissions. Our manuscript proposes two algorithms for online convex optimization
  to obtain a variation based regret bound. One of our algorithms is the same to the algorithm proposed in the other submission.
  A commmentary on the merged paper written by Satyen Kale can be found here.
  
  
  
• Online Kernel Selection: Algorithms and Evaluations [bibtex]
  Tianbao Yang, Mehrdad Mahdavi, Rong Jin, Jinfeng Yi, Steven Hoi
  In AAAI 2012 (oral presentation)
  
  
• Online AUC Maximization [bibtex]
  Peilin Zhao, Steven Hoi, Rong Jin, Tianbao Yang
  In ICML 2011
  
  
• A Kernel Density Based Approach for Large Scale Image Retrieval [bibtex]
  Wei Tong, Fengjie Li, Tianbao Yang, Rong Jin, Anil Jain
  In ICMR 2011
  
  
• Online Multiple Kernel Learning: Algorithms and Mistake Bounds [bibtex]
  with Rong Jin and Steven Hoi
  In ALT 2010
  
  
• Unsupervised Transfer Classification: Application to Text Categorization [bibtex]
  Tianbao Yang, Rong Jin, Anil Jain, Yang Zhou, Wei Tong
  In KDD 2010
  
  
• Learning from Noisy Side Information by Generalized Maximum Entropy Model [Supplementary Material][bibtex]
  Tianbao Yang, Rong Jin, Anil Jain
  In ICML 2010
  
  
• Directed Network Community Detection: A Popularity and Productivity Link Model [bibtex]
  Tianbao Yang, Yun Chi, Shenghuo Zhu, Yihong Gong, Rong Jin
  In SDM 2010.
  
  
• A Bayesian Framework for Community Detection Integrating Content and Link [bibtex]
  Tianbao Yang, Rong Jin, Yun Chi, Shenghuo Zhu
  In UAI 2009.
  
  
• Combining Link and Content for Community Detection: A Discriminative Approach [Codes][bibtex]
  Tianbao Yang, Rong Jin, Yun Chi, Shenghuo Zhu
  In KDD 2009.
  
  
• A Bayesian Approach Toward Finding Communities and Their Evolutions in Dynamic Social Networks [bibtex]
  Tianbao Yang, Yun Chi, Shenghuo Zhu, Yihong Gong, Rong Jin
  In SDM 2009.
  
  

Journal Publications

• (New! ): Multi-Output Distributional Fairness via Post-Processing.
  Gang Li*, Qihang Lin, Ayush Ghosh, Tianbao Yang
  TMLR
  
  
• (New! ): Unified Convergence Analysis of Adaptive Optimization with Moving Average Estimator.
  Zhishuai Guo*, Yi Xu, Wotao Yin, Rong Jin, Tianbao Yang
  ML
  
  
• (New! ): Stochastic Constrained DRO with a Complexity Independent of Sample Size
  Qi Qi, Jiameng Lyu, Kung sik Chan, Er Wei Bai, Tianbao Yang
  In TMLR
  
  
• (New! ): Attentional Biased Stochastic Gradient for Imbalanced Classification.
  Qi Qi*, Yi Xu, Rong Jin, Wotao Yin, Tianbao Yang
  In TMLR
  
  
• Fast Objective and Duality Gap Convergence for Non-convex Strongly-concave Min-max Problems.
  Zhishuai Guo*, Zhuoning Yuan*, Yan Yan*,Tianbao Yang
  In JMLR
  
  
• Memory-based Optimization Methods for Model-Agnostic Meta-Learning.
  Bokun Wang, Zhuoning Yuan, Yiming Ying, Tianbao Yang
  In JMLR
  
  
• Contrastive Learning and Subtyping of Post-Covid-19 Lung Computed Tomography Images.
  Frank Li, Xuan Zhang, Alejandro Pierre Comellas, Eric Hoffman, Tianbao Yang, Ching-long Lin
  In Frontiers in Physiology
  
  
• Advanced Graph and Sequence Neural Networks for Molecular Property Prediction and Drug Discovery. (including the winning result at MIT AICURES Challenge by Deep AUC Maximization)
  Zhengyang Wang, Meng Liu, Youzhi Luo, Zhao Xu, Yaochen Xie, Limei Wang, Lei Cai, Qi Qi, Zhuoning Yuan, Tianbao Yang, Shuiwang Ji
  In Bioinformatics
  
  
• First-order Convergence Theory for Weakly-Convex-Weakly-Concave Min-max Problems.
  Mingrui Liu, Hassan Rafique, Qihang Lin, Tianbao Yang
  In JMLR
  
  
• Non-Convex Min-Max Optimization: Provable Algorithms and Applications in Machine Learning.
  Hassan Rafique*, Mingrui Liu*, Qihang Lin, Tianbao Yang
  In Optimization Methods and Software
  
  
• A Data Efficient and Feasible Level Set Method for Stochastic Convex Optimization with Expectation Constraints.
  Qihang Lin, Selvaprabu Nadarajah, Negar Soheili, Tianbao Yang
  In JMLR, 2020
  
  
• Advancing Non-Convex and Constrained Learning: Challenges and Opportunities.
  Tianbao Yang
  AI Matters, 2019
  
  
• High-dimensional model recovery from random sketched data by exploring intrinsic sparsity.
  Tianbao Yang, Lijun Zhang, Qihang Lin, Shenghuo Zhu, Rong Jin
  In ML, 2019
  
  
• Accelerate Stochastic Subgradient Method by Leveraging Local Growth Condition.
  Yi Xu, Qihang Lin, Tianbao Yang
  In Analysis and Applications, 2019
  
  
• Analysis of Nuclear Norm Regularization for Full-rank Matrix Completion.
  Lijun Zhang, Tianbao Yang, Rong Jin, Zhi-Hua Zhou
  In JMLR, 2019
  
  
• A Simple Homotopy Proximal Mapping for Compressive Sensing.
  Tianbao Yang, Lijun Zhang, Rong Jin, Shenghuo Zhu, Zhi-Hua Zhou
  In Machine Learning (ML) Journal, 2018
  
  
• Continuous amperometric hydrogen gas sensing in ionic liquids.
  Yongan Tang, Jianxin He, Xiaoli Gao, Tianbao Yang, Xiangqun Zeng
  In Analyst, 2018
  
  
• RSG: Beating Subgradient Method without Smoothness and/or Strong Convexity.
  Tianbao Yang, Qihang Lin
  In Journal of Machine Learning Research (JMLR), 2018
  
  
• Distributed Stochastic Variance Reduced Gradient Methods by Sampling Extra Data with Replacement.
  Jason Lee, Qihang Lin, Tengyu Ma, Tianbao Yang
  In Journal of Machine Learning Research (JMLR), 2017
  
  
• On Data Preconditioning for Regularized Loss Minimization [bibtex]
  Tianbao Yang, Rong Jin, Shenghuo Zhu and Qihang Lin
  In ML, 2015
  
  
• Random Projections for Classification: A Recovery Approach
  with Lijun Zhang, Mehrdad Mahdavi and Rong Jin and Shenghuo Zhu
  In IEEE Transactions on Information Theory, 2014
  
  
• An Efficient Primal-Dual Prox Method for Non-Smooth Optimization [bibtex]
  Tianbao Yang, Mehrdad Mahdavi and Rong Jin and Shenghuo Zhu
  In ML, 2014
  
  
• Regret Bounded by Variation for Online Convex Optimization
  Tianbao Yang, Mehrdad Mahdavi, Rong Jin and Shenghuo Zhu
  In ML, 2014. The conference version received the Best Student Paper Award at COLT'12.
  
  
• Combining a popularity-productivity stochastic block model with a discriminative content model for detecting general structures
  Bian-fang Chai, Jian Yu, Cai-yan Jia, Tianbao Yang, Ya-wen Jiang
  In Physical Review E., 2013
  
  
• Improved Bound for the Nystrom's Method and its Application to Kernel Classification
  with Rong Jin, Mehrdad Mahdavi and Yu-Feng Li and Zhi-Hua Zhou
  In IEEE Information Theory, 2013
  
  
• Trading Regret for Efficiency: Online Convex Optimization with Long Term Constraints
  with Mehrdad Mahdavi and Rong Jin
  In JMLR, 2012.
  
  
• Online Multiple Kernel Classification
  Steven Hoi, Rong Jin, Peilin Zhao, Tianbao Yang
  In ML, 2012.
  
  
• Detecting Communities and Their Evolutions in Dynamic Social Networks: A Bayesian Approach [Codes][bibtex]
  Tianbao Yang, Yun Chi, Shenghuo Zhu, Yihong Gong, Rong Jin
  In ML 2011
  
  

Technical Reports

• Benchmarking Deep AUROC Optimization: Loss Functions and Algorithmic Choices
  Dixian Zhu, Xiaodong Wu, Tianbao Yang
  
  
• A Novel Convergence Analysis for Algorithms of the Adam Family and Beyond (presented at NeurIPS 2021 OPT workshop, this is an extended abtract)
  Zhishuai Guo, Yi Xu, Wotao Yin, Rong Jin, Tianbao Yang
  
  
• Randomized Stochastic Variance-Reduced Methods for Stochastic Bilevel Optimization.
  Zhishuai Guo, Quanqi Hu, Lijun Zhang, Tianbao Yang
  
  
• (TR): Adam+: A Stochastic Method with Adaptive Variance Reduction.
  Mingrui Liu, Wei Zhang, Francesco Orabona, Tianbao Yang
  
  
• (TR): Stochastic Primal-Dual Algorithms with Faster Convergence than $O(1/\sqrt{T})$ for Problems without Bilinear Structure.
  Yan Yan, Yi Xu, Qihang Lin, Lijun Zhang, Tianbao Yang
  
  
• (TR): Revisiting SGD with Increasingly Weighted Averaging: Optimization and Generalization Perspectives.
  Zhishuai Guo, Yan Yan, Tianbao Yang
  
  
• (TR): Doubly Stochastic Primal-Dual Coordinate Method for Empirical Risk Minimization and Bilinear Saddle-Point Problem.
  Adams Wei Yu, Qihang Lin, Tianbao Yang
  
  
• (TR): Frank-Wolfe Method is Automatically Adaptive to Error Bound Condition
  Yi Xu, Tianbao Yang
  
  
• (TR): On Noisy Negative Curvature Descent: Competing with Gradient Descent for Faster Non-convex Optimization.
  Mingrui Liu, Tianbao Yang
  (note: a short version was accepted to NIPS 2018)
  
  
• (TR): Unified Convergence Analysis of Stochastic Momentum Methods for Convex and Non-convex Optimization.
  Tianbao Yang, Qihang Lin, Zhe Li
  (note: a short version was accepted to IJCAI 2018)
  
  
• (TR): Analysis of Distributed Stochastic Dual Coordinate Ascent
  Tianbao Yang, Shenghuo Zhu, Rong Jin, Yuanqing Lin
  
  
• (TR): An Improved Bound for the Nystrom Method for Large Eigengap
  with Rong Jin, Mehrdad Mahdavi
  
  
• (TR): Efficient Constrained Regret Minimization
  with Mehrdad Mahdavi, Rong Jin
  
  

Workshop and Other Publications

• (New! ): AUC-Mixup: Deep AUC Maximization with Mixup (In NeurIPS workshop)
  Jianzhi Xu, Gang Li, Tianbao Yang
  
  
• (New! ): Deep AUC Maximization for Medical Image Classification: Challenges and Opportunities. (presented at Medical Imaging Meets NeurIPS 2021 workshop)
  Tianbao Yang
  
  
• SEP-Nets: Small and Effective Pattern Networks.
  Zhe Li*, Xiaoyu Wang, Xutao Lv, Tianbao Yang
  In Beyond ImageNet Large-Scale Visual Recognition Challenge 2017 Workshop
  
  
• A two-stage approach for learning a sparse model with sharp excess risk analysis
  Zhe Li, Tianbao Yang, Lijun Zhang, Rong Jin
  In NIPS workshop on Learning faster from easy data, 2015.
  
  
• Doubly Stochastic Primal-Dual Coordinate Method for Regularized Empirical Risk Minimization with Factorized Data
  with Adams Wei Yu, Qihang Lin
  In NIPS workshop on Optimization for Machine Learning, 2015.
  
  
• A Stochastic Gradient Method with Linear Convergence Rate for a Class of Non-smooth Non-strongly Convex Optimizations
  with Qihang Lin
  In NIPS workshop on Optimization for Machine Learning, 2015.
  [Note: The workshop version has an erroneous proof. Please see our new version with broader results here . ]
  
  
• In-situ Measurement and Prediction of Hearing Aid Outcomes Using Mobile Phones
  Syed Shabih Hasan, Ryan B. Brummet, Octav Chipara, Yu-Hsiang Wu, Tianbao Yang,
  In International Workshop on Smart and Connected Health (IWSCH 2015), 2015.
  
  
• A Directed Inference Approach towards Multi-class Multi-model Fusion
  Tianbao Yang, Lei Wu and Piero P. Bonissone
  In Multiple Classifier Systems, 2013.
  
  
• Community Detection by Popularity Based Models for Authored Networked Data
  Tianbao Yang, Prakash Mandaym Comar and Linli Xu
  In ASONAM, 2013.
  
  
• Combining Link and Content for Community Detection
  Tianbao Yang, Rong Jin, Yun Chi, Shenghuo Zhu
  In Encyclopedia of Social Network Analysis and Mining, Springer Verlag, 2013. (to appear)
  
  
• Online Stochastic Optimization with Multiple Objectives
  with Mehrdad Mahdavi, Rong Jin
  In NIPS workshop on Optimization for Machine Learning, 2012.
  
  
• Online Decision Making Under Stochastic Constraints
  with Mehrdad Mahdavi, Rong Jin
  In NIPS workshop on Discrete Optimization in Machine Learning, 2012.
  
  
• Learning Kernel Combination from Noisy Pairwise Constraints [bibtex]
  Tianbao Yang, Rong Jin, Anil K. Jain
  In IEEE SSP 2012
  
  
• Co-training for Large Scale Image Classification: An Online Approach [bibtex]
  Wei Tong, Tianbao Yang, Rong Jin, Anil Jain
  In ICPR 2010 workshop on Analysis and Evaluation Large-scale Multimedia.