Artificial intelligence (AI) and machine learning technologies are being used in high-stakes decision-making systems like lending decision, employment screening, and criminal justice sentencing. A new challenge arising with these AI systems is avoiding the unfairness they might introduce and that can lead to discriminatory decisions for protected classes. Most AI systems use some kinds of thresholds to make decisions. This project aims to improve fairness-aware AI technologies by formulating threshold-agnostic metrics for decision making. In particular, the research team will improve the training procedures of fairness-constrained AI models to make the model adaptive to different contexts, applicable to different applications, and subject to emerging fairness constraints. The success of this project will yield a transferable approach to improve fairness in various aspects of society by eliminating the disparate impacts and enhancing the fairness of AI systems in the hands of the decision makers. Together with AI practitioners, the researchers will integrate the techniques in this project into real-world systems such as education analytics. This project will also contribute to training future professionals in AI and machine learning and broaden this activity by including training high school students and under-represented undergraduates. This project focuses on advancing optimization for threshold-agnostic fair AI systems. The research activities include: (i) developing scalable stochastic optimization algorithms for optimizing a broad family of rank-based threshold-agnostic objectives; (ii) developing novel threshold-agnostic fairness measures including Receiver Operating Characteristic curve (ROC) fairness, Area under the ROC Curve (AUC) fairness, etc. and studying the relationship between them and the existing fairness measures; (iii) developing efficient stochastic methods for in-processing fairness-aware learning methods to directly optimize threshold-agnostic objectives subject to new threshold-agnostic fairness-ensuring constraints; and, (iv) investigating effective end-to-end deep learning framework that not only automatically learns the feature representations, but also satisfies the fairness constraints. The algorithms will be evaluated on multiple tasks, including image recognition, recommendation, spatial-temporal hazard prediction, and predicting students' performance.

 

Students

1. Quanqi Hu, Xiyuan Wei, Gang Li, Yao Yao, Boyang Zhang, Yankun Huang, Ayush Ghosh (high school student)

Publications

1. Xinyu Chen, Bokun Wang, Ming Yang, Quanqi Hu, Qihang Lin, Tianbao Yang. Stochastic Momentum Methods for Non-smooth Non-Convex Finite-Sum Coupled Compositional Optimization. NeurIPS, 2025. (PDF)
2. Ming Yang, Gang Li, Quanqi Hu, Qihang Lin, Tianbao Yang. Single-loop Algorithms for Stochastic Non-convex Optimization with Weakly-Convex Constraints. 2025 (under review). (PDF)
3. Yankun Huang, Qihang Lin, Yangyang Xu. Inexact Moreau Envelope Lagrangian Method for Non-convex Constrained Optimization under Local Error Bound Condition. 2025 (under review). (PDF)
4. Yutian He, Yankun Huang, Yao Yao, Qihang Lin. Enforcing Fairness Where it Matters: An Approach Based on Difference-of-Convex Constraints. 2025 (under review). (PDF)
5. Wei Liu, Qihang Lin, Yangyang Xu. Lower Complexity Bounds of First-order Methods for Affinely Constrained Composite Non-Convex Problem. Mathematics of Operations Research, 2025. (PDF)
6. Boyang Zhang, Quanqi Hu, Mingxuan Sun, Qihang Lin, Tianbao Yang. Learning to Rank with Top- Fairness. Transactions on Machine Learning Research (TMLR), 2025. (PDF)
7. Chenlang Yi, Zizhan Xiong, Qi Qi, Xiyuan Wei, Girish Bathla, Ching-Long Lin, Bobak Jack Mortazavi, Tianbao Yang. AdFair-CLIP: Adversarial Fair Contrastive Language-Image Pre-training for Chest X-rays. The Medical Image Computing and Computer Assisted Intervention (MICCAI), 2025. (PDF)
8. Gang Li, Qihang Lin, Ayush Ghosh, Tianbao Yang. Multi-Output Distributional Fairness via Post-Processing. Transactions on Machine Learning Research (TMLR), 2025. (PDF)
9. Quanqi Hu, Qi Qi, Zhaosong Lu, Tianbao Yang. Single-loop stochastic algorithms for difference of max-structured weakly convex functions. Advances in Neural Information Processing Systems (NeurIPS), 2024. (PDF)
10. Zhishuai Guo, Tianbao Yang. Communication-Efficient Federated Group Distributionally Robust Optimization. NeurPS 2024. (PDF)
11. Qi Qi, Quanqi Hu, Qihang Lin, Tianbao Yang. Provable optimization for adversarial fair self-supervised contrastive learning. Arxiv, 2024. (PDF)
12. Zhishuai Guo, Rong Jin, Jiebo Luo, Tianbao Yang. FeDXL: Provable Federated Learning for Deep X-Risk Optimization. ICML 2023. (PDF)
13. Bokun Wang, Zhuoning Yuan, Yiming Ying, Tianbao Yang. Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning. Journal of Machine Learning Research, 2023. (PDF)
14. Zhishuai Guo, Yan Yan, Zhuoning Yuan, Tianbao Yang. Fast Objective & Duality Gap Convergence for Non-Convex Strongly-Concave Min-Max Problems with PL Condition. Journal of Machine Learning Research, 2023. (PDF)
15. Qi Qi, Jiameng Lyu, Kung sik Chan, Er Wei Bai, Tianbao Yang. Stochastic Constrained DRO with a Complexity Independent of Sample Size. Transactions of Machine Learning Research, 2023. (PDF)
16. Qi Qi, Yi Xu, Rong Jin, Wotao Yin, Tianbao Yang. Attentional-Biased Stochastic Gradient Descent. Transactions of Machine Learning Research, 2023. (PDF)
17. Bang An, Xun Zhou, Yongjian Zhong, Tianbao Yang. SpatialRank: Urban Event Ranking with NDCG Optimization on Spatiotemporal Data. NeurIPS 2023. (PDF)
18. Dixian Zhu, Yiming Ying, Tianbao Yang. Label Distributionally Robust Losses for Multi-class Classification: Consistency, Robustness and Adaptivity. ICML 2023. (PDF)
19. Gang Li, Wei Tong, Tianbao Yang. Maximization of Average Precision for Deep Learning with Adversarial Ranking Robustness. NeurIPS 2023. (PDF)
20. Yao Yao, Qihang Lin, Tianbao Yang. Stochastic methods for auc optimization subject to auc-based fairness constraints. International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. (PDF)
21. Quanqi Hu, Dixian Zhu, Tianbao Yang. Non-Smooth Weakly-Convex Finite-sum Coupled Compositional Optimization NeurIPS, 2023. (PDF)
22. Quanqi Hu, Zi-Hao Qiu, Zhishuai Guo, Lijun Zhang, Tianbao Yang. Blockwise Stochastic Variance-Reduced Methods with Parallel Speedup for Multi-Block Bilevel Optimization. ICML 2023. (PDF)
23. Qi Qi, Shervin Ardeshir, Yi Xu, Tianbao Yang. Fairness via Adversarial Attribute Neighbourhood Robust Learning. Arxiv, 2022. (PDF)

Software

KSO-RED: This implemented the algorithms for learning-to-rank algorithms that optimize both ranking quality (NDCG) and fairness in top-K positions. [Code] [Documentation]

TAB: This implemented the algorithms for Multi-Output Distributional Fairness via Post-Processing. [Code] [Documentation]

ABSGD: Attentional Biased SGD for robust learning. [Code]

Acknowlewdgement and Disclaimer

This material is based upon work supported by the National Science Foundation under Grant No. IIS-2147253. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.