Raed Al Kontar
Publications | News | Group | Code & Talks
I lead a data science group at the Industrial & Operations Engineering (IOE) department at the University of Michigan. I am also an affiliate with both the Michigan Institutes for Data Science and Computational Discovery and Engineering. I work on developing data science methods for solving engineering problems. I also enjoy occasional theoretical endeavors beyond application.
My focus is on collaborative, distributed, and federated data analytics. Currently, my research aims to answer three questions:
- Descriptive: How to extract what is shared and unique across datasets?
- Predictive: How can multiple entities (such as hospitals or IoT devices) collaboratively improve the predictive power of their models while keeping their personal data private?
- Prescriptive: How to fast-track and improve optimal design by effectively distributing trial & error efforts across collaborating entities?
Check our paper “The Internet of Federated Things (IoFT),” which has inspired this research.
Outside of work, I enjoy soccer and mountain hikes.
Highlighted Paper
Personalized PCA: Decoupling Shared and Unique Features [LINK]
To understand heterogeneity across different data sources, we revisit one of the most fundamental and successful data analytics tools: principal component analysis (PCA). Specifically, we decompose heterogenous datasets into functions of global principal components (PCs) shared by all and local PCs unique to each dataset. At the heart of our approach is enforcing orthogonality across the shared and unique PCs. The orthogonality of PCs implies that the shared and unique features span different subspaces, thus describing different patterns in the data sources.
Unlike traditional PCA, such a model is not always identifiable: take the simple case where all datasets are the same, then decoupling the shared and unique features is impossible. Here we provide an identifiability condition based on the misalignment of local components. Roughly speaking, misalignment characterizes heterogeneity through the “minimal difference” among the subspaces spanned by the local components. Turns out that if local subspaces are slightly misaligned, our model becomes identifiable. What’s more interesting is that the estimation error gets smaller as heterogeneity gets more significant, and finding the PCs gets easier. This result lies in sharp contrast to all distributed and federated predictive theory, as it highlights that heterogeneity can be a blessing in disguise.
Youtube Channel
Check out our YouTube channel for talks on our recent work.
Teaching
- IOE 691: Modern Bayesian Data Science
- STAT/IOE 570: Design of Experiments
- IOE 265: Probability & Statistics for Engineers
Education
- Ph.D, Industrial and Systems Engineering, University of Wisconsin Madison, July 2018
- M.S, Statistics, University of Wisconsin Madison, May 2017
- B.E, Civil and Environmental Engineering, American University of Beirut, May 2014
Latest News
Sep 2023: Our group is a finalist at both the Quality, Statistics & Reliability (QSR) and Data Mining (DM) section at this year’s INFORMS. If you wish to attend, see details below:
- Best Paper Competition at QSR: Monday, October 16, 2023, 10:45 AM – 12:00 PM at CC-North 227C
- Best Paper Competition at DM: Monday, October 15, 2023, 2:15PM – 3:30 PM at CC-West 104A
- May 2023: Ph.D student Xubo Yue, just accepted an assistant professor job at Northeastern University
- May 2023: Ph.D student Seokyhun Chung just accepted an assistant professor job at the University of Virginia
- 2022: Our Lab received an NSF CAREER award titled: “From the Cloud to the Crowd: An Enabling Solution for the Internet of Federated Things”
- 2022: Our recent NIH grant was featured on the National Library of Medicine’s Youtube channel: check out the video
Group
- Xubo (Max) Yue: maxyxb@umich.edu, 2018-2023
- Upcoming Assistant Professor at Northeastern University
- Seokhyun Chung: seokhc@umich.edu, 2018-2023
- Upcoming Assistant Professor at the University of Virginia
- Naichen Shi: naichens@umich.edu, 2020 – present
- Qiyuan Chen: cqiyuan@umich.edu, 2022 – present
The Internet of Federated Things (IoFT)
- A critical change is happening in today’s Internet of Things (IoT). The computational power at the edge devices is steadily increasing. Mobile phones’ processing power is becoming comparable to laptops available for everyday use. Powerful AI chips now drive autopilot systems for new electric vehicles, and small local computers such as Raspberry Pis have become commonplace in manufacturing systems. This change opens a new paradigm of data analytics within IoT that exploits edge compute resources to distribute model learning and process more of the client’s data where it is created; at the edge.
- The Internet of Federated Things (IoFT) paper is a joint effort across faculty from multiple universities and expertise were we provide a vision for this new IoT paradigm along with a systematic overview of current efforts toward realizing this vision. The paper features a data directory for IoFT-based datasets. Real-life datasets where distributed and federated analytics can be tested are provided: IoFT Data. A comprehensive talk on IoFT can be found on YouTube Video
Publications
1PhD student advised by me
*Note: We are now building our Github repository and Youtube channel. As such, some videos and codes are pending.
Personalized PCA: Decoupling Shared and Unique Features
Naichen Shi1, Raed Kontar
[Preprint, Video pending, Code]
Personalized Tucker Decomposition: Modeling Commonality and Peculiarity on Tensor Data
Jiuyun Hu, Naichen Shi1, Raed Kontar, Hao Yan
[Preprint, Code]
Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling
Xubo Yue1, Raed Kontar
[Preprint, Video pending, Code]
Best student paper award finalist, physical and engineering sciences, joint statistical meeting (JSM), 2023
Best student paper award finalist, data mining (DM) section, INFORMS annual meeting, 2022
Heterogeneous Matrix Factorization: When Features Differ by Dataset
Naichen Shi1, Raed Kontar, Salar Fattahi
[Preprint, Video pending, Code]
Collaborative and Distributed Bayesian Optimization via Consensus: Showcasing the Power of Collaboration for Optimal Design
Xubo Yue1, Raed Kontar, Albert Berahas, Yang Liu, Zhenghao Zai, Kevin Edgar, Blake N. Johnson
[Preprint, Video pending, Code]
Rethinking Cost Sensitive Classification in Deep Learning via Adversarial Data Augmentation
Qiyuan Chen1, Raed Kontar, Maher Noueihed, Jessie Yang, Corey Lester
[Preprint, Code]
Towards designing human-centered artificial intelligence for computer vision tasks: A focus group study on preventing medication dispensing errors
Yifan Zheng, Brigid Rowell, Qiyuan Chen1, Jin Yong Kim, Raed Al Kontar, X. Jessie Yang, Corey A Lester
[Preprint]
Federated Multi-output Gaussian Processes
Seokhyun Chung1, Raed Kontar
Technometrics, 2023
[Link, Video pending, Code]
Best refereed paper award finalist, quality, statistics & reliability (QSR) section, INFORMS annual meeting, 2022
Personalized Dictionary Learning for Heterogeneous Datasets
Geyu Liang, Naichen Shi1, Raed Kontar, Salar Fattahi
Neural Information Processing Systems (NeurIPS), 2023
[Preprint]
Federated Predictive Analytics for Condition Monitoring Signals
Seokhyun Chung1, Raed Kontar
IEEE Transactions on Reliability, 2023
[Link, Code]
Optimize to Generalize in Gaussian Processes: An Alternative Objective Based on the R’enyi Divergence
Xubo Yue1, Raed Kontar
IISE Transactions, 2023
[Link, Code]
Fed-ensemble: Ensemble Models in Federated Learning for Improved Generalization and Uncertainty Quantification
Naichen Shi1, Fan Lai, Raed Kontar, Mosharaf Chowdhury
IEEE Transactions on Automation Science and Engineering (TASE), 2023
[Link, Code Pending]
Best theoretical paper finalist, data mining and decision analytics workshop, INFORMS annual meeting, 2021
A Continual Learning Framework for Adaptive Defect Classification and Inspection
Wenbo Sun, Raed Kontar, Jionghua Jin, Tzyy-Shuh Chang
Journal of Quality Technology (JQT), 2023
[Link]
SALR: Sharpness-aware Learning Rate Scheduler for Improved Generalization
Xubo Yue1, Maher Nouiehed, Raed Kontar
IEEE Transactions on Neural Networks and Learning Systems, 2023
[Link, Code]
Federated Data Analytics: A Study on Linear Models
Xubo Yue1, Raed Al Kontar, Ana Maria Estrada Gomez
IISE Transactions, 2023
[Link, Youtube video, Code]
Featured Article in the December 2023 issue of the Industrial and Systems Engineering (ISE) Magazine
Personalized Federated Learning via Domain Adaptation with an Application to Distributed 3D Printing
Naichen Shi1, Raed Kontar
Technometrics, 2023
[Link, Video pending, Code]
GIFAIR-FL: An Approach for Group and Individual Fairness in Federated Learning
Xubo Yue1, Maher Nouiehed, Raed Kontar
Informs Journal on Data Science, 2022
[Link, Youtube video, Code]
Best refereed paper award finalist, QSR section, INFORMS annual meeting, 2021
Parameter Calibration in Wake Effect Simulation Model with Stochastic Gradient Descent and Stratified Sampling
Bingjie Liu, Xubo Yue1, Eunshin Byon, Raed Kontar
Annals of Applied Statistics, 2022
[Link]
Best paper finalist, data analytics and information systems (DAIS) division, IISE annual conference, 2021
On Negative Transfer and Structure of Latent Functions in Multi-output Gaussian Processes
Moyan Li1, Raed Kontar
SIAM Journal on Uncertainty Quantification, 2022
[Link]
Gaussian Process Inference Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits
Hao Chen1, Lili Zheng1, Raed Kontar, Garvesh Raskutti
Journal of Machine learning Research (JMLR), 2022
[Link, Youtube video, Code]
Weakly Supervised Multi-output Regression via Correlated Gaussian Processes
Seokhyun Chung1, Raed Kontar, Zhenke Wu
Informs Journal on Data Science, 2022
[Link, Youtube video, Code]
Best paper finalist, quality control & reliability engineering (QCRE), IISE annual conference, 2021
A Multi-stage Approach for Knowledge-guided Predictions with Application to Additive Manufacturing
Seokhyun Chung1, Cheng-Hao Chou, Xiaozhu Fang, Raed Kontar, Chinedum Okwudire
IEEE Transactions on Automation Science and Engineering (TASE), 2022
[Link]
The Internet of Federated Things (IoFT)
Raed Kontar, Naichen Shi1, Xubo Yue1, Seokhyun Chung1, Eunshin Byon, Mosharaf Chowdhury, Judy Jin, Wissam Kontar1, Neda Masoud, Maher Noueihed, Chinedum E. Okwudire, Garvesh Raskutti, Romesh Saigal, Karandeep Singh, and Zhisheng Ye
IEEE Access, 2021
[Link, Youtube video, IoFT Dataset Directory]
Featured article of the year at IEEE access, 2021
Look-ahead Planning for Renewable Energy: A Dynamic “Predict and Store” Approach
Jingxing Wang, Seokhyun Chung1, Abdullah AlShelahi, Raed Kontar, Eunshin Byon, Romesh Saigal
Applied Energy, 2021
[Link]
Performance Evaluation of a Prescription Medication Image Classification Model: An Observational Cohort
Corey A. Lester, Jiazhao Li, Yuting Ding, Brigid Rowell, Jessie ‘Xi’ Yang, Raed Kontar
Nature: NPJ Digital Medicine, 2021
[Link]
Joint Models for Event Prediction from Time Series and Survival Data
Xubo Yue1, Raed Kontar
Technometrics, 2020
[Link, Code]
Best student paper finalist, QCRE, IISE annual conference, 2019
Why Non-myopic Bayesian Optimization is Promising and How Far Should We Look-ahead? A Study via Rollout
Xubo Yue1, Raed Kontar
Artificial Intelligence and Statistics Conference (AISTATS), 2020
[Link]
Stochastic Gradient Descent in Correlated Settings: A Study on Gaussian Processes
Hao Chen1, Lili Zheng1, Raed Kontar, Garvesh Raskutti
Neural Information Processing Systems (NeurIPS), 2020
[Link, Youtube video, Code]
Functional Principal Component Analysis for Extrapolating Multi-stream Longitudinal Data
Seokhyun Chung1, Raed Kontar
IEEE Transactions on Reliability, 2020
[Link, Code]
Minimizing Negative Transfer of Knowledge in Multivariate Gaussian Processes: A Scalable and Regularized Approach
Raed Kontar, Garvesh Raskutti, Shiyu Zhou
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020
[Link]
Best paper finalist, QSR section, INFORMS annual Meeting, 2018
A Simple Approach to Multivariate Monitoring of Production Processes with Non-Gaussian Data
Qianqian Dong, Raed Kontar, Min Li, Gang Xua, Jinwu Xua
Journal of Manufacturing Systems, 2019
[Link]
Remaining Useful Life Prediction Based on Degradation Signals Using Monotonic B-splines with Infinite Support
Salman Jahani, Raed Kontar, Shiyu Zhou
IISE Transactions, 2019
[Link]
Featured article, IISE Transactions, 2019
Honorable mention for the best paper in the IISE transactions focus issue on quality and reliability engineering, 2020
2018, 17, 16 (At UW Madison)
Statistical Monitoring of Multiple Profiles Simultaneously Using Gaussian Processes
Salman Jahani, Raed Kontar, Shiyu Zhou
Quality and Reliability Engineering International, 2018
[Link]
Nonparametric Modeling and Prognosis of Condition Monitoring Signals Using Multivariate Gaussian Convolution Processes
Raed Kontar, Shiyu Zhou, Chaitanya Sankavaram, Xinyu Du, Yilu Zhang
Technometrics, 2018
[Link]
Best student paper winner, QSR section, INFORMS annual meeting, 2017
Nonparametric-Condition-Based Remaining Useful Life Prediction Incorporating External Factors
Raed Kontar, Shiyu Zhou, Chaitanya Sankavaram, Xinyu Du, Yilu Zhang
IEEE Transactions on Reliability, 2018
[Link]
Remaining Useful Life Prediction Based on the Mixed Effects Model with Mixture Prior Distribution
Raed Kontar, Shiyu Zhou, Chaitanya Sankavaram, Yilu Zhang, Xinyu Du
IISE Transactions, 2017
[Link]
Estimation and Monitoring of Key Performance Indicators of Manufacturing Systems Using the Multi-Output Gaussian Process
Raed Kontar, Shiyu Zhou, John Horst
International Journal of Production Research, 2016
[Link]