I am a Ph.D. student at the University of Toronto advised by Colin Raffel, working in machine learning and NLP. I am interested in model merging and understanding why different merging methods behave the way they do. Previously, I worked on parameter-efficient fine-tuning and few-shot learning.
Publications
- Merging by Matching Models in Task Subspaces
Derek Tam, Mohit Bansal, Colin Raffel
TMLR March 2024
- TIES-Merging: Resolving Interference When Merging Models
Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal
Conference on Neural Information Processing Systems NeurIps 2023.
- Simple Weakly-Supervised Image Captioning via CLIP's Multimodal Embeddings
Derek Tam, Colin Raffel, Mohit Bansal
AAAI Workshop on Creative AI Across Modalities, 2023. February 2023.
- Evaluating the Factual Consistency of Large Language Models Through News Summarization
Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel
Findings of the Association for Computational Linguistics. ACL 2023 July 2023.
- Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Haokun Liu*, Derek Tam*, Mohammed Muqeeth*, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel
Conference on Neural Information Processing Systems NeurIps 2022.
- An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
Jiaao Chen*, Derek Tam*, Colin Raffel, Mohit Bansal, Diyi Yang
Transactions of the Association for Computational Linguistics TACL 2022.
- Isochrony-Aware Neural Machine Translation for Automatic Dubbing
Derek Tam, Surafel M. Lakew, Yogesh Virkar, Prashant Mathur, Marcello Federico
Interspeech 2022.
- Improving and Simplifying Pattern Exploiting Training
Derek Tam*, Rakesh R Menon*, Mohit Bansal, Shashank Srivastava, Colin Raffel
Empirical Methods in Natural Language Processing EMNLP 2021 (Short).
- Predicting Institution Hierarchies with Set-based Models.
Derek Tam, Nicholas Monath, Ari Kobren, Andrew McCallum
Automated Knowledge Base Construction AKBC
2020.
- Optimal Transport-based Alignment
of Learned Character Representations for String
Similarity.
Derek Tam, Nicholas Monath, Ari Kobren, Aaron Traylor, Rajarshi Das, Andrew McCallum
Association of
Computational Linguistics ACL
2019. (Oral)
About Me
I spent the first three years of my PhD at UNC Chapel Hill, where I also worked with Mohit Bansal. Before that, I received a M.S. in Computer Science at the University of Massachusetts Amherst while supervised by Andrew McCallum. I received a B.S. in Computer Science and Statistics from Carnegie Mellon University. Outside of research, I am also a member of Friendship Baptist Church.