Derek Tam

I am a Ph.D. student at the University of Toronto (and the Vector Institute) advised by Colin Raffel. I am interested in using principled machine learning methods for solving NLP problems related to collaboratively developing large languge models and efficient training. Specific examples include model merging and parameter-efficient fine-tuning.

Publications

• Merging by Matching Models in Task Subspaces
  Derek Tam, Mohit Bansal, Colin Raffel
  Under Submission
  
• TIES-Merging: Resolving Interference When Merging Models
  Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, Mohit Bansal
  Conference on Neural Information Processing Systems NeurIps 2023.
  
• Simple Weakly-Supervised Image Captioning via CLIP's Multimodal Embeddings
  Derek Tam, Colin Raffel, Mohit Bansal
  AAAI Workshop on Creative AI Across Modalities, 2023. February 2023.
  
• Evaluating the Factual Consistency of Large Language Models Through News Summarization
  Derek Tam, Anisha Mascarenhas, Shiyue Zhang, Sarah Kwan, Mohit Bansal, Colin Raffel
  Findings of the Association for Computational Linguistics. ACL 2023 July 2023.
  
• Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
  Haokun Liu*, Derek Tam*, Mohammed Muqeeth*, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel
  Conference on Neural Information Processing Systems NeurIps 2022.
  
• An Empirical Survey of Data Augmentation for Limited Data Learning in NLP
  Jiaao Chen*, Derek Tam*, Colin Raffel, Mohit Bansal, Diyi Yang
  Transactions of the Association for Computational Linguistics TACL 2022.
  
• Isochrony-Aware Neural Machine Translation for Automatic Dubbing
  Derek Tam, Surafel M. Lakew, Yogesh Virkar, Prashant Mathur, Marcello Federico
  Interspeech 2022.
  
• Improving and Simplifying Pattern Exploiting Training
  Derek Tam*, Rakesh R Menon*, Mohit Bansal, Shashank Srivastava, Colin Raffel
  Empirical Methods in Natural Language Processing EMNLP 2021 (Short).
  
• Predicting Institution Hierarchies with Set-based Models.
  Derek Tam, Nicholas Monath, Ari Kobren, Andrew McCallum
  Automated Knowledge Base Construction AKBC 2020.
  
• Optimal Transport-based Alignment of Learned Character Representations for String Similarity.
  Derek Tam, Nicholas Monath, Ari Kobren, Aaron Traylor, Rajarshi Das, Andrew McCallum
  Association of Computational Linguistics ACL 2019. (Oral)
  

About Me

I spent the first three years of my PhD at UNC Chapel Hill, where I also worked with Mohit Bansal. Before that, I received a M.S. in Computer Science at the University of Massachusetts Amherst while supervised by Andrew McCallum. I received a B.S. in Computer Science and Statistics from Carnegie Mellon University. Outside of research, I am also a member of Friendship Baptist Church.

---