Wenhu Chen

Faculty Member

Website Google Scholar

Assistant Professor, David Cheriton School of Computer Science, University of Waterloo

Canada CIFAR Artificial Intelligence Chair

Wenhu Chen is currently an assistant professor in the David R. Cheriton School of Computer Science at University of Waterloo. He obtained his PhD from the computing science department of University of California, Santa Barbara in 2021, and he spent a wonderful postdoctoral year at Google Research. His main research interests include natural language processing, large language models, vision-language interaction, image generation, etc.

Research Interests

Natural Language Processing
Multimodal Learning
Knowledge Reasoning and Grounding

Highlights

Canada CIFAR AI Chair in 2022
WACV best student paper honorable mention
UCSB CS Outstanding Dissertation Award
Tencent AI Gift Award

Publications

Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks

Wenhu Chen and Xueguang Ma and Xinyi Wang and William W Cohen

2022

Re-imagen: Retrieval-augmented text-to-image generator

Wenhu Chen and Hexiang Hu and Chitwan Saharia and William W Cohen

2022

Explanations from Large Language Models Make Small Reasoners Better

Shiyang Li and Jianshu Chen and Yelong Shen and Zhiyu Chen and Xinlu Zhang and Zekun Li and Hong Wang and Jing Qian and Baolin Peng and Yi Mao and Wenhu Chen and Xifeng Yan

2022

Large language models are few (1)-shot table reasoners

Wenhu Chen

2022

MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text

Wenhu Chen and Hexiang Hu and Xi Chen and Pat Verga and William W Cohen

2022

Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering

Wenhu Chen and Pat Verga and Michiel de Jong and John Wieting and William Cohen

2022

Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models

Xichen Pan and Pengda Qin and Yuhong Li and Hui Xue and Wenhu Chen

2022

HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Data

Kai Nakamura and Sharon Levy and Yi-Lin Tuan and Wenhu Chen and William Yang Wang

2022

Subject-driven Text-to-Image Generation via Apprenticeship Learning

Wenhu Chen and Hexiang Hu and Yandong Li and Nataniel Ruiz and Xuhui Jia and Ming-Wei Chang and William W Cohen

2023

DePlot: One-shot visual language reasoning by plot-to-table translation

Fangyu Liu and Julian Martin Eisenschlos and Francesco Piccinno and Syrine Krichene and Chenxi Pang and Kenton Lee and Mandar Joshi and Wenhu Chen and Nigel Collier and Yasemin Altun

2022

Controllable Dialogue Simulation with In-Context Learning

Zekun Li and Wenhu Chen and Shiyang Li and Hong Wang and Jing Qian and Xifeng Yan

2022

QA Is the New KR: Question-Answer Pairs as Knowledge Bases

Wenhu Chen and William W Cohen and Michiel De Jong and Nitish Gupta and Alessandro Presta and Pat Verga and John Wieting

2022

Using meta-information in neural machine translation

Evgeny Matusov and Wenhu Chen and Shahram Khadivi

2022

On the Risk of Misinformation Pollution with Large Language Models

Yikang Pan and Liangming Pan and Wenhu Chen and Preslav Nakov and Min-Yen Kan and William Yang Wang

2023

MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing

Kai Zhang and Lingbo Mo and Wenhu Chen and Huan Sun and Yu Su

2023

TheoremQA: A Theorem-driven Question Answering dataset

Wenhu Chen and Ming Yin and Max Ku and Elaine Wan and Xueguang Ma and Jianyu Xu and Tony Xia and Xinyi Wang and Pan Lu

2023

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

LI Yizhi and Ruibin Yuan and Ge Zhang and Yinghao Ma and Xingran Chen and Hanzhi Yin and Chenghao Xiao and Chenghua Lin and Anton Ragni and Emmanouil Benetos and Norbert Gyenge and Roger Dannenberg and Ruibo Liu and Wenhu Chen and Gus Xia and Yemin Shi and Wenhao Huang and Zili Wang and Yike Guo and Jie Fu

2023

Few-shot In-context Learning for Knowledge Base Question Answering

Tianle Li and Xueguang Ma and Alex Zhuang and Yu Gu and Yu Su and Wenhu Chen

2023

DreamEdit: Subject-driven Image Editing

Tianle Li and Max Ku and Cong Wei and Wenhu Chen

2023

MARBLE: Music Audio Representation Benchmark for Universal Evaluation

Ruibin Yuan and Yinghao Ma and Yizhi Li and Ge Zhang and Xingran Chen and Hanzhi Yin and Le Zhuo and Yiqi Liu and Jiawen Huang and Zeyue Tian and Binyue Deng and Ningzhi Wang and Wenhu Chen and Gus Xia and Wei Xue and Si Liu and Shi Wang and Ruibo Liu and Yike Guo and Jie Fu

2023

MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response

Zihao Deng and Yinghao Ma and Yudong Liu and Rongchen Guo and Ge Zhang and Wenhu Chen and Wenhao Huang and Emmanouil Benetos

2023

EDIS: Entity-Driven Image Search over Multimodal Web Content

Siqi Liu and Weixi Feng and Wenhu Chen and William Yang Wang

2023

MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning

Xiang Yue and Xingwei Qu and Ge Zhang and Yao Fu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen

2023

MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen

2023

CMMMU: A Chinese Massive Multi-discipline Multimodal Understanding Benchmark

Ge Zhang and Xinrun Du and Bei Chen and Yiming Liang and Tongxu Luo and Tianyu Zheng and Kang Zhu and Yuyang Cheng and Chunpu Xu and Shuyue Guo and Haoran Zhang and Xingwei Qu and Junjie Wang and Ruibin Yuan and Yizhi Li and Zekun Wang and Yudong Liu and Yu-Hsuan Tsai and Fengji Zhang and Chenghua Lin and Wenhao Huang and Wenhu Chen and Jie Fu

2024

Interactive Natural Language Processing

Zekun Wang and Ge Zhang and Kexin Yang and Ning Shi and Wangchunshu Zhou and Shaochun Hao and Guangzheng Xiong and Yizhi Li and Mong Yuan Sim and Xiuying Chen and Qingqing Zhu and Zhenzhu Yang and Adam Nik and Qi Liu and Chenghua Lin and Shi Wang and Ruibo Liu and Wenhu Chen and Ke Xu and Dayiheng Liu and Yike Guo and Jie Fu

2023

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Xichen Pan and Li Dong and Shaohan Huang and Zhiliang Peng and Wenhu Chen and Furu Wei

2023

LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT

Le Zhuo and Ruibin Yuan and Jiahao Pan and Yinghao Ma and Yizhi LI and Ge Zhang and Si Liu and Roger Dannenberg and Jie Fu and Chenghua Lin and Emmanouil Benetos and Wenhu Chen and Wei Xue and Yike Guo

2023

VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation

Max Ku and Dongfu Jiang and Cong Wei and Xiang Yue and Wenhu Chen

2023

Instruct-Imagen: Image Generation with Multi-modal Instruction

Hexiang Hu and Kelvin CK Chan and Yu-Chuan Su and Wenhu Chen and Yandong Li and Kihyuk Sohn and Yang Zhao and Xue Ben and Boqing Gong and William Cohen and Ming-Wei Chang and Xuhui Jia

2024

MagicLens: Self-Supervised Image Retrieval with Open-Ended Instructions

Kai Zhang and Yi Luan and Hexiang Hu and Kenton Lee and Siyuan Qiao and Wenhu Chen and Yu Su and Ming-Wei Chang

2024

Understanding the Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation

Xinyi Wang and Alfonso Amayuelas and Kexun Zhang and Liangming Pan and Wenhu Chen and William Yang Wang

2024

Augmenting Black-box LLMs with Medical Textbooks for Clinical Question Answering

Yubo Wang and Xueguang Ma and Wenhu Chen

2023

TIGERScore: Towards Building Explainable Metric for All Text Generation Tasks

Dongfu Jiang and Yishan Li and Ge Zhang and Wenhao Huang and Yuchen Lin and Wenhu Chen

2023

ImagenHub: Standardizing the evaluation of conditional image generation models

Max Ku and Tianle Li and Kai Zhang and Yujie Lu and Xingyu Fu and Wenwen Zhuang and Wenhu Chen

2023

UniIR: Training and Benchmarking Universal Multimodal Information Retrievers

Cong Wei and Yang Chen and Haonan Chen and Hexiang Hu and Ge Zhang and Jie Fu and Alan Ritter and Wenhu Chen

2023

OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement

Tianyu Zheng and Ge Zhang and Tianhao Shen and Xueling Liu and Bill Yuchen Lin and Jie Fu and Wenhu Chen and Xiang Yue

2024

E^ 2-LLM: Efficient and Extreme Length Extension of Large Language Models

Jiaheng Liu and Zhiqi Bai and Yuanxing Zhang and Chenchen Zhang and Yu Zhang and Ge Zhang and Jiakai Wang and Haoran Que and Yukang Chen and Wenbo Su and Tiezheng Ge and Jie Fu and Wenhu Chen and Bo Zheng

2024

ChatMusician: Understanding and Generating Music Intrinsically with LLM

Ruibin Yuan and Hanfeng Lin and Yi Wang and Zeyue Tian and Shangda Wu and Tianhao Shen and Ge Zhang and Yuhang Wu and Cong Liu and Ziya Zhou and Ziyang Ma and Liumeng Xue and Ziyu Wang and Qin Liu and Tianyu Zheng and Yizhi Li and Yinghao Ma and Yiming Liang and Xiaowei Chi and Ruibo Liu and Zili Wang and Pengfei Li and Jingcheng Wu and Chenghua Lin and Qifeng Liu and Tao Jiang and Wenhao Huang and Wenhu Chen and Emmanouil Benetos and Jie Fu and Gus Xia and Roger Dannenberg and Wei Xue and Shiyin Kang and Yike Guo

2024

ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation

Weiming Ren and Harry Yang and Ge Zhang and Cong Wei and Xinrun Du and Stephen Huang and Wenhu Chen

2024

Kun: Answer Polishment for Chinese Self-Alignment with Instruction Back-Translation

Tianyu Zheng and Shuyue Guo and Xingwei Qu and Jiawei Guo and Weixu Zhang and Xinrun Du and Chenghua Lin and Wenhao Huang and Wenhu Chen and Jie Fu and Ge Zhang

2024

COIG-CQIA: Quality is All You Need for Chinese Instruction Fine-tuning

Yuelin Bai and Xinrun Du and Yiming Liang and Yonggang Jin and Ziqiang Liu and Junting Zhou and Tianyu Zheng and Xincheng Zhang and Nuo Ma and Zekun Wang and Ruibin Yuan and Haihong Wu and Hongquan Lin and Wenhao Huang and Jiajun Zhang and Wenhu Chen and Chenghua Lin and Jie Fu and Min Yang and Shiwen Ni and Ge Zhang

2024

AnyV2V: A Plug-and-Play Framework For Any Video-to-Video Editing Tasks

Max Ku and Cong Wei and Weiming Ren and Huan Yang and Wenhu Chen

2024

Reward Guided Latent Consistency Distillation

Jiachen Li and Weixi Feng and Wenhu Chen and William Yang Wang

2024

DEEP-ICL: Definition-Enriched Experts for Language Model In-Context Learning

Xingwei Qu and Yiming Liang and Yucheng Wang and Tianyu Zheng and Tommy Yue and Lei Ma and Stephen W Huang and Jiajun Zhang and Wenhu Chen and Chenghua Lin and Jie Fu and Ge Zhang

2024

StructLM: Towards Building Generalist Models for Structured Knowledge Grounding

Alex Zhuang and Ge Zhang and Tianyu Zheng and Xinrun Du and Junjie Wang and Weiming Ren and Stephen W Huang and Jie Fu and Xiang Yue and Wenhu Chen

2024

SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval

Siwei Wu and Yizhi Li and Kang Zhu and Ge Zhang and Yiming Liang and Kaijing Ma and Chenghao Xiao and Haoran Zhang and Bohao Yang and Wenhu Chen and Wenhao Huang and Noura Al Moubayed and Jie Fu and Chenghua Lin

2024

More Publications