Kai Chen

I am leading a group of talented researchers and engineers to tackle challenges related to LLM, generative AI and multi-modal learning, as well as build impactful open-source projects. Some projects led by me include: OpenMMLab, InternLM, OpenCompass, OpenXLab.

Before joining Shanghai AI Lab, I was a director at SenseTime and led R&D teams for computer vision. I received the PhD in 2019 degree at MMLab, The Chinese University of Hong Kong, under the supervision of Prof. Dahua Lin and Chen Change Loy. Prior to that, I received the B.Eng. degree from Tsinghua University in 2015.

We have opening positions for full-time employees, interns, joint PhDs (with THU/SJTU/FDU/ZJU/USTC, etc).

news

Sep, 2024	7 papers accepted to NeurIPS 2024 and 4 papers accepted to NeurIPS 2024 Datasets and Benchmarks Track.
Sep, 2024	3 papers (1 main + 2 findings) accepted to EMNLP 2024.
Jul, 2024	6 papers accepted to ECCV 2024.
May, 2024	5 papers (2 main + 3 findings) accepted to ACL 2024.
May, 2024	2 papers accepted to ICML 2024.
Mar, 2024	2 papers accepted to NAACL 2024 (1 main + 1 findings).
Feb, 2024	7 papers accepted to CVPR 2024.
Jan, 2024	OpenCompass 2.0 is released and it now has a Chinese name 司南.
Sep, 2023	1 paper accepted to NeurIPS 2023.
Jul, 2023	3 papers accepted to ICCV 2023.
Apr, 2023	1 paper accepted to IJCAI 2023.
Feb, 2023	3 papers accepted to CVPR 2023.
Nov, 2022	2 papers accepted to AAAI 2023.
Sep, 2022	1 paper accepted to NeurIPS 2022.
Sep, 2022	A major update of OpenMMLab – OpenMMLab 2.0 is released.
Jul, 2022	1 paper accepted to ECCV 2022.
Mar, 2022	6 papers accepted to CVPR 2022.
Sep, 2021	2 papers accepted to NeurIPS 2021.
Apr, 2021	1 paper accepted to TPAMI.
Apr, 2021	We released MMTracking, MMOCR and mmgeneration.
Feb, 2021	2 papers accepted to CVPR 2021.
Dec, 2020	1 paper accepted to AAAI 2021.
Jul, 2020	We announced 6 new open-source projects on OpenMMLab. See our website for details.
Jul, 2020	1 paper accepted to ECCV 2020 (Spotlight).
Feb, 2020	1 paper accepted to CVPR 2020.
Oct, 2019	We won the 1st prize in COCO 2019 Object Detection Challenge (without external data).
Jul, 2019	1 paper accepted to ICCV 2019 (oral).
Mar, 2019	3 papers accepted to CVPR 2019.
Sep, 2018	We released MMCV and MMDetection .
Sep, 2018	We won the COCO 2018 Object Detection Challenge. (slides)
Feb, 2018	1 paper accepted to CVPR 2018.
Mar, 2017	1 paper accepted to CVPR 2017 (Spotlight).

selected publications (full list)

ANAH: Analytical Annotation of Hallucinations in Large Language Models

Ziwei Ji, Yuzhe Gu, Wenwei Zhang, and 3 more authors

In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) 2024

Bib PDF Code Website

@inproceedings{ji2024anah,
  title = {ANAH: Analytical Annotation of Hallucinations in Large Language Models},
  author = {Ji, Ziwei and Gu, Yuzhe and Zhang, Wenwei and Lyu, Chengqi and Lin, Dahua and Chen, Kai},
  booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
  year = {2024},
}

T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step

Zehui Chen, Weihua Du, Wenwei Zhang, and 8 more authors

In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) 2024

Bib PDF Code Website

@inproceedings{chen2024t,
  title = {T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Step},
  author = {Chen, Zehui and Du, Weihua and Zhang, Wenwei and Liu, Kuikun and Liu, Jiangning and Zheng, Miao and Zhuo, Jingming and Zhang, Songyang and Lin, Dahua and Chen, Kai and Zhao, Feng},
  booktitle = {Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)},
  year = {2024},
}

Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models

Zehui Chen, Kuikun Liu, Qiuchen Wang, and 5 more authors

In Findings of the Association for Computational Linguistics: ACL 2024

Bib PDF Code

@inproceedings{chen2024agent,
  title = {Agent-FLAN: Designing Data and Methods of Effective Agent Tuning for Large Language Models},
  author = {Chen, Zehui and Liu, Kuikun and Wang, Qiuchen and Zhang, Wenwei and Liu, Jiangning and Lin, Dahua and Chen, Kai and Zhao, Feng},
  booktitle = {Findings of the Association for Computational Linguistics: ACL},
  year = {2024},
}

MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark

Hongwei Liu, Zilong Zheng, Yuxuan Qiao, and 7 more authors

In Findings of the Association for Computational Linguistics: ACL 2024

Bib PDF Code

@inproceedings{liu2024mathbench,
  title = {MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark},
  author = {Liu, Hongwei and Zheng, Zilong and Qiao, Yuxuan and Duan, Haodong and Fei, Zhiwei and Zhou, Fengzhe and Zhang, Wenwei and Zhang, Songyang and Lin, Dahua and Chen, Kai},
  booktitle = {Findings of the Association for Computational Linguistics: ACL},
  year = {2024},
}

PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models

Yiming Zhang, Zhening Xing, Yanhong Zeng, and 2 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024

Bib PDF Code Website

@inproceedings{zhang2024pia,
  title = {PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models},
  author = {Zhang, Yiming and Xing, Zhening and Zeng, Yanhong and Fang, Youqing and Chen, Kai},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2024},
}

RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer

Jiahao Wang, Songyang Zhang, Yong Liu, and 6 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023

Bib PDF Code Website

@inproceedings{wang2023riformer,
  title = {RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer},
  author = {Wang, Jiahao and Zhang, Songyang and Liu, Yong and Wu, Taiqiang and Yang, Yujiu and Liu, Xihui and Chen, Kai and Luo, Ping and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2023},
}

Dense Distinct Query for End-to-End Object Detection

Shilong Zhang, Jiaqi Wang, Jiangmiao Pang, and 4 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023

Bib PDF Code

@inproceedings{zhang2023dense,
  title = {Dense Distinct Query for End-to-End Object Detection},
  author = {Zhang, Shilong and Wang, Jiaqi and Pang, Jiangmiao and Lyu, Chengqi and Zhang, Wenwei and Luo, Ping and Chen, Kai},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2023},
}

Group R-CNN for Weakly Semi-supervised Object Detection with Points

Shilong Zhang, Zhuoran Yu, Liyang Liu, and 3 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2022

Bib PDF Code

@inproceedings{zhang2022group,
  title = {Group R-CNN for Weakly Semi-supervised Object Detection with Points},
  author = {Zhang, Shilong and Yu, Zhuoran and Liu, Liyang and Wang, Xinjiang and Zhou, Aojun and Chen, Kai},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {9417--9426},
  year = {2022},
}

MMRotate: A Rotated Object Detection Benchmark Using PyTorch

Yue Zhou, Xue Yang, Gefan Zhang, and 9 more authors

In Proceedings of the 30th ACM International Conference on Multimedia 2022

Bib PDF Code

@inproceedings{zhou2022mmrotate,
  title = {MMRotate: A Rotated Object Detection Benchmark Using PyTorch},
  author = {Zhou, Yue and Yang, Xue and Zhang, Gefan and Wang, Jiabao and Liu, Yanyi and Hou, Liping and Jiang, Xue and Liu, Xingzhao and Yan, Junchi and Lyu, Chengqi and Zhang, wenwei and Chen, Kai},
  booktitle = {Proceedings of the 30th ACM International Conference on Multimedia},
  year = {2022},
}

K-Net: Towards Unified Image Segmentation

Wenwei Zhang, Jiangmiao Pang, Kai Chen, and 1 more author

Advances in Neural Information Processing Systems (NeurIPS) 2021

Bib PDF Code Website

@article{zhang2021k,
  title = {K-Net: Towards Unified Image Segmentation},
  author = {Zhang, Wenwei and Pang, Jiangmiao and Chen, Kai and Loy, Chen Change},
  journal = {Advances in Neural Information Processing Systems (NeurIPS)},
  volume = {34},
  pages = {10326--10338},
  year = {2021},
}

Temporal ROI Align for Video Object Recognition

Tao Gong, Kai Chen, Xinjiang Wang, and 5 more authors

In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) 2021

Bib PDF Code

@inproceedings{gong2021temporal,
  title = {Temporal ROI Align for Video Object Recognition},
  author = {Gong, Tao and Chen, Kai and Wang, Xinjiang and Chu, Qi and Zhu, Feng and Lin, Dahua and Yu, Nenghai and Feng, Huamin},
  booktitle = {Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)},
  volume = {35},
  number = {2},
  pages = {1442--1450},
  year = {2021},
}

Towards Balanced Learning for Instance Recognition

Jiangmiao Pang, Kai Chen, Qi Li, and 5 more authors

International Journal of Computer Vision (IJCV) 2021

Bib HTML

@article{pang2021towards,
  title = {Towards Balanced Learning for Instance Recognition},
  author = {Pang, Jiangmiao and Chen, Kai and Li, Qi and Xu, Zhihai and Feng, Huajun and Shi, Jianping and Ouyang, Wanli and Lin, Dahua},
  journal = {International Journal of Computer Vision (IJCV)},
  volume = {129},
  number = {5},
  pages = {1376--1393},
  year = {2021},
  publisher = {Springer US},
}

CARAFE++: Unified Content-Aware ReAssembly of FEatures

Jiaqi Wang, Kai Chen, Rui Xu, and 3 more authors

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021

Bib PDF

@article{wang2021carafe++,
  title = {CARAFE++: Unified Content-Aware ReAssembly of FEatures},
  author = {Wang, Jiaqi and Chen, Kai and Xu, Rui and Liu, Ziwei and Loy, Chen Change and Lin, Dahua},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)},
  year = {2021},
  publisher = {IEEE},
}

Feature Pyramid Grids

Kai Chen, Yuhang Cao, Chen Change Loy, and 2 more authors

arXiv preprint arXiv:2004.03580 2020

Bib PDF Code

@article{chen2020feature,
  title = {Feature Pyramid Grids},
  author = {Chen, Kai and Cao, Yuhang and Loy, Chen Change and Lin, Dahua and Feichtenhofer, Christoph},
  journal = {arXiv preprint arXiv:2004.03580},
  year = {2020},
}

Libra R-CNN: Towards Balanced Learning for Object Detection

Jiangmiao Pang, Kai Chen, Jianping Shi, and 3 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019

Bib PDF Code

@inproceedings{pang2019libra,
  title = {Libra R-CNN: Towards Balanced Learning for Object Detection},
  author = {Pang, Jiangmiao and Chen, Kai and Shi, Jianping and Feng, Huajun and Ouyang, Wanli and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {821--830},
  year = {2019},
}

Region Proposal by Guided Anchoring

Jiaqi Wang*, Kai Chen*, Shuo Yang, and 2 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019

Bib PDF Code

@inproceedings{wang2019region,
  title = {Region Proposal by Guided Anchoring},
  author = {Wang*, Jiaqi and Chen*, Kai and Yang, Shuo and Change Loy, Chen and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {2965--2974},
  year = {2019},
}

Hybrid task cascade for instance segmentation

Kai Chen, Jiangmiao Pang, Jiaqi Wang, and 9 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019

Bib PDF Code

@inproceedings{chen2019hybrid,
  title = {Hybrid task cascade for instance segmentation},
  author = {Chen, Kai and Pang, Jiangmiao and Wang, Jiaqi and Xiong, Yu and Li, Xiaoxiao and Sun, Shuyang and Feng, Wansen and Liu, Ziwei and Shi, Jianping and Ouyang, Wanli and Loy, Chen Change and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {4974--4983},
  year = {2019},
}

MMDetection: Open MMLab Detection Toolbox and Benchmark

Kai Chen, Jiaqi Wang*, Jiangmiao Pang*, and 22 more authors

arXiv preprint arXiv:1906.07155 2019

Bib PDF Code

@article{chen2019mmdetection,
  title = {MMDetection: Open MMLab Detection Toolbox and Benchmark},
  author = {Chen, Kai and Wang*, Jiaqi and Pang*, Jiangmiao and Cao, Yuhang and Xiong, Yu and Li, Xiaoxiao and Sun, Shuyang and Feng, Wansen and Liu, Ziwei and Xu, Jiarui and Zhang, Zheng and Cheng, Dazhi and Zhu, Chenchen and Cheng, Tianheng and Zhao, Qijie and Li, Buyu and Lu, Xin and Zhu, Rui and Wu, Yue and Dai, Jifeng and Wang, Jingdong and Shi, Jianping and Ouyang, Wanli and Loy, Chen Change and Lin, Dahua},
  journal = {arXiv preprint arXiv:1906.07155},
  year = {2019},
}

CARAFE: Content-Aware ReAssembly of FEatures

Jiaqi Wang, Kai Chen, Rui Xu, and 3 more authors

In Proceedings of the IEEE International Conference on Computer Vision (ICCV) 2019

Bib PDF Code

@inproceedings{wang2019carafe,
  title = {CARAFE: Content-Aware ReAssembly of FEatures},
  author = {Wang, Jiaqi and Chen, Kai and Xu, Rui and Liu, Ziwei and Loy, Chen Change and Lin, Dahua},
  booktitle = {Proceedings of the IEEE International Conference on Computer Vision (ICCV)},
  year = {2019},
}

Prime sample attention in object detection

Yuhang Cao, Kai Chen, Chen Change Loy, and 1 more author

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2020

Bib PDF Code

@inproceedings{cao2020prime,
  title = {Prime sample attention in object detection},
  author = {Cao, Yuhang and Chen, Kai and Loy, Chen Change and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {11583--11591},
  year = {2020},
}

Optimizing Video Object Detection via a Scale-Time Lattice

Kai Chen, Jiaqi Wang, Shuo Yang, and 4 more authors

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018

Bib PDF Website

@inproceedings{chen2018optimizing,
  title = {Optimizing Video Object Detection via a Scale-Time Lattice},
  author = {Chen, Kai and Wang, Jiaqi and Yang, Shuo and Zhang, Xingcheng and Xiong, Yuanjun and Loy, Chen Change and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages = {7814--7823},
  year = {2018},
}

Discover and Learn New Objects from Documentaries

Kai Chen, Hang Song, Chen Change Loy, and 1 more author

In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017

Bib PDF Website Dataset

@inproceedings{chen2017discover,
  title = {Discover and Learn New Objects from Documentaries},
  author = {Chen, Kai and Song, Hang and Loy, Chen Change and Lin, Dahua},
  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2017},
}