Welcome to my homepage! I am a master’s student at National University of Singapore (NUS). Currently, I am working at NLP group of Shanghai AI Lab as a research intern under the supervision of Dr. Zhiyong Wu, focusing on topics of LLMs. Prior to this, I was advised by Dr. Xiaoli Li at Institute for Infocomm Research (I²R), A*STAR, Singapore for NLP research. Before that, I completed my B.Eng in the School of Data Science and Engineering (DaSE) with distinction at East China Normal University, where I felt very privileged to be instructed by Prof. Weining Qian, Prof. Xuesong Lu and Prof. Xiang Li for research/engineering projects. My research interests cover neural code intelligence, efficient methods for NLP, representation learning and broad deep learning topics in general.

I will start my Ph.D. in Computer Science at HKU CS in 2024 Fall 🐱

🔥 News

• 2024.04:  📑📑 Check out our Code Intelligence Survey Paper 🔥
• 2023.12:  ⛱️⛱️ Attending EMNLP 2023 in SG 🇸🇬
• 2023.05:  🚀🚀 HugNLP Framework is ready for use! Please check our Paper, Repo and Blogs!
• 2023.05:  👏👏 We release SelfAware (ACL 2023 Findings) for benchmarking LLMs’ self-knowledge Slides.
• 2022.12:  🎉🎉 Our team won second prize (100k RMB) in the International Algorithm Case Competition: PLM Tuning Track.

📝 Publications

Survey

A Survey of Neural Code Intelligence: Paradigms, Advances and Beyond 🔥🔥

Qiushi Sun, Zhirui Chen, Fangzhi Xu, Chang Ma, Kanzhi Cheng, Zhangyue Yin, Jianing Wang, Chengcheng Han, Renyu Zhu, Shuai Yuan, Pengcheng Yin, Qipeng Guo, Xipeng Qiu, Xiaoli Li, Fei Yuan, Lingpeng Kong, Xiang Li, Zhiyong Wu

[Paper] | [Slides] | [Project] | [Video] |

Let me walk you through the development of neural code intelligence:

• Follow LMs for code as a thread to trace the field’s development 🚀
• Explore cross-domain synergies and opportunities 🌱
• Present a broad array of promising research avenues 💡

• Preprint KS-Lottery🎟️: Finding Certified Lottery Tickets for Multilingual Language Models, Fei Yuan, Chang Ma, Shuai Yuan, Qiushi Sun, Lei Li.
• Preprint SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents, Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, Zhiyong Wu. [LLMAgents @ ICLR 2024]
• Preprint Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models, Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao and Jun Liu.
• Preprint Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration, Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu and Lingpeng Kong. [LLMAgents @ ICLR 2024]
• Preprint Boosting Language Models Reasoning with Chain-of-Knowledge Prompting, Jianing Wang^*, Qiushi Sun^*, Xiang Li and Ming Gao.
• LREC-COLING 2024 TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills, Qiushi Sun, Nuo Chen, Jianing Wang, Xiang Li and Ming Gao.
• LREC-COLING 2024 Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives, Qiushi Sun, Chengcheng Han, Nuo Chen, Renyu Zhu, Jingyang Gong, Xiang Li and Ming Gao. | 🥈 100K RMB Award-winning Solution
• LREC-COLING 2024 Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models, Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Xipeng Qiu and Xuanjing Huang.
• EMNLP 2023 Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communication, Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang and Xipeng Qiu. Slides | Video
• EMNLP 2023 Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding, Jianing Wang, Qiushi Sun, Nuo Chen, Chengyu Wang, Xiang Li, Ming Gao and Jun Huang.
• EMNLP 2023 Pass-Tuning: Towards Structure-Aware Parameter-Efficient Tuning for Code Representation Learning, Nuo Chen, Qiushi Sun, Jianing Wang, Xiang Li and Ming Gao.
• CIKM 2023 (Demo) HugNLP: A Unified and Comprehensive Library for Natural Language Processing, Jianing Wang, Nuo Chen, Qiushi Sun, Wenkang Huang, Chengyu Wang and Ming Gao. | 🏆 Best Demo Paper Award
• ACL 2023 Do Large Language Models Know What They Don’t Know?, Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu and Xuanjing Huang. Slides | Video |
• ACL 2023 When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario, Chengcheng Han, Liqing Cui, Renyu Zhu, Jianing Wang, Nuo Chen, Qiushi Sun, Xiang Li and Ming Gao.
• EMNLP 2022 CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure, Nuo Chen^*, Qiushi Sun^*, Renyu Zhu^*, Xiang Li, Xuesong Lu and Ming Gao. Slides | Video |

*Denotes equal contribution. More working drafts / preprints under review will be released later ⌛️

🎖 Honors and Awards

• 2023.10 CIKM Best Demo Paper Award, ACM & SIGIR
• 2022.12 International Algorithm Case Competition: PLM Tuning, Second Prize, Guangdong-Hong Kong-Macao Greater Bay Area
• 2022.07 Shanghai Outstanding Graduates, Shanghai Municipal Education Commission
• 2022.05 Excellent Bachelor Thesis Award, East China Normal University
• 2021.11 Outstanding Student, East China Normal University
• 2021.10 First Class Scholarship, East China Normal University
• 2021.05 Finalist (Special Class Prize), Mathematical and Interdisciplinary Contest in Modeling

📖 Educations

• 2022.08 - 2024.01 (now), Master, National University of Singapore , Singapore.
• 2018.09 - 2022.07, Undergraduate, School of Data Science and Engineering, East China Normal University , Shanghai, China.
• 2011.09 - 2018.07, Middle School, Pudong Foreign Languages School , SISU, Shanghai, China.

💻 Internships

• 2023.06 - Present, NLP Research Intern, NLP Group, Shanghai AI Laboratory , Shanghai, China.
• 2023.01 - 2023.06, NLP Research Intern, Institute for Infocomm Research, A*STAR , Singapore.
• 2021.10 - 2022.07, Research Assistant, School of Data Science and Engineering , ECNU, Shanghai, China.

🧑‍🏫 Teaching

• (Fall 2021) Teaching Assistant @ Deep Learning for Computer Vision, ECNU, Shanghai, China.

🔍 Services

• I serve(d) as a conference reviewer for EMNLP2022, ACL2023, CIKM2023, EMNLP2023(Best Reviewer Award), ICLR2024, ACL Rolling Review

“What’s past is prologue.” – William Shakespeare (The Tempest)