Jiawei Liu

I am a PhD student at UIUC PLFMSE working with Lingming Zhang.

I am interested in programming systems and machine learning, specifically:

AI4Code: How to build and evaluate AI programmers to improve developer efficiency and improve software quality?
MLSys Correctness: How to detect, debug and mitigate errors in our ML systems, from silicon to compiler and cloud?

🤗 Feel free to drop me an email if you are interested in my research.

Papers Show All

NeurIPS’23

To Appear

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang

Thirty-seventh Conference on Neural Information Processing Systems. 2023

PAPER Bib Poster Slides

@inproceedings{liu2023is,
  title = {Is Your Code Generated by Chat{GPT} Really Correct? Rigorous Evaluation of Large Language Models for Code Generation},
  author = {Liu, Jiawei and Xia, Chunqiu Steven and Wang, Yuyao and Zhang, Lingming},
  booktitle = {Thirty-seventh Conference on Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=1qvx610Cu7},
}

ESEC/FSE’23

NeuRI: Diversifying DNN Generation via Inductive Rule Inference

Jiawei Liu, Jinjun Peng, Yuyao Wang, and Lingming Zhang

Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2023

🏆 ACM SIGSOFT Distinguished Paper Award

PAPER Bib Slides Artifact

@inproceedings{liu2023neuri,
  author = {Liu, Jiawei and Peng, Jinjun and Wang, Yuyao and Zhang, Lingming},
  title = {NeuRI: Diversifying DNN Generation via Inductive Rule Inference},
  year = {2023},
  isbn = {9798400703270},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3611643.3616337},
  doi = {10.1145/3611643.3616337},
  booktitle = {Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages = {657--669},
  numpages = {13},
  location = {San Francisco, CA, USA},
  series = {ESEC/FSE 2023},
}

ASPLOS’23

NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers

Jiawei Liu, Jinkun Lin, Fabian Ruffy, Cheng Tan, Jinyang Li, Aurojit Panda, and Lingming Zhang

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. 2023

🏆 Distinguished Artifact Award

PAPER Bib Poster Slides Artifact

@inproceedings{liu2023nnsmith,
  author = {Liu, Jiawei and Lin, Jinkun and Ruffy, Fabian and Tan, Cheng and Li, Jinyang and Panda, Aurojit and Zhang, Lingming},
  title = {NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers},
  year = {2023},
  isbn = {9781450399166},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3575693.3575707},
  doi = {10.1145/3575693.3575707},
  booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
  pages = {530--543},
  numpages = {14},
  keywords = {Deep Learning Compilers, Compiler Testing, Fuzzing},
  location = {Vancouver, BC, Canada},
  series = {ASPLOS 2023},
}

OOPSLA’22

Coverage-guided tensor compiler fuzzing with joint IR-pass mutation

Jiawei Liu, Yuxiang Wei, Sen Yang, Yinlin Deng, and Lingming Zhang

Proceedings of the ACM on Programming Languages 6 (OOPSLA1). Apr 2022

PAPER Bib Slides Artifact

@article{liu2022coverage,
  title = {Coverage-guided tensor compiler fuzzing with joint IR-pass mutation},
  author = {Liu, Jiawei and Wei, Yuxiang and Yang, Sen and Deng, Yinlin and Zhang, Lingming},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {6},
  number = {OOPSLA1},
  pages = {1--26},
  year = {2022},
  publisher = {ACM New York, NY, USA},
  url = {https://doi.org/10.1145/3527317},
  doi = {10.1145/3527317},
  month = apr,
  articleno = {73},
}

Pre-print

Relax: Composable Abstractions for End-to-End Dynamic Machine Learning

Ruihang Lai, Junru Shao, Siyuan Feng, Steven S. Lyubomirsky, Bohan Hou, Wuwei Lin, Zihao Ye, Hongyi Jin, Yuchen Jin, Jiawei Liu, Lesheng Jin, Yaxing Cai, Ziheng Jiang, Yong Wu, Sunghyun Park, Prakalp Srivastava, Jared G. Roesch, Todd C. Mowry, and Tianqi Chen

arXiv preprint arXiv:2311.02103. 2023

PAPER Bib

@article{lai2023relax,
  title = {Relax: Composable Abstractions for End-to-End Dynamic Machine Learning},
  author = {Lai, Ruihang and Shao, Junru and Feng, Siyuan and Lyubomirsky, Steven S. and Hou, Bohan and Lin, Wuwei and Ye, Zihao and Jin, Hongyi and Jin, Yuchen and Liu, Jiawei and Jin, Lesheng and Cai, Yaxing and Jiang, Ziheng and Wu, Yong and Park, Sunghyun and Srivastava, Prakalp and Roesch, Jared G. and Mowry, Todd C. and Chen, Tianqi},
  journal = {arXiv preprint arXiv:2311.02103},
  year = {2023},
}

Pre-print

White-box Compiler Fuzzing Empowered by Large Language Models

Chenyuan Yang, Yinlin Deng, Runyu Lu, Jiayi Yao, Jiawei Liu, Reyhaneh Jabbarvand, and Lingming Zhang

arXiv preprint arXiv:2310.15991. 2023

PAPER Bib

@article{yang2023white,
  title = {White-box Compiler Fuzzing Empowered by Large Language Models},
  author = {Yang, Chenyuan and Deng, Yinlin and Lu, Runyu and Yao, Jiayi and Liu, Jiawei and Jabbarvand, Reyhaneh and Zhang, Lingming},
  journal = {arXiv preprint arXiv:2310.15991},
  year = {2023},
}

NeurIPS’23

To Appear

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang

Thirty-seventh Conference on Neural Information Processing Systems. 2023

PAPER Bib Poster Slides

@inproceedings{liu2023is,
  title = {Is Your Code Generated by Chat{GPT} Really Correct? Rigorous Evaluation of Large Language Models for Code Generation},
  author = {Liu, Jiawei and Xia, Chunqiu Steven and Wang, Yuyao and Zhang, Lingming},
  booktitle = {Thirty-seventh Conference on Neural Information Processing Systems},
  year = {2023},
  url = {https://openreview.net/forum?id=1qvx610Cu7},
}

Pre-print

NPS: A Framework for Accurate Program Sampling Using Graph Neural Network

Yuanwei Fang, Zihao Liu, Yanheng Lu, Jiawei Liu, Jiajie Li, Yi Jin, Jian Chen, Yenkuang Chen, Hongzhong Zheng, and Yuan Xie

arXiv preprint arXiv:2304.08880. 2023

PAPER Bib

@article{fang2023nps,
  title = {NPS: A Framework for Accurate Program Sampling Using Graph Neural Network},
  author = {Fang, Yuanwei and Liu, Zihao and Lu, Yanheng and Liu, Jiawei and Li, Jiajie and Jin, Yi and Chen, Jian and Chen, Yenkuang and Zheng, Hongzhong and Xie, Yuan},
  journal = {arXiv preprint arXiv:2304.08880},
  year = {2023},
}

ESEC/FSE’23

NeuRI: Diversifying DNN Generation via Inductive Rule Inference

Jiawei Liu, Jinjun Peng, Yuyao Wang, and Lingming Zhang

Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 2023

🏆 ACM SIGSOFT Distinguished Paper Award

PAPER Bib Slides Artifact

@inproceedings{liu2023neuri,
  author = {Liu, Jiawei and Peng, Jinjun and Wang, Yuyao and Zhang, Lingming},
  title = {NeuRI: Diversifying DNN Generation via Inductive Rule Inference},
  year = {2023},
  isbn = {9798400703270},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3611643.3616337},
  doi = {10.1145/3611643.3616337},
  booktitle = {Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering},
  pages = {657--669},
  numpages = {13},
  location = {San Francisco, CA, USA},
  series = {ESEC/FSE 2023},
}

ASPLOS’23

NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers

Jiawei Liu, Jinkun Lin, Fabian Ruffy, Cheng Tan, Jinyang Li, Aurojit Panda, and Lingming Zhang

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. 2023

🏆 Distinguished Artifact Award

PAPER Bib Poster Slides Artifact

@inproceedings{liu2023nnsmith,
  author = {Liu, Jiawei and Lin, Jinkun and Ruffy, Fabian and Tan, Cheng and Li, Jinyang and Panda, Aurojit and Zhang, Lingming},
  title = {NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers},
  year = {2023},
  isbn = {9781450399166},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  url = {https://doi.org/10.1145/3575693.3575707},
  doi = {10.1145/3575693.3575707},
  booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
  pages = {530--543},
  numpages = {14},
  keywords = {Deep Learning Compilers, Compiler Testing, Fuzzing},
  location = {Vancouver, BC, Canada},
  series = {ASPLOS 2023},
}

OOPSLA’22

Coverage-guided tensor compiler fuzzing with joint IR-pass mutation

Jiawei Liu, Yuxiang Wei, Sen Yang, Yinlin Deng, and Lingming Zhang

Proceedings of the ACM on Programming Languages 6 (OOPSLA1). Apr 2022

PAPER Bib Slides Artifact

@article{liu2022coverage,
  title = {Coverage-guided tensor compiler fuzzing with joint IR-pass mutation},
  author = {Liu, Jiawei and Wei, Yuxiang and Yang, Sen and Deng, Yinlin and Zhang, Lingming},
  journal = {Proceedings of the ACM on Programming Languages},
  volume = {6},
  number = {OOPSLA1},
  pages = {1--26},
  year = {2022},
  publisher = {ACM New York, NY, USA},
  url = {https://doi.org/10.1145/3527317},
  doi = {10.1145/3527317},
  month = apr,
  articleno = {73},
}

ACMMM’21 (OSC)

Fast and Flexible Human Pose Estimation with HyperPose

Yixiao Guo, Jiawei Liu, Guo Li, Luo Mai, and Hao Dong

Proceedings of the 29th ACM International Conference on Multimedia. Apr 2021

PAPER Bib Website

@inbook{guo2021fast,
  title = {Fast and Flexible Human Pose Estimation with HyperPose},
  author = {Guo, Yixiao and Liu, Jiawei and Li, Guo and Mai, Luo and Dong, Hao},
  booktitle = {Proceedings of the 29th ACM International Conference on Multimedia},
  pages = {3763--3766},
  year = {2021},
  isbn = {9781450386517},
  publisher = {ACM New York, NY, USA},
  url = {https://doi.org/10.1145/3474085.3478325},
  doi = {10.1145/3474085.3478325},
}

*PLSE conferences like OOPSLA and ESEC/FSE do not badge for reproducibility at artifact evaluation as it requires third-party re-implementation. Nonetheless, we got all badges we can get. :D

Academic Services

Organizing: LLM4Code@ICSE'24

Reviewer: TSE, TOSEM, DCAA@AAAI'23

Artifact Evaluation Committee: PLDI'23, OSDI'22, ATC'22

Talks

LLMs for Software Testing (Guest Lecture)

GAI4SE, NC State University Nov. 2023

Automating DL Compiler Bug Finding with NNSmith

Apache TVM Conference Mar. 2023

Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation

SAMPL Lunch Talks, University of Washington May 2022