Pengcheng He

Deep Reinforcement Learning from Hierarchical Preference Design ICML 2025 DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models ICLR 2024 Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection EMNLP 2024 Learning Stackable and Skippable LEGO Bricks for Efficient, Reconfigurable, and Variable-Resolution Diffusion Modeling ICLR 2024 LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models ICLR 2024 Personalized Abstractive Summarization by Tri-agent Generation Pipeline EACL 2024 PROM: A Phrase-level Copying Mechanism with Pre-training for Abstractive Summarization COLING 2024 Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective ICLR 2024 Switchable Decision: Dynamic Neural Generation Networks ICML 2024 Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning ICLR 2023 DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing ICLR 2023 Diffusion-GAN: Training GANs with Diffusion ICLR 2023 DIONYSUS: A Pre-trained Model for Low-Resource Dialogue Summarization ACL 2023 Guiding Large Language Models via Directional Stimulus Prompting NeurIPS 2023 HyperTuning: Toward Adapting Large Language Models without Back-propagation ICML 2023 In-Context Learning Unlocked for Diffusion Models NeurIPS 2023 Less is More: Task-aware Layer-wise Distillation for Language Model Compression ICML 2023 LMGQS: A Large-scale Dataset for Query-focused Summarization EMNLP 2023 LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation ICML 2023 Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models NeurIPS 2023 POUF: Prompt-Oriented Unsupervised Fine-tuning for Large Pre-trained Models ICML 2023 Query Rewriting in Retrieval-Augmented Large Language Models EMNLP 2023 Truncated Diffusion Probabilistic Models and Diffusion-based Adversarial Auto-Encoders ICLR 2023 Z-Code++: A Pre-trained Language Model Optimized for Abstractive Summarization ACL 2023 ALLSH: Active Learning Guided by Local Sensitivity and Hardness NAACL 2022 CAMERO: Consistency Regularized Ensemble of Perturbed Language Models with Weight Sharing ACL 2022 Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention IJCAI 2022 MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation NAACL 2022 No Parameters Left Behind: Sensitivity Guided Adaptive Learning Rate for Training Large Transformer Models ICLR 2022 OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering NAACL 2022 PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance ICML 2022 Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach EMNLP 2021 ARCH: Efficient Adversarial Regularized Training with Caching EMNLP 2021 DEBERTA: DECODING-ENHANCED BERT WITH DISENTANGLED ATTENTION ICLR 2021 Generation-Augmented Retrieval for Open-Domain Question Answering ACL 2021 Reader-Guided Passage Reranking for Open-Domain Question Answering ACL 2021 Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization ACL 2021 Token-wise Curriculum Learning for Neural Machine Translation EMNLP 2021 UnitedQA: A Hybrid Approach for Open Domain Question Answering ACL 2021 Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning EMNLP 2020 On the Variance of the Adaptive Learning Rate and Beyond ICLR 2020 SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization ACL 2020 The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding ACL 2020 A Hybrid Neural Network Model for Commonsense Reasoning EMNLP 2019 Multi-Task Deep Neural Networks for Natural Language Understanding ACL 2019