Publications

TTVD: Towards a Geometric Framework for Test-Time Adaptation Based on Voronoi Diagram

Mingxi Lei, Chunwei Ma, Meng Ding, Yufan Zhou, Ziyun Huang, Jinhui Xu

The Thirteenth International Conference on Learning Representations (ICLR), 2025.

NSV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding

Jian Chen, Ruiyi Zhang, Yufan Zhou, Tong Yu, Franck Dernoncourt, Jiuxiang Gu, Ryan A. Rossi, Changyou Chen, Tong Sun

The Thirteenth International Conference on Learning Representations (ICLR), 2025.

Numerical Pruning for Efficient Autoregressive Models

Xuan Shen, Zhao Song, Yufa Zhou, Bo Chen, Jing Liu, Ruiyi Zhang, Ryan A. Rossi, Hao Tan, Tong Yu, Xiang Chen, Yufan Zhou, Tong Sun, Pu Zhao, Yanzhi Wang, Jiuxiang Gu

The 39th Annual AAAI Conference on Artificial Intelligence (AAAI), 2025.

ARTIST: Improving Generation of Text-rich Images by Disentanglement

Jianyi Zhang*, Yufan Zhou*, Jiuxiang Gu, Curtis Wigington, Tong Yu, Yiran Chen, Tong Sun, Ruiyi Zhang.

Improve text rendering ability of text-to-image diffusion model.

IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2025.

Customizing Language Models for Text-to-Layout Planning

Jian Chen, Ruiyi Zhang, Yufan Zhou, Jennifer Healey, Jiuxiang Gu, Changyou Chen.

Empirical Methods in Natural Language Processing (EMNLP), 2024.

Agent-DocEdit: Language-Instructed LLM Agent for Content-Rich Document Editing

Te-Lin Wu, Rajiv Jain, Yufan Zhou, Puneet Mathur, Vlad I Morariu.

First Conference on Language Modeling (CoLM), 2024.

Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models

Zihao Lin, Mohammad Beigi, Hongxuan Li, Yufan Zhou, Yuxiang Zhang, Qifan Wang, Wenpeng Yin, Lifu Huang.

Annual Meeting of the Association for Computational Linguistics (ACL), 2024.

Customization Assistant for Text-to-Image Generation

Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Tong Sun.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

An assistant which can generate creative images for specific user-input subject along with text explanation and elaboration in 2-5 seconds, without any fine-tuning.

TRINS: Towards Multimodal Language Models That Can Read

Ruiyi Zhang, Yanzhe Zhang, Jian Chen, Yufan Zhou, Jiuxiang Gu, Changyou Chen, Tong Sun.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints

Jian Chen, Ruiyi Zhang, Yufan Zhou, Changyou Chen.

The Twelfth International Conference on Learning Representations (ICLR) 2024.

Enhanced Visual Instruction Tuning for Text-Rich Image Understanding

Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, Tong Sun.

Workshop on Instruction Tuning and Instruction Following at NeurIPS 2023.

Shifted Diffusion for Text-to-image Generation

Yufan Zhou, Bingchen Liu, Yizhe Zhu, Xiao Yang, Changyou Chen, Jinhui Xu.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

We propose a method termed Corgi, which can better generate image embeddings from text inside multimodal embedding space.
It benefits both standard and language-free text-to-image generation. And yes, I do have a Corgi.

LAFITE: Towards Language-Free Training for Text-to-Image Generation

Yufan Zhou, Ruiyi Zhang, Changyou Chen, Chunyuan Li, Chris Tensmeyer, Tong Yu, Jiuxiang Gu, Jinhui Xu, Tong Sun.

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.

Our proposed work, Lafite, is the first work which can successfully train text-to-image generation model with image-only dataset.

TiGAN: Text-Based Interactive Image Generation and Manipulation

Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Chris Tensmeyer, Tong Yu, Changyou Chen, Jinhui Xu, Tong Sun.

AAAI conference on Artificial Intelligence (AAAI), 2022.

Meta-Learning with Neural Tangent Kernels

Yufan Zhou*, Zhenyi Wang*, Jiayi Xian, Changyou Chen, Jinhui Xu.

International Conference on Learning Representations (ICLR), 2021.

MixKD: Towards Efficient Distillation of Large-scale Language Models

Kevin J Liang, Weituo Hao, Dinghan Shen, Yufan Zhou, Weizhu Chen, Changyou Chen, Lawrence Carin.

International Conference on Learning Representations (ICLR), 2021.

Learning Manifold Implicitly via Explicit Heat Kernel Learning

Yufan Zhou, Changyou Chen, Jinhui Xu.

Conference on Neural Information Processing Systems (NeurIPS), 2020.

Weakly-supervised Brain Tumor Classification with Global Diagnosis Label (Oral)

Yufan Zhou, Zheshuo Li, Chunwei Ma, Mingchen Gao, Changyou Chen, Hong Zhu, Jinhui Xu.

IEEE International Symposium on Biomedical Imaging (ISBI), 2020.

Variational Adversarial Kernel Learned Imitation Learning (Spotlight)

Fan Yang, Alina Vereshchaka, Yufan Zhou, Changyou Chen, Wen Dong.

AAAI conference on Artificial Intelligence (AAAI), 2020.

Learning Diverse Stochastic Action-Generators by Learning Smooth Latent Transitions (Spotlight)

Zhenyi Wang, Ping Yu, Yang Zhao, Ruiyi Zhang, Yufan Zhou, Junsong Yuan, Changyou Chen.

AAAI conference on Artificial Intelligence (AAAI), 2020.

Holistic Brain Tumor Screening and Classification Based on DenseNet and Recurrent Neural Network

Yufan Zhou, Zheshuo Li, Hong Zhu, Changyou Chen, Mingchen Gao, Kai Xu, Jinhui Xu.

International Conference on Medical Image Computing and Computer Assisted Intervention, Brain Lesion Workshop (BrainLes, MICCAI), 2018.

Manuscripts/Preprints

SUGAR: Subject-Driven Video Customization in a Zero-Shot Manner

Yufan Zhou, Ruiyi Zhang, Jiuxiang Gu, Nanxuan Zhao, Jing Shi, Tong Sun.

A zero-shot method for video customization, which can generate creative videos for user-input subject image, with desired style, color, texture, background required by user-input text.

Toffee: Efficient Million-Scale Dataset Construction for Subject-Driven Text-to-Image Generation

Yufan Zhou, Ruiyi Zhang, Kaizhi Zheng, Nanxuan Zhao, Jiuxiang Gu, Zichao Wang, Xin Eric Wang, Tong Sun.

Efficient method to construct dataset for subject-driven T2I generation, which can save at least tens of thousands of GPU hours.

LLaVAR: Enhanced Visual Instruction Tuning for Text-rich Image Understanding

Yanzhe Zhang, Ruiyi Zhang, Jiuxiang Gu, Yufan Zhou, Nedim Lipka, Diyi Yang, Tong Sun.

Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach

Yufan Zhou, Ruiyi Zhang, Tong Sun, Jinhui Xu.

A novel framework for customized text-to-image generation without the use of regularization.
We can efficiently customize a large-scale text-to-image generation model on single GPU, with only one image provided by the user.