Zhongkai Xue

Introduction

I am currently pursuing my PhD in Computer Science at the Institute of Automation, Chinese Academy of Sciences (CASIA), supervised by Prof. Liang Wang. Previously, I have been fortunate to learn from Prof. Rex Ying and Prof. Tianshu Yu.

My research focuses on multimodal intelligence, particularly on how models perceive, remember, and interact with the world over time. I believe intelligence is inherently multimodal, thus capable AI systems should learn from and interact with the world through multiple modalities. My recent work has focused on these topics:

Agent Memory Systems: enabling agents to retain, organize, and use knowledge from long-term experience.
Streaming Omni Models: enabling real-time perception and interaction over continuous multimodal streams.
Multimodal Generation: connecting multimodal understanding with generation and world modeling.

I am always happy to discuss these topics, explore new ideas, and collaborate on interesting research questions!

Education

PhD Student in Computer Science @ Institute of Automation, Chinese Academy of Sciences

From Sep 2026

Exchange Student in Math & Statistics @ University of Oxford, St Hilda's College

Oct 2024 – Mar 2025

Bachelor in Financial Engineering @ The Chinese University of Hong Kong, Shenzhen

Sep 2022 – May 2026

Publications 108 Citations

Memory Systems

RoboMemory: A Brain-Inspired Multi-Memory Framework for Interactive Environmental Learning in Physical Embodied Systems

Under Review, 2026

Abstract: We introduce RoboMemory, a framework inspired by the brain that brings spatial, temporal, episodic, and semantic memory together for embodied agents. It maintains a dynamic spatial knowledge graph for consistent memory updates and places a planner and critic in a closed feedback loop, supporting adaptive decisions, planning over long horizons, and cumulative learning through repeated interactions.

[arxiv] [code]

Multimodal Interactions

The Underappreciated Power of Vision Models for Graph Structural Understanding

Conference on Neural Information Processing Systems, 2025

Abstract: We study how vision models understand graph structure and introduce GraphAbstract, a benchmark that tests holistic perception of organizational patterns, symmetry, connectivity, and critical elements. Vision models surpass GNNs on global structural tasks and generalize across scales, showing that visual perception offers a promising path toward graph foundation models.

[arxiv] [code]

MJ-VIDEO: Fine-Grained Benchmarking and Rewarding Video Preferences in Video Generation

Conference on Neural Information Processing Systems, Spotlight, 2025

Abstract: We introduce MJ-BENCH-VIDEO, a comprehensive benchmark with 28 criteria covering alignment, safety, fineness, coherence and consistency, and bias and fairness. Building on this benchmark, MJ-VIDEO uses a mixture of experts to select relevant evaluators dynamically, improving both overall and detailed preference judgments while helping video generation models better follow human preferences.

[arxiv] [code] [site]

Position & Reviews

When Vision Meets Graphs: A Survey on Graph Reasoning and Learning

International Joint Conference on Artificial Intelligence, 2026

Abstract: We present the first systematic survey of vision meets graphs, an emerging area that treats visual depictions as direct inputs for graph reasoning and learning. Through a framework of rendering, perception, and inference, we organize prior work around vision for graph reasoning, vision for graph learning, and scientific graphs, then outline a path toward models that perceive and reason about graphs as scientists do.

[techrxiv]

Political-LLM: Large Language Models in Political Science

arXiv, 2025

Abstract: We present Political-LLM, a framework for understanding how LLMs are integrated into computational political science. Its dual perspective taxonomy connects political tasks such as prediction, generation, simulation, and causal inference with methods for data preparation, fine tuning, and evaluation. We also identify challenges involving domain data, bias and fairness, human expertise, and evaluation.

[arxiv] [code] [site]