Research question
What can multimodal and language models understand about a driving scene, and how can their outputs support useful interaction? Collaborative research develops driving-scene benchmarks, instruction-based interfaces over established driving functions, and scene-grounded dialogue. Complementary studies investigate scenario generation, traffic simulation, and selective reasoning, each with its own evaluation task.
NuPlanQA provides multi-view driving-scene question answering and evaluates perception and spatial reasoning. PDB-Eval tests descriptions and explanations of personalized behavior. Concept-bottleneck control studies explanations through human-understandable concepts.
NuPlanQA, ICCV 2025 · PDB-Eval, IEEE IV 2025 · Echterhoff et al., WACV 2024
LaMPilot studies language-model programs over established driving functions. LLM4AD contributes a collaborative review, benchmark, and vehicle-platform experiments that examine language models in autonomous-driving applications.
SC-ADAS combines scene interpretation, multi-turn dialogue, and structured function calling. Driver-confirmed requests are converted to ADAS commands; evaluation examines interaction behavior and the latency and context costs of cloud-based generative models.
SIMSplat studies language-aligned driving-scenario generation with 4D Gaussian splatting, while NIVA studies learned closed-loop multi-agent traffic simulation.
Tonic meta-control maintains a persistent vigilance state to decide when costly reasoning modules run, evaluated with a synthetic testbench and highway driving simulation.
Coauthored studies explore agentic trip planning and input-ambiguity detection in language models.
Chen et al., IEEE IV 2026 · Chen et al., IJCNLP-AACL 2026
LaMPilot's published framework turns language instructions into executable driving programs for evaluation in simulation.
Yunsheng Ma and coauthors (2024), Figure 1, arXiv:2312.04372v2. CC BY-NC-SA 4.0. Rasterized and cropped from the author preprint; labels unchanged. Source for lampilot-framework CC BY-NC-SA 4.0
Tonic's simulation-study architecture separates persistent vigilance from surprise-triggered escalation.
Kyungtae Han and coauthors (2026), existing public Tonic Meta-Control project, architecture figure; unchanged. Source for tonic-architecture
A curated selection. Citations above link to each paper; see the full publication list for all work.
S. Park, C. Cui, Y. Ma, A. Moradipari, R. Gupta, K. Han, and Z. Wang
Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)
Y. Ma, C. Cui, X. Cao, W. Ye, P. Liu, J. Lu, A. Abdelraouf, R. Gupta, K. Han, A. Bera, J. Rehg, and Z. Wang
2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA
J. Wu, J. Echterhoff, K. Han, A. Abdelraouf, R. Gupta, and J. McAuley
2025 IEEE Intelligent Vehicles Symposium (IV), Cluj-Napoca, Romania
K. Han, Y. Chen, R. Gupta, and O. Altintas
2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC)
J. Lu, R. Gupta, A. Moradipari, K. Han, R. Zhang, and Z. Wang
2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
K. Han, Y. Chen, N. Ammar, and O. Altintas
2026 IEEE 29th International Conference on Intelligent Transportation Systems (ITSC)