Postdoctoral Scholar · UC Berkeley

Shi Chen

Computational Neuroscience Artificial Intelligence Visual Attention

Connecting artificial and biological intelligence through interpretable AI, neurophysiological recordings, and naturalistic behavior.

Placeholder portrait for Shi Chen

About

I am a postdoctoral scholar in the Department of Neuroscience at UC Berkeley, advised by Prof. Doris Tsao. My research interest lies at the intersection of artificial intelligence and neuroscience, with an emphasis on developing mechanistic interpretability methods to probe neural function. Alongside developing computational models, I design and conduct neurophysiological experiments to study episodic memory and embodied navigation in the primate brain.

Before transitioning to Neuroscience, I received my PhD in Computer Science at the University of Minnesota, working with Prof. Catherine (Qi) Zhao. My doctoral research focused on developing deep learning algorithms to understand human visual attention and reasoning, and translating these insights into computational models.

Research

Despite their distinct origins and learning mechanisms, artificial and biological intelligence both capture the structure of the same visual world. My research aims to bridge these two systems through a reciprocal approach: developing computational models to characterize neural representations and conducting neurophysiological experiments to uncover principles that can inform more generalizable AI.

01 Aligning foundation models and the brain

Foundation models learn rich representations and provide a useful basis for in silico experiments. Nevertheless, due to considerable differences in representation format (e.g., dense spatial features vs a handful of neurons) and available data (e.g., 2 hours of electrophysiological recordings), applying these models to interpret neural dynamics remains a major challenge. My research bridges this gap by developing self-supervised learning paradigm that anchors the two representations into a shared coordinate, preserving the structure developed in the foundation models while captureing the representational geometry in the neural activity.

02 Mechanistic interpretability methods for neural activity

The brain is complex and understanding it requires a systematic investigation. My research develops mechanistic interpretability methods to dissect neural activity from different perspectives, including sparse neural networks for selectivity identification, generative models for stimulus reconstruction, targeted perturbation to examine causal relationships, and dynamical systems analysis to understand the temporal evolution.

03 Neural codes of episodic experience and memory

Intelligence relies on the capability to consolidate episodic experiences into memory and retrieve it to guide future actions. How does the brain implement these computations? I approach the problems with a collection of naturalistic, embodied virtual-reality tasks and high-density Neuropixels recordings in nonhuman primates. Our studies reveal a compositional code in the hippocampus that represents episodic experiences, and discover coordinated memory ``replay'' in the hippocampus and the inferotemporal cortex.

04 Human visual attention and reasoning

Vision is the foundation of intelligence. My doctoral research uses visual attention as a window to study human perception and reasoning. Using eye-tracking experiments and symbolic reasoning models, I investigate how humans allocate their attention during the decision-making process, and integrate the learned insights into visual reasoning models. I also develop autism screening models based on gaze patterns recorded under diverse paradigms.

Publications

  1. 2026

    Prototypes of the Mind: A Unified Framework for Probing the Visual Brain

    Shi Chen, Seoyoung Ahn, Doris Y. Tsao

    NeurIPS 2026

  2. 2026

    Rapid Concerted Switching of the Neural Code in the Inferotemporal Cortex

    Yuelin Shi, Dasheng Bi, Janis K. Hesse, Frank F. Lanfranchi, Shi Chen, Doris Y. Tsao

    Nature 2026

  3. 2024

    Deep Learning to Interpret Autism Spectrum Disorder Behind the Camera

    Shi Chen, Ming Jiang, Qi Zhao

    IEEE TCDS 2024

  4. 2024

    Every Problem, Every Step, All In Focus: Learning to Solve Vision-Language Problems with Integrated Attention

    Xianyu Chen, Jinhui Yang, Shi Chen, Louis Wang, Ming Jiang, Qi Zhao

    IEEE TPAMI 2024

  5. 2023

    What Do Deep Saliency Models Learn about Visual Attention?

    Shi Chen, Ming Jiang, Qi Zhao

    NeurIPS 2023

  6. 2023

    Toward Multi-Granularity Decision-Making: Explicit Visual Reasoning with Hierarchical Knowledge

    Yifeng Zhang, Shi Chen, Qi Zhao

    ICCV 2023

  7. 2023

    Divide and Conquer: Answering Questions with Object Factorization and Compositional Reasoning

    Shi Chen, Qi Zhao

    CVPR 2023

  8. 2023

    Learning from Unique Perspectives: User-aware Saliency Modeling

    Shi Chen, Nachiappan Valliappan, Shaolei Shen, Xinyu Ye, Kai Kohlhoff, Junfeng He

    CVPR 2023

  9. 2022

    REX: Reasoning-aware and Grounded Explanation

    Shi Chen, Qi Zhao

    CVPR 2022

  10. 2022

    VisualHow: Multimodal Problem Solving

    Jinhui Yang*, Xianyu Chen*, Ming Jiang, Shi Chen, Louis Wang, Qi Zhao

    CVPR 2022

  11. 2021

    Attention to Action: Leveraging Attention for Object Navigation

    Shi Chen, Qi Zhao

    BMVC 2021

  12. 2021

    Attention in Reasoning: Dataset, Analysis, and Modeling

    Shi Chen*, Ming Jiang*, Jinhui Yang, Qi Zhao

    IEEE TPAMI 2021

  13. 2020

    AiR: Attention with Reasoning Capability

    Shi Chen*, Ming Jiang*, Jinhui Yang, Qi Zhao

    ECCV 2020 · Oral

  14. 2020

    Fantastic Answers and Where to Find Them: Immersive Question-directed Visual Attention

    Ming Jiang*, Shi Chen*, Jinhui Yang, Qi Zhao

    CVPR 2020

  15. 2019

    Attention-based Autism Spectrum Disorder Screening with Privileged Modality

    Shi Chen, Qi Zhao

    ICCV 2019

  16. 2019

    Shallowing Deep Networks: Layer-wise Pruning based on Feature Representations

    Shi Chen, Qi Zhao

    IEEE TPAMI 2019

  17. 2018

    Boosted Attention: Leveraging Human Attention for Image Captioning

    Shi Chen, Qi Zhao

    ECCV 2018

* Equal contribution

Service

Reviewer/program committee: NeurIPS, ICLR, ICML, CVPR, ICCV, ECCV, AAAI, AISTATS, TPAMI, IJCV, TNNLS, TMM.