Social interactions strongly impact the brain and the body. However, high-resolution descriptions of these important physical interactions and their neural correlates are lacking. This study described a hardware/software system and analysis pipeline that combines 3D videography, deep learning, physical modeling, and GPU-accelerated robust optimization with automatic analysis of neuronal receptive fields recorded in interacting mice. This system is capable of fully automatic multi-animal tracking with minimal errors during spontaneous social encounters with simultaneous electrophysiological recordings. The dataset includes several videos and electrophysiology data. This system can be useful for neurobehavioral studies of multiple animals interacting in complex low-light environments.