Domain adaptation to enhance (2 + 1)D CNN dynamic analysis of cell collective behavior in time-lapse microscopy videos

In recent years, 2D CNNs have excelled in analyzing single-frame video sequences, prompting the evolution of standard architectures toward full 3D CNNs. This transition, while enhancing the modeling of spatial and temporal information in video activities, demanded substantial data for effective training. To alleviate this challenge, we introduced a switched multitask training strategy. This approach involves factorizing 3D layers into (2 + 1)D convolutions, training only 2D (1D) layers for spatial (temporal) tasks and switching off the training of the 1D (2D) layers. Additionally, we addressed data scarcity by generating synthetic stylized video sequences. These were crafted using stochastic particle models of collective cell motions, further modified through neural style transfer to mimic real video data. Such a domain adaptation strategy facilitated the creation of training data impractical to obtain in the real world. Transferring knowledge from the switched (2 + 1) CNN to real video data, we encoded wound healing experiments of three distinct cell lines—human melanocytes cells M14, mouse neuroblastoma cells N1, and human prostate cells PC3—into deep features. Employing a novel feature selection strategy based on robustness to disturbances, we discriminated the three wound healing processes. Average classification accuracy of 92.91% (0.11%), 91.50% (0.37%), and 88.81% (0.29%) was obtained for the original real videos, the real videos with progressive altered levels of focus, and levels of brightness, respectively. The proposed approach proved to be a powerful tool for analyzing the spatiotemporal dynamics of biological systems, even in the presence of fluctuations.