Contour Tracking

Unsupervised Contour Tracking of Live Cells
by Mechanical and Cycle Consistency Losses

CVPR 2023

KAIST
Boston Children's Hospital
Harvard Medical School
KAIST
Imperial College London

Paper

Code

best viewed in Full HD

Abstract

Analyzing the dynamic changes of cellular morphology is important for understanding the various functions and characteristics of live cells including stem cells and metastatic cancer cells. To this end, we need to track all points on the highly deformable cellular contour in every frame of live cell video. Local shapes and textures on the contour are not evident, and their motions are complex often with expansion and contraction of local contour features. The prior arts for optical flow or deep point set tracking are unsuited due to the fluidity of cells and previous deep contour tracking does not consider point correspondence. We propose the first deep learning-based tracking of cellular (or more generally viscoelastic materials) contours with point correspondence by fusing dense representation between two contours with cross attention. Since it is impractical to manually label dense tracking points on the contour, unsupervised learning comprised of the mechanical and cyclical consistency losses is proposed to train our contour tracker. The mechanical loss forcing the points to move perpendicular to the contour effectively helps out. For quantitative evaluation, we labeled sparse tracking points along the contour of live cells from two live cell datasets taken with phase contrast and confocal fluorescence microscopes. Our contour tracker quantitatively outperforms compared methods by a large margin and produces qualitatively more favorable results.

Dense Correspondences on Live Cell Videos (best viewed in Full HD)

Dense Correspondences on Jellyfish

Ordered Contour Points shown with Rainbow Color

Citation

@InProceedings{Jang_2023_CVPR,
  author    = {Jang, Junbong and Lee, Kwonmoo and Kim, Tae-Kyun},
  title     = {Unsupervised Contour Tracking of Live Cells by Mechanical and Cycle Consistency Losses},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2023},
  pages     = {227-236}
}

Inspired by Im2Hands