Robust Interstory Visual Displacement and Rotation Measurement in Disaster Scenes Using a Learning‐Based Point‐Tracking Method

Accurate measurement of interstory displacement and rotation is essential for postdisaster structural safety assessment. While conventional sensor‐based monitoring systems, such as accelerometers, global positioning system (GPS), and displacement sensors, have been widely used, their deployment is often restricted by cost, installation constraints, and vulnerability to damage. Recent advances in computer vision have enabled structural displacement estimation directly from video, yet performance on real low‐cost surveillance videos remains limited by severe occlusion, image noise, and motion blur. This study introduces a learning‐based visual multipoint tracking framework capable of estimating both interstory displacement and in‐plane rotation from challenging disaster surveillance videos. The method builds upon the CoTracker3 model, a multipoint deep tracking model, which was fine‐tuned with the noise‐augmented MOVi‐F dataset to improve robustness under degraded visual conditions. Without requiring preinstalled markers, the approach automatically tracks the selected initial points across frames and estimates the displacement and in‐plane rotation from point trajectories. The framework was quantitatively evaluated through a full‐scale shaking‐table experiment, while the real‐earthquake videos were used as feasibility demonstrations. In the controlled shaking‐table experiments, Case 2 combined motion blur, noise, occlusion, and rotation. Using 200 initialized points, the fine‐tuned model reduced the rotation root‐mean‐square error (RMSE) from 0.226° to 0.137° and the Y ‐displacement RMSE from 1.389 to 0.941 mm compared with the original CoTracker3. The real‐earthquake videos are presented as feasibility demonstrations, indicating the potential of the framework for low‐cost postdisaster structural monitoring.