Computer Vision for Structural Dynamics and Health Monitoring. Dongming Feng

While most SHM studies are based on the measurement of structural acceleration responses, displacement responses more directly reflect overall structural stiffness and thus offer the potential for improved accuracy in the assessment of structural conditions. As shown in Figure 1.1, sensors currently available for measuring structural displacements can be classified as contact types, such as the linear variable differential transformer (LVDT); and string potentiometer and noncontact types, such as GPS, laser vibrometers, and radar interferometry systems. These displacement sensors suffer from many limitations for field applications. For example, it is costly and highly difficult, if not impossible, to install an LVDT or a string potentiometer, which requires a stationary reference point; noncontact laser vibrometers are generally accurate but are costly and have a short measurement distance because of safety regulations; GPS sensors are easier to install, but the measurement accuracy is limited; and an interferometric radar system allows remote measurements with good resolution but requires reflecting surfaces mounted on the structure, which can be difficult to install and maintain.

Figure 1.1 Common displacement sensors: (a) LVDT; (b) laser vibrometer; (c) GPS.

Rapid advances in cameras and computer vision techniques have made vision‐based sensing a promising alternative to conventional sensors for structural dynamic displacement measurement and health monitoring. As shown in Figure 1.2, a typical computer vision–based sensor system simply consists of one or more digital cameras and a computing unit such as a laptop or a tablet PC with measurement software installed. Video images of features on a structure, such as rivets and edges, are captured by the camera and streamed into the computer. By processing the digital video images using the measurement software, displacement time histories can be obtained at multiple locations simultaneously. The emerging vision‐based sensor offers significant advantages over conventional contact‐type and other noncontact‐type displacement sensors, as summarized next [16]:

      1 In contrast to a contact‐type sensor (such as an LVDT or a string potentiometer), which requires time‐consuming, costly installation on the structure and physical connections to a stationary reference point, a computer vision sensor requires no physical access to the structure, and the camera can be set up at a convenient remote location. This represents significant savings of both time and cost. For monitoring bridges, for example, no traffic control is required. In addition, each contact‐type sensor measures one‐dimensional (1D) displacement, but a single computer vision camera can measure two‐dimensional (2D) displacements simultaneously.Figure 1.2 Vision‐based remote displacement sensor.

      2 Compared with a noncontact GPS, which requires installation on the structure (but not a stationary reference point), a vision‐based sensor is far more accurate and less expensive. Depending on the cost, the GPS measurement error is typically in the range of 5–10 mm: more than an order of magnitude larger than that of a vision sensor.

      3 Unlike a noncontact laser vibrometer, which must be placed very close to the measurement target due to the limited allowable laser power, a vision sensor can be placed hundreds of meters away (with the help of an appropriate zoom lens) and still achieve satisfactory measurement accuracy.

      4 In contrast to conventional displacement sensors, almost all of which are point‐wise sensors, a single vision sensor can simultaneously track structural displacements at multiple points. More importantly, one can easily alter the measurement points after video images are taken, offering unique flexibility for achieving better SHM results.A comparison between commonly used vibration sensors and vision‐based displacement sensors is summarized in Table 1.1.

Table 1.1 Comparison of sensors for measuring structural vibrations.

Sensors	Measure	Pros	Cons
Wired or wireless accelerometer	Acceleration	Suitable for continuous monitoringHardware easily availableSensitive to high‐frequency vibrations	High cost of sensor systemHigh cost of installation and maintenanceContact sensorSingle‐point measurementAdditional mass on the structure may affect output
LVDT	Displacement	Hardware easily available	Difficult and costly to installContact sensorOne‐dimensional measurementSingle‐point measurement
Laser vibrometer	Velocity or displacement	NoncontactAccurate	High cost of sensor systemNot suitable for continuous monitoringLimited measurement distance
Computer vision sensor	Displacement	Noncontact, continuous monitoringLow‐cost industrial or consumer‐grade video camerasTwo‐ or three‐dimensional measurementMultiple flexible measurement points on the visible object surface	Accuracy affected by weather, light, and camera motion

      About 10 years ago, the research community started to develop computer vision–based sensor technology for displacement measurement of large‐size structures in controlled laboratory and challenging field environments. Modal analysis can be performed on the displacement data to extract natural frequencies and the mode shapes of a structure. Moreover, by analyzing the measured displacement time histories and modal analysis results, analytical models and parameters of the structure can be updated, damage detected, and structural integrity assessed. The adoption of vision sensors can significantly reduce the testing cost and time associated with conventional instrumentations. For example, Poozesh et al. [17] pointed out that testing a typical 50 m utility‐scale wind turbine blade requires approximately 200 gages (costing $35 000–$50 000) and about three weeks to set up a conventional strain gauge system, while by contrast, a multicamera system could streamline the blade‐testing process by eliminating the sensor instrumentation and reducing the setup time to two days.

      It should be noted that computer vision sensing has been attracting attention and gaining popularity in two major areas of structural engineering: (i) vision‐based sensors for displacement measurement and their SHM applications for modal/parameter identification, damage detection, force estimation, and model validation and updating; and (ii) visual monitoring of structural surface for defect detection and condition assessment, including the use of unmanned aerial vehicles (UAVs) and machine learning techniques. The emphasis of this book is on the former application.

1.3 Organization of the Book

      The goal of this book is to encourage the application of the emerging computer vision–based sensing technology not only in scientific research but also in engineering practice such as field condition assessment of civil engineering structures and infrastructure systems. This book may serve as a textbook for graduate students, researchers, and practicing engineers. Thus much emphasis has been placed on making computer vision algorithms and their applications in structural dynamics and SHM easily accessible and understandable. To achieve this goal, throughout the book, MATLAB computer code is provided for most of the problems that are discussed. Even though the book is conceived as an entity, its chapters