TY - JOUR
T1 - Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images
AU - Kim, Jong Hyun
AU - Batchuluun, Ganbayar
AU - Park, Kang Ryoung
N1 - Publisher Copyright:
© 2018 Elsevier Ltd
PY - 2018/12/30
Y1 - 2018/12/30
N2 - Existing studies using visible-light cameras have mainly focused on methods of pedestrian detection during daytime. However, these studies found it difficult to detect pedestrians during nighttime with low external light. The NIR illuminator has limitations in terms of illumination angle and distance, and the illuminator's power needs to be adjusted depending on whether an object is near or distant. Although, thermal cameras were used for nighttime pedestrian detection, thermal cameras are currently expensive and thus difficult to install in many places. To solve these problems, attempts have been made to use visible-light cameras for nighttime pedestrian detection. However, most of these attempts considered an indoor environment where the distance to the object was short. This study proposes a method of pedestrian detection at nighttime using a visible-light camera and faster region-based convolutional neural network (R-CNN). In addition, as pedestrians cannot be reliably detected from a single nighttime image, we combined deep convolutional features in successive frames. Using Korea advanced institute of science and technology (KAIST) open database, we conducted experiments and observed that the proposed method performed better than the baseline methods at all times (day and night). In addition, through the experiments with national ICT Australia Ltd. (NICTA) open database, we confirm that the proposed method is effective for pedestrian detection at all times. Finally, we present theoretical grounds for the proposed fusion.
AB - Existing studies using visible-light cameras have mainly focused on methods of pedestrian detection during daytime. However, these studies found it difficult to detect pedestrians during nighttime with low external light. The NIR illuminator has limitations in terms of illumination angle and distance, and the illuminator's power needs to be adjusted depending on whether an object is near or distant. Although, thermal cameras were used for nighttime pedestrian detection, thermal cameras are currently expensive and thus difficult to install in many places. To solve these problems, attempts have been made to use visible-light cameras for nighttime pedestrian detection. However, most of these attempts considered an indoor environment where the distance to the object was short. This study proposes a method of pedestrian detection at nighttime using a visible-light camera and faster region-based convolutional neural network (R-CNN). In addition, as pedestrians cannot be reliably detected from a single nighttime image, we combined deep convolutional features in successive frames. Using Korea advanced institute of science and technology (KAIST) open database, we conducted experiments and observed that the proposed method performed better than the baseline methods at all times (day and night). In addition, through the experiments with national ICT Australia Ltd. (NICTA) open database, we confirm that the proposed method is effective for pedestrian detection at all times. Finally, we present theoretical grounds for the proposed fusion.
KW - Faster R-CNN
KW - Fusion of deep convolutional features
KW - Nighttime image
KW - Pedestrian detection
UR - http://www.scopus.com/inward/record.url?scp=85050198155&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2018.07.020
DO - 10.1016/j.eswa.2018.07.020
M3 - Article
AN - SCOPUS:85050198155
SN - 0957-4174
VL - 114
SP - 15
EP - 33
JO - Expert Systems with Applications
JF - Expert Systems with Applications
ER -