Fuzzy system based human behavior recognition by combining behavior prediction and recognition

With the development of intelligent surveillance systems, human behavior recognition has been extensively researched. Most of the previous methods recognized human behavior based on spatial and temporal features from (current) input image sequences, without the behavior prediction from previously recognized behaviors. Considering an example of behavior prediction, “punching” is more probable in the current frame when the previous behavior is “standing” as compared to the previous behavior being “lying down.” Nevertheless, there has been little study regarding the combination of currently recognized behavior information with behavior prediction. Therefore, we propose a fuzzy system based behavior recognition technique by combining both behavior prediction and recognition. To perform behavior recognition during daytime and nighttime, a dual camera system of visible light and thermal (far infrared light) cameras is used to capture 12 datasets including 11 different human behaviors in various surveillance environments. Experimental results along with the collected datasets and open database showed that the proposed method achieved higher accuracy of behavior recognition when compared to conventional methods.