Heterogeneous information saliency features' fusion approach for machine's environment sounds based awareness

Human beings are more intelligent in dealing with sound which occurred in everyday life than robots or other kind of unmanned ground vehicles because of the instinct of sense or awareness, which is an ability to distinguish the most salient sound, object or events in the surrounding environment. Inspired by the biological acoustic awareness of human hearing system and the visual saliency talent of human vision, a heterogeneous information saliency feature fusion (HISFF) approach which simulates human awareness of environment sound for machine's awareness is proposed in this paper. The sound signal is visualized by using the Short-Time Fourier Transform (STFT) algorithm in order to convert the acoustic saliency into visual saliency, and the Mel-Frequency Cepstrum Coefficient (MFCC) is used to represent the human acoustic awareness. The proposed HISFF approach is tested by using the environment sound data which collected from the real world of both indoor and outdoor environment. The results show that this approach is able to extract the saliency signal from both long-term and short-term sound signal successfully and clearly, and conducts to very distinguishable features for machine's environment sounds based awareness.