An adaptive multi-sensor visual attention model

The emerging recurrent visual attention models mostly utilize a sensor to continuously capture features from the input, which requires a suited design for the sensor. Researchers usually need a number of attempts to determine optimal structures for the sensor and corresponding modules. In this work, an adaptive multi-sensor visual attention model (AM-MA) is proposed to enhance the recurrent visual attention model. The proposed model uses several sensors to observe the original input recurrently, while the number of sensors can be added adaptively. Each sensor generates a hidden state and is followed by a location network to provide the deployment scheme. We design a self-evaluation mechanism for AM-MA, by which it can decide whether to add new sensors during training. Besides, the proposed AM-MA leverages a fine-tune mechanism to avoid a lengthy training process. AM-MA is a parameter-insensitive model. That is, there is no need for researchers to pre-train the model for finding the optimal structure in the case of unknown complexity. Experimental results show that the proposed AM-MA not only outperforms the renowned sensor-based attention model on image classification tasks, but also achieves satisfactory results when given an inappropriate structure.