Adaptive path planning for wafer second probing via an attention-based hierarchical reinforcement learning framework with shared memory

In semiconductor manufacturing, wafer probing is a quality control process before packaging, usually performed by an automated machine with a fixed path. The unqualified grains in the first detection need to be confirmed again. The fixed path method is inefficient and requires manual intervention for the second wafer probing on randomly scattered grains. To this end, we propose a reinforcement learning-based adaptive path planning method for second wafer probing. To simplify decision-making in a large state space, we propose a novel attention-based hierarchical reinforcement learning method with shared memory (AHRL-SM) and introduce it into wafer probing for the first time. The high-level agent is responsible for focusing on the region with a large number of grains to be detected, while the low-level agent is responsible for planning the moving path of the probe in the specified sub-region. The soft attention mechanism and recurrent neural network are incorporated into the probing architecture to facilitate original image feature extraction and historical information acquisition, respectively. In addition, we propose a unique shared memory mechanism to further improve decision-making efficiency. The Markov decision process of the complete wafer second probing and the performance verification of the proposed method are thoroughly described in this work. Compared with the existing path planning methods for wafer probing, sufficient experimental results confirm that our method has obvious advantages in probing efficiency, grain surface protection, and generalization.