To solve the problem of passive sensor data association in multi-sensor multi-target tracking, a novel linear-time direct data assignment (DDA) algorithm is proposed in this paper. Different from existing methods which solve the data association problem in the measurement domain, the proposed algorithm solves the problem directly in the target state domain. The number and state of candidate targets are preset in the region of interest, which can avoid the problem of combinational explosion. The time complexity of the proposed algorithm is linear with the number of sensors and targets while that of the existing algorithms are exponential. Computer simulations show that the proposed algorithm can achieve almost the same association accuracy as the existing algorithms, but the time consumption can be significantly reduced.