This paper presents a comparison between the performance of Kalman filter and particle filter in the single object tracking system. The performance parameters of interest are the processing time of the algorithm and the accuracy of the tracking system. The algorithms are implemented using MATLAB and tested using two video sequences, and the results are compared and discussed.