In the present paper a numerical method based on fourth and sixth order finite difference with collocation method is presented for the numerical solution of partial integro-differential equation (PIDE). A composite weighted trapezoidal rule is manipulated to handle the numerical integrations which results in a closed-form difference scheme. The efficiency and accuracy of the scheme is validated by its application to one test problem which have exact solutions. Numerical results show that theses fourth and sixth-order schemes have the expected accuracy. The most advantages of compact finite difference method for PIDE are that it obtains high order of accuracy, while the time complexity to solve the matrix equations after we use compact finite difference method on PIDE is O(N), and it can solve very general case of PIDE.