In this paper, the singularity appearing in the variational boundary integral formulation for potential problem is addressed. The Taylor series expansion is used to expand terms inside the weakly and strongly singular integral kernels around the singular point. In this approach, the singularities involved in the integration kernels are analytically removed by excluding the singular point from the proposed integral and applying analytical integrations.