Searching for local minima, saddle points, and minimum energy paths (MEPs) on the potential energy surface (PES) is challenging in computational materials science because of the complexity of PES in high-dimensional space and the numerical approximation errors in calculating the potential energy. In this work, a local minimum and saddle point searching method is developed based on kriging metamodels of PES. The searching algorithm is performed on both kriging metamodels as the approximated PES and the calculated one from density functional theory (DFT). As the searching advances, the kriging metamodels are further refined to include new data points. To overcome the dimensionality problem in classical kriging, a distributed kriging approach is proposed, where clusters of data are formed and one metamodel is constructed within each cluster. When the approximated PES is used during the searching, each predicted potential energy value is an aggregation of the ones from those metamodels. The dimension of each metamodel is further reduced based on the observed symmetry in materials systems. The uncertainty associated with the ground-state potential energy is quantified using the statistical mean-squared error in kriging to improve the robustness of the searching method.