Resistive memories have drawn the attention of researchers due to their low power and single-cycle computation of vector-matrix multiplication (VMM), which is the main operation performed in neural networks. For performing VMM, one of the most desirable architectures is the memristor crossbar that has several advantages over other memory technologies, viz. in-memory computation, low power, and high density. However, faults present in the crossbar can introduce errors in the inference process during neuromorphic computations. Existing methods to handle faults using retraining and remapping incur overheads in terms of hardware, power, and delay. In this paper we explore and analyze the impact of faults on memristor-based crossbar for overall inference accuracy. We have observed that the accuracy is not significantly affected in the presence of a limited number of faults. Also, the inference quality and effect of faults depend on the number of neural network layers and storage resolution of memristors present in the crossbar. The introduced approach works in three phases, fault tolerance analysis, high-level fault detection, and low-level fault detection. In the first phase, we analyze the fault tolerance capability of the crossbar, which identifies how many faults can be tolerated for a given application. In the second phase, we estimate the percentage of faults, and if it is below a threshold the third phase can be skipped. In the third phase, an efficient method to determine the exact location of the faults is used. The proposed method is capable of performing parallel operations, thus requiring fewer read/write steps as compared to existing works. The proposed approach requires O(N) read/write operations as compared to \(O(N^2)\) operations required in existing works.