Exceptional points can be found for specific sets of parameters in thermoacoustic systems. At an exceptional point, two eigenvalues and their corresponding eigenfunctions coalesce. Given that the sensitivity of these eigenvalues to parameter changes becomes infinite at the exceptional point, their occurrence may greatly affect the outcome and reliability of numerical stability analysis. We propose a new method to identify exceptional points in thermoacoustic systems. By iteratively updating the system parameters, two initially selected eigenvalues are shifted toward each other, ultimately colliding and generating the exceptional point. Using this algorithm, we were able to identify for the first time a physically meaningful exceptional point with positive growth rate in a thermoacoustic model. Furthermore, our analysis goes beyond previous studies inasmuch as we employ a more realistic flame transfer function to model flame dynamics. Building on these results, we analyze the effect of exceptional points on the reliability of thermoacoustic stability analysis. In the context of uncertainty quantification, we show that surrogate modeling is not reliable in the vicinity of an exceptional point, even when large sets of training samples are provided. The impact of exceptional points on the propagation of input uncertainties is demonstrated via Monte Carlo computations. The increased sensitivity associated with the exceptional point results in large variances for eigenvalue predictions, which needs to be taken into account for reliable stability analysis.