Breast cancer is a prevalent form of cancer among women. It is associated with increased heat generation due to higher metabolism in the tumor and increased blood vessels resulting from angiogenesis. The thermal alterations result in a change in the breast surface temperature profile. Infrared imaging is an FDA-approved adjunctive to mammography, which employs the surface temperature alterations in detecting cancer. To apply infrared imaging in clinical settings, it is necessary to develop effective techniques to model the relation between the tumor characteristics and the breast surface temperatures. The present work describes the thermal modeling of breast cancer with physics-informed neural networks. Losses are assigned to random points in the domain based on the boundary conditions and governing equations that should be satisfied. The Adam optimizer in TensorFlow minimizes the losses to find the temperature field or thermal conductivity that satisfies the boundary conditions and the bioheat equation. Backpropagation computes the derivatives in the bioheat equation. Analyses of the three patient-specific cases show that the machine-learning model accurately reproduces the thermal behavior given by ansys-fluent simulation. Also, good agreement between the model prediction and the infrared images is observed. Moreover, the neural network accurately recovers the thermal conductivity within 6.5% relative error.