The intermittent and fluctuating nature of solar energy is the biggest challenge facing its widespread utilization. Implementing onsite photovoltaic (PV) systems as alternative energy sources has established the need for reliable forecasting procedures to improve scheduling and demand management. This paper presents solar energy forecasting combined with a demand-side prediction algorithm to optimize the utilization of available solar energy resources and manage the demand side accordingly. The algorithm utilizes support vector regression (SVR), a machine learning technique, validated using one-year energy consumption data collected from an office building instrumented as an experimental testbed facility. Power meters and temperature sensors collect the building’s internal climate and energy data, while a solar PV array and a weather station provide the external relevant data. The forecasting method uses the average power output of k-similar days as an added input to the SVR model to enhance its performance. The day-ahead prediction results show that this additional input contributes to higher forecasting efficiency, especially in the hot climate regions, where sunny weather conditions prevail throughout the year. The PV output prediction accuracy for sunny days is above 90%, which offers possibilities for optimized scheduling and leading to smart building energy management. Finally, this paper also proposes a temperature set point optimization algorithm for the building air conditioning system to minimize the difference between the building energy load and the generated solar PV power. Using 24 °C as the upper set point temperature limit reduces the energy demand (consumption) by up to 29% and the associated reduction in CO2 emissions.