Although a large part of Italy is characterized by a Mediterranean climate intrinsically highly suitable for olive cultivation, farmers may experience variable interannual yields with associated agronomic and management costs. A detailed overview of major climate stressors and their ongoing impacts on olive yield variability at a broad spatial–temporal scale across Italy could improve the understanding of how interannual olive yields are modulated by seasonal local climate and would enhance the development of actionable services to alert stakeholders of potential climate risks. We analysed aggregated olive yield data from the Italian National Statistics Institute (ISTAT) at the provincial level during 2006–2020, and several climatic variables from the Reanalysis v5 (ERA5) of the European Centre for Medium-Range Weather Forecasts (ECMWF) as a basis to (i) explore olive yield trends and inter-annual variations over the whole country; (ii) identify major seasonal climate stressors likely responsible for the largest variations in yield and (iii) develop a composite index that summarizes the risk of having exceptionally low yields due to the co-occurrence of multiple stressors. To this end, we defined two extreme classes of yield, namely, exceptionally low and high yields (LY and HY, respectively) and explored the climatic variables aggregated on a bimonthly time scale that influenced yield outcomes. Our analysis showed that exceptionally low yields have been erratically increasing since 2014 with temperature-related variables having the highest impact, especially a relatively warm winter and cool early autumn. These period-specific variables were major components of the resulting composite index predicting the likelihood of LY ranging from 28% to 49% due to increasing stress effects. Possible explanations of our findings are discussed, including the proliferation of the olive fly. We suggest the composite risk approach could lay the groundwork for an integrated meteorological seasonal forecasting system that provides timely insights on factors affecting within-season olive yield development.