Agricultural land abandonment is a common land-use change, making the accurate mapping of both location and timing when agricultural land abandonment occurred important to understand its environmental and social outcomes. However, it is challenging to distinguish agricultural abandonment from transitional classes such as fallow land at high spatial resolutions due to the complexity of change process. To date, no robust approach exists to detect when agricultural land abandonment occurred based on 30-m Landsat images. Our goal here was to develop a new approach to detect the extent and the exact timing of agricultural land abandonment using spatial and temporal segments derived from Landsat time series. We tested our approach for one Landsat footprint in the Caucasus, covering parts of Russia and Georgia, where agricultural land abandonment is widespread. First, we generated agricultural land image objects from multi-date Landsat imagery using a multi-resolution segmentation approach. Second, we estimated the probability for each object that agricultural land was used each year based on Landsat temporal-spectral metrics and a random forest model. Third, we applied temporal segmentation of the resulting agricultural land probability time series to identify change classes and detect when abandonment occurred. We found that our approach was able to accurately separate agricultural abandonment from active agricultural lands, fallow land, and re-cultivation. Our spatial and temporal segmentation approach captured the changes at the object level well (overall mapping accuracy = 97 ± 1%), and performed substantially better than pixel-level change detection (overall accuracy = 82 ± 3%). We found strong spatial and temporal variations in agricultural land abandonment rates in our study area, likely a consequence of regional wars after the collapse of the Soviet Union. In summary, the combination of spatial and temporal segmentation approaches of time-series is a robust method to track agricultural land abandonment and may be relevant for other land-use changes as well.