Agricultural expansion into subtropical and tropical forests causes major environmental damage, but its wider social impacts often remain hidden. Forest-dependent smallholders are particularly strongly impacted, as they crucially rely on forest resources, are typically poor, and often lack institutional support. Our goal was to assess forest-smallholder dynamics in relation to expanding commodity agriculture. Using high-resolution satellite images across the entire South American Gran Chaco, a global deforestation hotspot, we digitize individual forest-smallholder homesteads (n = 23,954) and track their dynamics between 1985 and 2015. Using a Bayesian model, we estimate 28,125 homesteads in 1985 and show that forest smallholders occupy much larger forest areas (>45% of all Chaco forests) than commonly appreciated and increasingly come into conflict with expanding commodity agriculture (18% of homesteads disappeared; n = 5,053). Importantly, we demonstrate an increasing ecological marginalization of forest smallholders, including a substantial forest resource base loss in all Chaco countries and an increasing confinement to drier regions (Argentina and Bolivia) and less accessible regions (Bolivia). Our transferable and scalable methodology puts forest smallholders on the map and can help to uncover the land-use conflicts at play in many deforestation frontiers across the globe. Such knowledge is essential to inform policies aimed at sustainable land use and supply chains.