District Swat is part of the high mountain Hindu-Kush Himalayan region of Pakistan. Documentation and analysis of land use change in this region is challenging due to very disparate accounts of the state of forest resources and limited accessible data. Such analysis is, however, important due to concerns over the degradation of forest land leading to deterioration of the protection of water catchments and exposure of highly erodible soils. Furthermore, the area is identified as hotspot for biodiversity loss. The aim of this paper is to identify geophysical and geographical factors related to land use change and model how these relationships vary across the district. For three selected zones across the elevation gradient of the district, we analyse land use change by studying land use maps for the years 1968, 1990 and 2007. In the high-altitude zone, the forest area decreased by 30.5 %, a third of which was caused by agricultural expansion. In the mid-elevation zone, agriculture expanded by 70.3 % and forests decreased by 49.7 %. In the lower altitudes, agriculture expansion was 129.9 % consuming 31.7 % of the forest area over the forty-year time period. Annual deforestation rates observed were 0.80, 1.28 and 1.86 % in high, mid and low altitudes, respectively. In the high-altitude ecosystems, accessibility (distance to nearest road and city) had no significant role in agriculture expansion; rather land use change appears significantly related to geophysical factors such as slope, aspect and altitude. In the low-elevation zone, accessibility was the factor showing the closest association with agriculture expansion and abandonment. The analysis illustrates that land use change processes vary quite considerably between different altitudinal and vegetation cover zones of the same district and that environmental constraints and stage of economic development provide important contextual information.