Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Structured Pruning and Lightweight Adaptation of Foundation Models for Resource-Efficient Earth Observation

AbstractStructured pruning has gained attention as a promising solution for the cost-effective deployment of large language, vision-language, and domain-specific foundation models. However, most prior work has focused on general-purpose language models and often lacks a systematic comparison of pruning strategies across model scales, architectures, and real-world multimodal applications. In this paper, we conduct a comprehensive evaluation of structured pruning methods, focusing on depth pruning, width pruning, and post-pruning recovery with parameter-efficient fine-tuning. We evaluate multiple open-source models, including Qwen3-0.6B, Qwen3-4B-Instruct, Qwen3-8B, LLaVA-OneVision-0.5B, and Phi-4-Mini-Instruct, across intrinsic metrics and downstream benchmarks such as BBH, MATH Hard, MuSR, and MMLU-Pro. Our findings reveal that (1) depth pruning generally provides a more stable trade-off between efficiency and performance than width pruning; (2) width pruning can reduce memory usage, but often gives limited throughput gains and less predictable accuracy retention; (3) reasoning-intensive benchmarks, particularly mathematical reasoning tasks, are more sensitive to structured pruning than some other evaluation tasks; (4) LoRA-based post-pruning adaptation can substantially recover performance after moderate pruning; and (5) pruning behaviors observed in general-purpose models transfer to Earth observation foundation models, as demonstrated on TerraMind, where depth pruning reduces latency and GPU memory usage while maintaining competitive EuroSAT and Bigearthnet task accuracy.

More from EMNLP 2026

All EMNLP 2026 papers →