StreamSense: A Content- and Resource-Aware Edge Agent for Adaptive Bitrate Streaming
Abstract
AbstractAdaptive Bitrate (ABR) decision-making is essential for optimizing Quality of Experience (QoE) in video streaming. Traditional ABR methods typically lack fine-grained content understanding and face challenges in balancing multi-dimensional QoE objectives under real-world edge device constraints. To overcome these limitations, we propose **StreamSense Agent**, a novel framework that reformulates ABR as a sequential decision-making process informed by content and network awareness. Our approach introduces three key innovations: (1) a Dynamic Content-Aware Entity Graph (D-CAEG) for structured video representation, (2) a composite QoE-driven optimization objective integrating perceptual quality, rebuffering, smoothness, and computational cost, and (3) a hierarchical resource-aware architecture enabling real-time control through an efficient lightweight actor. We further adopt an offline-to-online co-adaptation training paradigm with federated learning for continuous adaptation. Since effective ABR control requires the same core competencies as general sequential decision-making—multi-step planning across future video chunks, dynamic information gathering from heterogeneous network signals, and adaptive strategy adjustment under resource constraints—we validate these transferable capabilities on five established benchmarks. Specifically, multi-hop search tasks (BrowseComp, BrowseComp-ZH) mirror the integration of dispersed network-trace and content-complexity signals into a bitrate plan; complex navigation tasks (WebWalkerQA) correspond to sequential buffer-state traversal under volatile bandwidth; and information-extraction tasks (Xbench-DS, SimpleQA) reflect the precision required when reading real-time streaming telemetry. StreamSense Agent achieves state-of-the-art results, attaining accuracy of 16.7% on BrowseComp, 30.5% on BrowseComp-ZH, 42.1% on Xbench-DS, and 54.2% on WebWalkerQA. Deployment-oriented evaluations further confirm its computational efficienc