출처: arXiv · cs.AI원문 보기 ↗
원문 저작권은 출처에 있습니다. 이 사이트는 수집, 번역 또는 형식 정리만 합니다.
사실 흐름
해설과 영향
当前基于大语言模型的GUI智能体已在移动端、网页和桌面环境中崭露头角,但它们大多是为单一平台量身定制的。一篇新发表在arXiv上的论文提出了名为MAGA(Multi-Platform Self-Fusion of GUI Agents)的框架,旨在通过结构化动作蒸馏技术,让一个模型学会在不同操作系统和界面风格下自主执行任务。这项工作的核心思路是:不再为每个平台训练独立的智能体,而是将多个专家智能体的跨平台操作知识“蒸馏”进一个统一的学生模型中。
MAGA的技术关键在于将不同平台上的UI操作抽象为一种标准化的中间表示。这与近期一些相关研究思路相通,例如ShowUI项目曾提出用结构化的JSON格式来统一描述GUI动作,通过一套“README”式的系统提示,让模型理解不同设备的操作空间,从而以函数调用的方式动态执行动作[来源:marktechpost.com]。MAGA框架则进一步通过自我融合与蒸馏,使单个模型内化这种跨平台的通用能力。论文摘要指出,该方法能够有效应对现有智能体在平台迁移时面临的泛化瓶颈,原文未提供具体的性能基准数据。
从更广的视角看,这种通过蒸馏实现能力整合的策略并非孤例。在智能体搜索领域,有研究者提出了多智能体协议蒸馏(MAPD),将多个专有模型的协作式问题求解策略蒸馏到单个开源学生模型中[来源:arxiv.org]。MAGA将类似的思想应用于GUI操作,其价值在于有望降低部署多平台智能体的工程复杂度——开发者无需为手机、电脑和浏览器分别维护不同的模型,一个经过结构化动作蒸馏的智能体即可胜任跨端任务。
该研究目前处于预印本阶段,尚未经过同行评议。随着GUI智能体从实验室走向实际应用,如何让它们在碎片化的数字生态中保持一致的执行能力,正成为一个关键挑战。MAGA所代表的技术路线,或将推动更通用、更易部署的自主操作智能体早日落地。
참고 자료
출처 원문
MAGA: M ulti-Pl a tform Self-Fusion of G UI A gents via Structured Action Distillation
Hang Yan 1,2 Zhangxuan Gu 2 \corresponding Beitong Zhou 2 Jiaxuan Chen 2,3 Runze Li 2 Yusong Hu 2 Shuheng Shen 2 Changhua Meng 2 Work done during internship at Ant Group.
Abstract Graphical user interface (GUI) agents based on large language models are increasingly deployed across mobile, web, and desktop environments. However, existing agents are typically domain-specific, limiting the deployment and user experience. This motivates the consolidation of specialized models into a single cross-environment policy. Weight merging directly merges domain-specific experts but can corrupt executable actions under expert disagreement, while on-policy distillation (OPD) avoids conflicting teacher supervision yet still treats all response tokens equally during distillation, ignoring that action tokens are the only interface between the environment and the agent. To address this, We introduce Maga that re-allocates training signal according to the structured action. Based on the correctness of the generated action, it suppresses unnecessary or invalid distillation signals and focuses learning on erroneous actions. Besides, a training-only hint optimizes the supervision signal provided by domain-specific teachers without changing the student input. Across two model scales, Maga achieves the highest mean success rate, outperforming the strongest baseline by 2.0 % 2.0% at 8B and achieves almost the same average performance with teachers.
1 Introduction
Figure 1: Results from three benchmarks, including MobileWorld (MW), OSWorld (OSW), and WebVoyager (WV). (a) On 900 random samples, we identify 66 tasks where domain-specific models exhibited high disagreement, and weight merging reduces success rate of the merged model by 10%–24%. (b) Success rate of models based on Qwen3-VL-8B across three benchmarks. The horizontal dashed lines mark the domain-specific teachers. Maga is the strongest unified model on every benchmark and exceeds the corresponding domain-specific teacher on OSWorld and WebVoyager.
Recent advances in vision-language models (VLMs) (Zhu et al. 2024 ; Li et al. 2025 ; Zhang et al. 2026a ; Liang et al. 2026 ; Zhang et al. 2026b ) have improved visual understanding, multimodal reasoning, and grounded decision making. These capabilities support graphical user interface (GUI) agents (Zhou et al. 2025b ; Xu et al. 2026b ; Team et al. 2026 ; Tang et al. 2026b ) that interpret screen content and execute actions from natural language instructions. Earlier GUI agents typically focused on mobile applications (Sun et al. 2025 ; Cheng et al. 2026 ; Gong et al. 2026 ) , web browsers (Yao et al. 2022 ; Li et al. 2026a ; Zhang et al. 2026d ) , or desktop operating systems (Liu et al. 2025 ; Jia et al. 2025 ; Xue et al. 2026 ) separately. These approaches introduce deployment complexity and limit cross-domain user experience, so recent systems combine several of these domains into a single agent.
Previous methods combine abilities across domains either by merging expert parameters. Weight merging combines separately trained domain-specific experts, as in Model Soup (Wortsman et al. 2022 ) and TIES (Yadav et al. 2023 ) . GUI domains share action types such as Click and Scroll , but their domain-specific models can disagree on the corresponding decisions. These disagreements can degrade the merged model. As shown in Figure 1 (a), when evaluated on samples with high disagreement among domain-specific models, the action success rate of the merged model drops by 10 % 10% – 24 % 24% compared to the individual models.
Alternatively, per-sample routed on-policy distillation (OPD) (Lu and Lab 2025 ; Xiao et al. 2026 ; Yang et al. 2026b ; Wu et al. 2026a ) transfers each expert’s supervision to the student without combining their parameters. For each student-generated response, only the corresponding frozen domain-specific model, which serves as the teacher, scores the sampled tokens. However, ordinary OPD treats the response as a flat sequence and assigns equal weight to all tokens. This allocation is poorly matched to GUI scenarios, where the final short structured action is the only part executed by the agent. Moreover, GUI actions directly affect the interface state, and incorrect actions can significantly shift the system away from its intended state.
To address this, we calibrate the detailed distillation signal during training for GUI actions. A GUI action comprises an action type and its associated parameters(if required). Uniformly increasing supervision across all action tokens ignores this structure. Consequently, we introduce Maga , a distillation method that allocates additional training supervision according to the structure of the action. It operates on both the student and teacher sides. On the student side , Maga removes fully correct responses from distillation and focuses extra supervision on erroneous action components. It amplifies the full action span when the action type is correct. Otherwise, it amplifies only the type and masks the incompatible parameters. On the teacher side , we condition the domain-specific teacher on a hint of the correct action during training. The hint therefore provides a more reliable signal when the teacher scores the student-sampled tokens. The student never receives this hint, so its input and output remains unchanged.
We evaluate Maga on MobileWorld, OSWorld, and WebVoyager at two model scales. As shown in Figure 1 (b), at 8B it achieves a mean success rate of 51.2 % 51.2% , exceeding the strongest baseline by 2.0 % 2.0% and achieves almost the same average performance with teachers. Our contributions can be concluded as follows:
• We identify two limitations of existing approaches to unifying domain-specific GUI agents: weight merging degrades when the experts disagree, and ordinary OPD under-allocates signal to short structured actions.
• We introduce Maga , which re-allocates distillation signal according to the action structure through student-side conditional training signal re-allocation and a training-only teacher hint.
• Across three GUI domains and two model scales, Maga achieves the highest mean SR among unified methods. At 8B, it exceeds the strongest baseline by 2.0 % 2.0% and achieves almost the same average performance with teachers.
2 Related Work
GUI agents.
GUI agents now operate across increasingly diverse interfaces and tasks. Browser agents (Yao et al. 2022 ; Zheng et al. 2024 ; Zhang et al. 2026d ; Li et al. 2026a ) perceive and act on changing websites. Moving to mobile devices (Sun et al. 2025 ; Tang et al. 2026a ; Gong et al. 2026 ; Cheng et al. 2026 ) introduces persistent app state, cross-app dependencies, and longer action sequences. Desktop control agents (Jia et al. 2025 ; Liu et al. 2025 ; Yang et al. 2026a ; Xue et al. 2026 ) operate over applications, files, menus, and system tools. Despite these interface differences, practical deployment benefits from generalist agents (Cheng et al. 2024 ; Zhou et al. 2025a ; Gu et al. 2025 ; Wu et al. 2025 ; Hu et al. 2026 ; Team et al. 2026 ; Xu et al. 2026b ) that cover several interface families within one model. Following this line of work, we study how to combine separately trained per-domain GUI models into a single agent.
Model merging.
Training a single model on mixed-domain trajectories can improve one domain at the expense of another (Xiao et al. 2026 ; Zhang et al. 2026c ; Xu et al. 2026a ) , including under supervised fine-tuning (SFT) (Luong et al. 2024 ) or reinforcement learning (RL) (Shao et al. 2024 ) . Post-hoc merging instead combines specialized checkpoints, as in Weight Soup (Wortsman et al. 2022 ) and TIES (Yadav et al. 2023 ) . For GUI agents, domain-specific teachers share a structured action space of action types and optional parameters. Merging is largely harmless when they agree, but parameter disagreements can shift the prediction away from the corresponding domain-specific teacher (Figure 1 (a)). OPD (Wu et al. 2026b ; Yang et al. 2026c ; Xiao et al. 2026 ; Xu et al. 2026a ) avoids this conflict by scoring each student-generated token with that teacher. However, existing OPD method (Lian et al. 2026 ) distributes the original token-level signal across long reasoning traces and a few action tokens, leaving the latter under-supervised. Existing evaluation also covers only two domains, leaving broader scalability untested. We therefore introduce Maga to allocate signal according to the structured action and generalize it to three GUI domains.
Figure 2: Comparison of three strategies for unifying domain-specific GUI agents. (a) Parameter merging combines the weights of domain-specific teachers, which can alter executable decisions when the teachers disagree. (b) Per-sample routed on-policy distillation (OPD) trains a student through token-level teacher supervision, but still treats the response as a flat token sequence, leaving the action tokens under-supervised. (c) Maga allocates distillation signal according to the action grammar. On the student side, it removes fully correct responses from distillation. For an incorrect response with the correct action type, it amplifies the full action span. When the type is wrong, it amplifies only the type token and masks the downstream parameters. On the teacher side, a training-only hint improves supervision signal without changing the student input.
3 Method
This section first introduces the GUI agent interface and routed OPD (Figure 2 (b)). We then present Maga (Figure 2 (c)), which combines student-side conditional signal allocation (Section 3.2 ) with a teacher-side hint (Section 3.3 ) to provide targeted supervision for short, structured action outputs.
3.1 Preliminaries
GUI Agent Interface
During each interaction step, a GUI agent receives a task instruction u u , the current visual observation o o , and an optional interaction history H H . We write the resulting model input as x = ( u , o , H ) x=(u,o,H) . For each model call, the agent produces a response y = ( r , a ) y=(r,a) containing reasoning r r and one executable action a a .
The action is structured as a = ( z , p z ) a=(z,p{z}) , where z z is a discrete action type and p z p{z} contains the parameters required by that type. The action type z z selects the parameter schema p z p_{z} , which may be empty. These parameters may be screen coordinates, text, a URL, or a key combination. Coordinates are normalized to the shared [ 0 , 1000 ] [0,1000] interface, which are one parameter class used by spatial actions.
Routed On-Policy Distillation
Let d d denote a GUI domain, x ∼ 𝒟 d x\sim\mathcal{D}{d} an input, and T d T{d} a frozen domain-specific teacher. For a student π θ \pi{\theta} generating tokens y = ( y 1 , … , y | y | ) y=(y{1},\ldots,y{|y|}) with prefix y < t y{<t} , calculating the exact reverse KL divergence against T d T{d} requires summing over the full vocabulary. For efficiency, we approximate this using only the single sampled token y t y{t} following (Li et al. 2026b ) . We define the token-level distillation advantage as the gradient-stopped difference in log probabilities:
A ^ t OPD = sg [ log π T d ( y t ∣ x , y < t ) − log π θ ( y t ∣ x , y < t ) ] . \widehat{A}{t}^{\mathrm{OPD}}=\operatorname{sg}!\left[\log\pi{T{d}}(y{t}\mid x,y{<t})-\log\pi{\theta}(y{t}\mid x,y{<t})\right]. (1)
Routed OPD then optimizes the student using this advantage-weighted objective:
ℒ OPD ( θ ) = − 𝔼 d , x , y [ 1 | y | ∑ t = 1 | y | A ^ t OPD log π θ ( y t ∣ x , y < t ) ] . \mathcal{L}{\mathrm{OPD}}(\theta)=-\mathbb{E}{d,x,y}\Bigg[\frac{1}{|y|}\sum{t=1}^{|y|}\widehat{A}{t}^{\mathrm{OPD}}\log\pi{\theta}(y{t}\mid x,y_{<t})\Bigg]. (2)
3.2 Student-Side Conditional Training Signal Re-allocation
A GUI action is represented as a = ( z , p z ) a=(z,p{z}) , where the action type z z selects a type-specific parameter schema p z p{z} . For example, coordinates, text, URLs, and key combinations are different parameter classes within this structure. Let ℐ reason \mathcal{I}{\mathrm{reason}} , ℐ type \mathcal{I}{\mathrm{type}} , and ℐ param \mathcal{I}{\mathrm{param}} denote the reasoning, action-type, and parameter token positions, respectively. This decomposition yields three training cases, and we use a token weight w t w{t} to scale the routed-OPD advantage at token position t t .
Training Rule 1: Correct action type, correct parameters. When both action type z z and parameter p z p_{z} are completely correct, the rollout receives the maximum reward. We remove the whole trajectory from distillation by setting its advantage to zero:
w t = { 0 , t ∈ ℐ reason , 0 , t ∈ ℐ type , 0 , t ∈ ℐ param . w{t}=\begin{cases}0,&t\in\mathcal{I}{\mathrm{reason}},\ 0,&t\in\mathcal{I}{\mathrm{type}},\ 0,&t\in\mathcal{I}{\mathrm{param}}.\end{cases}
Thus, reasoning, action-type, and parameter tokens receive no distillation signal. The filter avoids further optimization of an action that the evaluator already accepts completely.
Training Rule 2: Correct action type, incorrect parameters. Unlike a fully successful rollout, this sample contains a remaining error that the teacher can correct. We assign
w t = { 1 , t ∈ ℐ reason , 1 + β , t ∈ ℐ type , 1 + β , t ∈ ℐ param . w{t}=\begin{cases}1,&t\in\mathcal{I}{\mathrm{reason}},\ 1+\beta,&t\in\mathcal{I}{\mathrm{type}},\ 1+\beta,&t\in\mathcal{I}{\mathrm{param}}.\end{cases}
Reasoning tokens retain their ordinary routed-OPD weight, while the complete action span, including both type and parameters, is amplified by 1 + β 1+\beta . This prevents the training signal of the action tokens from being diluted by the longer response.
Training Rule 3: Incorrect action type (regardless of parameters). When the action type is incorrect, the discrete type decision is the primary error to correct. Moreover, the predicted type selects the wrong parameter schema, so its downstream parameters are not semantically valid supervision. We therefore amplify the type tokens and mask the parameter tokens. For a rollout with an incorrect action type, we assign token weight w t w_{t} as follows:
w t = { 1 , t ∈ ℐ reason , 1 + β , t ∈ ℐ type , 0 , t ∈ ℐ param . w{t}=\begin{cases}1,&t\in\mathcal{I}{\mathrm{reason}},\ 1+\beta,&t\in\mathcal{I}{\mathrm{type}},\ 0,&t\in\mathcal{I}{\mathrm{param}}.\end{cases}
Thus, reasoning tokens retain their original signal, the incorrect action type receives additional correction, and downstream parameters receive no distillation signal.
3.3 Teacher-Side Hint
On the teacher side, we modify only the input prompt of the routed domain-specific teacher. Let 𝒫 ( x ) \mathcal{P}(x) denote the standard prompt constructed from the complete input x x , including the system instruction, interaction history, screenshot, and current task. Let h ( z ∗ ) h(z^{*}) be a hint containing only the reference action type, and let ⊕ \oplus append this hint to the end of the final user message. The teacher and student prompts are
𝒫 T ( x , z ∗ ) \displaystyle\mathcal{P}_{T}(x,z^{}) = 𝒫 ( x ⊕ h ( z ∗ ) ) , \displaystyle=\mathcal{P}!\left(x\oplus h(z^{})\right),
𝒫 S ( x ) \displaystyle\mathcal{P}_{S}(x) = 𝒫 ( x ) . \displaystyle=\mathcal{P}(x).
Thus, the teacher prompt differs from the student prompt only by the appended hint. All other input content remains identical. The hint contains no coordinates, text, URLs, keys, or reasoning and is never added to the student prompt.
The student rollout remains unchanged. The teacher does not decode a response and only scores the tokens sampled by the student. This changes the teacher term in the distillation advantage but does not alter the student-side weight w t w{t} assigned above. The updated advantage for token y t y{t} is
A ^ t hint \displaystyle\widehat{A}{t}^{\mathrm{hint}} = sg [ log π T d ( y t ∣ 𝒫 T ( x , z ∗ ) , y < t ) \displaystyle=\operatorname{sg}!\Big[\log\pi{T{d}}!\left(y{t}\mid\mathcal{P}{T}(x,z^{*}),y{<t}\right) (3)
− log π θ ( y t ∣ 𝒫 S ( x ) , y < t ) ] . \displaystyle\qquad-\log\pi{\theta}!\left(y{t}\mid\mathcal{P}{S}(x),y{<t}\right)\Big].
3.4 Overall Framework
Figure 2 (c) summarizes the complete framework. For each offline input, the student samples a one-step response and is assigned to one of the three allocation cases in Section 3.2 . The corresponding token weights w t w_{t} determine which parts of the routed OPD signal are retained, amplified, or masked. Meanwhile, the routed domain-specific teacher scores the same student-sampled tokens using the type-hinted prompt in Section 3.3 . Combining the student-side weights with the type-hinted teacher advantage gives the final objective:
ℒ Maga ( θ ) \displaystyle\mathcal{L}{\textsc{Maga}}(\theta) = − 𝔼 d , x , y [ \displaystyle=-\mathbb{E}{d,x,y}\Bigg[ (4)
1 | y | ∑ t = 1 | y | w t A ^ t hint log π θ ( y t ∣ x , y < t ) ] . \displaystyle\quad\frac{1}{|y|}\sum{t=1}^{|y|}w{t}\widehat{A}{t}^{\mathrm{hint}}\log\pi{\theta}(y{t}\mid x,y{<t})\Bigg].
During training, all domain-specific teachers remain frozen, and only the student is updated. When both student-side allocation and teacher-side hinting are disabled, the objective reduces to ordinary routed OPD. Detailed training algorithm is provided in Appendix A.3 .
Method MobileWorld OSWorld WebVoyager
Mean SR ↑ \uparrow
TNS (%) ↑ \uparrow
SR (%) ↑ \uparrow
Δ T \Delta T
SR (%) ↑ \uparrow
Δ T \Delta T
SR (%) ↑ \uparrow
Δ T \Delta T
Qwen3-VL-2B
\rowcolor gray!12 Domain-specific teacher
24.8
-
25.2
-
47.9
-
32.6
100.0
Student
3.4
-21.4
7.9
-17.3
7.9
-40.0
6.4
20.5
SFT
11.1
-13.7
20.6
-4.6
33.6
-14.3
21.8
65.6
GRPO
6.8
-17.9
11.1
-14.1
27.9
-20.0
15.3
43.3
Weight Soup
12.0
-12.8
21.7
-3.5
40.0
-7.9
24.5
72.6
TIES
14.5
-10.3
20.6
-4.6
34.3
-13.6
23.1
70.7
UI-MOPD
12.0
-12.8
21.4
-3.8
30.0
-17.9
21.1
65.3
Maga (ours)
14.5
-10.3
23.6
-1.6
37.9
-10.0
25.3
77.1
Qwen3-VL-8B
\rowcolor gray!12 Domain-specific teacher
37.6
-
42.8
-
72.1
-
50.9
100.0
Student
12.8
-24.8
20.9
-22.0
37.1
-35.0
23.6
44.8
SFT
33.3
-4.3
43.1
+0.3
68.6
-3.6
48.3
94.8
GRPO
17.9
-19.7
25.5
-17.3
60.0
-12.1
34.5
63.5
Weight Soup
29.1
-8.5
42.3
-0.5
72.9
+0.7
48.1
92.3
TIES
31.6
-6.0
41.5
-1.4
70.0
-2.1
47.7
92.7
UI-MOPD
30.8
-6.8
44.7
+1.9
72.1
0.0
49.2
95.4
Maga (ours)
34.2
-3.4
45.3
+2.4
74.3
+2.1
51.2
99.9
Table 1: Main comparison on MobileWorld, OSWorld, and WebVoyager across two model scales. For each domain, Δ T \Delta T is the method SR minus its domain-specific teacher SR. SRs are derived from integer success counts over 117, 369, and 140 tasks, respectively. TNS denotes the Teacher-Normalized Score defined in Eq. 5 . Among unified methods within each model scale, the best results are in bold and the second-best results are underlined . Uparrow indicates that a higher value is better
4 Experiments
In this section, we first describe the experiment implementation details, followed by presenting the main results of our method compared to various baselines and an ablation study. Then we provide deeper analysis of the trained student’s performance guided by six key questions, revealing fine-grained behavior beyond aggregate task success.
4.1 Implementation Details
Models, Benchmarks, and Baselines.
In our experiments, we use Qwen3-VL-2B and Qwen3-VL-8B (Yang et al. 2025 ) . For each model, we first mid-train the backbone on mixed data from all three domains to obtain a general student. This checkpoint initializes every trainable student in our experiments. We then apply SFT separately on each domain to obtain the three frozen domain-specific teachers. We test our method on the GUI tasks of MobileWorld (Kong et al. 2026 ) and full set of OSWorld (Xie et al. 2024 ) . We also evaluate a subset of WebVoyager (He et al. 2024 ) , which contains 140 sample tasks. More information about this subset can be found in Appendix A.1 . The main setting consolidates all three teachers in one run. The main comparison includes the domain-specific teachers, SFT, GRPO, Weight Soup (Wortsman et al. 2022 ) , TIES (Yadav et al. 2023 ) , and UI-MOPD (Lian et al. 2026 ) . In the ablation study, we remove the whole student side and its seperate rule. Besides, we also remove teacher side to demonstrate its effectiveness. And removing both student and teacher sides of Maga recovers ordinary per-sample routed OPD (Lu and Lab 2025 ) .
Training Details.
Our training set contains 343k examples, including 93k from mobile, 50k from desktop, and 200k from the web domain. Notably, the training data for GRPO, UI-MOPD, and Maga is constructed from the same SFT dataset by using the final step’s action as the ground-truth label. All runs are conducted on 32 H20 GPUs and optimize the trainable LLM parameters with AdamW, using 1e-5 as learning rate, batch size 128. The vision parameters remain frozen. Each prompt produces 4 rollouts. After each rollout, the rule-based reward and parsed action spans determine the student-side allocation, while the frozen domain-specific teacher scores the sampled tokens with a training-only hint. During inference, we use vLLM (Kwon et al. 2023 ) as rollout engine. Further details are provided in Appendix A .
Evaluation Protocol.
We evaluate fixed samples from each domain and use Success Rate (SR) as the primary benchmark metric. Throughout the paper, percentage differences denote absolute differences between percentage-valued metrics rather than relative changes. Moreover, to measure how much of each domain-specific teacher’s capability is retained, we first normalize a model merging method’s SR by the corresponding teacher SR in each domain and then average these ratios. We define the resulting Teacher-Normalized Score (TNS) for method m m as
TNS ( m ) = 100 | 𝒟 | ∑ d ∈ 𝒟 s m , d s T d , d , \operatorname{TNS}(m)=\frac{100}{|\mathcal{D}|}\sum{d\in\mathcal{D}}\frac{s{m,d}}{s{T{d},d}}, (5)
where s m , d s{m,d} and s T d , d s{T_{d},d} are the SRs of method m m and the domain-specific teacher on domain d d , respectively. The higher the TNS is, the more capability of the domain-specific teachers is retained by the student. A TNS of 100 % 100% matches the domain-specific teachers on average. Parameter-class breakdowns, parsing rules, and statistical tests are provided in Appendix B .
4.2 Main Results
Maga outperforms the other baselines at both model scales.
At 8B, Maga exceeds UI-MOPD on all three domains, improving mean SR by 2.0 % 2.0% and TNS by 4.4 % 4.4% . It also leads Weight Soup and TIES on both metrics. At 2B, although Weight Soup scores 2.1 % 2.1% higher on WebVoyager, Maga achieves the highest mean SR and TNS.
Maga is comparable to the teachers on average at 8B.
Across all domains, Maga obtains a TNS of 99.9 % 99.9% . Its point estimate is 3.4 % 3.4% below the MobileWorld teacher but 2.4 % 2.4% and 2.1 % 2.1% above the OSWorld and WebVoyager teachers, respectively. Its mean SR is 0.4 % 0.4% above the teacher mean.
Variant
MobileWorld
OSWorld
WebVoyager
\rowcolor gray!12 Maga
34.2
45.3
74.3
w/o student-side
30.8
41.5
72.9
w/o Rule 1
31.6
45.0
70.7
w/o Rule 2
32.7
43.4
74.3
w/o Rule 3
33.3
43.9
72.1
w/o teacher-side
31.6
45.3
73.6
w/o both sides
29.1
44.4
70.0
Table 2: Ablation study of Maga on Qwen3-VL-8B. We report SR on MobileWorld, OSWorld, and WebVoyager.
Maga reduces the imbalance of weight merging.
At 8B scale, Weight Soup exceeds the WebVoyager teacher by 0.7 % 0.7% but falls 8.5 % 8.5% below the MobileWorld teacher. Maga improves these margins to + 2.1 % +2.1% and − 3.4 % -3.4% , respectively, demonstrating more balanced cross-domain retention.
4.3 Ablation Study
In Table 2 , student-side removal lowers SR by 3.4 3.4 , 3.8 3.8 , and 1.4 1.4 points on MobileWorld, OSWorld, and WebVoyager . Teacher-side hint removal lowers MobileWorld and WebVoyager by 2.6 2.6 and 0.7 0.7 points, while OSWorld keeps unchanged. Moreover, ablation results on separate student-side rules also demonstrate their effectiveness.
Initial Student Trained Student Change
Domain Correct Type × \times Param. × \times Correct Type × \times Param. × \times
Δ \Delta Correct Δ \Delta Type × \times
Δ \Delta Param. × \times
MobileWorld 50.7 29.3 20.0 73.0 10.7 16.3
22.3 +22.3 − 18.6 -18.6 − 3.7 -3.7
OSWorld 40.7 26.3 33.0 63.7 13.0 23.3
23.0 +23.0 − 13.3 -13.3 − 9.7 -9.7
WebVoyager 35.3 37.0 27.7 53.0 20.0 27.0
17.7 +17.7 − 17.0 -17.0 − 0.7 -0.7
Overall 42.2 30.9 26.9 63.2 14.6 22.2
21.0 \mathbf{+21.0} − 16.3 \mathbf{-16.3} − 4.7 \mathbf{-4.7}
Table 3: Action outcomes of the initial and Maga -trained students, evaluated on 300 held-out examples per domain. Correct denotes a correct action type and all required parameters. Type × \times denotes responses without a correct action type. Param. × \times denotes a correct action type with at least one incorrect or missing required parameter.
Figure 3: Based on the trained student, we use 900 random samples (300 per domain) to generate response. (a) Each action falls into one of four mutually exclusive groups: matching its corresponding domain-specific teacher, matching only other domain-specific teacher, matching no teacher but is correct, or matching no teacher and is incorrect. Mean values are averaged across domains. (b) The heatmap compares how strongly each response matches its corresponding domain-specific teacher in the reasoning tokens action type tokens and action parameter tokens. Darker cells indicate a higher similarity to the teacher. All reported values are relative to the reasoning tokens, which serve as the zero-baseline for each domain.
4.4 Do Student-Side Rules Work as Designed?
All three student-side cases occur, and fully correct actions increase by 21.0%.
In Table 3 , we compare the initial student with the trained student on the same 900 held-out test examples, with 300 randomly sampled examples from each domain. These examples are disjoint from the training data.
Before training, 42.2 % 42.2% of actions are fully correct, 30.9 % 30.9% have a wrong action type and 26.9 % 26.9% have the correct type but an incorrect parameter. This indicates that all three student-side allocation cases occur on the held-out test set, with a majority of errors coming from the action type.
After training, the percentage of fully correct actions reaches 63.2 % 63.2% ( + 21.0 % +21.0% ), while parameter errors fall to 22.2 % 22.2% ( − 4.7 % -4.7% ) and type errors fall to 14.6 % 14.6% ( − 16.3 % -16.3% ). The increase is consistent across domains, with gains of + 22.3 % +22.3% on MobileWorld, + 23.0 % +23.0% on OSWorld, and + 17.7 % +17.7% on WebVoyager , which indicates that the student’s performance improves across all domains. Parameter-class results and task-level transitions from the initial to the trained student are provided in Appendix D .
4.5 Where Does Student’s Behavior Come From?
The trained student primarily matches the domain-specific teacher and also produces a small set of successful novel actions.
Based on 900 randomly sampled offline single-step tasks (300 per domain), we prompt the student and teachers to generate the corresponding answer action, and evaluate whether their action semantics align closely. Complete evaluation rules are provided in Appendix A.5 .
Figure 3 (a) shows that 61.5% of actions match the domain-specific teacher. In comparison, 14.8% match only other domain-specific teachers. Thus, the student more often reproduces behavior consistent with its routed domain-specific teacher than behavior found only in teachers from other domain teachers.
Most student actions that do not align with any teacher are incorrect, accounting for 20.6 % 20.6% of all evaluated tasks. However, 3.0 % 3.0% of these unmatched actions are actually correct. These successful yet unaligned behaviors demonstrate that the student is capable of generating valid, novel actions.
Figure 4: For initially incorrect actions, correcting solely the action type allows the model to automatically recover most of the associated parameters. The student model regenerates all parameters from scratch, without access to the ground-truth reference parameters. Each row represents a single response. The left section groups responses by the ground truth action type, while the right section indicates whether the regenerated action is correct.
4.6 On Which Tokens Do Student and Teacher Agree Most?
Action tokens generated by our trained student show stronger alignment with the corresponding domain-specific teacher than reasoning tokens. Figure 3 (b) compares the likelihood assigned by teachers to reasoning and action tokens generated by our trained student. The detailed calculation and statistical results are provided in Appendix A.5 .
Compared with reasoning tokens, action tokens show an average likelihood gain of + 0.072 +0.072 . Specifically, the type and parameter tokens also show positive average gains of + 0.068 +0.068 and + 0.073 +0.073 against reasoning tokens. This observation supports our action-aware approach to treat the action tokens as a distinct supervision target rather than weighting all response tokens uniformly.
4.7 Can Action Failures Be Recovered from the Type Alone?
Correcting only the action type recovers a substantial percentage of wrong actions.
From 900 sampled tasks across three domains, we isolate 136 136 responses with incorrect action types. For these, we replace the predicted action type with the ground truth and prompt the frozen student to regenerate parameters using only its original context and reasoning (details in Appendix A.5 ).
As shown in Figure 4 , correcting only the action type recovers 68.4 % 68.4% of the responses whose action type is wrong, while the remaining responses still fail due to wrong parameters. Specifically, 57 57 out of 87 87 actions requiring coordinate parameters are successfully recovered. This indicates that many wrong actions do not stem from a lack of capability in parameter generation. Instead, once the action type is corrected, the student model can often generate these parameters correctly. This finding also supports Maga ’s stronger supervision of the action type.
4.8 Can Maga Correct Errors on Samples with High-Disagreement?
Maga corrects weight-merge errors on both Click and Swipe cases.
To evaluate Weight Soup and Maga , we construct a case study of 66 held-out MobileWorld tasks (29 Click , 37 Swipe ). We filtered for spatial actions where all three teachers output valid coordinates with a maximum pairwise distance > 0.07 >0.07 .
Table 4 reports the corresponding change in accuracy. Weight Soup fails on 19 samples. Among them, Maga corrects 6 of the 13 Click errors and 4 of the 6 Swipe errors. One Click sample changes in the opposite direction, from a correct prediction to an incorrect Maga prediction. Overall, Maga corrects 10 Weight Soup errors while introducing one new error, increasing accuracy from 47 / 66 47/66 ( 71.2 % 71.2% ) to 56 / 66 56/66 ( 84.8 % 84.8% ), a gain of 13.6 % 13.6% .
4.9 Why Does the Student Not Consistently Surpass Its Teachers?
Reasoning tokens dominate the response, thereby diluting the action supervision.
As shown in Table 5 , we segment each response into reasoning and action spans. The action accounts for only 3.9 % 3.9% – 7.1 % 7.1% of response tokens, making it difficult for one student to exceed every specialized teacher.
Action Correct actions Change
Weight Soup Maga
Click 16/29 (55.2%) 21/29 (72.4%) +17.2%
Swipe 31/37 (83.8%) 35/37 (94.6%) +10.8%
Table 4: MobileWorld case study on high-disagreement samples. Model columns report correct actions over the subset size, with accuracy in parentheses. Change is Maga accuracy minus Weight Soup accuracy.
Token share (%) Mean interaction steps
Domain Reasoning Action Successful Failed Gap
MobileWorld 95.9 4.1 19.2 29.4 10.2
OSWorld 92.9 7.1 13.5 56.9 43.4
WebVoyager 96.1 3.9 15.8 39.2 23.4
Table 5: Response-token composition and average interaction steps for the 8B student. The step gap is the failed-trajectory mean minus the successful-trajectory mean.
Training on single-step data limits multi-step generalization.
Table 5 shows that failed trajectories are longer on average than successful ones, suggesting that long sequences are a bottleneck. We train Maga using only single-step supervision, but real-world GUI tasks require multi-step execution, misaligning the multi-step setting.
5 Conclusion
This work addresses how to consolidate multiple domain-specific GUI teachers into a single agent while retaining their specialized capabilities. Our analysis shows that weight merging degrades performance, particularly on high-disagreement spatial samples, while ordinary routed OPD provides only limited training signal to the short structured action. Moreover, the short action span carries domain-specific behavior, which significantly affects the environment state. Therefore, we propose Maga to address this mismatch with structured action signal re-allocation and a training-only teacher hint, directing additional supervision according to action correctness and structure. Across MobileWorld, OSWorld, and WebVoyager at two model scales, Maga achieves the highest mean success rate among unified methods within three domains and best preserves the capabilities of the domain-specific teachers. Specifically, at the 8B scale, it improves mean SR by 2.0 % 2.0% over the strongest baseline and achieves almost the same average performance with teachers, suggesting that robust multi-platform consolidation benefits from explicit supervision of structured actions.