MemGUI-Agent

MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management

Anonymous submission

MemGUI-Agent uses ConAct, a Context-as-Action interface that folds action history, updates UI memory, and emits the next GUI action in one structured response.

Code Repo MemGUI-3K MemGUI-8B-SFT Demo Videos

Anonymous Demo Videos

Case studies from the paper

Each task groups the baseline failure and our successful rollout, with benchmark source, task query, and rollout length.

MemGUI-Bench 路 8B 路 AP News Digest

Long-horizon reading and note creation

Task query: In the AP News app, find the three most recent articles from the Technology section and the three most recent from the Business section. Read each full article, remember its title and main content, then create a Joplin note titled Tech & Business Digest with sectioned summaries.

MemGUI-Bench 路 235B 路 Compare Product Specs

Collecting product specs and writing a note

Task query: Open Amazon, collect screen size, battery, and storage for iPhone 15 Pro, Galaxy S24 Ultra, and Pixel 8 Pro, then write all nine facts into a Joplin note named Phone Spec Matrix.

MobileWorld 路 8B 路 Alibaba Headquarters Contact

Finding a business phone number and creating a contact

Task query: Find the phone number of Alibaba's Hangzhou headquarters on Google Maps, and based on that, create a new contact named Kevin Zhang with the company.

MobileWorld 路 235B 路 Mastodon Contact Update

Updating contact details from a social post

Task query: Olivia left new phone and email information in her latest Mastodon post. Update Olivia in Contacts, then send her the text message: Hello, how are you. Set the email label to internet.

01 / Main Results

Context efficiency and task success

MemGUI-Agent improves both zero-shot 235B and trained 8B settings on long-horizon mobile GUI benchmarks.

Context efficiency and benchmark performance of MemGUI-Agent
Context efficiency and benchmark performance of MemGUI-Agent.

02 / Method

ConAct: Context-as-Action

Instead of carrying an ever-growing raw transcript, MemGUI-Agent makes context updates explicit and executable inside each model response.

ConAct framework
ConAct treats context management as part of the action space: the policy updates folded history, folded UI state, and the recent step record while producing the next GUI action.

Folded Action History

Compresses completed interaction spans while preserving task-relevant progress.

Folded UI State

Stores persistent UI facts such as names, prices, values, and form state.

Recent Step Record

Keeps local interaction continuity for the next GUI action.

03 / Dataset and Case Study

MemGUI-3K and case study

MemGUI-3K supports SFT and offline analysis. The case study follows one long mobile workflow end to end.

MemGUI-3K dataset statistics
MemGUI-3K contains 2,956 trajectories and 64,430 reasonable-step SFT samples.
MemGUI-Agent case study
Paper case study showing compact context over a long mobile interaction.

04 / BibTeX

Reference

Author identities and publication links are withheld during double-blind review. 

@inproceedings{anonymous2026memgui,
  title={MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management},
  author={Anonymous Authors},
  booktitle={Under double-blind review},
  year={2026}
}