SCREENCOMPUTER USE
Software Task Screenflows
Desktop task recordings that connect interface state, user action and task outcome across a continuous workflow.
View datasetSCREENNARRATED WORKFLOWS
Guided Software Walkthrough Sessions
A narrated workflow contains two streams of information at once: what the person says and what the interface is doing. Guided Software Walkthrough Sessions connect those streams so spoken explanation can be grounded in screen state and task progression.
The resulting data can align narration with transcript segments, actions and visible interface text where useful. This creates examples in which language is tied to the procedure being demonstrated rather than existing as a separate description after the fact.
Technical structure
- Screen video and narration linked through shared session identifiers
- Narration transcripts aligned to the relevant part of the workflow
- Task steps and action context represented on the same timeline
- OCR-derived interface text where visible text contributes to the model task
Typical applications
Multimodal assistants, instruction grounding, screen-to-language models, procedural understanding and documentation systems.
Delivery note
Alignment granularity is defined around how closely the project needs language, interface state and action to correspond.
Delivery
The exact package depends on the collection scope. Where relevant, delivery can include task-specific records, manifests, provenance fields, stable identifiers, SHA-256 hashes, MLCommons Croissant 1.0 metadata and loading instructions.