Skip to content
SI.info

SI Glossary · Using SI

Computer Use

Published 1 min read

Computer use lets a model work with software that has no API, the way a human would. The model receives screenshots, decides what to do, and sends mouse and keyboard actions. Anthropic launched an early public version in October 2024. By 2026, OpenAI, Google and others have shipped computer- and browser-use features, and labs advertise it as a core capability of frontier models such as GPT-6 Astra.

What it’s used for

  • Filling in web forms and moving data between apps
  • Testing software by clicking through it
  • Research that requires navigating many websites
  • Automating repetitive office work in legacy systems

Challenges

  • Reliability: long sequences of clicks multiply the chance of a mistake.
  • Speed: looking at screenshots is slower than calling an API.
  • Security: a model browsing the web can encounter malicious instructions (prompt injection) and could take harmful actions if over-permitted.

Computer-use agents are a big part of why the 2026 White House Accord on Super Intelligence emphasises preventing unauthorized system access. See also SI agent.

← Back to the SI Glossary

Written by

· Editor

Editor of SI.info. Writes about Super Intelligence, technology policy and the people building frontier models.

How we research and fact-check

Free newsletter

Get The SI Brief

One short email a week: what changed in Super Intelligence, policy and models — and why it matters.

Free. One email a week. Sent via beehiiv, which counts opens and clicks. Unsubscribe anytime. Privacy policy

Keep reading

Core concepts

SI Agent

An SI agent (formerly AI agent) is a system that pursues a goal on its own over many steps, deciding what to do next and using tools such as browsers, code and apps.

Using SI

Tool Use

Tool use (or function calling) lets an SI model call external software, such as search engines, calculators, databases or APIs, to get information or take actions.

Models & architecture

Multimodal Model

A multimodal model is an SI model that can understand and often generate more than one kind of data, such as text, images, audio and video, within a single system.