Microsoft launches a new feature within Copilot Studio that allows AI agents to use websites and desktop apps independently.
Microsoft expands Copilot Studio with a new feature: ‘computer use’. The addition, now available in an early preview, allows AI agents to interact directly with websites and desktop applications through their graphical user interface. Recently, Microsoft launched a similar feature Copilot Actions that enabled the AI agent to surf the web for you.
Computer Use
Thanks to the new capability, Copilot Studio agents can perform tasks in applications for which no API is available. By clicking on elements on the screen, filling in fields, or selecting menus, agents act in a manner similar to human users. This approach allows for the automation of manual processes such as data entry, market research, or invoice processing.
Computer use works with popular browsers such as Edge, Chrome, and Firefox, and runs on infrastructure hosted by Microsoft. This reduces maintenance costs for organizations, while keeping business data within Microsoft Cloud and not using it for model training. The feature also automatically adapts when applications or websites change, using built-in reasoning models.
Traditional RPA
The feature also represents a new step towards more accessible robotic process automation (RPA). Unlike traditional RPA, which is often vulnerable to changes in UI elements, computer use continues to function when interface adjustments are made. Users can describe the desired actions in natural language, while the agent makes smart choices based on what appears on the screen.
read also
Microsoft 365 Copilot gets AI-powered research tools
Microsoft emphasizes that this new approach to automation is not only more efficient but also suitable for users without a technical background. Through Copilot Studio, organizations can now build agents faster that perform complex tasks in a visual user environment, with more flexibility and fewer technical barriers.