Accessing Chrome Internal Ui Elements for Clicking or Mouse Over events.

Example scripts and tips (replaces Old Scripts & Tips archive)

Moderators: Dorian (MJT support), JRL, Phil Pendlebury

Post Reply
ocnuybear
Pro Scripter
Posts: 100
Joined: Sun Jul 15, 2018 5:14 pm

Accessing Chrome Internal Ui Elements for Clicking or Mouse Over events.

Post by ocnuybear » Fri Sep 30, 2022 5:52 pm

Chrome is a very common browser that can be automated, but using it and MS now since 2018, it seemed only outside elements was exposed when you are not using chromedriver like back, forward, reload ,etc. But I just discovered a very handy way to set Chrome so that most internal elements can be found using the following procedure:

1. Type in Chrome URL:

Code: Select all

chrome://accessibility/
2. Make sure all 5 checkboxes on the upper left (accessibility modes) like " Native accessibility API support" down to "HTML" is checked.

3. Close all your Chrome sessions, and create a new shortcut to force this accessibility mode:

Code: Select all

"C:\Program Files\Google\Chrome\Application\chrome.exe" --force-renderer-accessibility
4. Start Chrome and go to a website that you want to automate, to quickly check if it is working start MS and select Tools->View System Windows, find the Chrome application using the website name or search for Chrome which may look like this:

Code: Select all

3016894 - Chrome_WidgetWin_1 "website - Google Chrome"
Select this line and right click->Accessible Object Tree.
Another window will popup called "Accessible Objects"
In here you will should find most button names or elements with their coordinates which you can retrieve using:

Code: Select all

UIAccessibleList>nHandle,res
You wil have to split this list up obviously to get to your elements, as soon as you clicked somehere you just have to reload this to get the new updated elements with their coordinates.
Hope this will be helpful to many people here.

Post Reply
cron
Sign up to our newsletter for free automation tips, tricks & discounts