how to recognize dynamic text on java applet

[mfauzim]

I need to interface to mainframe application which use java applet.
So far I manage:
1. open the applet and login following standard sequence. this just standard keyboard enter. at the end, I will reach the inquiy screen
2. I enter switch id in the enquiry screen and press specified key.

the screen will display list of telephone depending on the status. My issue is I how do I compare telephone list extracted from another db against list display on the java applet screen. if display has the tel no in the db list, i need to select the tel for next action. After reading imagelib, i'm not sure whether it can compare text content( from another db ) against the graphic image on the java app. What I understand is the image lib can only compare against another predefine image.

Appreciate any guide on all the pro here

[mfauzim]

Any guide is really appreciated. I really 'blank' on how I can apply image recognition in this issue.

[Marcus Tettmar]

The trouble with Java apps is that they don't expose objects in the same way that regular Windows GUIs do, so we can't hook into the GUI and access the text associated with the objects. Image recognition is a problem since you are not looking for, or comparing, images, but you want to get text.

Usually the best way to get text in this case is to use the clipboard. It may be possible to send keystrokes to the applet to copy the text that it is displaying to the clipboard. So, with a bit of luck you can send keystrokes to the Java app to copy each item of text to the clipboard, and then use the GetClipboard function to retrieve it into a script variable.

Another approach is to use optical character recognition. Such an approach which recognises, and extracts, text on the screen is described here:
http://www.mjtnet.com/blog/2006/06/06/s ... ical-text/

If we could see a screen shot of the applet we may have some other ideas.

[mfauzim]

Thanks marcus for the lead, I will try the texttract and will post any progress.

regards