Image Recognition

[diavola]

Hi,

I've been using this software for a month or two now its great, however got stuck with this issue I have, mainly because I'm no coder. Was wondering if anyone would be kind enough and help me out?

I have this form which is VB based, the submit button stays the same size and color. There are two of them, one which goes inside the green box and the other inside the red but are never in the same spot twice. Basically I want it to identify the button which is inside the Red box and click on it. Here's a very bad paintshop image I drew earlier to give more insight of what it might look like.

http://imgur.com/V38DLym

[diavola]

Oh and I forget to mention

the red side, can be where the green side is, and vice versa, the only thing that are constant are the sizes of the boxes and buttons/button color remains the same

[diavola]

Oh and another thing, the Image is an example of 2 different instances of the form, there is only one red part and 1 green part

[Dorian (MJT support)]

Can the Find Object Wizard see the buttons? If so, do they always have the same properties? If so, try using that and see if you can reliably click on your target button.

If not, then we'll go back to image recognition.

For this, we can use a combination of GetPixelColor, and the Image Recognition Wizard.

Aside from the tops of the boxes swapping color, are the boxes always in the same position on the screen, or can they be made to be in the same position by moving a window? Ideally, we need the answer to be yes.

First, use the Image Recognition Wizard, using :

"Find the position of an image to interact with",
and
"search within this rectangle".

When drawing the rectangle, include one entire side of the left box.

Follow through, and allow the wizard to generate the code.
You should be left with something like this :

Code: Select all

ScreenCapture>13,3,613,379,%TEMP_DIR%\screenrect.bmp
FindImagePos>%BMP_DIR%\image_7.bmp,%TEMP_DIR%\screenrect.bmp,0.7,1,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
  MouseMove>{%XArr_0%+13},{%YArr_0%+3}
  LClick
Endif

Notice there are four coordinates there. You can use this to look at both boxes, just by copying the code and editing the co-ordinates so they're looking at the other box.

We know the top left of the box is at 13,3, so lets check the color.

Code: Select all

GetPixelColor>13,3,LeftColor

You'll need to figure out the numerical value for the red, or the green. This will be stored in the variable LeftColor.

Code: Select all

GetPixelColor>13,3,LeftColor
mdl>LeftColor

For example, using this on my desktop, the value of LeftColor was 16777215.

So let's say 16777215 is the color match for the box you want to click in.

We'll use IF, ELSE, and ENDIF, like this :

IF...
the left box pixel color is red...
look in the left box
find the button and click it
ELSE
if it isn't red,
look in the right box
find the button and click it
ENDIF

So the finished product will end up looking something like this. (Don't use mine, as the mouse co-ordinates, colors, and button sizes will be different).

Code: Select all

//Get the color of the left box header pixel
GetPixelColor>13,3,LeftColor

//If it's red
If>LeftColor=16777215

//Look for the button in the left box
ScreenCapture>13,3,613,379,%TEMP_DIR%\screenrect.bmp
FindImagePos>%BMP_DIR%\image_7.bmp,%TEMP_DIR%\screenrect.bmp,0.7,1,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
  MouseMove>{%XArr_0%+13},{%YArr_0%+3}
  LClick
Endif

Else
//If it wasn't red
//Look for the button in the right box... make sure to edit the coordinates below so they look in the right box.
ScreenCapture>XX,XX,XXX,XXX,%TEMP_DIR%\screenrect.bmp
FindImagePos>%BMP_DIR%\image_7.bmp,%TEMP_DIR%\screenrect.bmp,0.7,1,XArr,YArr,NumFound,CCOEFF
If>NumFound>0
  MouseMove>{%XArr_0%+13},{%YArr_0%+3}
  LClick
Endif

Endif