M-Rique commited on
Commit
976945e
·
1 Parent(s): 3defe27

Add crosshair

Browse files
Files changed (1) hide show
  1. e2bqwen.py +5 -4
e2bqwen.py CHANGED
@@ -50,7 +50,8 @@ The desktop has a resolution of <<resolution_x>>x<<resolution_y>> pixels, take i
50
  Use precise coordinates based on the current screenshot for mouse movements and clicks.
51
  Whenever you click, MAKE SURE to click in the middle of the button, text, link or any other clickable element. Not under, not on the side. IN THE MIDDLE, else you risk to miss it.
52
  In menus it is always better to click in the middle of the text rather than in the tiny icon. Calculate extremelly well the coordinates. A mistake here can make the full task fail.
53
- Sometimes you may have missed a click, so never assume that you're on the right page, always make sure that your previous action worked. In the screenshot you can see if the mouse pointer is off of the targeted element, pay special attention to it.
 
54
  </click_guidelines>
55
 
56
  <task_resolution_example>
@@ -409,9 +410,9 @@ class E2BVisionAgent(CodeAgent):
409
 
410
  image_copy = image.copy()
411
 
412
- # if getattr(self, "click_coordinates", None):
413
- # print("DRAWING MARKER")
414
- # image_copy = draw_marker_on_image(image_copy, self.click_coordinates)
415
 
416
  self.last_marked_screenshot = AgentImage(screenshot_path)
417
  print(f"Saved screenshot for step {current_step} to {screenshot_path}")
 
50
  Use precise coordinates based on the current screenshot for mouse movements and clicks.
51
  Whenever you click, MAKE SURE to click in the middle of the button, text, link or any other clickable element. Not under, not on the side. IN THE MIDDLE, else you risk to miss it.
52
  In menus it is always better to click in the middle of the text rather than in the tiny icon. Calculate extremelly well the coordinates. A mistake here can make the full task fail.
53
+ Sometimes you may have missed a click, so never assume that you're on the right page, always make sure that your previous action worked.
54
+ In the screenshot you will see a green crosshair displayed over the position of your last click: this way can inspect if the mouse pointer is off of the targeted element, pay special attention to it.
55
  </click_guidelines>
56
 
57
  <task_resolution_example>
 
410
 
411
  image_copy = image.copy()
412
 
413
+ if getattr(self, "click_coordinates", None):
414
+ print("DRAWING MARKER")
415
+ image_copy = draw_marker_on_image(image_copy, self.click_coordinates)
416
 
417
  self.last_marked_screenshot = AgentImage(screenshot_path)
418
  print(f"Saved screenshot for step {current_step} to {screenshot_path}")