How to "send keys" to a canvas element for longer duration?

Question

My aim:

I am trying to make a python bot to win chrome's dino game.

The game allows 2 types of jumps:

• short jumps
• long jumps

Using main_body.send_keys(Keys.SPACE) (as shown in the code below) gives short jumps.

My problem:

I am having difficulty executing long jumps.

My approach:

Currently, for long jumps, I am using the Keyboard library:

keyboard.press(keyboard.KEY_UP)

This, unfortunately, requires the browser windows to be in focus all the time. Later on, I wish to run this program headlessly, so this approach won't work.

Alternatively:

I tried ActionChains:

ActionChains(driver) \
.key_down(Keys.SPACE) \
.pause(0.2) \
.key_up(Keys.SPACE) \
.perform()

But this ends up scrolling the entire page. And doesn't fulfill the intended purpose.

I simply wish to "send" these actions to the canvas element directly instead of performing them on the entire page...

I wish to do something like this:

main_body.key_down(Keys.SPACE) 
time.sleep(0.2)
main_body.key_up(Keys.SPACE) 

Although this will, of course, give me this error: AttributeError: 'FirefoxWebElement' object has no attribute 'key_down' because canvas has no attribute key_down or key_up.

Here is an MCVE:

NOTE: In the code, the dino will keep jumping continuously but that is not the point, this is just to check the hights of the jumps and not to win the game.

import keyboard
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains

driver = webdriver.Firefox()
driver.get('https://chromedino.com/')
canvas = driver.find_element_by_css_selector('.runner-canvas')
main_body = driver.find_element_by_xpath("//html")

try:
    canvas.click()
except:
    main_body.send_keys(Keys.SPACE)    


while True:
    #short jump
    main_body.send_keys(Keys.SPACE)

    #long jump
    ActionChains(driver) \
    .key_down(Keys.SPACE) \
    .pause(0.2) \
    .key_up(Keys.SPACE) \
    .perform()

    #long jump using keyboard:
    keyboard.press(keyboard.KEY_UP)

Please see the effect of each type of jump by commenting out the code for others.

If possible suggest some other alternative way of getting a long jump without using Keyboard and without scrolling the entire page...

Answer 1

Unfortunately I cannot see a reproducible behaviour for jumps in that game. When i press UP o SPACE, I randomly see short or long jumps, so I cannot be really sure if my approach will work for you.

However, I think that, with a small effort, you could create a suitable event that will fit you needs. Basically, since Selenium can execute arbitrary javascript, my approach here is to send a keydown event to the canvas element (tested with Firefox 77).

A screenshot is made for each iteration to ensure the dino actually jumps.

Have fun.

from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time

options = FirefoxOptions()
options.add_argument("--headless")
driver = webdriver.Firefox(options=options)
driver.get('https://chromedino.com/')

canvas = driver.find_element_by_css_selector('.runner-canvas')
main_body = driver.find_element_by_xpath("//html")

try:
    canvas.click()
except:
    main_body.send_keys(Keys.SPACE)

while True:
    driver.execute_script('''
    var keydownEvt = new KeyboardEvent('keydown', {
        altKey:false,
        altKey: false,
        bubbles: true,
        cancelBubble: false,
        cancelable: true,
        charCode: 0,
        code: "Space",
        composed: true,
        ctrlKey: false,
        currentTarget: null,
        defaultPrevented: true,
        detail: 0,
        eventPhase: 0,
        isComposing: false,
        isTrusted: true,
        key: " ",
        keyCode: 32,
        location: 0,
        metaKey: false,
        repeat: false,
        returnValue: false,
        shiftKey: false,
        type: "keydown",
        which: 32,
    });
    arguments[0].dispatchEvent(keydownEvt);
    ''', canvas)
    driver.get_screenshot_as_file('proof_%s.png' % int(time.time()))
    time.sleep(0.2)

driver.quit()

Answer 2

You were so close. However a few words:

• The T-RexDinosaur within T-Rex Chrome Dino Game only animates within the canvas
• The control, i.e. pressing the space bar doesn't needs to be sent within the canvas but only can be initiated from the keyboard.

• In your code block for short jumps you don't need to send Keys.SPACE to any element.
• In your code block for long jumps as well you don't need to send Keys.SPACE to any element and the variable keyboard wasn't defined as well.

---

Solution

As a solution once you open the url you need to induce WebDriverWait for the visibility_of_element_located() of the canvas element and you can use either of the following Locator Strategy based solutions.

---

Short Jumps

For short jumps you can use:

driver.get("https://chromedino.com/")
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "canvas.runner-canvas")))
while True:
    #short jump
    ActionChains(driver).key_down(Keys.SPACE).key_up(Keys.SPACE).perform()

I was consistently able to score 69-72

Snapshot:

---

Long Jumps with 0.2 seconds pause

For long jumps with pause(0.2) you can use:

driver.get("https://chromedino.com/")
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "canvas.runner-canvas")))
while True:
    #short jump
    # ActionChains(driver).key_down(Keys.SPACE).key_up(Keys.SPACE).perform()

    #long jump
    ActionChains(driver).key_down(Keys.SPACE).pause(0.2).key_up(Keys.SPACE).perform()

I was consistently able to score 65

Snapshot:

---

Long Jumps with 0.5 seconds pause

For long jumps with pause(0.5) you can use:

driver.get("https://chromedino.com/")
WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "canvas.runner-canvas")))
while True:
    #short jump
    # ActionChains(driver).key_down(Keys.SPACE).key_up(Keys.SPACE).perform()

    #long jump
    ActionChains(driver).key_down(Keys.SPACE).pause(0.5).key_up(Keys.SPACE).perform()

I was consistently able to score 57-60

Snapshot: