Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Submit form with no submit button in rvest

I'm trying write a crawler to download some information, similar to this Stack Overflow post. The answer is useful for creating the filled-in form, but I'm struggling to find a way to submit the form when a submit button is not part of the form. Here is an example:

session <- html_session("www.chase.com")
form <- html_form(session)[[3]]

filledform <- set_values(form, `user_name` = user_name, `usr_password` = usr_password)
session <- submit_form(session, filledform)

At this point, I receive this error:

Error in names(submits)[[1]] : subscript out of bounds

How can I make this form submit?

like image 764
hfisch Avatar asked Nov 24 '15 04:11

hfisch


People also ask

Is it possible to submit a form without a submit button?

The form can be submitted without using submit button by implementing a specific event attribute or by clicking the link. This task can be done by using the OnClick event attribute or by using the form. submit() method in Javascript. Approach: The form.

Why submit button is not working in Javascript?

Sometimes the problem is caused by old versions of the Javascript files, cached by your browser and can be fixed by clearing the browser cache. You can use the browser console of your browser for debugging. After the Javascript error is fixed, the submit button will automatically be enabled.

How can I submit a form without using Javascript?

Use the <noscript></noscript> tags to define a "normal" submit-button when javascript is disabled. Show activity on this post. You could maybe use the <noscript> tag and encapsulate the above code with the button type as submit. If the client has js, the code inside the noscript will be ignored.

Can you submit a form with a button?

You can tie a submit button to a form that the button doesn't live inside of. The trick is to give the form an id and then reference that id with the button's form property. With this setup, clicking the Submit button will cause the form to be submitted.


1 Answers

Here's a dirty hack that works for me: After studying the submit_form source code, I figured that I could work around the problem by injecting a fake submit button into my code version of the form, and then the submit_form function would call that. It works, except that it gives a warning that often lists an inappropriate input object (not in the example below, though). However, despite the warning, the code works for me:

session <- html_session("www.chase.com")
form <- html_form(session)[[3]]

# Form on home page has no submit button,
# so inject a fake submit button or else rvest cannot submit it.
# When I do this, rvest gives a warning "Submitting with '___'", where "___" is
# often an irrelevant field item.
# This warning might be an rvest (version 0.3.2) bug, but the code works.
fake_submit_button <- list(name = NULL,
                           type = "submit",
                           value = NULL,
                           checked = NULL,
                           disabled = NULL,
                           readonly = NULL,
                           required = FALSE)
attr(fake_submit_button, "class") <- "input"
form[["fields"]][["submit"]] <- fake_submit_button

user_name <- "user"
usr_password <- "password"

filledform <- set_values(form, `user_name` = user_name, `usr_password` = usr_password)
session <- submit_form(session, filledform)

The successful result displays the following warning, which I simply ignore:

> Submitting with 'submit'
like image 199
Tripartio Avatar answered Oct 20 '22 00:10

Tripartio