Shell: Connecting to a website and accessing a field

Question

I want to write a script that takes an argument which is text, opens a connection to a specific website and input the arg into text field using the field's ID. Is this possible? How would I do this? I'm a total shell noob

Edit:

Exact flow:

- start script with string
- input string into text field on web page
- click form button
- wait for processing
- click hyperlink

Answer 1

If you know exactly which field you need to fill, then this can be done using lynx. Assume you get the string S with your script as an input argument. Then you create a command script, which will guide lynx through its behaviour.

For example, suppose S=foo, and your field is the second field in the web page. After that, there are two more fields, and then the submit button. After that you wait for the page to load and click the hyperlink (after that you exit). The web page is www.something.com.

The command script would be in a file bar.txt:

key <tab> //get to first field
key <tab> //get to second field
key f     //input f
key o     //input o
key o     //input o
key <tab> //get to third field
key <tab> //get to fourth field
key <tab> //get to sumbit button
key ^J    //click submit and wait for load
key <tab> //get to hyperlink
key ^J    //click hyperlink and wait for load
key Q     //exit
key y     //confirm exit

The main command would then be lynx www.something.com -accept_all_cookies -cmd_script=bar.txt

Now all you need to do is dynamically create the input string.

#!/bin/bash
script=bar.txt
input=$1
webpage=www.something.com
len=${#input}
echo 'key <tab>' > $script
echo 'key <tab>' >> $script
for i in `echo $input|fold -w1` 
do
    echo 'key '$i >> $script
done
echo 'key <tab>' >> $script
echo 'key <tab>' >> $script
echo 'key <tab>' >> $script
echo 'key ^J' >> $script
echo 'key <tab>' >> $script
echo 'key ^J' >> $script
echo 'key Q' >> $script
echo 'key y' >> $script

lnyx $webpage -accept_all_cookies -cmd_script=bar.txt

Now all you need to do is save the script, modify it to be executable and call it ./script your_string

Answer 2

To get you started, here is my script to order today's lunch from our local canteen:

URL="https://lunch.com/lunch/cgi-bin/order.cgi"

O="order=Order"
A="amount_%d=%%d&amount_foil_container_%d=%%d"

function order_lunch() {
  if [[ -n "$@" ]]; then
    curl -u "$USER":"$PASSWORD" \
         -d $(printf $(printf "$O&$A&$A&$A&$A" 0 0 1 1 2 2 3 3) \
                     "${@:2:8}") \
         "$URL";
  else
    echo "Nothing to order.";
  fi;
}

Where input is a string in the following format

2012-08-23 1 0 0 0 0 0 0 0

where each field denotes a different dish, i.e. a 1 in the first position after the date is "1 pasta"

Good luck.

Answer 3

... "opens a connection to a specific website and input the arg into text field using the field's ID" ...

You mean you want to fill & send a HTML <form> ... </form>, right?

I would use curl (http://curl.haxx.se/). With curl you can automate HTTP POST requests very easily, suppose you have website with following form (excerpt from: http://curl.haxx.se/docs/httpscripting.html):

<form method="POST" action="junk.cgi">
      <input type=text name="birthyear">
      <input type=submit name=press value=" OK ">
</form>

this command would fill & send the form (let's pretend that form is available on http://www.example.com/when.cgi):

curl --data "birthyear=1905&press=%20OK%20" http://www.example.com/when.cgi