Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Logging into a website via C#, navigating to another web page, and then output the source code as a string

Tags:

c#

.net

c#-4.0

I am relatively new to working with web pages in C#. What I am trying to do is log into a particular website ( https://www15.swalife.com/PortalWeb/portal/cwaLogon.jsp ) and allowing the page to be redirected to the default page, and then navigating from there to (https://www15.swalife.com/csswa/ea/plt/accessELITT.do) and downloading the source code and output it to a string.

I have figured out how to download the source code via HTTPWebRequest and HTTPWebResponse but am having trouble on coding the logging in function. I assume I will have to do something with POST? I have looked at http://www.dreamincode.net/forums/topic/152297-c%23-log-in-to-website-programmatically/ also.

Thanks in advance!!

EDIT:

The code supplied by jimmyjambles works flawlessly, except it doesn't quite get me the source code of the page I wanted. The code suggests that the log-in process failed, but I believe with a little bit of tweaking I could get it to work...also to everyone having issues with:

ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback(AcceptAllCertifications);

Try changing your "public string" and "public bool" functions to "public static string" and "public static bool" respectively :)

EDIT 2:

Response HTML:

"<!DOCTYPE HTML PUBLIC \"-//W3C//DTD HTML 4.01 Transitional//EN\">\n<HTML>\n<HEAD>\n\n\n\n\n\n\n<META http-equiv=\"Content-Type\" content=\"text/html; charset=ISO-8859-1\">\n<META name=\"GENERATOR\" content=\"IBM WebSphere Studio\">\n<TITLE>endSession.jsp</TITLE>\n<LINK rel=\"stylesheet\" href=\"eipPortletStyles/swalife.css\" type=\"text/css\">\n\t<script type=\"text/javascript\" language=\"JavaScript\" \n\t\tsrc=\"eipCommonJavaScript/eipGeneralFunctions.js\"/> </script>\n\t\t\n<script type=\"text/javascript\">\n\n\tfunction refreshParent()\n\t{\n\t    if(window.parent)\n\t    {\n\t    if(window.parent.name == 'appMainFrame')\n\t       window.parent.location = \"/csswa/ea/plt/logout.do\";\n\t    //  alert('Your session has expired.  Please login again. ');\n\t    }\n\t}\n\n</script>\n</HEAD>\n<BODY onload=\"refreshParent();\">\n \n\t \t<div class=\"eipErrors\">\n  \t\t\t<div class=\"legendLabel\">Message</div>\n  \t\t\t\n  \t\t\t    <div class=\"errorsHeader formTitle\">You Have Exited Out of Crew Web Access.<br>  \t\t\t \n  \t\t\t    </div>\n  \t\t\t \n  \t\t\t<div class=\"errorsHeader formTitle\"> Please Close this Window and <font  size=\"+1\">Log Out of SWALife</font> to Complete the Log Out Process.  </div>\n  \t\t<div class=\"errorsText\">\n  \t\t  &nbsp;\n  \t\t\t\t\n  \t\t</div>\n  \t\t\n  \t\t\t\n  \t\t\n  \t\t<div class=\"errorsFooter\">You will need to log back in before continuing.</div>  \t\n  \t\t\n \t</div>\n  \n</BODY>\n</HTML>\n"
like image 393
Harrison Totty Avatar asked Jun 08 '12 18:06

Harrison Totty


1 Answers

In order to use HttpWebRequest to access a secondary URL after login you will need to keep in mind a few things.

Firstly as Casperah has mentioned you will need to inspect the login form and identify the "name" attribute of the controls used to receive the login data.

Once you have done this you will need to format a post string accordingly and provide it to the WebRequest.

The last consideration is that once you have logged in, you will need to store and maintain the cookie assigned to you from the server that keeps you logged in.

I have taken a WebRequest snippet from this msdn article and modified it to perform a second page request after the login.

        string loginurl = "http://www.gmail.com";
        string secondurl = "http://mail.google.com/prefs";

        string username = "[email protected]";
        string password = "12345";
        GetSecondaryLoginPage(loginurl, secondurl, username, password);



    public string GetSecondaryLoginPage(string loginurl, string secondurl, string username, string password, string cookieName = null)
    {
        // Create a request using a URL that can receive a post. 
        HttpWebRequest request = (HttpWebRequest)WebRequest.Create(loginurl);
        // Set the Method property of the request to POST.
        request.Method = "POST";

        CookieContainer container = new CookieContainer();

        if (cookieName != null)
            container.Add(new Cookie(cookieName, username, "/", new Uri(loginurl).Host));

        request.CookieContainer = container;

        // Create POST data and convert it to a byte array.  Modify this line accordingly
        string postData = String.Format("username={0}&password={1}", username, password);

        ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);

        byte[] byteArray = Encoding.UTF8.GetBytes(postData);
        // Set the ContentType property of the WebRequest.
        request.ContentType = "application/x-www-form-urlencoded";
        // Set the ContentLength property of the WebRequest.
        request.ContentLength = byteArray.Length;
        // Get the request stream.
        Stream dataStream = request.GetRequestStream();
        // Write the data to the request stream.
        dataStream.Write(byteArray, 0, byteArray.Length);
        // Close the Stream object.
        dataStream.Close();
        // Get the response.
        WebResponse response = request.GetResponse();
        // Get the stream containing content returned by the server.
        dataStream = response.GetResponseStream();
        // Open the stream using a StreamReader for easy access.
        StreamReader reader = new StreamReader(dataStream);
        // Read the content.
        string responseFromServer = reader.ReadToEnd();

        using (StreamWriter outfile =
        new StreamWriter("output.html"))
        {
            outfile.Write(responseFromServer.ToString());
        }

        // Clean up the streams.
        reader.Close();
        dataStream.Close();
        response.Close();

        request = (HttpWebRequest)WebRequest.Create(secondurl);
        request.CookieContainer = container;

        response = request.GetResponse();
        // Get the stream containing content returned by the server.
        dataStream = response.GetResponseStream();
        // Open the stream using a StreamReader for easy access.
        reader = new StreamReader(dataStream);
        // Read the content.
        responseFromServer = reader.ReadToEnd();

        // Clean up the streams.
        reader.Close();
        dataStream.Close();
        response.Close();

        return responseFromServer;
    }


    public bool AcceptAllCertifications(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certification, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
    {
        return true;
    }

The only added lines are for the postData and cookies.

You will need to modify the line

string postData = String.Format("username={0}&password={1}", username, password);

based on your controls on the form, since you posted the site your trying to work with I can guess that you are probably looking for

string postData = String.Format("uid={0}&portalBase=cwa&password={1}", username, password);
like image 85
jimmyjambles Avatar answered Oct 12 '22 23:10

jimmyjambles