python requests post to url query parameter requires special character

Question

I am using python requests to submit a post request to a specific url. This url has a query parameter that contains the actual filepath of the file I am uplaoding, with the actual colon (':') and backslashes('\') (it will not accept %3A, or %5C). I have no idea what to do.

Here is my code:

with requests.Session() as s:

 payload_10_vo_params = {
    'hidInputControlValues': this_hidInputControlValues10,
    'hidFormNames': this_hidFormNames10,
    'hidToken': this_hidToken10,
    'hidFileNm': r"C:\fakepath\{}".format(this_FileName),
    'hidFileSourceId': '',
    'ReUploadInd': ''
}

payload10_mpe_vo = MultipartEncoder(
    [
        ('hidLoanTranSaveFlag', 'TRUE'),
        ('hidUpfrontMIPct', ''),
        ('radLeadRequestOptBtnImportRetNum', 'LUPL'),
        ('txtCorrCompId', ''),
        ('cboLeadRequestFileTypCd', 'DU32'),
        ('txtLeadRequestSelectFile', (this_FileName, open(this_FileName, 'rb'), 'application/octet-stream')),
        ('txaLoanTranComments', ''),
    ], boundary=ct_disp_bndry_10_final_str)

headers10 = {
    "Accept": "application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, */*",
    "Accept-Language": "en-US",
    "User-Agent": "Mozilla/5.0 (compatible; MSIE 7.0; Windows NT 6.1; Win64; x64; Trident/7.0; .NET CLR 2.0.50727; SLCC2; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; Tablet PC 2.0)",
    "UA-CPU": "AMD64",
    "Content-Type": "multipart/form-data; boundary={}".format(ct_disp_bndry_10_final_str),
    "Referer": "referer string",
    "Host": "host string"}

resp10 = s.post(url10, headers=headers10, params=payload_10_vo_params,  data=payload10_mpe_vo)    

Here is the value of the response url from fiddler, please see the final parameter(the value for hidFileNm), everything else is matching fine, (those special characters are for the special character §, which was encoded perfectly by requests).

POST /SRVCorrBP/LeadMgmt/Application/Lead/LeadSubmitNew.jsp?hidInputControlValues=�DU32�Y�LUPL�C%3A%5CFAKEPATH%5CMAYRA%20CORTEZ.FNM�&hidFormNames=�frmLoanTran~TRUE~�&hidToken=1508961847871LSXG_EDTLeadSummary&hidFileNm=C:\fakepath\filename&hidFileSourceId=&ReUploadInd=

This is what I get:

/SRVCorrBP/LeadMgmt/Application/Lead/LeadSubmitNew.jsp?hidInputControlValues=%C2%A7DU32%C2%A7Y%C2%A7LUPL%C2%A7C%3A%5CFAKEPATH%5CBOPPPMVO.FNM%C2%A7&hidFormNames=%C2%A7frmLoanTran~TRUE~%C2%A7&hidToken=1512952710233LSXG_EDTLeadSummary&hidFileNm=C%3A%5Cfakepath%5Cfilename&hidFileSourceId=&ReUploadInd=

As you can see, the hidFileNm parameter is supposed to be

hidFileNm=hidFileNm=C:\fakepath\filename

But my request is posting as:

hidFileNm=C%3A%5Cfakepath%5Cfilename

I've used requests many times before but this is the first time I have run into this issue.

Answer 1

EDIT: it worked before but it seems they change something and it doesn't work any more.

---

requests automatically escapes parameters before it adds them to url.

You have to manually create string with url and all parameters.

create strings key=value

args = ['{}={}'.format(k, v) for k,v in payload_10_vo_params.items()]

create one string key1=value1&key2=value2

args_in_one_string = "&".join(args)

create url with parameters

url10 = url10 + "?" + args_in_one_string

---

EDIT: there is some prepared-request and example in request issues

from requests import Request, Session

s = Session()
req = Request('GET', url)
prepped = s.prepare_request(req)
prepped.url = prepped.url.replace('.', '%2E')
resp = s.send(prepped)

but I don't know if it can help.

Maybe you will have to use urllib