COLDFUSION Copy from Word/Outlook causing question marks

Question

We have a web app that was created in CF and when we copy from word/outlook and paste into out comment field to submit. It saves to the database with some kind of encoding that causes "?".

I have been trying figure out what the mysterious coding so i can perform a replace function. I have tried so many replaces:

str = RemoveHTML(str);
//Replace Tabls
str = Replace(str, Chr(9), " ", "All");
//Replace Newlines and carriage returns
str = Replace(str, Chr(10), " ", "All");
str = Replace(str, Chr(13), " ", "All");
str = Replace(str,"[^0-9A-Za-z ]","","all");
str = Replace(str, "[^\x20-\x7E]", "", "ALL");

str = Replace(str, Chr(32), " ", "All");



//Replace two or more blanks with one space
str = ReReplaceNoCase(str, "[[:blank:]]{2,}", " ", "All");
str = Trim(str);
str = Left(str, 5000);
return str;

Any help going in the right direction would be greatly appreciated.

Thanks so much!

Answer 1

The character encoding is not going through. The content of the Word document contains characters (like the curly quote marks) that the DB isn't recognizing. Check the DB table's column data type to see if it's VARCHAR or NVARCHAR.

• NVARCHAR allows Unicode
• VARCHAR doesn't

There are some RTF editors that magically convert text when pasted from MS Word. You can try to just capture the paste event and call some JS code to address the common special characters from Word documents.

https://developer.mozilla.org/en-US/docs/Web/API/Element/paste_event

Or try an RTF editor for the <textarea> that should make that conversion for you.

https://quilljs.com/