Insert regular double quotes in LaTeX mode with AUCTeX

Question

How do I rebind the double quote key to simply insert a double quote in a LaTeX buffer with AUCTex enabled?

I tried redefining TeX open and close quote, but that didn't seem to work.

(add-hook 'LaTeX-mode-hook
          '(progn
             (setq-default TeX-close-quote "\"")
             (setq-default tex-close-quote "\"")
             (setq-default TeX-open-quote "\"")
             (setq-default tex-open-quote "\"")
             (setq-default TeX-quote-after-quote t)))

Update

The above code and the accepted answer would have worked, except that I have smartparens enabled. Smartparens helpfully redefines the quote key to insert LaTeX quotes. The code to use normal quotes is below:

(eval-after-load 'latex
  '(progn
     (require 'smartparens-latex)
     ;; removes the double quote trigger binding. Now smartparens will 
     ;; insert a regular double quote
     (sp-local-pair 'latex-mode "``" "''" :trigger "\"" :actions :rem)))

Answer 1

You can unset the auctex binding as:

(defun my-hook ()
  (local-unset-key "\""))
(add-hook 'LaTeX-mode-hook 'my-hook)

Alternately, if you want to use the smart quotes most of the time but occasionally insert a literal double quote, just do C-q ".