When should Emacs #'function syntax be used?

Question

Basically, when should I use Emacs Lisp's function procedure? I haven't found any examples in which there's a difference in behavior if you pass functions as arguments 'like-this or #'like-this. In fact, if I evaluate (eq 'goto-char #'goto-char) it returns t.

The Emacs Lisp code that I've come across rarely uses function/#'; the authors just quote/' everything.
Example: (add-hook 'emacs-lisp-hook 'turn-on-eldoc-mode)

However, I can find a few counterexamples. Here's one from the source code of Emacs 24.3's electric.el:

(add-hook 'post-self-insert-hook           #'electric-indent-post-self-insert-function           'append) 

Guesses and further questions:

• Is it just a Lisp-2 stylistic convention?
• Does it have to do with byte-compilation?
• Does it only matter for library writers? Like, if you intend for your code to be run under a huge number of environments and Emacs versions? (The corollary would be if you're just "dotfile-tinkering" then you don't need to worry about all this.)
• When should I quote lambda-expressions? When can I leave them unquoted?
  E.g., (do-something '(lambda …
  versus (do-something (lambda …
• Was there some limitation in an earlier version of Emacs that gave rise to these facets of elisp? Like, can I mostly ignore the difference between ' and #' as long as I'm using a version of Emacs more recent than X?

Answer 1

function (aka #') is used to quote functions, whereas quote (aka ') is used to quote data. Now, in Emacs-Lisp a symbol whose function cell is a function is itself a function, so #'symbol is just the same as 'symbol in practice (tho the intention is different, the first making it clear that one is not just talking about the symbol "symbol" but about the function named "symbol").

The place where the difference is not just stylistic is when quoting lambdas: '(lambda ...) is an expression which evaluates to a list whose first element is the symbol lambda. You're allowed to apply things like car and cdr to it, but you should not call it as if it were a function (although in practice it tends to work just fine). On the contrary #'(lambda ...) (which can be written just (lambda ...)) is an expression which evaluates to a function. That means you can't apply car to it, but the byte-compiler can look inside #'(lambda ...), perform macro-expansion in it, warn you if what it finds doesn't look kosher, etc...; For lexical-binding it even has to look inside in order to find the free variables to which that function refers.