NLTK. Find if a sentence is in a questioning form

Question

I am trying to detect if a sentence is a question or a statement. Apart from looking for a question mark at the end of the sentence, is there another way to detect this? I am processing Twitter posts and people are not necessarily following good practises like question marks on Twitter.

Reference to other libraries is also ok with me if nltk does now work.

Answer 1

One simple way to do this is to parse a sentence and look for the tag assigned to it. For example, parsing the sentence "Is there any way to do this?" with Stanford parser will return:

(ROOT
  (SQ (VBZ Is)
    (NP (EX there))
    (NP
      (NP (DT any) (JJ other) (NN way))
      (S
        (VP (TO to)
          (VP (VB do)
            (NP (DT this))))))
    (. ?)))

where SQ denotes "Inverted yes/no question, or main clause of a wh-question, following the wh-phrase in SBARQ". Another example:

(ROOT
  (SBARQ
    (WHNP (WP What))
    (SQ (VBZ is)
      (NP
        (NP (DT the) (NN capital))
        (PP (IN of)
          (NP (NNP Scotland)))))
    (. ?)))

where SBARQ denotes "Direct question introduced by a wh-word or a wh-phrase". It's pretty straightforward to call an external parser from Python and process its output, for example check this Python interface to Stanford NLP tools.