How can I split a text into sentences using the Stanford parser?

Question

How can I split a text or paragraph into sentences using Stanford parser?

Is there any method that can extract sentences, such as getSentencesFromString() as it's provided for Ruby?

Answer 1

You can check the DocumentPreprocessor class. Below is a short snippet. I think there may be other ways to do what you want.

String paragraph = "My 1st sentence. “Does it work for questions?” My third sentence."; Reader reader = new StringReader(paragraph); DocumentPreprocessor dp = new DocumentPreprocessor(reader); List<String> sentenceList = new ArrayList<String>();  for (List<HasWord> sentence : dp) {    // SentenceUtils not Sentence    String sentenceString = SentenceUtils.listToString(sentence);    sentenceList.add(sentenceString); }  for (String sentence : sentenceList) {    System.out.println(sentence); } 

Answer 2

I know there is already an accepted answer...but typically you'd just grab the SentenceAnnotations from an annotated doc.

// creates a StanfordCoreNLP object, with POS tagging, lemmatization, NER, parsing, and coreference resolution  Properties props = new Properties(); props.put("annotators", "tokenize, ssplit, pos, lemma, ner, parse, dcoref"); StanfordCoreNLP pipeline = new StanfordCoreNLP(props);  // read some text in the text variable String text = ... // Add your text here!  // create an empty Annotation just with the given text Annotation document = new Annotation(text);  // run all Annotators on this text pipeline.annotate(document);  // these are all the sentences in this document // a CoreMap is essentially a Map that uses class objects as keys and has values with custom types List<CoreMap> sentences = document.get(SentencesAnnotation.class);  for(CoreMap sentence: sentences) {   // traversing the words in the current sentence   // a CoreLabel is a CoreMap with additional token-specific methods   for (CoreLabel token: sentence.get(TokensAnnotation.class)) {     // this is the text of the token     String word = token.get(TextAnnotation.class);     // this is the POS tag of the token     String pos = token.get(PartOfSpeechAnnotation.class);     // this is the NER label of the token     String ne = token.get(NamedEntityTagAnnotation.class);          }  } 

Source - http://nlp.stanford.edu/software/corenlp.shtml (half way down)

And if you're only looking for sentences, you can drop the later steps like "parse" and "dcoref" from the pipeline initialization, it'll save you some load and processing time. Rock and roll. ~K