Sentence Boundary Detection? about spacy-api-docker

[jgontrum]

I saw a twitter conversation today, where Matthew wrote that sentence boundary detection (that does not require dependency parsing) is on the roadmap for spaCy v2. (https://twitter.com/honnibal/status/860395803826937856).

But if you have a Python module that detects sentence boundaries on a raw text, It would be no problem to integrate this in the API as separate entry point.

[roschler]

Hi Johannes,

Thanks for replying. I'm a bit confused. I thought the current version of spaCy already does sentence boundary detection?:

explosion/spaCy#23

"Improved sentence segmentation now included in the latest release. Docs are updated with usage."

I'd like to be able to submit an entire document to the server and have it return a JSON array of all segmented sentences.

Same for me. /dep will return a list of annotated tokens but it would be nice to have something like this:

{
  'sentence': [
    {'index': 0, 'dep': [...]},
    {'index': 1, 'dep': [...]},
  ]
}

Similar to the JSON result that's provided by the Stanford CoreNLP Server.

[jgontrum]

I agree this is probably a better a format when working with the output.
However, I'm currently not using this API in any of my projects, so development has a rather low priority for me.

But if you want to change the format, have a look at this part:

spacy-api-docker/displacy_service/parse.py

Lines 27 to 49 in 78585e8

	def to_json(self):
	words = [{'text': w.text, 'tag': w.tag_} for w in self.doc]
	arcs = []
	for word in self.doc:
	if word.i < word.head.i:
	arcs.append(
	{
	'start': word.i,
	'end': word.head.i,
	'label': word.dep_,
	'text': str(word),
	'dir': 'left'
	})
	elif word.i > word.head.i:
	arcs.append(
	{
	'start': word.head.i,
	'end': word.i,
	'label': word.dep_,
	'text': str(word),
	'dir': 'right'
	})
	return {'words': words, 'arcs': arcs}

I'm always happy to receive pull requests if you make any changes :)