pysummarization.tokenizabledoc package¶
Submodules¶
pysummarization.tokenizabledoc.mecab_tokenizer module¶
-
class
pysummarization.tokenizabledoc.mecab_tokenizer.
MeCabTokenizer
[source]¶ Bases:
pysummarization.tokenizable_doc.TokenizableDoc
Tokenize string.
Japanese morphological analysis with MeCab.
-
part_of_speech
¶ getter
-
pysummarization.tokenizabledoc.simple_tokenizer module¶
-
class
pysummarization.tokenizabledoc.simple_tokenizer.
SimpleTokenizer
[source]¶ Bases:
pysummarization.tokenizable_doc.TokenizableDoc
Tokenize delimited sentence with a blank.