Vietnamese treebank construction and entropy-based error detection
Treebanks, especially the Penn treebank for natural language processing (NLP) in English, play an essential role in both research into and the application of NLP.
However, many languages still lack treebanks and building a treebank can be very complicated and difficult.
This work has a twofold objective.
Firstly, to share our results in constructing a large Vietnamese treebank (VTB) with three levels of annotation including word segmentation, part-of-speech tagging, and syntactic analysis.
Major steps in the treebank construction process are described with particular regard to specific Vietnamese properties such as lack of word delimiter and isolation.
Those properties make sentences highly syntactically ambiguous, and therefore it is difficult to ensure a high level of agreement among annotators.
Various studies of Vietnamese syntax were employed not only to define annotations but also to systematically deal with ambiguities.