There are various corpora of ‘gold standard’ syntactically analysed natural language. Below is a sample from the ‘Penn TreeBank’, analysing the sentence ‘W.R. Grace holds three of Grace Energy ‘s seven board seats’
( (S
(NP-SBJ (NNP W.R.) (NNP Grace) )
(VP (VBZ holds)
(NP
(NP (CD three) )
(PP (IN of)
(NP
(NP (NNP Grace) (NNP Energy) (POS 's) )
(CD seven) (NN board) (NNS seats) ))))
(. .) ))
There are a variety of computational tasks on might undertake that take such a corpus as input.
Certain transformation phenomena are typically noted in syntax courses and linguistics text-books, as motivation for ‘constituents’, for example that long multi-word ‘noun phrases’ can typically be replaced by single word ‘pronouns’ , predicting for example from the above structure thatt ‘Grace Energy ‘s seven board seats’ could be replaced by ‘them’. Code could be developed implementing such tree transformations, and one could evaluate whether or not the result — as just a sequence of words — after the transformation is indeed acceptable, thereby using the corpus to assess a linguistic claim. There are many variants of this. One might also look at how easily, or not, it might be to implement correctly the same transformations without making reference to the proposed syntax structure.
One may have a parsing program (or contending annotators) , which constructs trees in the same format as the corpus and may wish to assess how closely the parser reproduces the ‘gold standard’ (resp. the annoators agree with each other). One way to approach this is via the so-called ‘tree distance’ measure (a tree-analogue of the Levenshtein tree distance). One could implement this, and possibly other measures, and compare them.