This stage flags vertices with “Skip-Sentence”. The vertex flag is the start of the sentence. This can be used to ignore a complete sentence by a later stage. The conditions evaluated by the processor are: - Sentence length, given by the token count, not vertices.
- A list of tags that work as an exception to the count, meaning that if the tag is found within the sentence the count is irrelevant and the sentence is not flagged (whitelisting).
- A list of tags that if found in the sentence it should be flagged (blacklisting).
Blacklisting a tag always has precedence over the other values, so any sentence with a blacklisted flag will always be flagged as “SKIP_SENTENCE”. Whitelisted tags will always have precedence over the token limit restriction. And finally token limit restriction is on effect. Note |
---|
Sentence Filter will flag the initial vertex of the sentence with a "SKIP_SENTENCE" flag, it will not remove the sentence from the interpretation graph. |
|