Stopper?

Some months in the past, back in April-June when I was dealing with the foo]] problem, I made a note to myself:

Handler valid_ending_pos woefully inadequate for Tags

But by the time I got around to reading my note, I couldn't fathom what the problem was. Clearly past-me had seen something, but present-me didn't see anything.

Today-me found it. On the Grammarian status code list, there is only one code specific to Tags.  In section 3000:

400 Trigger or verbatim found in tag content

There is no testing for status 400 anywhere in the current suite. There is also no code which sets a status 400. Consider the following.

Lorem ipsum [foo bar;baz [[quux dolor sit amet, consectetuer
adipiscing]] elit.] Sed massa mi, dapibus.

This is doubly-illegal. It's forbidden by the grammar rule embodied in status 3400 as well as the rule which says that the opening markup of triggers must be the first thing on a line. Well, the trigger-specific infringement will be caught (status 3200, as it happens), but the 3400 violation isn't caught by any code at all. And what about...

Lorem ipsum [foo bar;baz
[[quux dolor sit amet, consectetuer
adipiscing]]
elit.] Sed massa mi, dapibus.

There. Now the 3200 violation goes away (this code is well-formed) AND the 3400 violation isn't caught. Problem! Isn't it?

That depends on whose job it is to know what's inside of what. This specific example could be caught by Tag handling code, but moving the word dolor outside the trigger...

Lorem ipsum [foo bar;baz dolor
[[quux sit amet, consectetuer
adipiscing]]
elit.] Sed massa mi, dapibus.

changes all that, because now the trigger is seen by the Text handler, not the Tag handler.

Now we see that, at its root, this problem is the same as the problem which led to the creation of the VerbatimText handler: a handler must be able to introspect the layers "above" it to make the correct decision in one specific case. And now, like then, I feel the proper thing to do is to stick with the design as it stands.

This means that the proper place for code to detect this grammar violation and throw the associated status code is the Grammarian itself. The Grammarian must be responsible for all such issues where knowledge of the node context stack is needed to make the right decision (just as it is in the case of VerbatimText, which the Grammarian will explicitly call instead of the regular Text handler when a Verbatim context is atop the stack).