HOWTO: Define token length constraints

[drush]

Looking for an approach to be able to define length constraints on tokens. As a canonical example, a math language that only allows variables to be a maximum of 4 characters would work.

GOOD:
var + 6 = var2

BAD:
var + 16 = otherVariable

Error: OtherVariable is too long (variables can be at most 4 characters in length)

[pdubroy]

Hey Darren,

It's not very pretty, but this can be done using a sequence. E.g.:

year = digit digit digit digit
zipCode = digit digit digit digit digit

etc.

I've considered adding first-class support for this, but I haven't yet seen a use case that really warrants it.

(Aside: Thanks for the inaugural post in our Github discussions! 🎉)

[pdubroy]

Just realized that this doesn't quite address the example you gave :-)

To implement a max identifier length, you can make all but the first character optional:

ident = letter alnum? alnum? alnum?

[drush]

This works in the short case, but allowing this to be parametric is important. ie Postgres column name length is max 63 characters.

[pdubroy]

Thanks for the use case.

I think the best way to handle that today is to raise an error in a semantic action. Would that work for your use case? Or is there a reason that it absolutely needs to be in the grammar?

[keturn]

Today's Advent of Code puzzle specifies that values be 1–3 digits. I didn't think of digit digit? digit?; my implementation ended up looking like this:

int1000 = 
      digit int1000_tens -- hundreds
      | digit int1000_ones  -- tens
      | digit -- ones

actions = {
    int1000_hundreds(hundreds, tens) { return parseInt(hundreds.sourceString) * 100 + tens.eval() },
    int1000_tens(tens, ones) { return parseInt(tens.sourceString) * 10 + ones.eval() },
    int1000_ones(ones) { return parseInt(ones.sourceString) }
}

It's terribly verbose and clumsy, and brought me here to find out if there's a better idiom.

I admit I can't think of anything that would make this truly critical to be in the grammar. I think it was mostly that it was my first encounter with ohm, I was writing the grammar before I knew how the semantics would be evaluated, so I tried to define as much of the spec as possible in the grammar. Easier to keep the test cases together that way too.

On the other hand, being a pattern language with + and * operators, I did have some hope that regex's {n,m} quantifier would work as well.