4158ef0237
Initializing character tokens, and the NSString objects they use, results in a relatively high overhead. The tokenizer now accumulated all successive characters in on accumulator token until a non-character token is emitted. This change reduces the execution time of the performance test on the local machine by 48% (reference to baseline)