The point of forking Pikchr was to add responsive color in a single diagram. This wasn’t the initial impetus, but it rapidly became the point. Porting it to Zig was mostly to have something to develop the Zitron language server against, and also just to see what would happen; with that accomplished, it sure was nice not to have to roll up a hashmap.
OG Pikchr does have a dark mode, or rather, it has dark diagrams. The code will generate one or the other, not both. There isn’t a way to specify dark colors separately either, the mapping to darkness is automatic.
It’s pretty good for what it is, which is an empirical mapping within RGB space. Without any special knowledge of how this mapping came about, I guess that DRH wrote something plausible, then tuned it until it suited his use case.
I wanted something which preserves the quality of more colors, which is basically infeasible in RGB space. It didn’t take long to find Björn Ottoson’s Oklab color space, which has become industry standard, making the decision pretty easy. If you happen to need to do color mapping in Zig, you’ll find the implementation in memex, which is a paraphyletic grab bag of miscellaneous code which I suspect might come in handy in more than one place.
The new Pikchresque color map uses OkLch, which is simply the cylindrical variant of OkLab. This makes the project feasible, but it’s not automatic either, there was a process of tuning the mapping dimensions until I was reasonably satisfied with the result. This includes specific handling of background, foreground, and text color, as well as a distinct curve in the near-grayscale range.
I think it looks pretty OK, and pretty OK is as good as I can get without radical changes to how Pikchresque operates. This is because the mapping works on color in isolation, while really solid perceptual color mapping has to take account of which colors are layered onto what.
The possibility of allowing the user to specify two colors remains, but this also entails significant changes to the language, and at some point this is no longer a mutually intelligible superset of Pikchr, it would need its own name at least. This might happen, but if I’m going to do it, I want to do a good job.
The original color mapping is still available as one of the flag options for invoking Pikchresque, on the command line or otherwise.
A Pikchr is worth a thousand words, surely, and all that introduction is thus worth a bit less than half a pikchr. Let’s take a look!
It will not surprise you to learn that this blog has a responsive color scheme. Pikchresque can generate diagrams in just one color, which is about to come in handy, along with a bit of HTML to give them appropriate backgrounds.
Here’s a nice saturated palette:
Yes, the yellow text on white background is illegible, that’s not the point.
Classic Pikchr does ok here:
Each on its own, not a bad diagram, except the text: the swap works for cyan and magenta, not for blue and violet. You can also see that foreground and background colors receive separate treatement, also true of the OK version.
Here’s the new school:
Clearly this is closer to “same diagram, just in dark mode”. As a matter of interest, yellow hues get subtly re-curved, with a very subtle bump toward redness, more luminance, and the effect more pronounced as chroma saturates. A mapping which gives equal treatment lands the yellow closer to the OG pikchr; to my eye, too green.
You may think the orange could use a bit of that as well; I see it too, but this is an artifact of it sitting right between a fat block of red and yellow. Eyes are weird!
The difference here is much more pronounced.
As I figure it, an author using pastels is doing so on purpose.
With the classic map, this happens:
This is basically unavoidable with a mapping within RGB colorspace. RGB is how pixels work, it’s nowhere near to how our eyes work.
Here’s the OK version:
Pastels? Pastels.
Note that I’m deliberately trying to split the difference between what a color means, and what looks nicer in each mode. This comparison is inherently artificial, because one doesn’t look at both side by side like this. The goal is a diagram which works in both modes, without surprises. Someone else might want to match perceptual tone as closely as possible, there is nothing objective about any of this.
Same thing, other direction.
Old school:
New school:
It’s important to note that color is holistic, and these kinds of artificial tests themselves cause a visual impact which is somewhat misleading as to the qualities of the result. For that matter, some of you are seeing this on a light background, some on a dark one, and that too impacts what you see. Indeed, this is the very point!
You may enjoy switching from one mode to the other, and re-examining the sample palettes in whatever mode was not your first read. To my eye, the deep color light mode looks harsh in dark mode, and the deep color dark mode looks wan and washed out in light mode, while both look just fine in the mode they’re designed for. QED, and YMMV as always.
In an actual diagram, colors (we may hope) carry some kind of meaning. One might select a pastel, or a darker shade, or garish primary saturation, to convey something to the audience. The goal here is to keep the hue, and the heaviness or lightness of the color, while adjusting its actual luminance to closer fit its intended background.
I think I’ve done an OK job.
The keen-eyed have likely spotted something hinky going on with black and white here.
It’s true: I’ve juked them. Original:
Juked:
Judgement call: gray means something which is not simply “degree of luminous difference from the background.”. Dark should be dark, light should be light. Note that the contrast for the palest three is bad, due to a decision which is not far from “black text is white, and white text is white also”, with the same for lines, but clearly, not for fills.
That, right there, is the better job I could do if Pikchr diagrams knew what colors were stacked how. I’ll settle for this, in large part because using white as a background color is probably a mistake, and the background should be transparent for that component instead. For technical diagrams, two shades of gray is probably the correct maximum, just pick them between silver and charcoal and all will be well. I like 5s and 9s, which are not on the diagram.
The original clamps dark grays to the same shade of middle gray, for backgrounds, but not for strokes and text. The new one does something similar for foreground colors, but not backgrounds. I think this works better for more diagrams, but time will tell.
This exposes one quirk of Pikchr, which I have not changed: if a box has an identical color and fill, the algorithm decides that they’re both in effect the background: that keeps a brim from showing up in one mode and not another, but it can be the case that a diagram looks better if said brim does appear. Even a single bit of difference in any color channel cancels this effect, so that Weird Trick is available should you need it.
The main thing I left out of the initial release of zitron-ls was a formatter. This wasn’t an accident, grammar files have a particular sort of conventional layout and it took time to come up with a rule-based system which would look right.
This even affects the Zig code: for instance, it’s not uncommon to put two or three short actions on a single line following a rule, here’s an example from pikchr:
boolproperty ::= LARROW. { p.cur.larrow = 1; p.cur.rarrow = 0; }
Formatting the Zig AST wants to put those two on their own lines: good for Zig, not as good for Zitron.
Once I had worked out a vocabulary and decision criteria for the formatting, Astra had it working in an hour: ten minutes of asking me questions, 25 minutes of planning, 33 minutes of implementation.
Followed by several rounds of tweaking, of course. I called it a day when I liked how pikchr.zy is laid out after formatting, more than I liked it before. YMMV, of course, in which case:
// zitron fmt: off
Knock yourself out!
The format functionality is implemented as a standalone-ish module, which uses the Zitron tree-sitter. It seems less than traditional for formatting to be tied directly to a language server, so I wanted to keep my options open.
In classic yacc, one can use characters directly as tokens: '+' is a valid token name. Bison extends this with strings, so you can do ":=" if you’d like.
These are accomplished with hacks, basically. The character one ain’t so bad, at least back in the day: '+' is just a way of saying 0x2b, depending on locale naturally, and since it’s just a number, a lexer can return it as such.
This kind of sucks anyway, because the parser table wants to be dense: 0 for the accept action, then terminals, then nonterminals, without skipping anything. So a tax is paid: yacc translates these oh-so-convenient character tokens down into the numbers which its parser needs to see.
The Bison strings are even worse, because there’s no obvious way to turn ":=" into a number, and it doesn’t provide a non-obvious way. Instead, you get to tell it what you mean: %token ASSIGN ":=", so that lexer can read an enum named ASSIGN and return it when appropriate. Window-dressing, basically.
Lemon has never supported this. Terminals are given numbers in the order in which they appear, and are referred to using the CONSTANT_CASE which is good C style for an enum. As such there’s no translation, no epicycles where every glyph still needs a name and now you have two ways to refer to it.
The %token directive in Lemon gives the author control over the order of token assignment, which has some value: related tokens grouped into a sequence can be tested for as a range, for example.
It was the natural choice for Zitron to follow this example, and make a Zig enum out of the tokens. This does deviate from the official style: the token ASSIGN ends up as the enum literal .ASSIGN, when the style guide would prefer .assign. This is ok in my book, and comes with certain advantages: a token WHILE can be the enum .WHILE, but in “good style” would have to be .@"while". The grammar’s grammar is case sensitive, I entirely refuse to make nonterminals capitalized, and the tortuous mechanism which I would need, in order to end up with a lowercased enum, is a perversity which I refuse to countenance.
That’s the thing though: it’s Zig, .@":=" is a cromulent enum literal, so why not support string tokens as well?
Well, now we do. My suggestion is to use them only for strings which zig fmt would not un-at, so to speak. Zitron will accept and use "smol_token" as a token, and refer to it as .@"smol_token" - However, this would be formatted into just smol_token by the formatter, and I have put no effort into preserving this quirk in the language server.
This little nicety was always intended to obviate the need to invent names for glyphs. A production like
expr ::= LP expr RP.
Is better written
expr ::= "(" expr ")".
As now it can be.
Single-quoted terminals are not supported. There is no equivalent enum literal form, so this would be a mere complication, without any gain in expressive power.
The Zitron parser has a more tolerant concept of a string than the Zig parser happens to. Of note, newlines and tabs may be directly embedded in a string, this will be passed directly to the enum, and the emitted Zig will not be valid. The new formatter would replace the direct tab with four spaces, as it happens; this may or may not improve the situation.
The original parser would have interpreted the sequence "\"" as a string containing \, followed by a (presumably unterminated) second string. This is no longer the case. So long as you stick to the subset of this string syntax which Zig will understand, the expected will result.
As often happens with a project of this nature, this blog has been ‘almost’ ready to publish for a couple of months. Just sitting here on my laptop, legible to no one but myself.
So I’m brushing off the last of the sawdust, filling out the parts I’ve been putting off, and kicking it out the door.