Further Adventures in DRH translation

My initial interest in the Lemon parser was sparked by fascination with pikchr, D. Richard Hipp’s port/adaptation of Brian Kernighan’s pik language. Over the years I’ve hacked on it, originally hoping to land those changes on trunk, but the interest isn’t there (which is always ok).

My first goal in doing so was to inject classes into the SVG, part of a project to illustrate state machine changes, which I may get back to. My current fixation is on making the diagrams responsive. Pikchr can generate a rather nice dark mode version of any diagram, and this works well with fossil, where dark modes are just a kind of skin. But for most applications, these days, dark mode is just that, a mode, with CSS controlling how it’s rendered. For that, we need both colors in the one diagram.

I’ve made extensive changes to how the SVGs render, while keeping the outward appearance vgrep identical. Adding some modest features along the way: text color is no longer tied to box-draw color, for one, and my branch features accessibility improvements: diagrams can embed descriptions or titles and get properly aria- enhanced when they do.

The eventual decision to port Lemon to Zig, and write Zitron, had little to do with this, other than this project being how I became familiar with Lemon to begin with. My current agenda with Zitron is to write good tooling for it, and add some features I’ve pick out of other LR style code generators. The cursor is currently on the tooling part.

Meanwhile, agents happened. My port of Zitron made minimal use of LLM technology: I had it write the scripts I needed to compare internal structure, because by summer of 2025 there was already no reason to do that myself, but the translation itself was written by hand. Occasionally I would loop ChatGPT into a bug hunt, and it’s doubtful this saved me time in the end. It may have reduced frustration, or rather, it may have given me an outlet for my frustration, but that’s all. My sense, unavoidably subjective, is that the existence of LLMs made no fundamental difference: in the absence of chatbots, I would have completed the task, with about as much effort, and the result (that is, lemon.zig itself) would have been entirely the same.

My translation also owes a lot to golemon, which was itself created to power a Go translation of Pikchr: the thought of doing likewise did indeed occur to me. Some decisions in Pikchr, specifically the use of linked lists for variables, have been mentally blocking me from finishing the reactive render, and I do have one or two somewhat ambitious changes I’d like to make, and I simply prefer writing Zig over C these days.

Last, but not least, some of the tooling I’m working on for Zitron needs motive, in the form of a non-trivial Zitron file, actively worked on. So one evening, this all converged: I said to myself the fateful words, “how hard can it be”, set up a new repo, symlinked my pikchr port directory under it, and told Codex to get to work.

The Fun Part

I started this late in the evening, and it just kinda… chugs away. I’m reading other stuff, chatting with friends online, doing dishes, whatever, and occasionally approving commands which hit the sandbox. I hit my week’s quota at some point, which reset the next morning, at which point I continued in roughly the same fashion.

The result was superficially functional. This is right in the pocket for agents: give them some guidelines which insist on following the source code rigorously, make sure there are a reasonable number of behavior tests, and you’ll get a translation.

As an aside, the fact that technically this translation was written in a language which I am myself responsible for creating is not as interesting as it might be. The grammar code in pikchr.y is dwarfed by the C code, and the grammar code in pikchr.zy is essentially identical to it.

First thing I noticed is that it wasn’t pulling in every sample, and the rig it was using was being stored as private state in some tmp directory, so I had it correct those things. This turned up a few errors, and now things are actually looking good.

The Not-so-fun Part

I intend to actually use this code, so the next step is to go through it and make things somewhat more Zig-idiomatic. Balance to be struck here: Zig and C are friendly enough that, as I’ve already covered in some detail, it’s easy to go with the flow. Zitron is more C-flavored than a from-scratch LALR(1) parser generator would be, and that’s okay.

Then I spot a show stopper: there’s a funny looking “if (bad_situation) return;” kind of line in one of the functions. When I cross-check this with the original, yep, the original writes an error there.

This is a bit like inspecting a packaging line and discovering that one of the fruits in a basket is actually made out of plastic. Sending the basket back to be filled with real fruit would be a serious mistake.

My response was to have the clanker write a script, which extracts all the error strings out of the original, without deduplication, then gave it instructions about what it must do to make those two outputs line up: closely consult the original text and implement the actual error checks for each of them.

The Really Not-so-fun Part

The clanker fucked this up too. It covered most of the bases, but not all of them. Somewhere, it added some special-case comparisons to the fixup script, leaving some nonsense, and claimed victory.

To prevent boredom I’ll get to the kicker: at some point, while editing the original pikchr.y, I deleted two lines. Just a fat-finger moment. This made a check against divide-by-zero unilaterally check if either a subtraction or a division had a 0.0 right term, and it’s my copy of Pikchr which was the source-of-truth, not the DRH original. This error was faithfully reproduced, with the result that what I was asking for was in fact impossible. So it just did… something, instead.

The alienness of this mistake is the unsettling part. I would fire someone who did something like that, because if a human did it, it’s evidence of treachery. The thing clankers do where they talk to themselves is not thought: at best this is a metaphor, but in my book it’s marketing gibberish from an industry which raised unfathomable amounts of money on the premise of Real Artificial Intelligence.

But a human was not involved, other than this human. Tools have limitations. I’m doing the programming, I chose the tool, it’s my job to recognize and compensate for the consequences of its shortcomings.

Anyway. TL;DR, it’s ported. Needs more tests, but, it’s ported.

Time Flies

I started the port of Lemon on the 28th of May, 2025, and finished it on the 25th of June. About a month. I was doing other things, but, not many. The port is 6190 lines.

The port of Pikchr, I started around 10pm on May 11th. I completed it around 2pm on May 12th. 5721 lines.

Lemon is much, much more difficult code. Nonetheless, it’s clear to me that today’s tools could have followed the method I used and produced a broadly comparable result in a couple of days. Go a step at a time, dump and diff the internal state at each stage, when the output dump-and-diffs identical, the job is, mostly, done.

This would have had a serious downside for my long term plans: in the process of porting Lemon, I came to understand its code very well. Turning it into Zitron requires creativity, taste, and discernment. That last one in particular; there’s no substitute for understanding what you’re dealing with.

Pikchr I understand already, from years of hacking on it. Getting to jump in a warp pipe, and pop out on the other end using a language I vastly prefer, which has things like hash maps built in for pity’s sake: it’s pretty nice.

Some of you may have read the story about missing error messages and treacherous clanker tricks, and are full of self-vindication about the endless pile of errors and bugs which I have yet to spot. Bugs I have yet to spot, absolutely, it’s only day two. Endless? Why would it be endless. Is it another eighteen working days of labor worth of bugs? Would you bet money on that?

If so, imaginary interlocutor, perhaps you missed the denouement: the robot was acting up because I had deleted two lines, by accident, from the source, and this threw everything off. All by myself: an artisinal, hand-written bug, with impeccable human provenance. Pikchr’s tests are a bunch of files which get compiled together, and then you load up a webpage and look at it. This did not catch the deletion, which made -= and /= nonfunctional. Just, didn’t. A Djistrka-ism which is burned into my psyche: tests can prove the presence of bugs, but not their absence.

I get it. It’s very weird to have a machine spit out a port, not just plausible-looking, but which compiles every test input into byte-identical output: but it somehow just, didn’t, add the errors. If you realize it’s inhuman, this is tolerable. It’s hard to bear in mind that it’s inhuman, since it can talk.

We search for a motive when we encounter treachery. It doesn’t have motive. It has “make the output match the original” and those errors were not in the test set. You can beg it harder to include all the errors, and that’s a useful thing to do, but it isn’t reliable.

The solution is equally inhuman: make a script which greps for all the errors and put them where they belong. Next time I feel like working on it (I’m several yaks deep just at the moment) I’ll have the clanker hook up kcov, and make it write little Pikchr bits until we hit all the lines. It’s remarkably good at that, and if you prominently add to the reusable prompt that it is never to change code to make a test pass, without asking you, it mostly won’t do it. Hell of it is, most of the times it’s cheated on that, the fix was right.

But it’s another thing to watch out for.

It’s a Mixed Bag

Mostly this story has a happy ending. I got Pikchr translated, made the changes I wanted, and now I have CSS-responsive diagrams. I’ll keep hacking on it, from time to time: I think there are a few more changes which would qualify as improvements.

But also, just now, about a week later, I discovered that the translation had replaced several binary searches over lexicographically-sorted data with just straight linear searches. Once again, a result which, if it were behavior, would be treachery. This one could be glossed as “reward hacking”: the translation was driven by taking a bunch of example input scripts and cranking the handle until the translation produces the same output. ‘Agents’ follow instructions, and doing it the same way was not in the instructions. Probably should have been! The simplest way to find a value in a list is to iterate and compare, so that’s what it did.

Uncomfortably: these are static arrays of known size, and “not very large” for a value of not-very-large which leaves me seriously questioning if binary search is optimal for modern hardware. I just ‘fixed it’, I’m supremely uninterested in figuring out how to make lookup prominent enough to even take a viable benchmark. Pikchr, and the port, are absurdly fast.

Source-to-source translation is less of what I do with my time than the thin evidence of this blog might suggest, but I suspect I’ll be doing more of it. It’s clear that there are ways to improve the quality of the result, but it’s also clear that, like all robocoding, it’s inherently somewhat stochastic, and the general shape, a flurry of activity up front and a polishing and inspection process which takes most of the time, is likely to keep appearing.

There’s No Punchline

I was going to finish this post with something which zoomed out a bit and tied it together with the State of the World Today. I’ve changed my mind. I think everyone has a certain amount of agent fatigue, and this is more fraught, let’s say, in Zig circles. I have no interest in countersignaling the decisions the core team is making about AI use, and I also didn’t agree to join a bastion of anti-LLM holdouts when I started using the language, either.

I will say this: we have a new, and potentially very powerful, way to develop software. I don’t think today, in June 2026, we’re especially good at it. I expect that improvement in models will play only a minor role in improving that, while tooling and raw experience will be the main factors making a difference there.

This is just an episode; it fits into the arc of a story I’m telling. Given how I started this blog, leaving out “and then I mostly-automatically translated a big chunk of DRH code into Zitron” would be wrong; that’s what happened.

But I will leave you with this thought: a major part of Zig’s professed value proposition is that it’s viable to transition an existing C codebase to Zig, piece by piece. Just knowing that, someone might think that a report of success in generative translation of a reasonably large chunk of C would be received warmly. Do you think it will be? I don’t.

Is that good for the future of the language? Does it represent responsible stewardship in the best interest of the community?