How do we measure and compare prediction quality in a symbolic/deterministic context that also punish for complexity? I struggled on this in my phd thesis and had to brew my own Minimum Description Length implementation because I found nothing applicable. The statisticians I interrogated were like "don't try to do model selection on symbolic models, make them probabilistic and work from there"
Are you familiar with NSGA-2? That was the one I used in my PhD work.
A lot of research in GP/SR went toward methods for balancing density and selection from the pareto front.
One of the bigger problems is growth of meaningless equation material to counteract other pressures in the algorithm, which itself leads to premature convergence on suboptimal solutions. I had some thoughts at the time about ways to penalize this during the growth phase in my own algo, but never got around to them. Mainly needs bookkeeping for contribution to the final equation output.
Very good. I have a listening experience close to elevenlabs. But sometimes you can get "failed" segments so you need to detect them and retry/repair automatically. One example is the fading volume.
Yep, they are quite bad without stockfish. You can test it with the playchess skill in my repo. Maybe 1200/1300, who knows? Still I do think that they can, with enough time, explore multiple variations where they confront their naivety to stockfish and build up a compact picture on why move Y should have been played instead of move X. That was my intuition when building this skill.
Have you actually tested this with quiet positions, where understanding needs to be more about different strategic or positional choices? They can explain concrete tactical lines with stockfish analysis at hand, but so can an analysis board. In my experience, even with fairly deep and accurate engine analysis, they cannot do this accurately with quiet positions. I tried to build a product around this idea and I'm convinced that even if it is possible it would require hundreds of thousands - or millions - spent on post-training to even confirm that possibility.
Good point. I had to tweak a bit the system to get more positional analysis because this is something I wanted too get. I think it's still shying away on this aspect. When it does dwell on it, I feel that when the output describe what should be the plan of both camps, it's quite convincing but only a much stronger player (or me with stockfish) could really assess this.
Anyway I am with you that deep positional appraisal is very hard and we should not expect too much from this set of skills on that front.
Thx for tips. I just saw there are some courses on chess.com, that will be nice to compare claude understanding with Silman commentary as the gold standard.
Edit: I am pushing a small attempt on this position. I fed this one as a screenshot https://www.chess.com/lessons/roots-of-positional-understand... . I forgot to tell that it was black to move but that was fine. The analysis quality seems ok. What puzzles me is that it recognized the Carlsbad structure and talked a bit the plans around the minority attack, not only the correct explanations on g6 and why it should be played. This is where we benefit from positional stuff in the training data.
No need to apologize. I like reading HN for comments that don't beat around the bush.
It's true that it's costly. I never tried to optimize it. In a way I feel this hacky project doesn't deserve its place on the front page. It's just me hacking around on Claude Code to get something I like. Take it as a proof of concept if you will. I'd be happy to see lighter alternatives.
> Then presenting it as a project or tool of value to share to others is delusional
Here I see a gap in your reasoning. Lazy and costly, sure. But useless, I'm not so sure. In my world, a vibecoded tool can be useful enough to be shared, despite the risks. Claude Code itself is almost entirely written by Claude, according to its creator. Yet I use it every day.
No it does not help the quality of the analysis from what I saw, but it makes the analysis much more interesting, because it can challenge my wrong judgement and answer the questions that I asked myself out loud.
Very interesting project, is Maia used in this platform?
This is crazy. My claude chess stuff is (edit: was) currently near yours on the front page and noticed your post. I have a project very close than yours that I hesitated to share. From a quick glance, we went for a similar approach. I just open sourced it so that you can compare implementation notes. https://github.com/brumar/whiteboard-agents . It's not thoroughly tested but can be interesting to check.
TIL
reply