Same for me. In addition to that, having breakfast actually caused me to get hungry again pretty quickly and generally start feeling terrible if I then didn't eat again, whereas without it, I could pretty much go all day, no problem.
This has been my experience over many years of restricting my consumption window. I also find I get mid-morning and after-lunch energy wells if I have an early breakfast, not so much if my first meal is after 10am.
Breakfast actually made me feel physically ill for a while when I was in my early 20s, so I skipped it out of necessity. I think it had something to do with insomnia. Keeping down half an apple was a struggle. So I basically starting skipping entirely.
These days a single coffee with some milk is enough until lunch, whenever that is. I can eat breakfast (or at least brunch) decently well now, but like you say I just get hungry again anyway. And if I'm going to the gym any time before 1pm I try to eat nothing beforehand or I just feel worse.
Not an expert, but from reading studies I suspect that the specifics like which meal to skip can vary by person. Which makes sense since we all have unique physiology. Eg, some people are natural night owls vs morning people. There’s plenty about the body we don’t fully understand, but executing discipline to follow a generally-healthy eating regiment likely has both direct physical benefits and benefits from a sense of control and well-being.