You can build a site of your own without using a single framework.

But, the thing is even if you don't use one, you end up writing one for yourself, maybe even a tiny one to better manage your code.

I don't know that if you can call this little piece of code "a framework", but maybe something like a "micro framework" (something like preact, hyperHTML, lit-html).