When I first read about the NTK I was motivated to try an experiment to see if the kernel depended on where in the parameter space one evaluated it. I was surprised to see in my simple example the answer was "no". Here is a write up I did:

https://arxiv.org/pdf/2104.05874.pdf