Formulating hypothesis first, even with little knowledge, is fine as long as the point is to invalidate them (basically you assume they are wrong and you’re fine spending time to prove it).
It can help focus the data gathering phase, and you get two results: you can ignore your first hypothesis from now on, and you have further data to look at to formulate a decent hypothesis to expand on.
I agree in practice, but I think my point is to resist the dichotomy you are implying - that there is a "first hypothesis with little data"... vs ... the other ones..?
Isn't all our knowledge little knowledge - even if it seems like a lot? Wouldn't your advice apply equally as well to your most knowledgable topic?
The hard part isn't having knowledge, it's finding the hypothesis to show how your knowledge is little knowledge.
(Aside: I would love to hear the experiences you are forming your comment around! I'm sure there's a lot of interesting stuff there.)
I think we fundamentally agree, and probably have the same approach, with just a different “marketing”.
For the background of these, on a professional level I try to sell the experiments with some angle, whatever it is, even if it is intuitively wrong.
An exemple of that was a discussion around wording for a relatively delicate part of our site. We didn’t know how to approach it, and it hasn’t been touch for a very long time. So, the random hypothesis taken was that nobody reads the text anyway. We set hooks and tracking in and out of the sections to have a decent view of what’s happening, and we got enough data to come with new insights and propositions; Next phase was AB testing two changes, with expectations to have positive effect, and the tracking we set in place was to validate it went as planned. This also didn’t go fully as expected, but not that far from the mark, and we iterated.
In hindsight we could surely have just done the data collection and make hypothesis from it, but I feel the data would have been more vague and less exploitable. I don‘t trust “let’s just collect data” approaches much.
It can help focus the data gathering phase, and you get two results: you can ignore your first hypothesis from now on, and you have further data to look at to formulate a decent hypothesis to expand on.