This doesn't properly explain what it says it explains. To explain it correctly, you have to explain why the dot product of two vectors computed as the sum of the products of the coefficients of an orthonormal basis is a scalar equal to the product of the Euclidean magnitudes of the vectors and the cosine of the angle between them. The Wikipedia article on dot product explains this reasonably well, so just read that.
So I first learned about cosine similarity in the context of traditional information retrieval, and the simplified models used in that field before the development of LLMs, TensorFlow, and large-scale machine learning might prove instructive.
Imagine you have a simple bag-of-words model of a document, where you just count the number of occurrences of each word in the document. Numerically, this is represented as a vector where each dimension is one token (so, you might have one number for the word "number", another for "cosine", another for "the", and so on), and the magnitude of that component is the count of the number of times it occurs. Intuitively, cosine similarity is a measure of how frequently the same word appears in both documents. Words that appear in both documents get multiplied together, but words that are only in one get multiplied by zero and drop out of the cosine sum. So because "cosine", "number", and "vector" appear frequently in my post, it will appear similar to other documents about math. Because "words" and "documents" appear frequently, it will appear similar to other documents about metalanguage or information retrieval.
And intuitively, the reason the magnitude doesn't matter is that those counts will be much higher in longer documents, but the length of the document doesn't say much about what the document is about. The reason you take the cosine (which has a denominator of magnitude-squared) is a form of length normalization, so that you can get sensible results without biasing toward shorter or longer documents.
Most machine-learned embeddings are similar. The components of the vector are features that your ML model has determined are important. If the product of the same dimension of two items is large, it indicates that they are similar in that dimension. If it's zero, it indicates that that feature is not particularly representative of the item. Embeddings are often normalized, and for normalized vectors the fact that magnitude drops out doesn't really matter. But it doesn't hurt either: the magnitude will be one, so magnitude^2 is also 1 and you just take the pair-wise product of the vectors.
Oh well, it was good while it lasted. But now we need a brand new hype to replace the malfunctioning AI neon sign. Maybe "neural meta computing", or "quantum hyper reasoners"?
You see, a calm, factual, truthful and informed conversation of different technologies actual maturity, merrits and risks is in nobody's interest. /s
Good food for thought and hopefully an inspiration for people to build on.
The wiki movement has stagnated for a long time. Some reasons are internal e.g., overpromising on semantic technologies and never really delivering a step change in functionality that will delight people the way the original wikis did.
But most of the reasons for the wiki no-show are external, driven by the dominant economics of the modern digital experience which despite the optimistic early book with the same title, is not Wikinomics.
Yet the cognitive stress from information overload is not getting less. If we ever manage to get away from the user-as-product paradigm the design and function of wikis will be a key ingredient for the next digital revolution.
Funny you bring up logistics and (data) ontologies. I'm a PM at a logistics software company and I'd say the lack of proper ontologies and standardized data exchange formats is the biggest effort driver for integrating 3rd party carrier/delivery services such as DHL, Fedex etc.
It starts with the lack of a common terminology. For tool A a "booking" might be a reservation e.g. of a dock at a warehouse. For tool B the same word means a movement of goods between two accounts.
In terms of data integration things have gotten A LOT worse since EDIFACT is de facto deprecated. Every carrier in the parcel business is cooking their own API, but with insufficient means. I've come across things like Polish endpoint names/error messages or country organisations of big Parcel couriers using different APIs.
IMHO the EU has to step in here because integration costs skyrocket. They forced cellphone manufacturers to use USB-Cs for charging, why can't they force carriers to use a common API?
If your debt grows faster than your income you are blackpilled. If you run a derivatives market in your head and nothing you imagine can be boostrapped, you are blackpilled.
Crypto gets a lot of shit but social contracts made analyzable as smart contracts is promising for ethical financial markets. e.g. you could detect a debt spiral as a flow/loop/void structure with renormalization groups and persistent homology. When you can spot entities trying to create wage slaves from 100 miles away you can focus your energy on being creative.
Heidegger is against it, yes, but he's also clear that we're stuck within technology as a historical destining of Being. We can't just escape technology by thinking differently about it, but must wait heedfully for the historical development of Being (as he lays out in The Turning, just after QCT). In fact, the fulfilment of technology's own reductionism is a necessary step, as only the full concealment of Being in technology allows us to recognize that oblivion for what it is and thus to overcome it. As such, I think it's a reasonable argument to make that AI taking away the work of ordering for humans may help us escape the incomplete agency of ordering the standing-reserve so that we're no longer forced to merely see the world as standing-reserve.
By the way, I saw your comment about Heidegger and Asian religious philosophy below - are you familiar with the Kyoto school? I've signed up for the Halkyon Guild's upcoming course on Nishitani Keiji, and am vaguely hoping his theory of the self-overcoming of nihilism can be brought to bear on the questions Heidegger raises about technology.
