Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

To get a bit into the theory: a string is a sequence of symbols drawn from some "alphabet", and we can define operations like concatenation, counting, matching and substring on it. In particular, a natural operation on strings is lexicographic comparison, whereby you compare the first characters of two strings, if they are different, the two strings compare that way, if they match, you proceed to look at the second character, etc.

Encoding translates a string from one language to another, and it's often a reversible translation. The obvious case is to translate from Unicode to binary and back. In Unicode, the alphabet is the set of all Unicode codepoints. In binary, you have a stream of octets and the alphabet is the set of integers between 0 and 255. So UTF-8 is a standard that specifies how to perform such a translation.

Since we're often going back and forth between Unicode and binary, and because real processors work with bytes, it's reasonable to call "encoding" translating a string to binary and "decoding" translating binary to a string. The math doesn't require that, though.

Collation represents a string as an expression (usually also a string) that is trivially comparable. For instance, supposed I wanted an English-friendly way to compare band names. I might map all codepoints that look like A or E to "A", then K, S and C to C, etc., discarding any accents and maybe throwing out leading junk like "The" or "A".

That's why a case-insensitive collation won't let you have Foo and FOO, because they would both collate as FOO. It also means, unlike encoding, collation is expected to be one-way.



> Collation represents a string as an expression (usually also a string) that is trivially comparable. For instance, supposed I wanted an English-friendly way to compare band names. I might map all codepoints that look like A or E to "A", then K, S and C to C, etc., discarding any accents and maybe throwing out leading junk like "The" or "A".

So how does collation algorithmic instability break an application?

Edit: What I mean is-- what are a set of reasonable assumptions a dev would make in the application layer that would be good practice on the one hand, yet also be wholly dependent on collation algorithm stability to keep from breaking?


All the assumptions that make a database index work will break.

If you think about any kind of binary tree based structure, it will have invariants like "the left child node will be less than the parent node, and the right child node will be greater than the parent node."

If those are violated just a little bit, at best data will simply disappear from the index. It's possible for updates to delete stuff, or for operations on the index to hang.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: