Well it's being done through the API, which is presumably OAuth'd under your name, but again, this API is generally used by other sites to interact with your photos library, and it's very easy for someone to just approve the OAuth permissions for sharing photos, not realizing they are also implicitly sharing their location.
One scenario is you're at a party at someone's house. If you post a captioned photo from the party that's geotagged, you're identifying where they live.
