They don’t have your location. That is passed in an encrypted form from device to apps like Find My Phone. Apple’s servers don’t store it, nor have access.
This is false. The device transmits all visible Wi-Fi access points and their signal strengths (which is analogous to your location) to Apple at all times. The API returns location data to the handset.
There is no way to verify that Apple's servers don't store this information.
They absolutely have access to it, as they provide the location information to the phone based on the Wi-Fi data.
Apple says [0]: "These calculations are performed live on the iPhone using a crowd-sourced database of Wi-Fi hotspot and cell tower data that is generated by tens of millions of iPhones sending the geo-tagged locations of nearby Wi-Fi hotspots and cell towers in an anonymous and encrypted form to Apple."
The phone uses a local database of wifi locations to help triangulate on-device. Separately, the locations of wifi APs are uploaded to apple to maintain the database, but the source of the data is not saved, just the location of the WAP. A subset of the database is cached on-device.
They tried to clear this up in a 2011 press release but of course it just comes down to trusting it works the way apple says it does:
