Look for what users have said about them, and their own published uptime/event history. Some big-name services are surprisingly unreliable.
Agree with codegeek about sandboxes. With some vendors it's hard to test your code because their sandbox doesn't look like their live service, or hard to come up with instructions in those cases where you need to ask your customers to interact with a vendor. Some vendors make our tests that integrate against them flaky because the sandbox is flaky :/
Look carefully for things that smell either like too little attention to detail or great attention to detail. For example, for a while a big payment gateway had, basically, a webhook for telling you about payments/payment failures on recurring transactions, but there was no retry mechanism, so if a connection was lost for any reason, so was your data. At the other extreme, it's a good sign if they go above and beyond in some areas, e.g. Twilio and Stripe have great documentation, their approaches to deprecation/compatibility are good, and Stripe offering the idempotency-key mechanism showed they'd spent time solving the whole not-charging-people-twice-because-of-an-inconveniently-timed-network-glitch thing.
This is probably a quirky personal preference, but I'm encouraged when vendors offer APIs for single-item and bulk operations and the bulk ops perform OK with a lot of items. We have some generally alright vendors whose APIs turn out to be painfully slow when we sync data for our largest customers back to our servers. This is a problem that would not necessarily have shown up from testing a high request rate; it's a matter of scaling well with amount of data _stored_.
Some services have "old" and "new" APIs that are not fully integrated, e.g. auth credentials needed and object IDs might differ between them. Sometimes you have to use both to get all of the functionality you want. This hasn't tended to correlate with a good developer experience. On the other hand needing to make your new API distinct is probably inevitable sometimes due to business history or other practicalities--I wouldn't make a decision just based on that.
Comprehensible pricing is a good sign, though of course that's more achievable some times than others. Something like AWS or GCP has accumulated a lot of cost/pricing details to sort through since there are a lot of types of resources they offer, and there aren't really shortcuts to get around that.
Agree with codegeek about sandboxes. With some vendors it's hard to test your code because their sandbox doesn't look like their live service, or hard to come up with instructions in those cases where you need to ask your customers to interact with a vendor. Some vendors make our tests that integrate against them flaky because the sandbox is flaky :/
Look carefully for things that smell either like too little attention to detail or great attention to detail. For example, for a while a big payment gateway had, basically, a webhook for telling you about payments/payment failures on recurring transactions, but there was no retry mechanism, so if a connection was lost for any reason, so was your data. At the other extreme, it's a good sign if they go above and beyond in some areas, e.g. Twilio and Stripe have great documentation, their approaches to deprecation/compatibility are good, and Stripe offering the idempotency-key mechanism showed they'd spent time solving the whole not-charging-people-twice-because-of-an-inconveniently-timed-network-glitch thing.
This is probably a quirky personal preference, but I'm encouraged when vendors offer APIs for single-item and bulk operations and the bulk ops perform OK with a lot of items. We have some generally alright vendors whose APIs turn out to be painfully slow when we sync data for our largest customers back to our servers. This is a problem that would not necessarily have shown up from testing a high request rate; it's a matter of scaling well with amount of data _stored_.
Some services have "old" and "new" APIs that are not fully integrated, e.g. auth credentials needed and object IDs might differ between them. Sometimes you have to use both to get all of the functionality you want. This hasn't tended to correlate with a good developer experience. On the other hand needing to make your new API distinct is probably inevitable sometimes due to business history or other practicalities--I wouldn't make a decision just based on that.
Comprehensible pricing is a good sign, though of course that's more achievable some times than others. Something like AWS or GCP has accumulated a lot of cost/pricing details to sort through since there are a lot of types of resources they offer, and there aren't really shortcuts to get around that.