188M input / 80M output tokens per hour was per node I thought?
Reversing out these numbers tells us that they're paying about $2/H100/Hour (or $16/hour for a 8xH100 node).
Disclaimer (one of my sites) https://www.serversearcher.com/servers/gpu - says that a one month commit on a 8XH100 node goes for $12.91/hour. The "I'm buying the servers and putting them in COLO rate" usually works out at around $10/Hour, so there's scope here to reduce the cost by ~30% just by doing better/more committed purchasing.
Reversing out these numbers tells us that they're paying about $2/H100/Hour (or $16/hour for a 8xH100 node).
Disclaimer (one of my sites) https://www.serversearcher.com/servers/gpu - says that a one month commit on a 8XH100 node goes for $12.91/hour. The "I'm buying the servers and putting them in COLO rate" usually works out at around $10/Hour, so there's scope here to reduce the cost by ~30% just by doing better/more committed purchasing.