That's pretty dishonest. I would never charge like that. I would rather be completely upfront with a client and charge them $500 per hour for 1 hour, if I know that I can get it done that fast, and it's really providing that much value.
It seems like the gap between your two ways of thinking mostly stems from how silly "hours worked" is as a measure of work done. You're of course right to be skeptical of being dishonest about the number of hours worked, but he has a point as well: it makes no sense to get paid less for a job simply because you're more productive (and make no mistake, most clients would lower the total price of the same task based on your ability to complete it in fewer hours).
The way people typically do it is charge by value of project -- "this component is worth X dollars". Businesses want to solve problems costing D-dollars in T-time for themselves.
They couldn't care less if you spent 1 vs 10 hours on it if they can't do it themselves, so why bother oversharing your time expenditure?
