I do something similar but I just pre-define the names of lights I have in Home Assistant (e.g. "lights.living_room_lamp_small" and "lights.kitchen_overhead") and a smart enough LLM handles it.

If you just say "the lamp" it asks to clarify. Though I hope to tie that in to something location based so I can use the current room for context.