Sometimes I can be a perfectionist myself, so I understand. I dont think you should stop with this at all, you definately have my support if you want to keep perfecting this song . It keeps getting better everytime I hear it!
By the way, I think the "average listener" sometimes doesnt know anything about music. To be honest, I dont think many listeners care about quality whatsoever. I've heard people say some songs are "the best they ever heard" and those song cant even come close to this one quality-wise.
My advice; if people love what you do they will support you. The people who dont support you, screw them. All in all, just keep doing what you love to do, after all, art is a form of expression, it cant adapt to what the "average listener" wants.
Anyway, Im loving the song, once again. It was amazing the first time Ive heard this, and it keeps getting better.
- Damian