Great story and thanks for sharing.
... Now this is where I failed. ...
I disagree, and second-guessing on this aspect is unhealthy. And I don't think
anyone on this forum would think $40 for that was overpriced. (Yes, I realize a "negotiation" and "value" aren't actually directly related.)
But I'll go along, if only to mention that if you'd have gone much lower he might have thought you were trying to rip him off, and anything much higher might have raised suspicions as to your intent.