My experience is somewhat different from yours, although that is not to mitigate your story at all!
I had some benefits that you did not: I was a full time youth minister for 18 years before I transitioned into my 12-year stint in preaching ministry--expectations can be so different. So, I had perhaps more realistic expectations and "best practices" in place when I entered into preaching. For one, I did not take calls during meals! Cell phones have never been allowed at our supper table--everything is to be in silent mode. Of course, this was just as cell phones were being developed, but even then, the answering machine took the calls. Meal time was sacred. No apologies.
Parishoners did not call me to complain (that was a mixed blessing/curse). They called my Elders! And of course, the leadership didn't have the courage to say, "Well, what did he say when you went to him?" I did have the occasional few that would come to me personally during office hours. Those actually worked out quite well. But most folk prefer to triangle leadership rather than go to the source (one of the best books I had ever read was Generation to Generation by Edwin Freidman--every beginning pastor should be required to read the book).
I don't know if this was your experience, but among our tribe preachers rarely received benefits (you had to be in a mega-church to get benefits). So, what some people thought was a good wage was off-set by the fact many of us had to find our own health-care and retirement. When I left local ministry 12 years ago and took a position in a non-profit at roughly the same salary, it was like getting a $12,000 raise because they covered my health care! So, a minister in my tradition who was making $60,000 was actually only making $45,000-$48,000 (or even less).
There were several good moments as well as several bad moments--a lot depends on the church you serve. When I hired on to my last youth ministry I mentioned to my elders that I took Fridays off as my family day. On the rare occasion when an elder or member called me on a Friday it was with immediate apology: "I am sorry! I know this is family day. This will only take a few minutes, I promise!" I really appreciated the attitude--and they were usually true to their word. (Needless to say, when I left six years later to enter into preaching ministry, it was a mutually sad experience--the church was much more traditional than I, but our experience together was mostly wonderful and we still have mutual love).
When challenged to "preach the party line" that I honestly disagreed with, I respecfully declined. That is something I could not do. Fortunately, no one demanded me to go against my conscience in that regard. (Again, this is probably because I was likely older than you were--with ministry experience--when I started preaching ministry and felt comfortable in drawing the line).
One true similarity is the loneliness. As a youth minister, I always had a ready-made peer group: youth workers/volunteers and parents. But when I went into the Senior minister position in another church and state, I wasn't prepared for the isolation. It was as if I was some "other"--not fully human. That confounded me. (To be fair, there were some who befriended me--but it was very different from my experiences in youth ministry).
And yes, the expectations to work beyond 45-60 hours was high (of course, I expected late night hospital calls, visits, and sitting up late with someone who was dying--like you, I considered those holy moments and an honor--I did sign up for that...).
Eighteen to twenty hours is pretty standard for sermon prep. For me, I would take my laptop--or before laptops, my yellow legal-pad--to a coffee shop and work for about two hours a day during office hours. Not everyone could get away with that--but that was one nice thing I didn't get much push-back on. Also, I really tried to front load my week in sermon prep. My goal was to have sermons completely finished by Thursday (I have always taken Fridays off--I actually preferred working on Monday, though many ministers of my generation took Mondays off after the adrenaline let-down following Sundays).
For me (and I can only speak for me), I think it was worth it. Much for the same reasons you mentioned in your final paragraph. There are so many people, so many teens (as a youth minister) who are now incredible adults and friends (and yes, moms and dads had more to do with that than me), there are amazing friends I still have--people I call family whom I would have never known otherwise, there are those whose lives were made better because God somehow worked through this very flawed human being.
Your last paragraph is perfect:
"The value of a pastor’s ministry is in the people that he helped and the lives that he touched, and — even though it can be hard to measure — that is a legacy that is remembered long after the words of his sermons have faded into the air."