On the whole, I think EMS is a bit paranoid when it comes to fear of lawsuits.
That said, I am personally fearful of lawsuits. It does not matter if a case has substance or not, if you become a plaintiff in a lawsuit you are in for a ride in many cases. You'll need a lawyer and time to go to court, both of which could potentially cause a great deal of personal and financial anguish. Being found not liable doesn't mean you get off scott-free. Often times a counter-suit is needed to pick up the costs of a frivolous lawsuit, costing you even more time. I think this aspect of our legal system is horrifying, but in the absence of significant change, being named in a lawsuit means you will be losing some amount of money.
If anyone chooses to make the argument for not-rendering aid off-duty based on the above, I have zero problem with that.
I have no rule for how I act in these situations. If I'm a rural area and no one's on scene, odds are I'll stop. Even if I cannot care for the person, it's easier to tell 911 whats going on when you're still there and not 5 miles away. I would never, ever stop if Fire or EMS was on scene. I have stopped at the same time LE arrived, and the cop gleefully gave me his first in bag and promptly left me and some other untrained bystanders to manage the patient for 20 minutes until the ambulance finally arrived.
Oddly enough the last two times I've on-sighted anything and held c-spine the crew insisted I take the head for the c-spine process, which was a bit unexpected. I'm happy enough to give a report and get out of there.