It depends to a huge degree on where you are, laws vary WILDLY by state, and even county.
I'm not going to talk about the business end of things, because I'm really not qualified to.
On the medical legal requirements:
As someone earlier said, you're going to need both medical malpractice insurance and workman's comp insurance (Yes, even if it's just you, you still need to be covered when you're working.)
You will need a medical director, the odds are good that you can't glom onto the county's/state's director, but that's gonna be region specific. You'll have to pay to develop your protocols, then to administer them. If you have access to a reasonable set of protocols that you can copy (or if your state publishes protocols), that might cut down the expenses significantly, but you'll still have to go over them carefully and modify them. Ranges for medical director salaries I've seen have been from 500USD/month to 2,500USD/month...that being said, those were all for ambulance services, not standby services, I don't know how that might affect things.
You'll need a business license, which depending on area might be a huge pain in the *** or might be quite easy.
You'll probably need a clinic license to dispense drugs, and a pharmacy license to buy them (Obviously you won't need the pharmacy license to buy OTCs, but even oxygen may require one.), you may also have to get a consulting pharmacist, they tend to go for 50-100USD an hour.
If you're able to work at the ALS level, you might be required to carry all the drugs in scope, in which case you'll need a mid level provider DEA license.
and, of course, as someone else mentioned, a supplier. Good news here is that boundtree will give pretty decent price breaks to even miniscule services, so once you're all registered, that part should be easy.
That's what I can think of off the top of my head, and as noted, doesn't touch on the business side of things. Ultimately though, you should go talk to your state EMS office, they can tell you exactly what you need to do in order to be able to operate.
All told, your better bet may be to associate with a current service, and negotiate with them to let you work under their umbrella in exchange for a cut of the profits (possibly still needing to carry your own insurances.)