Sounds like the orientation/new hire process right? The big thing to remember is that every place is going to go about this slightly differently.
New employees at my organization go through a ~4-6 month long "process" progressing towards state licensure as a paramedic. During that time, they are a third person on the truck with an FTO at all times. Before they ever get to the street, they take a state protocol test, review certain clinical skills with our education coordinator, have an airway class, and undergo equipment familiarization for what we carry. Later in the process, all candidates meet with one of our two medical directors for a review on RSI concepts and separately shadow one of them for a shift in the ED.
The first stage is known as Observation. This only lasts a couple shifts, and as you would likely guess, involves the candidate observing the FTO and his partner running all calls and then watching as his FTO provides primary treatment during transport. The candidate is not being scored on these calls but may perform basic skills with the FTO. Basically, this stage is to demonstrate how our system operates and get their feet wet as we tend to do things slightly differently operationally than most other places.
The next stage is known as Mentoring. In this stage, the candidate is running every call with the FTO monitoring and providing input as required. They are allowed up to two "prompts" from the FTO during this stage to still "pass" the call. A prompt can be anything from a reminder to use capnography during treatment of a patient with SOB to protocol violations or problems in assessments or scene management. Each call is graded as pass/fail with comments from the FTO and an evaluation form filled out with an opportunity for the candidate to respond.
Once the candidate and their FTO feels they are ready, the candidate moves on to the Evaluation phase. At this point, the FTO steps back a little bit more and provides less input to the candidate on the call. They only step in if something egregious is going wrong or there is somehow true need for an additional set of hands on a critical patient. The candidate functions with their paramedic partner as the FTO simply observes on scene and during transport. The candidate is now required to pass each call with a maximum of one prompt. The candidate has to have an overall pass rate of 90% I believe with a certain number of "high priority" calls before they are ready to move on.
The final stage before release to function independently moves the candidate to being the second person on a truck rather than a third (extra) person. They still work with an FTO exclusively for this portion, but they transport calls alone without their FTO accompanying them. This step is primarily focused on the operational component and ability to function as a single medic. At the conclusion of this portion, the FTO, FTO supervisor, and medical director all agree that this candidate is ready and they are released for normal shift work.
Edited to add: I forgot our scenario component. There are various times throughout the process where the candidate goes through various scenarios with our simulation team and sim man mannequin. We're still getting this part figured out as we go.