Yeah, I realized I could just hit the Y button a while after I posted this thread, but what I just had to do to fix everything was to beat one of the side quests, and Magus would ask me to join again, allowing me to change my party.

"Stephen Wolfram is the creator of Mathematica and is widely regarded as the most important innovator in scientific and technical computing today." - Stephen Wolfram