Just a warning this will all be very verbose and would be easier to explain in person. Sorry, this game has my nerd mode at 100% erect, so you asked for it
Expanding on Dan's post, or possibly just regurgitating, What about people signing on to squads within each faction? Each faction has missions, like we did this time, get the samples, get the explosives, some of those required other things, a rad suit for a plant sample, find arms the dynamite or make contact with those delightful hills people. To take it one step further so each squad also has missions that build towards the overall objective. Squad A happens to find a disembodied brain in a jar, that squad B just so happens to need to activate a robot that once activated can now startup a vehicle to aid the brotherhood on their journey home. Could even be the same quests each or locations for each mission with different names and essentially each squad is tasked the missions in different order and rotates (missing girl for squad a is now missing water chip for squad b) All the missions give the squads a task beyond simply fighting other teams, the combat OR creative roleplaying (temporary truces with raiders hiring locals for extra guns etc) occurs when the squads come across other enemy squads on the field on their way to their objective. Another combat mechanic is....
What about making the super mutants an actor based faction? the Super mutants can be a player faction but they have very specific roles, different from the BoS and Raiders. Be big stupid berserkers that happen to be around important quest items. At random times the muties can send a squad to attack the city or soemthing. Im not familiar with the muties objectives from the game really, but what I saw were few people acting as berserk bringers of death, they were too tactical. This could still be a player team, but perhaps smaller and each squad is led by an actor that will lead and help encourage people to play in character. Having only one actor per say, 5 players on the muties would help keep the numbers up with out having to scrounge for actors (not sure how hard that was for you, but simple is always better right?) This will allow plenty of shooting opportunities as well as some more roleplay fun.
If mutants are made into a more acting position, this leads to my next suggestion-Off the top of my head I cant remember where the game takes place on the timeline, but perhaps it is time to introduce the Enclave as a player faction. Very different objectives from the raiders and Bos, in many ways a common enemy as well. While both wear power armor there are color differences, make one tan based,the other green, or one acu and desert, the other marpat and multicam or however you wanna work it.
Forums specifically for each faction, provide threads or find people willing to produce threads on cheap/effective costume ideas, lore for the faction, etc etc, stuff that gets people in the mindset.
Ill also suggest having the locals merely be quest givers if possible. Only have a specific few be fighters or hired guns.