We have a flat floor camper trailer we took over the whole length of the Binns Track (2100kms+ of rough outback tracks and dirt) last October. Our FJ has 2" Ironman suspension, Ironman bull bar and a winch. The rear suspension handled the camper without drooping and when you do hit that hidden pot hole or washout, the suspension handles it all, with no real discomfort or dangerous situations.
As stated earlier, only use 5th gear when going downhill or have a decent tail wind on the flat, everything else 4th gear.
We always travel in 4WD high range on the dirt, has much better control on the loose stuff and the number of times you hit a pile of very loose sand and stones in a corner, or the mud patch just round the corner, means you are less likely to get yourself into trouble.