If you haven't been to the Lookout for a while(a few years) then things have changed. The "family ride out surrounded by dog walkers" is true if you use the main fire access "roads" (wide forest paths) through Swinley Forest but the true MTB stuff used to be all the bits in-between.
The map that bockers posted shows the new "official" trails, created on the understanding that if you go "off piste" then you can't come crying to the Crown Estate and sue them for you falling off your bike, as used to happen. No need to buy the little yearly tags any more.
The green trail is for kids, I've even seen people jogging round with their toddlers
and I've done the blue trail several times with the kids, and with me on my cyclocross bike it's a little bit challenging in places (I've not got an MTB). I've not tried the red trail, I suspect that will be more up your street, it crosses my commuting route through the Forest a couple of times. And you can still go off piste, I suspect that many of the old tracks are still there.
Worth a look if you haven't been for a while.