Buy a new fuse box/foam gasket now, as in my experience something will break or you'll find something that means you want to replace the current one anyway.
There should be enough slack in the wire loom to let you 'pop' the fusebox out of the bulkhead and in to the engine bay, which makes access a lot easier. The box is held to the bulkhead by, from memory 4 plastic tabs on the back/inside car side of the box. In theory you squeeze these together and it just pops out. In my experience this sort of works but at least 1 of the tab will have gone brittle and just snap (new fuse box required #1).
The wires terminate in special plugs that slot in to the fuse box, each one forming a single blade and 4 of them (2 each side) forming the contacts that grip the metal ends of the fuse. With the box in the engine bay you can see what you're doing and find the little 'tang' that holds each blade/plug in to the box and release them. Now if nothing broke releasing the fuse box from the bulkhead sod's law states one of the tangs will break at this point (new fuse box required #2).
With the contacts out you can clean them up and inspect them and either re-insert in to the box (making sure they all go back to the same places) and re-insert the box in to the bulkhead. If nothing's broken you'll probably find some overheating damage from some point in your car's past. There was some melting of the plastic around one of the plug/blades in my box for example (new fuse box required #3).
If you do decide/need to go to a new fuse box but your plug/blades are in good condition you just need to feed the loom out of the old box and in to the new one, then plug all the blades in to the new box. Mine hadn't weathered the years so well but there was plenty of slack in the loom to snip them off and fit the new ones (included with the new fuse box) in their place, then slot in as before.