Downriver from Woolwich Foot Tunnel, rather a long way, is Dartford Crossing. It consists of a tunnel, effectively the northbound crossing of the M25; and a bridge, the southbound. You bike it either way by turning up at the Crossing Control and getting a free lift across in a Range Rover painted like a poisonous insect. The exciting process is covered in detail in a recent post.
It's a mile or two from Dartford town centre. You'll have to find it by the traditional cycle-touring method: misread a map, ask people who don't know, and stumble on it by chance going the other way to what you thought.
The tunnel was started in 1938, opened in 1963, doubled to two tunnels in 1981, and finally augmented with a bridge - the Queen Elizabeth II Bridge - in 1991. Cycle paths connect the northside Crossing Control with civilisation, and Essex, but there isn't much to do so you'll probably come back over the bridge quite swiftly and return to the south side.
Retrace your steps to Dartford town centre and take a grateful train home. This is a good place to end of the Thames Tidal Crossing bike trip: 32 cycle traverses between Teddington and Dartford by bridge, tunnel and ferry, staying alongside the river virtually all the way, most of it traffic-free. The whole thing is a lovely way to explore London, and makes a great two-day trip, split somewhere in the very centre around Westminster Bridge.
If you're up to collect the entire set, there's one final estuary crossing. From Dartford town centre it's 8-10 miles along National Cycle Route 1 (not alongside the river, though) to the Tilbury-Gravesend ferry.