Downriver from Woolwich Ferry, only a few yards, is Woolwich Foot Tunnel. It's very similar to Greenwich Foot Tunnel, with similar drawing-room lifts and round entrance huts, but the addition of a few barriers to hinder attempts to cycle. (Shame! They allow it in the Tyne Tunnel, which has a dedicated tunnel for bikes.)
It's a bit longer than Greenwich at 504m, was opened later in 1921, and is a bit leakier.
From here there's no sensible or pleasant way downriver on the north bank. On the south bank, though, is National Cycle Route 1, which takes you all along the riverside, traffic-free, as far as Erith.
Immediately after the tunnel is Firepower, an artillery museum, which has a cafe. The building complex is the old Woolwich Arsenal, which gave its name to a football team which used to be in London. There's a bunch of life-sized metal male figures that look like Gormleys but aren't, presumably standing around waiting for Woolwich to gentrify. They're very rusty.
Just a few hundred yards on, London feels to have come to a sudden stop, and the scenery gets suddenly broad and estuarial. After Erith you go past a sailing club and alongside fields with horses and get your first view of the Queen Elizabeth II Bridge shimmering hazily in the distance. A pebbly track cuts inland and then, leaving Route 1, it's a couple of miles of roads into Dartford town centre. You'll have to head north up a hill and through housing estates to find the cycle path access to Dartfor's bridge/tunnel crossing system. From Woolwich Foot Tunnel it's about 12 miles to Dartford Crossing.