Downriver from Twickenham Bridge, a few hundred yards along the riverside lane, is Richmond Lock. You can wheel your bike across the ironwork pedestrian bridge over it except at night, when the bridge is closed and anyway you should be tucked up in a pub. There are several steep steps up to and down from the bridge. It's a Grade II listed structure, built in 1894 with a lot of nuts and bolts.
Your map here is London Cycling Guide No. 6, one of the excellent TfL series available free in bike shops or online.
Cross the bridge to the south bank and continue along the gravelly towpath. You go past several riverbank views that look more Severn than Thames, and eventually see across to Brentford's redeveloped marina area. On your right is the Old Deer Park and then Kew Gardens; you might hear the screech of parakeets flitting about the trees, and of children who have fallen off their bikes on a family ride. Across the water is Syon Park. Join Kew Bridge on its upriver side. Or its downriver side.