Downriver from Teddington Footbridge, the first cycle crossing of the Thames is Hammertons (or Ham) Ferry, which connects Marble Hill Park and Ham House. It's a tiny, on-demand shuttle boat that runs February-October, 10am-6pm.
It costs £1 per adult plus 50p per bike. That works out as something like £30 per mile, which is more expensive than the Orient Express, so go on and treat yourself.
You ride a bit of history here as you cross to the north bank. Hammertons Ferry was set up as a rival to the nearby Twickenham Ferry in 1908. A legal battle established the right of Hammertons Ferry to operate (more detail in the comments below).
Its victory was celebrated in a song that was published called 'Ferry to fairyland', which seems to us a rather ambitious description of Marble Hill. Still, the Twickenham Ferry's chief claim to fame hitherto seems to have been a tedious mention in Dickens's Little Dorrit, which clearly couldn't compete with a comic music-hall ditty.
Continue a mile or so past all the smart moored boats along the towpath on the north bank to Richmond Bridge.