There's a big difference between 'safe numbers on track' and 'being able to ride as you wish with clear track ahead of you'.
If the groups out on track were any bigger, people would be spending half their on-track time riding around other bikes instead of getting in clear laps. While safety may be able to maintained with larger groups, rider enjoyment would go down markedly.
I second DMNTD's comments; I've done three MotoTT days this year, and at each of them I've been out for at least five sessions. My trip meter generally shows slightly over 160km for the day (which, on Betty, is barely do-able on one tank of gas at track riding pace), and by selecting the correct group, I find myself spending most of my time with nobody holding me up and nobody parked up my arse.
Maybe I'm not the riding god you are, Mort, but I know that by the end of one of those days, I'm well knackered and pretty much ready to park up and collapse.
In terms of safety, smaller group sizes aren't going to stop people giving themselves the learn and falling off of their own accord, but they are going to reduce the chance of multiple-bike incidents, which can get much nastier than just going for a slide off the track on one's arse. I think MotoTT have the right idea with the format they're running.
Bookmarks