It wasn't until I talked to Jason, a South African on the bus with me, that I found out why this was. It turns out that not only do you pay for the bags but if you turn them in you can get a small amount back. "There used to be so many plastic bags along the roads we used to call them the national flower of South Africa," he said.
The contrast with Swaziland, where I've just been, is striking. There, plastic bags still litter the countryside. Here in South Africa, you can see some other litter, but not a plastic bag in sight.
It's a little thing but it seems to work.