One of the hardest trolley problems involves deciding between killing a single person, or doubling it and giving it to the next person to decide. There's also the variations where you're one of the people further down the line. If you don't pull it, at some point, you will certainly also be on the track. Also, just because the first few don't pull it, it doesn't mean someone later down won't. Someone might do it to prevent more potential deaths. Someone might just want to do it.