President Ulysses S. Grant was arrested for speeding on a horse while in office and had to pay a fine.