No you should not swim with contact lenses in because if you do swim with contact lenses in the pool water and chlorine can cause irritation to the eyes while the contact lenses are in.
The chlorine and pool water could also damage the contact lenses so it's best to remove the contact lenses from your eyes before you swim.
Even if you do use swimming goggles water can still get into the eyes and affect the contact lenses and eyes so it's best to remove the contact lenses before swimming.
The last time I tried swimming with contact lenses in my eyes I had eye irritation and burning eyes afterwards and the eye doctor said it was due to the chlorine and other chemicals in the pool water affecting the contact lenses and my eyes.
He said to avoid irritation that I should remove the contact lenses and after doing so I never had any other eye irritation.