An oxygen sensor on a car or other vehicle can indeed be cleaned after you remove the oxygen sensor from the vehicle.
Gasoline can usually clean up a dirty oxygen sensor really good and have it back to working order after allowing it to sit in the gasoline for 8 hours or overnight.
You could take a coffee can or other small container and put some gasoline into the container and then let the oxygen sensor soak in the container of gasoline for a minimum of 8 hours.
If you let the oxygen sensor sit in the gasoline overnight then it would clean it much better.
Then wipe the gasoline off and you might want to use an old toothbrush or other brush to clean the oxygen sensor off further.
Then install it back into the vehicle and the oxygen sensor should work better.
Sometimes you can just buy a new oxygen sensor for around $20.00 to $30.00 which is what I would do if I had the money since I have to remove the old oxygen sensor anyway.
But I've had good luck cleaning an oxygen sensor when gasoline when I could not afford to replace the oxygen sensor.