Yes reverse osmosis water is okay to use in batteries.
In fact reserve osmosis and distilled water is what's commonly used in batteries since regular tap water that hasn't been distilled could harm the battery.
So as long as you're not using regular tap water in the battery then it will be fine.