If you're wearing a retainer and take the retainer out of your mouth for just a day then nothing will usually happen.
After the few months to 12 months it's generally safe to take the retainer out of your mouth for a day or two but then you must wear the retainer most other days.
When you get a retainer to keep your teeth aligned you have to wear the retainer full time for at least the first 12 months after getting your braces removed.
Then after the first 12 months you can switch to wearing the retainer only to 3 to 5 nights per week.
But if you want to keep your teeth aligned with the retainer you need to wear the retainer forever or for the rest of your life at least 3 to 5 times per week.
So you can remove the retainer if you want too for a few days per week but you should put the retainer back in your mouth and wear it for the rest of your life.
Now you can choose to say the heck with the retainer and allow your teeth to possibly get out of alignment again if you want too.
But if you want to keep your teeth in the shape they were in after getting braces and retainer then the retainer must be worn forever.