Yes small splinters will eventually work their way out of your skin without needing to do anything.
It may take a month or two but overtime you shouldn't feel the splinter anymore after the skin starts to heal up from getting the splinter.
Usually the splinter will works it's way naturally out of your skin.