Yes a vet can legally keep your dog or other pet until you pay the bill you owe them.
Some vets will put a lien on your pet which is legal for them to do and will make it so that the vet can hold your dog and not release the dog or other pet to you until all the debt is paid.
Some vets have worked with me on payment plans especially when I had a good credit rating and let me pay some here and there monthly until I paid them off.
But not all vets will work with you so it's a good idea to have pet health insurance in place to take care of any vet visits such as emergency vet visits.
Not all pet insurance will pay for vet checkups but they will cover all emergency costs for emergency medical visits.
But yes veterinarians can have a lien for services, boarding, etc and then keep the pet legally until all debt is paid.