If you're allergic to cows milk or other dairy products then you're likely allergic to goats milk as well.
Goats milk contains the same proteins as cows milk and sheep milk so that would make the goats milk bad for someone with a dairy allergy or milk allergy.
I would avoid drinking any goats milk if you're allergic to cows milk or other dairy products.
You can try drinking some almond milk or other non dairy milk if you're allergic to dairy products or cows milk because goats milk is a dairy product as well.
However some people with a milk allergy or dairy allergy are perfectly fine drinking goats milk so I would check with the doctor first before you drink some goats milk.
Although you could drink a bit of goats milk and see if you have an allergic reaction to it.