Yes your cash App Card can be used at any ATM for withdrawing cash from the Cash App card.
Although Cash App doesn't charge any fees for withdrawing money from the ATM with the Cash App Card you may still be charged the ATM fee from the owner of the ATM for withdrawing money.