If your battery was dead when you changed the alternator then yes you should charge the battery back up before starting the vehicle otherwise the dead battery could ruin the new alternator.
When changing an alternator you always want to disconnect the battery and then reconnect the battery after you charge replace the alternator and charge it back up.
The battery in your vehicle should have the correct capacity for your vehicle and be fully charged up to prevent damage to your alternator.