To remove the head unit, you will need 4 stereo removal tools. You can purchase these through your local dealer, or from a stereo shop. For this FAQ, I used tools purchased from Crutchfield, Part #120869001. Double check that that tool is for a 99 VW Golf. Crutchfield hasn't updated their database for the Double Din Symphony so if you ask for Audi you get the wrong part. Since you only get two tools per pack, you will need to order two sets. This particular tool has three different tools for different head units. I used the longest of the three tools to remove the head unit. Make sure, before you start this, that you have your radio security code!

Also, it goes without saying, do this at your own risk folks.

First, locate the 4 slots for the removal tools, mine are above and below the buttons on each side.
Slide the tool in with the slanted tooth pointing in towards the middle of the stereo. (On both sides, tooth towards the middle) After you feel the tool lock, continue with the other three tools.
Once all four tools are inserted, gently pull each corner until the head unit slides out.
Once you have the head unit pulled out, you will need to remove the tools. To do this, you need to press the clip in towards the middle of the head unit. This picture shows what the clip looks like without the tool inserted.
Here you see the tool inserted and how it holds the clip back. Gently push on the clip and pull the tool out.

After you have your head unit pulled out, you will need to remove the connections on the rear. It's fairly straightforward; there are four plugs on the back left, and three connections on the right. Just remove those connections and there's only one thing left to do.

There is a round connector in the middle of the back of the head unit. It looks like it's just pushed onto the back, but it's actually screwed on. So, you now have two choices. First, you can cut the little plastic connector holding the wire (Like I did...), or you can rotate the head unit until the connector comes off.

And that's it, not so bad after all.