NewToAudi94

I just bought a '99 a4 (against my better judgement really), with 77k on it. it is the 2.8 5 speed and today I drove it for about an hour and stopped for gas. When I got back into the vehicle to start it, it would not fire. After pushing into a parking spot and pulling the fuel pump relay to tap it in case it was stuck, the car started and I drove it with no problems 2 hours home. I let it sit for about the time it did when I got gas, and went back out to start it. It started like usual. Fast forward an hour I went to drive again and it started, sputtered and died. I bought a new relay but tried to start it after pulling the old one and putting the old one back in again and it fired up perfectly. I am hesitant to install the new relay because I believe it may be an overheating pump that works when it is cooled down. Has anyone encountered anything like this in their vehicles? My personal thoughts are an overheating fuel pump that needs to be replaced. But I don't know how to test for this.