On key on engine off my fuel pressure regulator jumps to 42psi, and then if I key back off (turning the fuel pump off), it falls to about 30. If your fuel pressure never comes back up, it sounds like the pump is only running for a second and then not running ever again after that. That, to me, would point to an issue with the pump or relay.
If I were to pressurize my fuel system, then unplug the fuel pump, then start it, it might fire for a second, but wouldn't run for more than a few seconds like that. If you can still manage to idle for a minute, then that, to me, would point to an issue with the relay.
Are you using all new parts? If it will idle, can you pull the vacuum line to the regulator and see if that changes anything? I've read about some crazy regulator failures before.
One additional note, on my car, on key on engine off the fuel pump will run for about 5 seconds, and then turn off, and doesn't turn back on until you key off key out for about a minute, or send a crank signal (key to start). The fuel pump will not run continuously upon key on engine off. I idle at about 36-38psi of fuel pressure.