Multi-GPU issue with the ASUS ROG Strix Z690-E Gaming WiFi Motherboard

I have a bit of a niche/particular issue, so I'll try to be as detailed as possible, bear with me lol.

I have the "ASUS ROG Strix Z690-E Gaming WiFi 6E LGA 1700(Intel 12th Gen) ATX Gaming Motherboard(PCIe 5.0,DDR5,2.5 Gb LAN,Thunderbolt 4,5xM.2,1xPCIe 5.0 M.2,ROG Hyper M.2 Card,Front Panel USB 3.2 Gen 2x2 Type-C)" Motherboard. The specs to this motherboard are available here:
https://rog.asus.com/motherboards/rog-strix/rog-strix-z690-e-gaming-wifi-model/spec/

I am using this motherboard for a deep learning setup. I have three NVIDIA RTX A4000 GPUs. My CPU is the intel 13900K
cpu. I have 64 GB of DDR5 ram (CORSAIR Vengeance 64GB (2 x 32GB) 288-Pin PC RAM DDR5 5200 (PC5 41600) Desktop Memory Model CMK64GX5M2B5200C40). My power supply is a EVGA SuperNOVA 1300 G+, 80+ GOLD 1300W, Fully Modular, 10 Year Warranty, Includes FREE Power On Self Tester, Power Supply - 220-GP-1300-X1. I am using a single SSD, (
Samsung 980 PRO SSD 2TB PCIe NVMe Gen 4 Gaming M.2 Internal Solid State Hard Drive Memory Card, Maximum Speed, Thermal Control, MZ-V8P2T0B)


The issue I'm having is that when I connect more than 2 GPUs at once, I get a d4 error code and the computer will not boot. Here is a list of things I have tried. I have used my GPUs in another computer (all three at once), so I can confirm they are not broken. I also have swapped the GPUs in every possible configuration of PCIe slots using 2 at once (1st and 3rd, 2nd and 3rd), so clearly neither the GPUs nor the PCIE express lanes are the issue since they all work when not three are used at once. I have also tried using various VGA cables from my power supply, again same issue. It really is just something strange about having 3 GPUs at once.

I am on the latest version of the bios.

- Enabling/disabling fast boot/secure boot
- Using the default bios settings
- I have tried enabling/disabling CSM

Any thoughts for what to try next to fix this issue much appreciated. I'd be happy to give any additional information. I really don't know what to do other than RMA the motherboard, which I'd really like to avoid.