Real3D released a PCI accelerator card called R3D/100. Not much is known about this device and it is definitely different than the Pro-1000 but likely an evolution of it. This architectural overview contains a very interesting blurb about the geometry engine, which is claimed to be a RAM and ROM microcode-programmable numerical processor. Figure 2 suggests this was not an off-the-shelf part because of the integrated raster, texture, memory, and PCI blocks. I'm not sure whether it was possible to purchase "IP cores" in the mid 90's.
The Pro-1000 SDK contains two ROM images to program the device with. One is clearly for the PowerPC CPU. The other is called the "loop ROM" and contains what looks to me like mostly meaningless tables of repetitive values. There is some other data in there but it is sparse and does not look like code to me.
Given that Real3D is known to have used Fujitsu and Analog Devices DSPs in their previous designs, I would expect them to use an existing design for the Pro-1000 rather than rolling their own custom processor. A good candidate would have been the TMS320C80 Media Video Processor which was a 5-core design introduced in the early 90's. (Her'es a manual for the main core). From a cursory glance at the loop ROM, this doesn't seem to be it, though.
If it was truly a custom part, there isn't anything we can do to reverse engineer it. At some point, I would like to try one last experiment: checking VROM usage (by marking used addresses) to determine if there is any untouched data there that may be a DSP program.