Network code

[Spindizzi]

Hello,
Thanks SailorSat for your infos
I Had only reserved a total memory of 192ko (0xc0000000-0xc002ffff), so it must be 256ko (0xc0000000-0xc003ffff)

The disassembled network code of daytona2 can be found here : https://yadi.sk/i/RwcOD_vM3GtVGH (only the 1st part is correctly disassembled, the 2nd part needs to be re-origine to be rightly disassembled : actually the dc.b dc.w) . Already posted in the old section about 68k cpu : http://www.supermodel3.com/Forum/viewtopic.php?f=7&t=1363&start=10#p12358 and http://www.supermodel3.com/Forum/viewtopic.php?f=7&t=1363&start=10#p12388

I know that when the program is transferred by the PowerPC, it will appear a bit mangled. I think it transfers 32-bit words, with half of the word being empty or something like that, and then a word swap needs to be done. Should be easy to dump the program once it's been uploaded.

Correct, this is what I've done in the code

I notice in the disassembled code and in the debugger running the code, that there are accesses from the network card to memory 0x080000-0x0bffff (256ko)(reg A4) and there are references to 0x040000 (reg A5)(don't know the range) and 0x0c0000 (reg A6)(don't know the range)
What are they ? a sort of mirror of the addresses seen by the ppc ? or simply the receive/send ram ?

[Bart]

Hello,
Thanks SailorSat for your infos
I Had only reserved a total memory of 192ko (0xc0000000-0xc002ffff), so it must be 256ko (0xc0000000-0xc003ffff)

The disassembled network code of daytona2 can be found here : https://yadi.sk/i/RwcOD_vM3GtVGH (only the 1st part is correctly disassembled, the 2nd part needs to be re-origine to be rightly disassembled : actually the dc.b dc.w) . Already posted in the old section about 68k cpu : http://www.supermodel3.com/Forum/viewtopic.php?f=7&t=1363&start=10#p12358 and http://www.supermodel3.com/Forum/viewtopic.php?f=7&t=1363&start=10#p12388

I know that when the program is transferred by the PowerPC, it will appear a bit mangled. I think it transfers 32-bit words, with half of the word being empty or something like that, and then a word swap needs to be done. Should be easy to dump the program once it's been uploaded.

Correct, this is what I've done in the code

I notice in the disassembled code and in the debugger running the code, that there are accesses from the network card to memory 0x080000-0x0bffff (256ko)(reg A4) and there are references to 0x040000 (reg A5)(don't know the range) and 0x0c0000 (reg A6)(don't know the range)
What are they ? a sort of mirror of the addresses seen by the ppc ? or simply the receive/send ram ?

Thanks for posting this!

I haven't looked into this at all so I'm not up to speed but the first thing to do would be to figure out the 68K and PowerPC memory maps. It's been a while for me but I seem to recall that the 68000 has its vector table at address 0. Subsequent versions (68010 and others) may have had an ability to relocate the vector table elsewhere? My memory is fuzzy. It's important to know how the 68K is booted up because that will allow you to establish where the vector table is. I would also expect that there is some sort of register that can be used to reset the network 68K. This might not be in the 0xC0xxxxxx space (network board RAM) but rather in the Model 3 system register space. I would take a look at what system registers are being written before and after the network program is transferred. There could be a register that is used to trigger a reset pulse and there could also be a register (or maybe the same register, different bit(s)) that lock the 68K bus while data is being written to shared memory. This sort of bus arbitration can be simple or ridiculously complex depending on how the designers implemented it, so I can't tell you what to expect. Model 3 tends to be pretty simple and cleanly designed, though.

Also, I am pretty sure the 68K cannot access PowerPC RAM. The model is likely a single RAM region for the 68K to use that the PowerPC can also access. The PowerPC could have access to the entire 68K RAM space, and then it's just a matter of some convention they use on how to communicate. Presumably, some fixed location in the 68K RAM is used to communicate back and forth between the 68K and PowerPC. IRQs may be involved as well. There is a network board IRQ, IIRC, and what probably happens is that either the 68K explicitly triggers it by writing to some hardware register that is only mapped in its space, or the network controller ASIC automatically triggers an interrupt that the PowerPC listens to. Then there is probably some software protocol between the two devices governing how data should be updated.

I recall Nik mentioning the protocol appeared very simple. The PowerPC would write some data, the 68K would send that to the next link, and then some data would be read back.

In summary, I would expect the 68K address map to consist of two or more blocks: a RAM region that the PowerPC sees, and a register region that probably only the 68K sees for accessing the comms device. There may be a second RAM region that the PowerPC does not see but that the comms device can read data from. In that case, you would see the 68K copying data to some special memory location that could be outside the window that the PowerPC can see.

I hope that helps?

[Bart]

I just took a quick peek at the 68K programming manual and there's just a single vector table at address 0. But in your disassembly, it looks to me like offset 0 is the program entry point. There isn't a vector table? The vector table contains the initial stack pointer address at offset 0 and the program counter location at offset 4.

I vaguely recall that the Atari Jaguar had some really bizarre way of booting up its 68K. I think they designed the address decoding to intercept the reset vector and always return some particular address. The vector table is 1024 bytes. Should be hard to miss. If the PowerPC isn't copying that over, then either it's already there as part of some ROM, or they generate the data dynamically whenever the 68K requests those addresses. I can see in your disassembly that exception handlers do exist (the 'rte' instruction is return-from-exception) but they are at some arbitrary location that must have been specified in the vector table.

[SailorSat]

The 68k code most likely is rather simple. As soon as the link-up process is done, the host game usually won't check anything at all and just read/write it's data to the share memory.

The network irq most likely is only used on sync-up or lost link

[Spindizzi]

What amount of informations to digest need pills
Yes it's a priority to determine memory maps clearly
The 68k vector can be found in the start of the network code dump. In IDA, I've set the origine of the dissassembly to an other address not 0x0 (can't remember, it was a long time ago, maybe 0x500 or 0x800) this is why you can't see the vector in the disassembled code
Concerning the reset board, I agree
actually, I do myself a netboard reset when code is fully loaded (sort of magical number like you do for the driveboard), but I can see right after the code fully copied, there are writes to 0xc0010110, 0xc0010180, 0xc0010000. And I suspect a new vector to be written at 0x800-0x804-0x808 in the 68k memory

Network code copy ending
RESET NetBoard PC=000000
W32 *********************** @800<-0
Network code copy ending
RESET NetBoard PC=00083A
W32 *********************** @804<-f03e0000
W32 *********************** @808<-50030000
W32 *********************** @10180<-0
W32 *********************** @10110<-800000
W32 *********************** @10110<-800000
W32 *********************** @10180<-ffffffff
W32 *********************** @10110<-b00000
W32 *********************** @10110<-b00000
W32 *********************** @10000<-e407b800

right after, I've a memory test/flush (from 0x0000 to 0xffff) with values 55555555 then aaaaaaaa and then 0 for each dword

and after 2 writes to 0xc0010110 (this is where supermodel says : this is a master controller (or live/slave depending the nvram settings)

W32 *********************** @10110<-900000
W32 *********************** @10110<-900000

after that, ppc turns in loop as well as the 68k
May be an irq is set to let pass supermodel in game. The really first sync as SailorSat said

[Jiterdomer]

Will you able to connect it to TCP/IP?

[Ian]

It should be simple enough to write the code for TCP if these guys can figure out how the network board works. Not even sure it needs to be multithreaded as could probably just put a read/write time out in there. Realistically I think it'll only work over local network. Even a latency of 16ms is a whole frames worth of delay.

[Bart]

Latency will definitely be a problem. These games used a low-latency fiber optic link and expected to be synced almost frame-by-frame, although it probably comes down to how the individual games were implemented. They also use a ring topology,meaning that the round trip had to be made between all machines. Maybe SailorSat can comment further on Model 1 and 2 latency requirements?

[Bart]

actually, I do myself a netboard reset when code is fully loaded (sort of magical number like you do for the driveboard), but I can see right after the code fully copied, there are writes to 0xc0010110, 0xc0010180, 0xc0010000. And I suspect a new vector to be written at 0x800-0x804-0x808 in the 68k memory

Network code copy ending
RESET NetBoard PC=000000
W32 *********************** @800<-0
Network code copy ending
RESET NetBoard PC=00083A
W32 *********************** @804<-f03e0000
W32 *********************** @808<-50030000
W32 *********************** @10180<-0
W32 *********************** @10110<-800000
W32 *********************** @10110<-800000
W32 *********************** @10180<-ffffffff
W32 *********************** @10110<-b00000
W32 *********************** @10110<-b00000
W32 *********************** @10000<-e407b800

right after, I've a memory test/flush (from 0x0000 to 0xffff) with values 55555555 then aaaaaaaa and then 0 for each dword

I'm not sure when you log "RESET" -- the first one must be just when you start up Supermodel and try bringing up the 68K with a blank memory region? How big is the program and at what memory range is it loaded (0x0000-????). Addresses in the 0x8xx look to be within RAM but these 0x10xxx addresses could be device registers.

and after 2 writes to 0xc0010110 (this is where supermodel says : this is a master controller (or live/slave depending the nvram settings)

W32 *********************** @10110<-900000
W32 *********************** @10110<-900000

When do these happen? Does the PowerPC first read something out of the network board RAM (i.e., did it wait for the 68K to say that it was ready)?

after that, ppc turns in loop as well as the 68k

Should be easy to figure out what's going on. If the 68K is looping checking some memory region, you just need to find what writes that region. First thing to check is the IRQ routines. If it doesn't look like they write it, then the next thing would be to check the PowerPC. There is a network-related interrupt on the PowerPC side as well.

Usually, it's a good idea to try firing these IRQs on both the 68K and PowerPC side and observing how each side reacts to them. There are a number of possibilities: 1) network device triggers 68K IRQ periodically or when done sending/receiving, 2) network device triggers both PowerPC and 68K IRQs, 3) network device triggers only the PowerPC (unlikely), 4) the PowerPC can trigger an IRQ on the 68K by writing a register, 5) the 68K can trigger an IRQ on the PowerPC by writing to a register. I think there is only one network-related IRQ on the PowerPC. Not sure what's going on on the 68K side.

[Bart]

Actually, I still have Nik's emails. It's more complicated than I thought. Let me post some excerpts. Later I can try to collect up these emails and upload them. I'm sure he wouldn't mind.

1/14/2013:

Code: Select all

I had some time over the w/e to look again at the network board in Supermodel.  I'm pleased to say that I've made a bit of progress...  but before you get too excited, by that I mean that I've just got the board to initialise properly.  I've not got any game to reach the attract screen yet but I have got them to the point where the PPC to network interaction starts to happen via the shared network buffer, rather than through the I/O registers (which are used by network initialisation but don't seem to get utilised that much afterwards, save for a bit of error reporting).

As it happens, the network initialisation is quite interesting.  After various hardware checks the total number of machines is counted on the network and each one is assigned an id.  This is done using a handful of special messages that are sent over the wire.  I think also at this point some timing adjustment is done on the slave machines to make sure they are running exactly in lock-step with the master machine.  All this is handled by the 68K program, which I've pretty much mapped out and understand now.  The bit I was missing before was that during initialisation the PPC expects to receive back from the 68K program the number of machines found on the network and its own assigned node id via some bits in one of the I/O registers.  Without this the PPC just times out.

After a successful initialisation, the 68K program settles into a very simple loop that continually sends the shared network buffer (or just some part of it, I'm not sure yet) over the link.  Hence, I think the final key will be, as you say, to work out how to interpret the data that is read/written from/to the shared buffer by the PPC (in a similar way to SailorSat's work on the MAMEWorld forums).  I already have a good idea of the buffer's layout:  0x0000-0x0080 stores various status bytes, 0x0080-0x0100 are used by the special messages I mentioned above which count the network nodes and assign them ids (what I don't know yet is whether these bytes get used again for the in-game messages) and 0x0100-0x2000 contains the "ring buffer" (as SailorSat calls it) which is used to send/receive/store the messages sent by the running game.  It looks like this ring buffer is divided into blocks of 0x0400 bytes (1K), with (at least) one block per machine node.  What I don't know yet is whether the blocks are ordered by node id or whether the whole buffer is rotated with each frame (ie like a FIFO queue of received messages).  According to SailorSat, the Model 1 board uses a setup with the first ordering whereas Model 2 uses the second.  By playing around with different node ids and different numbers of nodes in the network I should be able to work all this out.  I'm hoping it won't be too hard though and that, once done, we should be able to get something up and running pretty quickly!

Lastly, a quick note about the networking code in Supermodel itself:  I've already written some code (using SDL_net) that connects different instances of the emulator together.  However, I've chosen to preserve the ring topology of the machines, rather than using a client/server model.  All the user has to do is specify the IP (and/or port) of the next client in the ring.  I prefer this model as it keeps things simple and removes the need to run a separate server.  Also, I believe this is how ElSemi does things with his Model 2 emulator which means users will be more familiar with it (I think the client/server model you mention is what SailorSat has been using, rather than ElSemi).

Talk to you later!  Nik.

1/16/2013:

Code: Select all


Thanks!  I've actually made a little more progress since my last e-mail.  I think I've worked out the exact layout of the shared network buffer.  The clue was in the 68K program.  It configures the layout via a range I/O addresses which tell the hardware the size of the network messages, where incoming messages should be buffered and where the next message to be sent should be read from.  I had completely missed this before, but what got me looking for it was the fact that the various control messages that the 68K program sends/receives (to count the number of nodes, assign node ids etc) use a different region of the buffer (0x0080-0x0100) to the messages that the game sends/receives (which use the "ring buffer" at 0x0100-0x3D00).  It turns out that the 68K program reconfigures the layout before sending or receiving a message of either type through the use of these I/O registers.  The values used for the game's messages confirm that the buffer is pretty much laid out as I was expecting before (those guesses were based on the reads/writes I had observed from/to the shared network buffer by the PPC):

 0x0100-0x0500 - next message to send (message size 0x0400 bytes)
 0x0500-0x3D00 - FIFO buffer of received messages (series of 14 messages, each 0x0400 bytes)

I was hoping that, having worked this out, I might have been able get things to progress to the attract screen in order to be able send you a screenshot of a successful link-up - but unfortunately I'm still not there yet  !!!  However, I'm convinced I'm getting close and hopefully I'll have something to show you soon (although I probably won't get a chance to look at things again until the w/e...)

Something else interesting about the shared network buffer is that it looks like it is banked so that it acts like a double buffer.  My guess is that this is to allow the network hardware to write new received messages into the "back" buffer as soon as they arrive without having to worry about affecting the running game, which can simply carry on processing the messages in the "fore" buffer.  I'm not sure though when exactly the buffers are "flipped" (it's handled by some code in one of the 68K interrupt handlers, but I don't know what drives that particular interrupt yet).  Luckily we probably won't have to worry about double buffering at all when simulating the board as we'll have the luxury of being able to choose when exactly we write new messages into the buffer (eg at the beginning or end of the frame).
 
>> As it happens, the network initialisation is quite interesting.
>> ...
>> some bits in one of the I/O registers.  Without this the PPC just
>> times out.
>
> Interesting.  Are you emulating this by running the 68K program?

Yes, that's correct, although I've already made a start on the simulation.  It's looking like it will be pretty simple to implement.
 
>> After a successful initialisation, the 68K program settles into a
>> ...
>> out.  I'm hoping it won't be too hard though and that, once done,
>> we should be able to get something up and running pretty quickly!
>
> Nice!  It sounds like simulating the functionality of the 68K should
> be pretty straightforward then, right?  In other words, you probably
> don't have to be aware of the format of the actual data being sent, and
> only need to push things around "blindly" to make it work.  At least
> initially.  I'm also guessing that the games communicate every frame
> because of their low-latency link.

Yes, I think so.  The overall timing is still something that I'm not 100% sure about though.  It may even be the last thing that is preventing me from getting things working.  I'm uncertain what "drives" the network board, as it were, as I don't fully understand the timing of some of the 68K interrupts.  I don't know if they are driven by an external timer, are invoked by the PPC itself or just get triggered by real network events (eg data arriving).  One thing I wanted to ask you about is the PPC's Network IRQ - do you know what triggers this?   I'm not sure if it's triggering at all at the moment in Supermodel.  The PPC handler routine for it looks very simple.  In scud, for example, it simply writes a single byte to one of the network board's I/O addresses and that's it.  I'm wondering if this is what drives one of the network board's interrupts?


1/17/2013:

Code: Select all

> I've never really dealt with any communications hardware other than
> UARTs but I would imagine that the hardware has some way of knowing
> when a transmission is finished or has been received and triggers
> interrupts accordingly.

Yes, I think at least one of the 68K interrupts is triggered by something like that.  There are four interrupts in all: IRQ2, IRQ4, IRQ5 & IRQ6.  IRQ2 occurs whenever the PPC writes a command to the network board via the I/O register while IRQ6 is triggered by the "network carrier".  I know this because if I don't trigger IRQ6 periodically the message "NO NETWORK CARRIER" appears during boot-up.  I'm guessing the network carrier is something like the token in a token ring network (which I understand the Model1/2/3 links are based on, right?)  I don't know with what frequency it should be triggering though.  We could probably work it out from the speed of the real Model 3 link, if that is documented somewhere.  The last two interrupts IRQ4 and IRQ5 are more of an unknown.  Their timing relative to each other and to IRQ6 is important though because if they don't follow a certain pattern the board errors.  I'm starting to think IRQ5 is the main driver of the board as the 68K program spends most of its main loop waiting for this interrupt to occur.  IRQ5 is also the interrupt that "flips" the double buffering of the shared network buffer.  For that reason I think it must be linked with the PPC board somehow.  It would only make sense for the buffer flipping to happen at a time that is convenient for the PPC.

> You mentioned that IO registers are use to configure
> memory addresses into which the hardware reads and writes.  I presume
> that the length is encoded in there as well?

If you mean the message length then no, not directly.  The only length that is passed via the I/O registers is the total length of the received messages buffer.  I derive the message size from the other values passed, ie "start address of received messages buffer" - "start address of buffer holding message to send".  I assume this must have been how the hardware did it.

> Cool.  I think simulation is definitely the way to go but having it
> emulated 'properly' is still quite valuable if only for validation
> and documentation purposes.  The MAME project is big on emulating things
> 'properly' and I would think that they would be excited to see your
> network board code.

Yep, that was what I was hoping!  Besides, it's been fun trying to figure out how the 68K program works :-)...

> What the PowerPC writes back is either an acknowledgement that clears
> the IRQ line to prevent the interrupt from re-triggering, an
> acknowledgement that simply clears a status flag, or triggers an IRQ
> on the 68K side.

That makes sense, although it does strike me a bit odd that that is all the IRQ handler does.  Maybe they just chose to not utilise the Network IRQ in Scud Race.  I should take a look at some of the other games to see if they do anything more interesting with it.


2/12/2013:

Code: Select all

Hey Bart,

I've figured out what I was missing with the network board status registers.  I needed to increment one of them with each frame during network initialisation.  Doing so fixes the problem with the game freezing if no network is found and it's now possible to enter the service menu to configure things at boot up (certainly much easier than the convoluted steps needed to get things working that I sent you this morning!). 


He also mentioned that when he got Daytona 2 running in networked mode initially, it was running at 50% speed. He at first mistook this as being due to his machine being too slow to run two instances of Supermodel but it turned out that the networking code exposed a frame timing bug which he then fixed. That's how he calibrated the Real3D status bit flip time that Supermodel currently uses. I've been trying to fix this up properly but I still don't have the correct behavior, so we may be in for some fun if these two paths of development happen in parallel