Network code

[Ian]

Writing some async network functions.
With the receive method, currently it's a vector array which contains the size of the data.
What if it's just a pointer to a block of memory ?
Will the receiver always know the size or should I include it

[Bart]

Writing some async network functions.
With the receive method, currently it's a vector array which contains the size of the data.
What if it's just a pointer to a block of memory ?
Will the receiver always know the size or should I include it

I haven't looked at Supermodel's network code but I would assume you are prefixing the messages with the total packet size. I would also include a message ID in case you want to support Supermodel-specific control messages down the road (e.g., a check to see that the remote side is running a compatible version of the emulator, etc.) I would recommend something like BSON or MessagePack for the payload, since they don't require a separate schema compiler, and am personally partial to using a GUID (there are ways to generate 48- or 64-bit short GUIDs) as the message type identifier. E.g., 4 bytes size + 8 bytes ID + payload.

[Ian]

Well currently it just appends the packet size to the start of the data.
That makes receiving the whole data chunk much simpler
I think I'm just going to return a vector of vectors, hopefully that won't blow Spindizzi's mind
Just trying to work out how to properly use std::move so as to avoid pointless copies

[Bart]

Well currently it just appends the packet size to the start of the data.
That makes receiving the whole data chunk much simpler
I think I'm just going to return a vector of vectors, hopefully that won't blow Spindizzi's mind
Just trying to work out how to properly use std::move so as to avoid pointless copies

If it's multi-threaded, not sure you can really avoid copies, otherwise you risk sending while the buffer is being mutated. But there should be no more than one copy in Supermodel (the OS network layer is likely to perform a second copy).

[Ian]

Gonna put each packet into it's own vector array. Then put each packet vector array into a vector.

Basically std::vector<std::vector<char>>
Can use std::move to move each packet into a new array when the user asks for data. Std::move basically just swaps the heap pointers.

[gm_matthew]

At the moment the netboard RAM has alignment issues that cause certain bugs such as cars in Scud Race not being oriented correctly. I was able to make changes that fixed those issues in my simulated netboard build, but those changes would break the old emulated netboard. I'd like to discuss how to to fix these alignment issues in both Model3.cpp and and NetBoard.cpp to get the emulated netboard working better; I can easily adapt my simulated netboard.

Most memory regions accessed by the emulated PowerPC are 32-bit byte-reversed, and regions accessed by an emulated 68K are 16-bit byte-reversed. If we would like to preserve the 16-bit alignment of netBuffer/CommRAM, I've figured out what needs to be done to get everything to map correctly:

32-bit write to netBuffer - swap bytes in each 16-bit word
16-bit write to netBuffer - swap bytes, apply XOR 2
8-bit write to netBuffer - apply XOR 2

32-bit write to CommRAM - swap 16-bit words
16-bit write to CommRAM - direct
8-bit write to CommRAM - apply XOR 1

In the current SVN, the following operations do not use the required XOR operator and so are not aligned correctly:

- mainboard 16-bit and 8-bit operations to netBuffer
- netboard 8-bit operations to RAM, CommRAM and ioreg

I added the XOR operator to each of these in order to fix the alignment issues, but then games mysteriously stopped working in linked mode! After hours of searching for the cause, I discovered that the culprit was in the code when making send/receive commands. Here's the code for receiving data:

Code: Select all

            if (recv_size < 0x0019) // must find a better condition
            {
               //auto recv_data = udpReceive.ReadData(5000);
               //DebugLog("-> nb recu : %x\n", recv_data.size());
               //memcpy(CommRAM + recv_offset, recv_data.data(), recv_data.size());
               //DebugLog("receive enable off=%x size=%x\n", recv_offset, recv_size);


               DebugLog("receive enable off=%x size=%x\n", recv_offset, recv_size);
               auto &recv_data = netr->Receive();
               memcpy(CommRAM + recv_offset, recv_data.data(), recv_data.size());
            }
            else
            {
               DebugLog("receive enable original value off=%x size=%x\n", recv_offset, recv_size);

               slot = (recv_size >> 12) & 0x0f;
               recv_size = recv_size & 0x0fff;
               recv_size = (recv_size << 4) | ((recv_size >> 8) & 0x000f);
               recv_size = recv_size & 0x0fff;
               if (slot != 0)
               {
                  recv_size = recv_size * slot;
               }
               recv_offset = (recv_offset << 8) | (recv_offset >> 8);

               //DebugLog("receive enable off=%x size=%x slot=%x\n", recv_offset, recv_size, slot);

               //auto recv_data = udpReceive.ReadData(5000);
               auto &recv_data = netr->Receive();
               DebugLog("-> nb recu : %x\n", recv_data.size());
               memcpy(CommRAM + recv_offset, recv_data.data(), recv_data.size());
            }

If the send/receive size is small, the parameters are used unmodified, otherwise they are manipulated in a way that seemed completely bizarre until I learned that it was actually just a workaround for the alignment issues! When performing small send/receive operations, the netboard code loads the parameters directly and so they would work fine, but when performing larger send/receive operations it would load values from RAM, getting the two bytes the wrong way round.

Games had stopped working after my alignment fixes because this code that was "fixing" misaligned send/receive parameters was now corrupting them, so I deleted it and games started working again. With the alignment issues fixed, the Scud Race car orientation bug no longer occurs and I was even able to get Le Mans 24 working by changing $904 on the netboard from 0x40 to 0x01 (preventing a TCP deadlock).

I've attached my updated versions of Model3.cpp and NetBoard.cpp.

[Ian]

Just tested scud and it works great
Do you want me to push these changes? Also did you want to get added to the project so you can commit your own changes?

[gm_matthew]

Just tested scud and it works great
Do you want me to push these changes? Also did you want to get added to the project so you can commit your own changes?

You can go ahead and push the changes; I haven’t run into any problems during testing.

I’d love to be added to the project; I’ve already e-mailed Bart requesting it

[model123]

Hello developer

I'm not a programmer so I don't know the details
I was hoping that the netboard RAM alignment would fix the spikeout FE communication error, but that error still occurs.
This error is not always in the same area. Occurs randomly at the start of the area.

Google translate

[Ian]

I wonder if the send and receive commands are swapped over. There must be some reason why the emulation blocks for an entire frame waiting for the data to arrive. If we can figure this out it should work at basically full speed.