Tilegen inside a fragment shader

Technical discussion for those interested in Supermodel development and Model 3 reverse engineering. Prospective contributors welcome.
Forum rules
Keep it classy!

  • No ROM requests or links.
  • Do not ask to be a play tester.
  • Do not ask about release dates.
  • No drama!

Re: Tilegen inside a fragment shader

Postby gm_matthew » Mon Sep 11, 2023 11:52 am

Bart wrote:I need to take a look at this again at some point. Matthew: I'll look around the area of your patch but do you happen to have the code that updates the window layer (I call it 'stencil' in TileGen.cpp but it's actually 'window' in Sega terminology, similar to the window layer on Sega Genesis) disassembled for reference?

Here's a link to Discord where I posted the code that calculates the scrolling value and generates the mask before calling the subroutines that actually write to tilegen RAM:
https://discord.com/channels/6094235930 ... 8666574970
gm_matthew
 
Posts: 224
Joined: Fri Oct 07, 2011 7:29 am
Location: Bristol, UK

Re: Tilegen inside a fragment shader

Postby Ian » Tue Sep 12, 2023 6:32 am

Because the nametable is packed like

The data is actually arranged in 32-bit chunks in little endian format, so that tiles 0, 1, 2, and 3 will be stored as 1, 0, 3, 2


I wonder if the tilegen is just loading odd numbered tiles. So the odd number gives you the data for the even number.
If you apply this logic drawing works pretty much normally ..

But in the case of the rolling start the because the adjacent tile is actually from a different layer .. perhaps it loads that instead of the start of the secondary layer. I can't think of any other reason for this bug :p
Ian
 
Posts: 2044
Joined: Tue Feb 23, 2016 9:23 am

Re: Tilegen inside a fragment shader

Postby gm_matthew » Tue Sep 12, 2023 12:35 pm

Ian wrote:Because the nametable is packed like

The data is actually arranged in 32-bit chunks in little endian format, so that tiles 0, 1, 2, and 3 will be stored as 1, 0, 3, 2


I wonder if the tilegen is just loading odd numbered tiles. So the odd number gives you the data for the even number.
If you apply this logic drawing works pretty much normally ..

But in the case of the rolling start the because the adjacent tile is actually from a different layer .. perhaps it loads that instead of the start of the secondary layer. I can't think of any other reason for this bug :p

If the tile name table is arranged like that then it makes even more sense for tiles to be loaded in pairs and not individually.

I reckon what the real Model 3 tilegen chip is doing is using the mask table and the scrolling table/registers to determine the 64 tiles (or 32 tile pairs) it should load for each line. Suppose that the "ROLLING START" banner is scrolled left by 264 pixels; the calculated mask for the non-patched ROM would result in drawing 256 pixels from the plane A window layer (layer A'), followed by 256 pixels from the plane A background layer (layer A) which is blank. The tilegen calculates an offset of 16 tile pairs (264 / 16 = 16.5, rounded down to 16), so it would load tiles #33 to #64 from the window layer and would not load tile #1 at all; instead it would load tiles #33 and #34 from the background layer, followed by #35 and #36 and so on up to tiles #63 and #64.

Because there is a further 8-pixel offset not yet accounted for, the tilegen has to scroll left by another 8 pixels which means that tile #33 from the window layer is not drawn (as it would be outside the display area) and it draws tiles #34 to #64 - so far 248 pixels have been drawn. It then draws tile #33 from the background layer as this is the next tile in the list, even though the mask specifies that it should switch from the window layer to the background layer 8 pixels later. After drawing tile #33 from the background layer it checks the mask, reloads the scrolling value and then draws tile #33 AGAIN before drawing the rest of the tiles as normal.

This is my best guess as to how the scrolling glitch does not occur on the actual Model 3 tilegen chip. If no one else can come up with a better theory, we basically have three choices from this point:

1. Leave the tilegen code alone and keep the patch that I made; not a particularly elegant solution but it works.
2. Remove the patch and reimplement the tilegen code to follow the behavior described above; we're basically guessing that this is how the real hardware works. (Need to test to make sure it doesn't break other games.)
3. Run a test on real hardware to check if my theory is correct; making the last four tiles in each line of the plane A window layer non-blank would be enough to verify.
gm_matthew
 
Posts: 224
Joined: Fri Oct 07, 2011 7:29 am
Location: Bristol, UK

Re: Tilegen inside a fragment shader

Postby Bart » Tue Sep 12, 2023 5:17 pm

gm_matthew wrote:
Ian wrote:Because the nametable is packed like
Because there is a further 8-pixel offset not yet accounted for, the tilegen has to scroll left by another 8 pixels which means that tile #33 from the window layer is not drawn (as it would be outside the display area) and it draws tiles #34 to #64 - so far 248 pixels have been drawn. It then draws tile #33 from the background layer as this is the next tile in the list, even though the mask specifies that it should switch from the window layer to the background layer 8 pixels later. After drawing tile #33 from the background layer it checks the mask, reloads the scrolling value and then draws tile #33 AGAIN before drawing the rest of the tiles as normal.


It's possible that something like this is happening. Not sure I completely follow the logic. But, it seems clear that the window mask exists because the tilegen cannot fetch 4 patterns or 4 name table entries at a time to render 4 true layers (otherwise they would just do that). That suggests that the window mask is pre-loaded at the start of a line (during the HBL period) and during active display, is used to control which name table entry is read for the next set of 16 pixels (IIRC, each mask bit corresponds to 2 tiles or 16 pixels).

The window is not supposed to be affected by scrolling but it must be performing some sort of adjustment for scrolling to know what to fetch. I wonder if in fact a coarse scrolling value (hscroll & ~15, for the 16 pixel window "column") is used to assess whether the next two tiles will be A or A', which could have the effect of the window actually appearing to scroll right for up to 16 pixels when the layer scrolls left. This would mask out the right side but could eat into the left side. I don't remember how wide the rolling start mask actually is.

1. Leave the tilegen code alone and keep the patch that I made; not a particularly elegant solution but it works.
2. Remove the patch and reimplement the tilegen code to follow the behavior described above; we're basically guessing that this is how the real hardware works. (Need to test to make sure it doesn't break other games.)
3. Run a test on real hardware to check if my theory is correct; making the last four tiles in each line of the plane A window layer non-blank would be enough to verify.


I think getting to the bottom of this would be great. It's a real pain for me to run code on my VF3 board but I could do it. I really wish I had invested time in devising a PC link system to load stuff into RAM. Maybe I should?

This could be done entirely in C. My Model 3 repo has a C program that measures CPU clock and frame timing and prints to the screen. Trivial to write a C program that could run a bunch of scenarios with the window mask. I'm going on vacation for two weeks tomorrow and only bringing my Mac, so I won't have my PowerPC tool chain available but I'll see if I can get access to a PC if needed. Could be a fun project.
User avatar
Bart
Site Admin
 
Posts: 3086
Joined: Thu Sep 01, 2011 2:13 pm
Location: Reno, Nevada

Re: Tilegen inside a fragment shader

Postby gm_matthew » Fri Sep 15, 2023 4:39 am

I found this note about a bug in the Mega Drive/Genesis VDP when switching from the window to plane A mid-line when the latter is scrolled:

Code: Select all
There is a bug in the window processing. This occurs when the window is
showing partially on the left side of the screen, specifically when
the VDP is drawing the 2-cell column from plane A that immediately proceeds
the last column the window was drawn on. (i.e. WHP+1)

If the lower four bits of the horizontal scroll value for the current
scan line are zero, then the name table attribute data for the current
column are fetched correctly.

If the lower four bits of the horizontal scroll value for the current
scan line are nonzero, the name table attribute data are fetched from
next column. (WHP+2)

In effect, you'll have N columns of the window plane, 1 column that has
identical patterns as the next column, then the remainder of the display
is drawn correctly.

Here's a diagram to illustrate this:
w   = window tiles
abc = tile columns

D3-D0 of scroll value == 0
wwwwwwwwwwwwwwwwaabbccddeeffgghh

D3-D0 of scroll value != 0
wwwwwwwwwwwwwwwwbbbbccddeeffgghh

It makes me wonder if a similar thing is happening on the Model 3 tilegen, only in reverse: it's switching from a scrolling layer to a non-scrolling layer, but it doesn't draw the last tile of the scrolling layer, similar to how the Mega Drive/Genesis VDP doesn't draw the first column.

Also worth noting that positive horizontal scroll values on the VDP (and also System 24) move the plane towards the right, whereas on Model 3 tilegen they move the plane towards the left.
gm_matthew
 
Posts: 224
Joined: Fri Oct 07, 2011 7:29 am
Location: Bristol, UK

Re: Tilegen inside a fragment shader

Postby Ian » Mon Sep 18, 2023 4:21 am

Sounds like a very similar bug. It wouldn't surprise me if all the sega h/w from this era basically worked off a similar design
Ian
 
Posts: 2044
Joined: Tue Feb 23, 2016 9:23 am

Previous

Return to The Dark Room

Who is online

Users browsing this forum: No registered users and 2 guests