This page is a mirror of Tepples' nesdev forum mirror (URL TBD).
Last updated on Oct-18-2019 Download

Nine sprites overflow doesn't work in the beginning

Nine sprites overflow doesn't work in the beginning
by on (#193184)
I found some strange behavior:

When you put nine sprites on a scanline and check for the sprite overflow flag (bit 5 in register $2002) during rendering, then it tends not to work if you do it too early in the game.

I.e. the game freezes at startup, but it works after a reset.

The freezing only happens on a real cartridge. Not in emulators and not on a PowerPak.

It only happens at startup.

If the game is already playing for some seconds and you only check for the sprite overflow then, it never freezes.
I.e. it doesn't spontaneously happen in the middle of the game. Seems to have more to do with the PPU not being ready yet.

(That's probably also the reason why you never see this on a PowerPak: Once you navigated through the menu to get to your game, the time when this freeze can happen has long gone.)

However, the same issue does not happen if you check for the sprite 0 flag instead of the sprite overflow flag, even if you keep everything else the same.
(Don't forget to put a non-transparent background tile at the location of sprite 0 in this case, so that it works at all.)

I put a sample source code and the ROM that demonstrates the behavior into the attachments.


The issue was originally discussed here: viewtopic.php?f=2&t=15737

But this is not a problem regarding my game anymore.
(I simply skipped the oveflow check when I'm in text screens and only checked for it during gameplay.)

Instead, this is about general analysis of the NES' behavior.
If I haven't made an obvious mistake, this is maybe actually undocumented behavior that belongs to the NESDev wiki.

Also, the last thread was more about shots in the dark (trying this, trying that) while the current one is about the definite source of the freeze, with a test ROM and code.

So, we don't need to speculate whether the sprites in the PPU are properly set etc.
If you have an idea what might be wrong, you can just have a look at the source code of the minimalistic test ROM and see for yourself whether I actually made the mistake you're suspecting.

That's why I decided this warrants a new thread, so that people don't have to scroll through all the other stuff from the old thread that turned out not to be the issue.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193185)
Source code for quick view:

Code:
.segment "HEADER"

   .byte "NES", $1A
   .byte 1
   .byte 1
   .byte 1


.segment "ZEROPAGE"

   Pointer: .res 2
   WaitForNmi: .res 1
   Counter: .res 1

.segment "SPRITES"

   Sprites: .res $100

.segment "CODE"

Reset:

   ; Initialization
   SEI
   CLD
   LDX #$40
   STX $4017
   LDX #$FF
   TXS
   INX
   STX $2000
   STX $2001
   STX $4010

   ; VBlank 1
   BIT $2002
@waitForVBlank1:
   BIT $2002
   BPL @waitForVBlank1

   ; RAM to 0
   TXA
   TAY
   STA Pointer
   STA Pointer + 1
@initializeRamOuterLoop:
@initializeRamInnerLoop:
   LDA #0
   STA (Pointer), Y
   INY
   BNE @initializeRamInnerLoop
   INC Pointer + 1
   INX
   CPX #$08
   BNE @initializeRamOuterLoop

   ; VBlank 2
   BIT $2002
@waitForVBlank2:
   BIT $2002
   BPL @waitForVBlank2

   ; Sprites out of screen
   LDX #0
   LDA #$F4
@spritesLoop:
   STA Sprites, X
   INX
   BNE @spritesLoop

   ; Nine sprites (not sprite 0) in one row
   LDX #4
@nineSpritesLoop:
   LDA #88
   STA Sprites, X
   INX
   LDA #$0F
   STA Sprites, X
   INX
   LDA #0
   STA Sprites, X
   INX
   STA Sprites, X
   INX
   CPX #40
   BNE @nineSpritesLoop


   ; Sprites for counter
   LDA #120
   STA Sprites + 40
   STA Sprites + 43
   STA Sprites + 44
   LDA #0
   STA Sprites + 42
   STA Sprites + 46
   LDA #120 + 8
   STA Sprites + 47

   ; First thing to run: NMI
   LDA #1
   STA WaitForNmi

   ; NMI enabled
   LDA #%10010000
   STA $2000

   ; Game logic
@gameLogic:

   LDA WaitForNmi
   BNE @gameLogicEnd

   ; Wait for sprite overflow bit
@waitForNotOverflow:
   LDA $2002
   AND #%00100000
   BNE @waitForNotOverflow
@waitForOverflow:
   LDA $2002
   AND #%00100000
   BEQ @waitForOverflow

   ; Update counter on screen
   LDA Counter
   LSR
   LSR
   LSR
   LSR
   STA Sprites + 41
   LDA Counter
   AND #%00001111
   STA Sprites + 45

   INC Counter

   ; Wait for NMI
   LDA #1
   STA WaitForNmi

@gameLogicEnd:

   JMP @gameLogic


Nmi:

   PHA
   TXA
   PHA
   TYA
   PHA

   LDA WaitForNmi
   BEQ @end

   LDA #0
   STA WaitForNmi

   ; PPUMASK enabled
   LDA #%00011110
   STA $2001

   ; Sprite OAMDMA
   LDA #$00
   STA $2003
   LDA #>Sprites
   STA $4014

   ; Palette
   LDA $2002
   LDA #$3F
   STA $2006
   LDA #$11
   STA $2006
   LDA #$0D
   STA $2007
   LDA #$05
   STA $2007
   LDA #$02
   STA $2007

   ; Scrolling
   LDA #0
   STA $2005
   STA $2005

   ; Name table
   LDA #%10010000
   STA $2000

@end:

   PLA
   TAY
   PLA
   TAX
   PLA

   RTI


.segment "VECTORS"

   .word Nmi
   .word Reset
   .word 0


.segment "CHARS"

   .incbin "Graphics.chr"



Config file:

Code:
MEMORY
{
   HEADER:  type = ro, start = $0000, size = $0010, file = %O, fill = yes;
   PRG_ROM: type = ro, start = $C000, size = $4000, file = %O, fill = yes;
   CHR_ROM: type = ro, start = $0000, size = $2000, file = %O, fill = yes;
   ZP:      type = rw, start = $0001, size = $00FF, file = "";
   RAM:     type = rw, start = $0200, size = $0400, file = "";
}

SEGMENTS
{
   HEADER:   load = HEADER,  type = ro;
   CODE:     load = PRG_ROM, type = ro;
   RODATA:   load = PRG_ROM, type = ro;
   VECTORS:  load = PRG_ROM, type = ro,  start = $FFFA;
   CHARS:    load = CHR_ROM, type = ro;
   ZEROPAGE: load = ZP,      type = zp;
   SPRITES:  load = RAM,     type = bss, align = $0100;
   BSS:      load = RAM,     type = bss;
}
Re: Nine sprites overflow doesn't work in the beginning
by on (#193193)
Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize? Hopefully someone will be able to test this properly.

BTW, if you're using the sprite overflow to time raster effects, you may want to make the wait loop tighter in order to minimize jitter:
Code:
   LDA #%00100000
@waitForOverflow:
   BIT $2002
   BEQ @waitForOverflow
Re: Nine sprites overflow doesn't work in the beginning
by on (#193197)
Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame:
Code:
   LDA #%10100000
@waitForOverflow:
   BIT $2002
   BEQ @waitForOverflow
Re: Nine sprites overflow doesn't work in the beginning
by on (#193200)
tokumaru wrote:
Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize?

That's probably what it leads up to.

tokumaru wrote:
BTW, if you're using the sprite overflow to time raster effects, you may want to make the wait loop tighter in order to minimize jitter

Yeah, I've heard this somewhere.

That one only works for this specific bit, right? It wouldn't work for sprite 0?

In my actual game code, I used the "wait for certain $2002 bit" functionality as a macro, so that I only had to pass the actual value since I called this both, for sprite overflow and sprite 0 on two different locations on the screen.

Also, in my specific case, this didn't make a practical difference. The place where the scrolling change happened was a scanline where the whole line consists of one continuous color. (Top: Status bar. Middle: Fog.) So, graphical artifacts like in "Journey to Silius" were a non-issue.

But yeah, in general, this might be a nice tip.

tepples wrote:
Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193204)
DRW wrote:
It wouldn't work for sprite 0?

For sprite 0 hits you can use BVC, just like you use BPL for the vblank flag. There's no need to bother setting up a mask, since BIT copied bits 7 and 6 to the N and V flags respectively, so you can just check the V flag. But if you want to keep using a generic macro, there's nothing stopping you from using %01000000 as a mask.

One way to wait for the sprite 0 hit:
Code:
WaitForSpriteHit:
  bit $2002
  bvc WaitForSpriteHit

Another way to do it, and the loop is just as fast:
Code:
  lda #%01000000
WaitForSpriteHit:
  bit $2002
  beq WaitForSpriteHit
Re: Nine sprites overflow doesn't work in the beginning
by on (#193207)
DRW wrote:
tokumaru wrote:
Could this mean that it takes longer than the typical PPU warm up time for sprite behavior to stabilize?

That's probably what it leads up to.

In particular, I'm under the impression that some of the OAM DRAM refresh circuitry doesn't stabilize until one frame has been completely rendered, or at least one scanline. A test ROM similar to OAM reset might help characterize this.

DRW wrote:
tepples wrote:
Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.

Some mistakes cannot be fixed; they must be worked around. One example of such a mistake is a mistake in the design of OAM DRAM refresh. Time is money, and it might take less time for you to put a workaround like this in place and get your game out the door than to wait until this misbehavior is nailed down.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193210)
tepples wrote:
Some mistakes cannot be fixed; they must be worked around.

Well, I did:

In my game, text screens don't actually need any scanline splits, only the level does. In my old version, the sprites for the split were still there, though.

So, I simply changed it to the following:
When we initialize a text screen, we put the 10 "system sprites" out of the screen. Only in levels are they set to their correct location.

At the code where the overflow and sprite 0 bit is waited for, I simply check: If address of Sprites + 0 = $F4 (i.e. if the y position of sprite 0 is out of the screen), then we skip the scanline split waits entirely.
I had to do a similar thing anyway, even in the old version: If PPUMASK = 0, then skip them as well.
So, the new version simply has one more check.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193216)
DRW wrote:
tepples wrote:
Or better yet, include the vblank flag so that it'll fail safe even if the sprite overflow doesn't turn on in a given frame

I would advise against this. If the flag doesn't turn on, there's some mistake and the code needs correction. I wouldn't want to hide this.

There isn't really a danger that it will "hide" the problem because it still produces a visual corruption of the screen from missing the split point.

It does, however, make the difference between a total hang/crash and just some temporary visual problems and/or slowdown until the issue resolves. As a player, if I've been playing something for an hour I'd much prefer a few seconds of glitchy graphics that I can recover from than a hard crash I have to completely restart from.

The PPU isn't totally reliable even in the best of conditions. Vibrations, power fluctiuations, dirt in the pins, etc. can and do cause occasional CHR corruptions in otherwise "bug free" games. I test for vblank in my sprite-0 waits myself for this reason- I don't trust the hardware completely.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193248)
O.k., yeah, this is a good argument.

However, in this specific thread, the optimal way to wait for sprite 0 or failsafe ways to wait for scanline splits is really not my focus.
As I said, in my actual game, the issue is already circumvented by simply not checking for sprite overflow when you are in text screens.
The publisher already has the new ROM, so I won't work on the game anymore.


Instead, this thread was not created out of an immediate problem, but because I'm genuinely interested in the actual technical cause of this behavior.

Because this might be a discovery about the NES' PPU that isn't documented yet, so maybe people who have more knowledge than me and who own the required equipemt can analyze the issue.


The sprite overflow fails to be set at an early point in the game, unless you use a soft reset.

If you check for sprite 0 in exactly the same way (plus putting an opaque tile at the required location, of course), this does not fail.
It's only the overflow bit, not the sprite 0 bit.

Why is this the case?

Is there an oversight in my code (see above) or is this some wiki-worthy piece of information about the PPU that hasn't been discovered yet because nobody ever relied on the sprite overflow bit being set right at the start?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193259)
DRW wrote:
If you check for sprite 0 in exactly the same way (plus putting an opaque tile at the required location, of course), this does not fail.
It's only the overflow bit, not the sprite 0 bit.

Are you sure that the sprite 0 hit *never* fails? Or could it simply be that OAM corruption is more likely to affect 1 out of 9 sprites than 1 out of 1? What about the overflow, does it *always* fail on cold boot?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193264)
If the behavior is that the display list entries for sprites 0 and 1 overwrite the display list entries for another randomly chosen even-odd pair of sprites, then allow me to describe a situation in which sprite overflow fails.

Say sprites 1-9 are for the overflow trigger, and sprite 0 is at least 16 pixels below sprites 1-9. If sprites 0 and 1 overwrite sprites 2 and 3, 4 and 5, 6 and 7, or 8 and 9, then you'll have only eight sprites on that line. But because sprite 0 is not overwritten, it will still be displayed.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193270)
How can we reliably test this to confirm it as a new discovery?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193273)
For the purposes of the test, maybe give the 9 sprites numbered tiles and spread them out horizontally so all of them can be seen individually. You might be able to see which one has failed this way when it hangs.

Edit: have made this, is attached.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193282)
tokumaru wrote:
Are you sure that the sprite 0 hit *never* fails?

When I had the problem, I asked my publisher to check some ROM versions that I created out of my game. In one of them, I disabled the overflow check, in one I disabled the sprite 0 check.

Disabling sprite 0 check still produced the error. (Because the overflow is the problem.)
Disabling overflow check, but keeping sprite 0 check, worked correcly.

Of course, I can't be sure. I assume he didn't test it 50 times. This would have to be done by someone who has the equipment, time and motivation.

But since sprite 0 check is pretty common in actual games (I assume "Super Mario Bros." also has it right at the beginning, doesn't it?) while no game really checks for overflow right at the start, this result is actually believable to me.

tokumaru wrote:
What about the overflow, does it *always* fail on cold boot?

Mostly, but not always. Just like I also had rare instances of my game still not working after a reset.

One interesting thing about this: The publisher sent me two cartridges of my game. One of them is more likely to fail at cold boot than the other one. One of them actually had a pretty high success rate (maybe 30-50 %) compared to the other one that almost never worked at startup.
So, the frequency of the error occuring also seems to be related to the individual boards.

Also:
No issue whatsoever with the same cartridges on a PAL system. (I tested this myself. I own an Amercian and a European NES.)
And a clone system that the publisher owns doesn't produce the behavior either.
Likewise, the problem occurs regardless of whether you use a front loader or top loader NES. (I have the front loader. And some person from NintendoAge confirmed me the bug on a top loader.)

tepples wrote:
Say sprites 1-9 are for the overflow trigger, and sprite 0 is at least 16 pixels below sprites 1-9. If sprites 0 and 1 overwrite sprites 2 and 3, 4 and 5, 6 and 7, or 8 and 9, then you'll have only eight sprites on that line. But because sprite 0 is not overwritten, it will still be displayed.

Huh? I'm not sure whether I get this. What do you mean with "overwriting"?
From a pure code logic, I can guarantee you that nothing overwrites anything. I reserved the first 10 sprites for this specific purpose. Neither in my game, nor in the above test code are they ever used for anything else but for the scanline splits, and sprite 0 is never part of the overflow sprites.
If you are referring to some internal NES-specific behavior: Maybe, maybe not. I don't know

By the way, is there a way to set the PowerPak into a mode that a game is already loaded at startup instead of the PowerPak going into its own menu first?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193283)
OAM is organized as 36 words of DRAM, each 58 bits in length. Each 58-bit word corresponds to an even-odd pair of display list entries: sprites 0 and 1, sprites 2 and 3, sprites 4 and 5, ..., sprites 62 and 63, secondary OAM entries 0 and 1, 2 and 3, 4 and 5, and 6 and 7.

Because merely reading a word of DRAM causes it to decay, the refresh circuitry reads a word and writing it back to the same address. But if the address it reads and the address it writes back differ, the data at the address it writes back is lost. For example, it normally refreshes the word for sprites 0 and 1 by reading the word for sprites 0 and 1 and writing it back to the word for sprites 0 and 1. But before it has stabilized, it may mistakenly read the word for sprites 0 and 1 and write it back to, say, the word for sprites 6 and 7. This means instead of sprite 0 being on one set of scanlines and sprites 1, 2, 3, 4, 5, 6, 7, 8, and 9 being on another set of scanlines, you end up with sprites 0 and 6 being on set of scanlines and sprites 1, 2, 3, 4, 5, 7, 8, and 9 being on another set of scanlines, because the data for sprite 6 was overwritten with the data for sprite 0.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193285)
Dug out my EPROM writer today and tested this on both my NES and my Famicom. I can't seem to reproduce the problem, it runs fine on both. Tried power off for various lengths of time, tried both the original ROM and the slightly modified one I just made.

So... I dunno. Only affects some systems, I guess?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193286)
@tepples:

Thanks for the explanation.

Can this problem happen anywhere or does it only happen at the start?

Because I never encountered an issue where the game randomly crashed during gameplay. It's only either at startup or not at all.

So, if the decaying and the writing back at incorrect locations can happen anytime, this is probably not the issue at hand. Otherwise, I think the bug would have occured many times in all possible situations. Because in my actual game, during a level, the game checks for sprite overflow and requires a positive result in every single frame.

If it's an issue related specifically to startup:
I guess this can be tested by simply putting all sprites on the same scanline instead of just nine and see whether the bug still happens.

rainwarrior wrote:
Dug out my EPROM writer today and tested this on both my NES and my Famicom. I can't seem to reproduce the problem, it runs fine on both. Tried power off for various lengths of time, tried both the original ROM and the slightly modified one I just made.

Well, for the test ROM, I had to take the publisher's word for it that the error still occurs. I couldn't try it out myself.
If you promise me not to give it to anybody, I can send you the old ROM of "City Trouble", i.e. the game that I can confirm for myself has the error, so that you can see whether the test ROM from above actually doesn't have the problem anymore and the publisher told me the wrong thing or whether your game board is simply one of the more tolerant ones where the error doesn't occur as frequently.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193288)
DRW wrote:
rainwarrior wrote:
Dug out my EPROM writer today and tested this on both my NES and my Famicom. I can't seem to reproduce the problem, it runs fine on both. Tried power off for various lengths of time, tried both the original ROM and the slightly modified one I just made.

Well, for the test ROM, I had to take the publisher's word for it that the error still occurs. I couldn't try it out myself.
If you promise me not to give it to anybody, I can send you the old ROM of "City Trouble", i.e. the game that I can confirm for myself has the error, so that you can see whether the test ROM from above actually doesn't have the problem anymore and the publisher told me the wrong thing or whether your game board is simply one of the more tolerant ones where the error doesn't occur as frequently.

If you want to send me a ROM I'd be willing to test it. The board I'm using is an old RetroUSB ReproPak with old UV eproms, pretty minimal, nothing "weird" I don't think.

Did your publisher test the test ROMs or just the game? (I'm just wondering if anyone has independently verified the test ROM to hang in the way you expect.)

I'm a little disappointed nothing unusual happened with the test ROMs for me. ;) I was hoping to learn something new.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193290)
The publisher told me that he also tested the test ROM and that it produced the same error. But I have no way of confirming this.

If there's a way to set the Power Pak to "load a specific game immediately and skip your own menu", I could see whether all of this can be reproduced with a PowerPak.

I'm sending you the "City Trouble" ROM in a few minutes.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193291)
DRW wrote:
If there's a way to set the Power Pak to "load a specific game immediately and skip your own menu", I could see whether all of this can be reproduced with a PowerPak.

I believe some people have tried replacing the menu software to automatically boot a game at power-on, but it still needs to program the FPGA and copy the game from the CF card to its RAM. There's no non-volatile storage available to execute the game's code immediately.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193292)
Having been given the City Trouble test ROM, it has no problem starting up on my NES or Famicom.

So... it's at least consistent with the test ROMs for me in that it's a "negative" result. So... I dunno, all I can say is the problem may not extend to all NES units, or maybe not to all boards? I dunno.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193294)
That was the result I was hoping for. If "City Trouble" had frozen, but the test ROM not, then the publisher's test would have been false and the posted test code wouldn't actually have included the issue to analyze.

Memblers was able to recreate the issue with "City Trouble". Maybe he could check the test ROM. I still have one message to him that he hasn't replied to yet. When he gets back to me, I can ask him.

Besides, I assume it's more "the issue doesn't appear on all boards" instead of "the issue doesn't appear on all NES units". I'm pretty sure if you tried my faulty cartridge of "City Trouble" on your NES, you would see the issue because the two cartridges already produced different results on the same console.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193298)
The other thing I'm noticing is that in your City Trouble game the first $4014 write occurs like 10 frames after power on. Even in the test example, it's a good 4 frames before the test is done.

Like, maybe this has nothing at all to do with "PPU warm up", or any period of time really, but instead just what happens on the first frame with rendering on. Garbage left in the sprite system at power on that normally gets flushed out on the first rendered frame? (This is probably the first time anyone has ever tried to use the sprite overflow flag on the first frame?)

You mentioned that PAL is not subject to this, which I think has been established always does some sort of sprite refresh even when rendering is off?

If you can, get my variant ROM posted above tested on a system where it fails, and see if there's any visual indication of and of the 9 sprites missing?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193299)
rainwarrior wrote:
Like, maybe this has nothing at all to do with "PPU warm up", or any period of time really, but instead just what happens on the first frame with rendering on. Garbage left in the sprite system at power on that normally gets flushed out on the first rendered frame?

I'm not sure whether this can be the issue, in case I'm understanding you correctly.

Have a look:
Code:
   ; First thing to run: NMI
   LDA #1
   STA WaitForNmi

   ; NMI enabled
   LDA #%10010000
   STA $2000

   ; Game logic
@gameLogic:

   LDA WaitForNmi
   BNE @gameLogicEnd

Before enabling NMI, I set the boolean WaitForNmi to 1 (true). Then I enable the NMI.
As long as WaitForNmi is 1, the game logic (which includes the overflow check) is skipped completely.
I.e. the game logic doesn't start until NMI has actually run for the first time.

And in NMI, the PPUMASK is set and the sprites are always updated anyway:
Code:
   ; PPUMASK enabled
   LDA #%00011110
   STA $2001

   ; Sprite OAMDMA
   LDA #$00
   STA $2003
   LDA #>Sprites
   STA $4014

So, as far as I see, the overflow check cannot suffer from any garbage data because it is guaranteed that all sprites are written to OAMDMA before the overflow check is done.

NMI, not the game logic, is always the first thing that gets executed due to the WaitForNmi variable being initialized with 1 and the game logic checking:
if WaitForNmi != 0 then goto gameLogicEnd

Does this disprove your assumption or is there anything that I overlooked?

rainwarrior wrote:
If you can, get my variant ROM posted above tested on a system where it fails, and see if there's any visual indication of and of the 9 sprites missing?

I can ask my publisher to do this, sure.
Although I was hoping that someone on the forum has the equipment to do quick tests. The publisher always takes a while until he gets to it.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193300)
DRW wrote:
rainwarrior wrote:
Like, maybe this has nothing at all to do with "PPU warm up", or any period of time really, but instead just what happens on the first frame with rendering on. Garbage left in the sprite system at power on that normally gets flushed out on the first rendered frame?

...
Does this disprove your assumption or is there anything that I overlooked?

I wasn't talking about garbage in the OAM data. I was talking about some internal state of the sprite system that somehow causes sprite overflow to fail on the first frame, something you can't actually initialize via code, but has to be flushed by the process of rendering a frame.

I'm reminded of how turning off rendering at an unlucky point during the visible frame will corrupt the appearance of sprites on the next frame. Maybe the power-on state of the sprite system is in a similar way.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193301)
That's maybe possible, but of course I have no way of confirming or disproving this. How can such a thing reliably be validated?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193302)
I noticed similar strange behavior after pressing the Reset button during development of Concentration Room: two sprites were missing from the logo of the Party (since replaced with that of Hombon). This was an early version that cold-opened on the story, from before I added the title screen.

Try OAM reset, then press Reset a few times to see what drops out. If sprites 02-09 ever drop out, then there's a chance of misbehavior of the sprite overflow flag.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193303)
tepples wrote:
Try OAM reset, then press Reset a few times to see what drops out. If sprites 02-09 ever drop out, then there's a chance of misbehavior of the sprite overflow flag.

This ROM seems to do the OAM DMA around scanline 80 of the third frame, and then finally enables rendering via $2001 maybe 12ms later. That interval by itself should be subject to OAM decay. Is it supposed to test anything else besides that?

I don't think OAM decay is the issue at hand... shouldn't any test we do make sure to rule it out? (i.e. it would be a very good test if it executed OAM DMA during vblank along with the initial $2001 enable)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193304)
Here's a modification of tepples' OAM reset ROM to do what I said (i.e. wait for vblank and synchronize OAM turning on and first frame of rendering, eliminating OAM decay)

There's a huge difference in behaviour for the two of them, for me:

1. In this new patched version, on some resets exactly two consecutive sprites are sometimes missing (seemingly randomly selected; sometimes none are missing).

2. In the original, on each reset, often two consecutive ones are missing, sometimes others are missing in addition to that, and sometimes the whole set of sprites is messed up.

So I think in the original we had two things happening together, one being OAM decay (from that long interval before rendering is turned on) on top of the corruption due to being reset in the middle of sprite evaluation? In the patched version I think it's demonstrating just the corruption from reset.

...and what this might suggest is that the first frame of sprites are not 100% reliable after reset, and I think on some systems it might have the same problem on power-on (i.e. having a somewhat consistent power-on state for that system that reliably corrupts in the same way, but unique to that system)?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193305)
...and testing again City Trouble with this knowledge, I note that I can crash it on startup via reset. Even though it is consistently safe at power on for me, reset is not. (I was not testing reset before.)

So I think on some systems, the power on state will give a ~100% chance of failure of the test, and on other systems (like mine) it's ~100% chance of success.

Reset, on the other hand seems to have like a ~10% chance of failure, now that I'm looking at it! (This probability is likely consistent across a lot of systems.)

I think this is actually a sufficient explanation for the problem!
Re: Nine sprites overflow doesn't work in the beginning
by on (#193307)
So, the rule is: Don't do anything important with sprites during the first frame?

Besides, what does the altered test ROM (where the nine sprites are laid out and numbered) show if you crash it via reset?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193308)
DRW wrote:
So, the rule is: Don't do anything important with sprites during the first frame?

Besides, what does the altered test ROM (where the nine sprites are laid out and numbered) show if you crash it via reset?

It will show 2 consecutive sprites missing from the overflow line. (The oam_reset2.nes ROM demonstrates this better though, since it shows the whole OAM not just the ones used for that overflow test.)

You can actually do the reset test on PowerPak too. I can crash your City Trouble test ROM via reset on my PowerPak. (PowerPak is perfectly fine for RESET testing, just not POWER.)

After burning an EPROM of oam_reset2.nes to test the power on state, on my NES it comes up clean on every power-on, but I'd say 50% or so of resets have 2 consecutive sprites missing. (The 2 consecutive sprites always begin on an even number, I tihnk. I neglected to mention this before.)

My Famicom also always has a clean power-on oam_reset2.nes but it seems interesting to note that it is still subject to the reset problem despite the PPU not getting a reset? That one surprises me. Why does this still happen on the Famicom? (The 2-sprite reset corruptions seem less frequent, I get clean resets more often, I think, but they still happen regularly, like maybe 25% of the time instead of 50%.)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193309)
rainwarrior wrote:
My Famicom also always has a clean power-on oam_reset2.nes but it seems interesting to note that it is still subject to the reset problem despite the PPU not getting a reset?

Because the PPU gets a mid-scanline LDX #0 STX $2001, which may have the same effect on the sprite evaluation and refresh state machine as a reset.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193310)
Aha! That explains it. :)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193311)
So if you wait for vblank on the famicom before disabling rendering (or just don't disable it at all) you shouldn't get any glitches after resetting, right?
Re: Nine sprites overflow doesn't work in the beginning
by on (#193312)
tokumaru wrote:
So if you wait for vblank on the famicom before disabling rendering (or just don't disable it at all) you shouldn't get any glitches after resetting, right?

Yes, this appears to be correct. (With oam_reset3.nes no sprite deletions appear to occur on reset for my Famicom, and sprite deletions definitely still occur on my NES.)


So, the thing I can't verify with my systems is whether other systems have consistent corruption on power-on. It would be useful if people could run (non PowerPak) oam_reset2.nes on various machines to see their power-on state. (Especially DRW's publisher's machine.)

tepples, you seemed to know more precisely what's happening? You mentioned corruption of specific 8 byte blocks earlier. (Do the blocks get filled with all $FF? I never seem to see stray/random sprites, just deleted ones...?)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193314)
One line of RAM (i.e. sprite pair) should get copied over another ... unfortunately there seems to be no way (other than running out of sprites) to tell when two slots have the exact same data.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193315)
lidnariq wrote:
One line of RAM (i.e. sprite pair) should get copied over another ... unfortunately there seems to be no way (other than running out of sprites) to tell when two slots have the exact same data.

Hm, with oam_reset2.nes I always see exactly two sprites missing, never more.

If a low number sprite had been copied over a later one, there should be additional missing sprites from 8-per-line dropout, right? I'm not seeing this, though.

So... does this suggest that the sprites are always copied from somewhere in 56-63? Or possibly are always copied from somewhere in the same 32-byte "quad-line"? Or... I don't know how to explain this.

I've seen missing sprite pairs as low as 02-03 and as high as 62-63... still haven't caught 00-01, if that's even possible, but I think I've probably seen them everywhere else.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193317)
Oh wait, this means it's always copying from 00-01 to the "missing" sprites, doesn't it.

(Unsure if that's because the program loads $2003 with 0 before the DMA? Can it copy from other locations?)


Edit: added another variation ROM that verifies/demonstrates that there are extra sprites on 0/1 with a diagonal organization of the sprites. In this example the top right corner 63 will always drop out from the extra 00, and the bottom left corner 56 will usually drop out (unless the copied sprite is 57+).

Sorta funny, but I guess that means if DRW had used sprite 0-8 instead of 1-9 for overflow this problem wouldn't have come up. ;)

(I hope tepples doesn't mind that I've patched his ROM half a dozen times tonight.)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193320)
rainwarrior wrote:
You can actually do the reset test on PowerPak too. I can crash your City Trouble test ROM via reset on my PowerPak. (PowerPak is perfectly fine for RESET testing, just not POWER.)

Since on my NES "City Trouble" only had problems during startup, but not during reset (except for some rare occasions), I assume the test ROM would be the same.
Unfortunately, I have borrowed the PowerPak away to my colleague, so I cannot test it in the moment anyway.

rainwarrior wrote:
So, the thing I can't verify with my systems is whether other systems have consistent corruption on power-on. It would be useful if people could run (non PowerPak) oam_reset2.nes on various machines to see their power-on state. (Especially DRW's publisher's machine.)

I can send him the ROM, but I don't think that's necessary because I assume we already have the answer:
The old ROM of "City Trouble" freezes on my machine, on the publisher's machine, on Memblers' machine and on one or two confirmed machines from people on the internet when they user power on.
"City Trouble" always relied on the nine overflow sprites being there.
Therefore, "City Trouble" freezing means that missing sprites occur on these persons' machines as well.

(The only interesting thing might be whether these are always the same sprites that disappear, but I don't know whether I should bother my publisher with such an issue. After all, this question which sprites disappear is too general-purpose now and not really tied to our specific game and original problem anymore.
I think Memblers would be the better tester here.)

Your machine is the exception so far since it's the only one that was tested where power on works, but not reset.

tokumaru wrote:
So if you wait for vblank on the famicom before disabling rendering (or just don't disable it at all) you shouldn't get any glitches after resetting, right?

When I read various wiki entries, I was told that rendering should always be enabled and disabled in NMI anyway, so that the game doesn't start drawing mid-frame.

So, I'm surprised mid-frame writing of PPUMASK to enable and disable rendering actually came up here.

In my program, I always have a buffer value named PpuMaskValue that gets the corresponding rendering status. And then in NMI, I set PpuMaskValue to $2001. I never write $2001 anywhere else (except right in the beginning of course, when it is initialized with 0), I always use the buffer variable.

rainwarrior wrote:
Sorta funny, but I guess that means if DRW had used sprite 0-8 instead of 1-9 for overflow this problem wouldn't have come up. ;)

In reality, i.e. in "City Trouble", this is not just a funny oversight where I can say: "Dammit, why didn't I simply use sprite 0-8?"

Using sprites 0-8 would have never happened.

Because the only reason why I use the overflow check at all is to have an additional scanline split, apart from sprite 0 split.

In the test program, I only did the overflow check, but in "City Trouble", both checks are used: Overflow for the status bar and sprite 0 for the parallax scrolling.

So, it would have always been 1-9 because sprite 0 has its own separate purpose.
Therefore, running into the error was pretty much unavoidable for me.

In the old "City Trouble" ROM, I just kept the 10 sprites and the bit checks on the text screens for consistency's sake. So, the text screens also had those two checks, despite not needing them since text screens have no scrolling.

That's why the bug was easy to fix once I knew what caused it:

Put the 10 sprites out of the screen whenever a text screen is initailized and only set it to the correct locations when a level is initialized.

And before each of the two checks, make sure that the variable at address Sprites + 0 does not have the value $F4 (my chosen out-of-screen y position value), otherwise skip the check.

Since the game always starts with a text screen (the credits), this makes sure the game cannot freeze because of this issue anymore.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193321)
Important question:

Does the issue of the missing sprites only happen when rendering is enabled for the first time? Or can it also happen every time when rendering is enabled after it was disabled?

Because if it can happen anytime when rendering is turned on again, not just after the very first time, then I might have to change my "City Trouble" code again. Because this means the game could freeze whenever a level is loaded if the nine overflow sprites aren't properly set.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193323)
DRW wrote:
Does the issue of the missing sprites only happen when rendering is enabled for the first time? Or can it also happen every time when rendering is enabled after it was disabled?

Only the first time unless you later disable rendering in the middle of the screen somewhere. As long as you only disable rendering during vblank you're safe. (You can disable just sprites, or just background mid-screen safely since both must be turned off to disable rendering entirely, or if you're very careful you can disable rendering on some part of the scanline where sprites are not being evaluated.)

DRW wrote:
tokumaru wrote:
So if you wait for vblank on the famicom before disabling rendering (or just don't disable it at all) you shouldn't get any glitches after resetting, right?

When I read various wiki entries, I was told that rendering should always be enabled and disabled in NMI anyway, so that the game doesn't start drawing mid-frame.

Setting it to 0 just after reset is an exception to this rule. Unless you are targetting Famicom only and not NES there is not much to be gained by waiting for vblank to initially write 0 to $2001. You've got to deal with this "one bad frame" issue always with the NES reset, apparently. tokumaru was only suggesting it as a test to make sure the reset issue was caused by the thing we thought was causing it, I think.

DRW wrote:
Your machine is the exception so far since it's the only one that was tested where power on works, but not reset.

I'm pretty certain that every NES/Famicom system should have the reset issue (excepting PAL?). It's only the behaviour at power-on that I suspect is different on other machines. (running oam_reset2.nes on them will diagnose this best)
Re: Nine sprites overflow doesn't work in the beginning
by on (#193325)
rainwarrior wrote:
Only the first time unless you later disable rendering in the middle of the screen somewhere. As long as you only disable rendering during vblank you're safe.

In this case, I'm safe.

rainwarrior wrote:
Setting it to 0 just after reset is an exception to this rule.

Alright, yeah, this is of course the big exception. I do this as well.

rainwarrior wrote:
I'm pretty certain that every NES/Famicom system should have the reset issue (excepting PAL?). It's only the behaviour at power-on that I suspect is different on other machines. (running oam_reset2.nes on them will diagnose this best)

Well, I can tell you for certain that my own original front loader NTSC NES often has issues reading "City Trouble" while pressing Reset usually fixes it.
You on the other hand experienced the exact opposite: Power on is usually fine, reset usually fails.

But since this is an infrequent thing anyway (sometimes power on works immediately, sometimes reset freezes as well, different carts produce different frequency of working/non-working), I'm not surprised that different machines behave differently.

If this was an issue where one machine always does this and the other machine always does that, then it would be strange. But in our case, my NES produced all four situations (power on working and non-working, reset working and non-working) at certain points in time, so differences between different systems are not surprising to me.

And since even the cartridge itself makes a difference, maybe your development cart is another factor in the calculation. If you lived in Germany, I would quickly send you over one of my faulty "City Trouble" carts to check whether their behavior changes on your NES or whether your NES' behavior changes with the cart.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193344)
This really isn't something that should be affected by the cartridge.

The frequency of failure at reset shouldn't be affected by different machines either, since it should be based entirely on the timing of pressing reset within the frame. (essentially "random" timing from human input)

Only power-on has the potential to be different, in the same way that uninitialized RAM can have a "signature" of power-on tendencies. I don't think the other factors should change things.

PAL should be different, though, since it has different sprite refresh behaviour.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193358)
Apologies if this information has been mentioned before, but the last time I remember hearing anything regarding the OAM DRAM refresh bug, this information was how the bug worked.

TL;DR: When rendering is disabled in the middle of a scanline (between pixels 64-255) that has any of sprites 0-5 on it, the DRAM refresh bug occurs.

It's really interesting to hear that pressing the reset button interrupts the sprite evaluation logic in the same way that disabling rendering does, to where the DRAM refresh bug occurs the first time the game switches rendering back on, but it makes sense and I suppose it should've been a logical conclusion. :P

A workaround to fix the title screen would be very easy, just display one blank frame as part as your reset routine. A full vblank-to-vblank frame, to be sure that the relevant sprite logic has run and is terminating safely before rendering is turned back off.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193364)
Drag wrote:
When rendering is disabled in the middle of a scanline (between pixels 64-255) that has any of sprites 0-5 on it, the DRAM refresh bug occurs.

What do you mean by sprites 0-5?

By the frequency of errors during the reset test on oam_reset2.nes, I'd say it corrupts something like 30-50% of the time. I don't think this would be possible if it was limited only to scanlines with sprites 0-5. It seems more or less the probability I'd expect if any interruption on the active half of a scanline counts.
Re: Nine sprites overflow doesn't work in the beginning
by on (#193367)
This is the original thread where this glitch was discussed. Here is where I got the "only sprites 0-5 trigger the glitch" figure, unless I've misinterpreted Blargg, which is always possible.

I worded the 64-255 pixel range thing clumsily in my old thread, so the correct information is that pixels 64-255 of a scanline are the "safe" region to disable rendering and not cause the DRAM refresh bug, but it's only safe when no sprites intersect the scanline, or when intersecting sprites intersect on their bottom-most row. Disabling rendering outside of the 64-255 "safe" region will always cause the bug.

Math time! Assuming NTSC NES.
There are 341 dots in a scanline. Dots 64-255 are safe for disabling sprites, subject to the condition of there not being any sprites intersecting the scanline underneath the one where you disable rendering. Therefore, there are 151 "bad" dots on each scanline, dots where you should not disable rendering under any circumstance.

Disabling rendering outside of the visible scanlines does not trigger the bug, so only the visible 240 scanlines can trigger the bug. 240 dangerous scanlines * 151 bad dots = 36240 bad dots that will trigger the bug. There are 89341.5 dots in a frame (.5 because one dot is skipped during odd frames).

Assuming a screen with no sprites, using the above figures, this bug has a 40.56% chance of occuring when you disable rendering at a random point in the frame (such as by pressing the reset button).

Assuming only sprites 0-5 trigger the glitch, the probability is 42.26% at best (all 6 sprites have the same Y position), and 50.77% at worst (all 6 sprites are spread out and do not intersect each other on the Y axis).

If all sprites trigger the glitch, the probability is 42.26% at best (all visible sprites have the same Y position), and 91.60% at worst (every single visible scanline has a sprite on it)

Never the less, these percentages seem to match what you experienced, so good catch on my blunder. :P
Re: Nine sprites overflow doesn't work in the beginning
by on (#194740)
DRW wrote:
If the game is already playing for some seconds and you only check for the sprite overflow then, it never freezes.

Some things depends on chip temperature:
If set spr adr to $80 instead of $00 before doing sprites dma. bugs happens, but disappears after some time (on famicom model hvc-102): https://youtu.be/QgJMzyEbko8
If change spr addr to $80 after dma: bugs changes to other bugs after some time (until power off) -> https://youtu.be/Y-v0STBK_Co OR bugs also fully disappers after some time - https://youtu.be/XrvyQM43QhI
Re: Nine sprites overflow doesn't work in the beginning
by on (#194759)
Ti_ wrote:
DRW wrote:
If the game is already playing for some seconds and you only check for the sprite overflow then, it never freezes.

Some things depends on chip temperature:
If set spr adr to $80 instead of $00 before doing sprites dma. bugs happens, but disappears after some time (on famicom model hvc-102): https://youtu.be/QgJMzyEbko8
If change spr addr to $80 after dma: bugs changes to other bugs after some time (until power off) -> https://youtu.be/Y-v0STBK_Co OR bugs also fully disappers after some time - https://youtu.be/XrvyQM43QhI

OAM Decay is definitely affected by temperature.

DRW's problem wasn't OAM decay, though. It is some uninitialized sprite evaluation flag, which I don't think really does have to do with temperature.
Re: Nine sprites overflow doesn't work in the beginning
by on (#194762)
rainwarrior wrote:
OAM Decay is definitely affected by temperature.

I gave links for same thing as "oamaddr bug" option in mesen emulator, not oam decay.
Re: Nine sprites overflow doesn't work in the beginning
by on (#194764)
What's the specific PPU version in your videos?
Re: Nine sprites overflow doesn't work in the beginning
by on (#194765)
Ti_ wrote:
rainwarrior wrote:
OAM Decay is definitely affected by temperature.

I gave links for same thing as "oamaddr bug" option in mesen emulator, not oam decay.

Oh, I misunderstood what those ROMs are supposed to test. (Where are those ROMs available?)

So you're saying the effect of sprite evaluation copying one tile over another is subject to temperature variation?

The related effect on reset from my tests above did not seem to change over time, or go away after warming up... They appeared to have a pretty consistent rate of occurrence.
Re: Nine sprites overflow doesn't work in the beginning
by on (#194810)
roms:
https://yadi.sk/d/p1_zFOK43HXiZH
(if you turn on after 1 hour been off, you may need to wait about 3-5 minutes to see changes).
you need to turn off power at least for 1minute to see effects goes back.
(roms oam_addr_80 and oam_dma_80)

lidnariq wrote:
What's the specific PPU version in your videos?

RP2C02E-0
Re: Nine sprites overflow doesn't work in the beginning
by on (#195737)
Some other glithces (on this ppu version?):
If a new frame starts while rendering been off somewhere in previous frame, some of sprites flashes:
https://yadi.sk/i/ic0hpjsT3J8A8J
Which of sprites depends on time where it has been off, when sprdma starts, and slightly by a chip temperature.

Games where it is seen, Somari:
https://yadi.sk/i/NQgOpc603J8Akf