This page is a mirror of Tepples' nesdev forum mirror (URL TBD).
Last updated on Oct-18-2019 Download

Best practice when updating entire nametable

Best practice when updating entire nametable
by on (#223254)
Hi all,

I understand that nametable changes may only occur within vblank / NMI, since that is when the PPU is free, and that there is not enough time to update the entire nametable during one vblank.

What I've done in the past is disable rendering *and* NMI together, then start my updating within NMI immediately afterwards. This seemed to work OK for a simple game going from a splash screen to the game screen.

For example: If nametable needs to change I disable NMI and rendering (probably don't need to disable rendering since I'm already in NMI?), and I update the entire nametable immediately afterwards while still in the last NMI call, then I re-enable NMI and rendering at the end of the current NMI call. So in theory I'm assuming that even though it takes a while to update, no new NMI calls will be made during the process.

Generally, if updating the entire nametable, what is the best practice around it and where should the updating code be?

Thanks
Re: Best practice when updating entire nametable
by on (#223255)
In many cases I prefer to leave the NMI running even if rendering is off, not touching the PPU, but still playing music.

You should only turn rendering on or off in your NMI handler during vblank, otherwise you will get a partially corrupt frame.

Once rendering is off, you can do it anywhere, the timing doesn't matter*. It's probably a lot more convenient to do this in your "main" thread rather than trying to operate within your NMI (which should be more oriented toward partial vblank-sized updates). Also, like I mentioned above, if you have an NMI routine that lets you skip the PPU and just keep updating sound, you don't get those unpleasant music skips when you move from room to room.

* Exception: writing palettes will cause colour to appear in the middle of the screen, even if rendering is off. Palettes should only be uploaded during vblank/NMI. Sprites also should only be uploaded during vblank, but for a different reason (the RAM "decays" over time and will corrupt if you don't OAM DMA immediately before the picture resumes).
Re: Best practice when updating entire nametable
by on (#223256)
Keep NMI enabled, just use a flag to tell it not to do any PPU stuff.
That way you can keep Music playing for example.
Re: Best practice when updating entire nametable
by on (#223257)
So the "main" thread would be like my Forever loop right? (for lack of a better word). The JMP that just keeps going until interrupts occur?

It sounds like if the nametable needs to change I can disable rendering in NMI and set a flag which the main thread can check. It updates the nametable and updates a flag so NMI knows to turn rendering back on? Should the disabling and enabling of rendering be located in any specific section of NMI? Like at the head or tail of the interrupt?

Thanks for the tip on palettes, I did not know that. (and music).
Re: Best practice when updating entire nametable
by on (#223258)
OP: there is a Wiki page on this subject. It speaks in "general tone" for development, so that the concept/implementation can be understood:

https://wiki.nesdev.com/w/index.php/The_frame_and_NMIs

Section "When to Turn Off PPU, NMIs" may be relevant to your interests (don't let the section title mislead you), but the entire page seems highly relevant to what's being discussed.
Re: Best practice when updating entire nametable
by on (#223262)
I've always implemented NMI handlers as an indirect jump, like this:
Code:
nmi_handler:
    jmp (nmi_ptr)

To change the behavior of the handler, you change the code 'nmi_ptr' points to. I find this cleaner to use than flags.

The second useful thing to have is a subroutine that blocks until an NMI occurs. You likely already have code to do this in your main loop, but it's even better to make it a subroutine:
Code:
wait_for_nmi:
    lda nmi_counter
waitLoop:
    cmp nmi_counter
    beq waitLoop
    rts

When you want to disable rendering and/or change 'nmi_ptr', you first call wait_for_nmi and then do your thing:
Code:
jsr wait_for_nmi
lda #0
sta PPUMASK
lda #.lobyte(some_handler)
sta nmi_ptr+0
lda #.hibyte(some_handler)
sta nmi_ptr+1

You can use this subroutine all over the place. I like to use it for implementing fade-ins and fade-outs, for example.
Re: Best practice when updating entire nametable
by on (#223263)
Thanks koitsu, that was a good read and I'm glad I read that before trying to separate game logic and draw logic.

pubby, so would you put all the NMI logic at whatever the pointer points to? So the callback for the interrupt is just a JMP?

Thanks
Re: Best practice when updating entire nametable
by on (#223266)
casprog wrote:
So the "main" thread would be like my Forever loop right? (for lack of a better word). The JMP that just keeps going until interrupts occur?

The more "robust" kind of program structure doesn't have an empty forever loop, it has a main thread that takes care of game logic and buffering video updates, and the NMI thread, which carries out the PPU updates, plays audio and does anything else that absolutely must run at 60Hz (e.g. raster effects).

Quote:
It sounds like if the nametable needs to change I can disable rendering in NMI and set a flag which the main thread can check.

It's actually the main thread that's supposed to signal stuff to the NMI thread. The main thread can't do PPU stuff whenever it wants, so it has to signal the intent to do those things so the NMI can effectively do them when vblank hits. The main thread goes "I want to turn rendering off", and waits for the NMI. When vblank hits, the NMI thread then obeys, and doesn't do any other PPU operations from that point on in order to not interfere with PPU operations that the main thread might be doing, such as updating an entire name table, but can keep doing other timely tasks without "freezing" the program. When the main thread is done, it signals the NMI thread to turn referring back on.

Quote:
Should the disabling and enabling of rendering be located in any specific section of NMI? Like at the head or tail of the interrupt?

As long as it's during vblank, it doesn't really matter. Whatever works best with your logic.
Re: Best practice when updating entire nametable
by on (#223277)
casprog wrote:
pubby, so would you put all the NMI logic at whatever the pointer points to? So the callback for the interrupt is just a JMP?

Yeah. And to change the NMI behavior you'd change the pointer.
Re: Best practice when updating entire nametable
by on (#223290)
Good to know about switching the PPU during the frame. I've just been ignoring the glitch for now, but was wondering if it would go away if I did it during vblank.

Side note:
If you are using a jump pointer, make sure that you change the pointer early in the frame. You would crash the CPU if the NMI was triggered while changing the pointer.
Re: Best practice when updating entire nametable
by on (#223291)
A four-write sequence should avoid a crash:
1. Change JMP to RTI
2-3. Change low and high bytes of destination address
4. Change back to JMP
Re: Best practice when updating entire nametable
by on (#223299)
I had assumed pubby's 0 to PPUMASK ($2001) was actually supposed to be PPUCTRL ($2000) to disable the NMI and make it safe to change the pointer. Though looking back, the safety is actually provided by waiting for vblank right before doing it. So pubby's given technique was already safe because that jsr wait_for_nmi means NMI won't fire on the next few (thousand) instructions.

For tepples' generic "change your NMI jump pointer whenever you like" (do we really need this capability?) suggestion, I would probably have used these "writes" instead:
1. Disable NMI ($2000)
2-3. change pointer
4. Enable NMI ($2000)

i.e. directly address the cause of the problem (NMI might happen while you're changing the pointer) instead of doing something even more advanced to transmute the problem into a skipped frame. (...and I think we've neglected to mention that tepples' method additionally requires relocating your NMI to RAM? That's another non-trivial advanced technique to learn about. I'm gonna assume that tepples chose the self modifying method to avoid another even more obscure and complicated issue... but that's another unstated premise. :P)


But now we've gotten to the point where we're talking about how to safely write self-modifying NMI handler code with someone who is asking for beginner advice, and I'm kinda thinking better advice would be: don't do it this way right now.

Yes you can write an NMI that starts out with an indirection, but it's one method of a billion equally valid ones for controlling what your NMI does, and the fiddly details of it are a little bit complicated. I don't recommend it if you're at the stage where you're asking OP's question.


Here's another way that I think has less complicated ramifications:

Just use a variable (here called skip_ppu to tell your NMI to stop messing with the PPU), set it to 1 whenever you need to do your nametable updates in the main thread, and back to 0 when you're ready for it to start doing its thing again:
Code:
nmi:
   pha ... etc.
   lda skip_ppu
   bne skip
   ... ; PPU stuff
skip:
   ... ; music
   ... pla etc.
   rti
Re: Best practice when updating entire nametable
by on (#223308)
$2000 writes mid-frame are discouraged because they can cause the "SMB glitch", which is when one scanline is rendered with the wrong name table setting.
Re: Best practice when updating entire nametable
by on (#223334)
Quote:
I don't recommend it if you're at the stage where you're asking OP's question


I completely agree but I already got myself into trouble lol So I took Pubby's idea and applied it to my main game logic. It seems to be working OK. Upon start press I swap the intro nametable with a character select nametable and change the pointer to point to another game state handler. I think I will leave the NMI alone and not use pointers with that (will try to keep it generic).

Quote:
You should only turn rendering on or off in your NMI handler during vblank, otherwise you will get a partially corrupt frame.


Would it be OK to turn it off as I'm doing here outside of NMI as long as I know vblank hit? In the article koitsu shared I read about setting flags and acting upon them during the next NMI, which I can do but I'm just curious.

Code:
Main:
   JSR Read_controllers
   JMP [game_logic_ptr]   
game_logic_end:   
   JMP Main



Code:
Game_logic_intro:
   LDA ctrl_1_state
   AND #%00010000 ; check for start button
   BEQ gli_controls_done
   ; start pressed
   ; wait for vblank
   JSR Wait_vblank
   ; disable ppu
   LDA #%00000000
   STA $2001
   ; update nametable
   LDA #LOW(select_nametable)
   STA nametable_ptr
   LDA #HIGH(select_nametable)
   STA nametable_ptr + 1
   JSR Load_nametable
   ; wait for vblank
   JSR Wait_vblank
   ; turn ppu back on
   LDA #%00011110
   STA $2001
   ; change game logic pointer
   LDA #LOW(Game_logic_select)
   STA game_logic_ptr
   LDA #HIGH(Game_logic_select)
   STA game_logic_ptr + 1
gli_controls_done:
   JMP game_logic_end
Re: Best practice when updating entire nametable
by on (#223337)
casprog wrote:
Would it be OK to turn it off as I'm doing here outside of NMI as long as I know vblank hit? In the article koitsu shared I read about setting flags and acting upon them during the next NMI, which I can do but I'm just curious.

Yes, if you know your code is executing withing vblank, it is an OK time to turn on/off rendering without any visual problem. The exact mechanism of your "Wait_vblank" routine might be important. If it's waiting on your NMI routine to increment a counter, you might need to ensure that the NMI actually returns quickly enough to still be in vblank. (e.g. if your NMI routine also plays music, this may not be guaranteed.)

...which is why my general recommendation was to do the turning off directly in the NMI handler. It's usually much easier to ensure that things happening in that routine occur within vblank than stuff in the main thread.


The basic concept I recommend is:

Buffer everything you want to send to the PPU (the mask value to be written to $2001, the scroll position $2000/$2005/$2005, OAM to be uploaded by DMA, palettes, any nametable updates, etc.), and then when you're done buffering the next frame, you set a flag variable to tell the NMI it's ready and then enter a loop that waits for the NMI handler to use it.

The NMI handler itself would check that flag at the beginning, and if it's not set, just do music. If it is set, it knows the main thread is waiting for it and should apply everything in the buffers, then unset that flag (letting the wait loop know it's finished).


(This is orthogonal with pubby's update method, the JMP indirection is just a way to have different modes of behaviour for the nmi handler, which you could do still within this buffering scheme.)
Re: Best practice when updating entire nametable
by on (#223434)
Quote:
Yes, if you know your code is executing withing vblank, it is an OK time to turn on/off rendering without any visual problem. The exact mechanism of your "Wait_vblank" routine might be important. If it's waiting on your NMI routine to increment a counter, you might need to ensure that the NMI actually returns quickly enough to still be in vblank. (e.g. if your NMI routine also plays music, this may not be guaranteed.)


gotcha, thanks, and yes I am relying on a counter changing.

I've been thinking about the pros and cons of swapping the entire nametable or only updating what needs to change via some buffer during NMI. I have found that for what I'm building maybe half the tiles will need to change level to level, and so I can go either way but I was concerned about space if I store full tables, but this would keep it simple.. Currently the nametable updates once per level, it will not change again until the next level. My space concern is I have 8KB of PRG ROM to work with using MMC1 correct? So in theory that would be 8 full nametables with no room left for palettes and sprites?
Re: Best practice when updating entire nametable
by on (#223436)
casprog wrote:
I've been thinking about the pros and cons of swapping the entire nametable or only updating what needs to change via some buffer during NMI. I have found that for what I'm building maybe half the tiles will need to change level to level, and so I can go either way but I was concerned about space if I store full tables, but this would keep it simple.. Currently the nametable updates once per level, it will not change again until the next level. My space concern is I have 8KB of PRG ROM to work with using MMC1 correct? So in theory that would be 8 full nametables with no room left for palettes and sprites?

MMC1 can address 512k of PRG-ROM. (16k at a time in the lower half.) Did you read PRG-RAM by mistake? It can optionally have PRG-RAM as well.
Re: Best practice when updating entire nametable
by on (#223445)
Looks like I did indeed cross the ROM with RAM :| thank you

In that case, since I don't need many levels, I will keep it simple and just generate the various tables and do a clean swap.
Re: Best practice when updating entire nametable
by on (#224139)
So I sort of successfully applied the suggestions in this thread to avoid flashes when changing screens in my game. I write the desired change to my mask copy variable, and then wait for the NMI. The very first thing my NMI routine does is to apply mask/ctrl. This only made the flashes at least stay at the top of the screen, but didn't quite go away.

The two workarounds I've found is to stop calling famitone update (the last thing the NMI handler does), or to wait for two full vsyncs after the mask is set before writing to the PPU. Am I missing something?
Re: Best practice when updating entire nametable
by on (#224140)
If you've disabled rendering (e.g. write 0 to $2001) then you can write anywhere in CHR without visual glitches EXCEPT palettes. Whenever $3F00-3FFF is written to it outputs that colour directly to the screen while the write address remains in that location.

You should not have to disable music while updating (I recommend leaving it on to avoid the unpleasant skip), and you definitely don't need to wait for extra vsyncs. If the problem was the palettes, I would only guess that doing either of these things coincidentally moved the timing of the palette writes off the visible portion of the screen.
Re: Best practice when updating entire nametable
by on (#224141)
I do change the BG color of the palette when switching screens, but gets run as part of my normal NMI handler. I already knew to avoid setting that outside of vblank.

My full NMI handler is: Check the NMI ready flag (set by my wait_nmi routine), write mask/ctrl, write buffered data (including palette changes), write scroll, and finally update famitone. If NMI ready is not set, it skips to only update famitone. Seems pretty standard right?

I agree that avoiding skips in the music is desirable. Waiting for two vblanks is working ok, even during animated transitions without causing a very noticeable delay really. It just seemed really weird that I need to do it. I must be doing something else wrong, I just don't see what it could be...
Re: Best practice when updating entire nametable
by on (#224142)
Maybe if you could screenshot or describe the flashes better? Or post a ROM, that would make it very easy to diagnose.
Re: Best practice when updating entire nametable
by on (#224151)
Here is a ROM that replicates the issue. Press up/down to select characters.
http://files.slembcke.net/temp/FlickerBug.nes

The character animations are almost 2k per, so I have to decompress them directly into CHR RAM, and thus need to disable to the PPU to do it. The garbage happens at the very top of the screen, but only when music is playing. I don't understand why, since the ctrl / mask registers are set before famitone even runs.

Related code snippets:
https://gist.github.com/slembcke/6a53fd ... 5729a98bf7
Re: Best practice when updating entire nametable
by on (#224157)
slembcke wrote:
The character animations are almost 2k per, so I have to decompress them directly into CHR RAM, and thus need to disable to the PPU to do it.

I haven't checked the ROM yet because I'm on my phone, but blanking the screen during character selection is not very good for the presentation of your game. In this case, it would be much better to update the tiles progressively each vblank and keep rendering on. If you copy a modest 128 bytes per frame, it'll take 16 frames to transfer the whole 2KB, which is still less than 0.3 seconds, fast enough to feel nearly "instantaneous" in a menu (definitely not during gameplay though).
Re: Best practice when updating entire nametable
by on (#224158)
I personally think the design of the character selection is fine.

However, the problem you are seeing is indeed palette writes happening outside of vblank. The actual colours of that artifact are a telltale sign that it's palettes, since they match the palettes you're using.

These palettes don't seem to be written by any code in that snippet, though. You have something that does LDA #$3F directly in the ROM, which is not in your snippet.

Mesen is really good for debugging when exactly writes happen. You can use its Event Viewer to visually see PPU writes overlaid on the screen, and also the "step back" feature of its debugger can help figure out what happened that made your palette writes so late.
Re: Best practice when updating entire nametable
by on (#224170)
tokumaru wrote:
but blanking the screen during character selection is not very good for the presentation of your game


Well, the character animations are all lz4 compressed. They don't need to be if I just put less of them in the game, but what fun would that be? Really it was a joke that I implemented the character select at all, but that's not really the point.

The point is that I turn the screen off to write some new sprites to the CHR RAM and rewrite the character's bio text. I could have picked any of the other screen transitions in the game that have the exact same problem without being "pointless", but this one was easy to trigger repeatedly.

rainwarrior wrote:
However, the problem you are seeing is indeed palette writes happening outside of vblank. The actual colours of that artifact are a telltale sign that it's palettes, since they match the palettes you're using.


Oooh! Are you saying that any palette writes, and not just the BG color is what can be causing the issue? I do blit the final palettes to VRAM outside of NMI. I was setting the BG color outside of that block because the majority screen color is often not the BG color for reasons of how the palettes work.

I'll try pushing the final palette load into another display list buffer.
Re: Best practice when updating entire nametable
by on (#224174)
Yes, basically whenever the PPU address points to $3F00-3FFF it will be constantly outputting the palette colour at that address (not just colour 0). The whole palette should be written within vblank.
Re: Best practice when updating entire nametable
by on (#224177)
slembcke wrote:
Oooh! Are you saying that any palette writes, and not just the BG color is what can be causing the issue?


Aurgh! Yeah, that seems to fix it. Pushing all of the palette writes into the NMI routine instead of just the BG color seems to work.

I guess you said exactly what I needed to hear before, "writing palettes will cause colour to appear in the middle of the screen". Since I was already writing the BG color separately in NMI to avoid half colored frames, I guess I just read that to mean what I wanted it to mean. -_-

Thanks again!

(I fixed the download link on the game's page to be transition glitch free: viewtopic.php?f=2&t=17611 :D )
Re: Best practice when updating entire nametable
by on (#224181)
rainwarrior wrote:
I personally think the design of the character selection is fine.

I checked the ROM now and I see that there's an animated transition, so yeah, that looks fine. It could be even smoother with progressive updates, but it doesn't look bad as is.