This page is a mirror of Tepples' nesdev forum mirror (URL TBD).
Last updated on Oct-18-2019 Download

Compression and mappers


by on (#48665)
In reply to this post

I always though codemasters amazing. Maybe a good compression could make some games fit on NES memory without need mappers?

by on (#48668)
Yeah, so long as you had enough RAM to decompress it to. That would probably require WRAM if you wanted to not greatly reduce the amount of RAM available for the program to use, which would require a mapper or a modification to NROM. But that's an interesting concept, though I suspect there are a lot of cons involved in doing something like that. But with code, there are a lot more patterns than with graphical data, or so I'd assume. LDA/STA combos make up a HUGE chunk of my code.

by on (#48671)
Siloucos wrote:
I always though codemasters amazing. Maybe a good compression could make some games fit on NES memory without need mappers?

The biggest game I know of with a mapper is 40 KiB (NROM 256 Kbit PRG + 64 Kbit CHR), and the smallest game I know of without a mapper is 48 KiB (CNROM 256 Kbit PRG + 128 Kbit CHR). CHR compression works only with CHR RAM, so you would have to put the CHR data in PRG space, which would bloat PRG to 32 KB and thus require a mapper. Or was there a CNROM game with 128 Kbit PRG and >= 128 Kbit CHR?

by on (#48672)
Siloucos wrote:
Maybe a good compression could make some games fit on NES memory without need mappers?

I don't think there's much you can do with the small amount of RAM the NES has. It's barely enough to hold all the variables, arrays and such of complex games, there is really no room for decompressed code/data. So we are limited to simple compression schemes that can be decompressed on the fly as the data is read from the ROM, which usually results in modest compression ratios, but is still better than no compression.

I have nothing against mappers in general, but I became a supporter of the simpler mappers, the ones that use discrete logic. None of them have good CHR bankswitching capabilities, so I became a CHR-RAM supporter too, and good CHR compression (like this one from Codemasters) makes all the difference.

I like UNROM and UOROM a lot, because their PRG bankswitching is as simple as it gets, and real carts can be built fairly easily. I find WRAM too complicated to add to a cart, so I tend to avoid it.

by on (#48674)
tokumaru wrote:
I like UNROM and UOROM a lot, because their PRG bankswitching is as simple as it gets


I actually am a really big fan of the MMC3's bankswitching capabilities. I really enjoy that it's one write to bankswitch, and that the register is 8 bits wide so you can have a lot of PRG space (256 * 8k, or am I missing something?). In one of my projects, I have a CHR tile updating routine that copies 10 tiles from ROM into CHR RAM every frame (or it can). With MMC3's bankswitching, I can really quickly switch banks of CHR data which is really important in limited Vblank time. Maybe I'm crazy, I dunno. Though I haven't ever seen UNROM or UOROM's bankswitching capabilities...

by on (#48679)
Celius wrote:
I actually am a really big fan of the MMC3's bankswitching capabilities.

Well, the fact that one of the pattern tables is divided in 4 while the other is divided in 2 can be annoying. PRG bankswitching is nice, yes, but the fact that the mapper is only available in original carts is a big turn off for me...

I guess I'll change my mind only when a FME7-level mapper (I would hardly be interested on an MMC3 clone, because of it's crappy scanline IRQ) is available for everyone, but nobody that has the technical knowledge for that seems to be interested.

Quote:
Though I haven't ever seen UNROM or UOROM's bankswitching capabilities...

It's as simple as it can be... writes to anywhere in ROM selects a 16KB page at $8000-$BFFF. $C000-$FFFF is hardwired.

@mod: Maybe it's time for a split...

by on (#48680)
Celius wrote:
I actually am a really big fan of the MMC3's bankswitching capabilities.

You won't be once you finish a game and find that nobody has cloned MMC3 in a CPLD yet.

Quote:
Though I haven't ever seen UNROM or UOROM's bankswitching capabilities...

  • UNROM and UOROM are comparable to SGROM's default mode: $8000-$BFFF switchable, $C000-$FFFF fixed to last bank, PPU $0000-$1FFF is RAM.
  • Depending on what kind of mirroring you want, BNROM and AOROM are comparable to SGROM's other mode: $8000-$FFFF switchable, PPU $0000-$1FFF is RAM.
  • There's a third mode on SGROM: $8000-$BFFF fixed to first bank, $C000-$FFFF switchable, PPU $0000-$1FFF is RAM. I don't know if any game uses this mode, but Crazy Climber's board has the same logic, and it would be good for a game with a lot of DPCM audio.


Looking for best split point... done.

by on (#48682)
Compression doesn't always require a sliding window. You can still do amazing stuff when constrained.
For instance, Dragon Warrior games never decompress the entire overworld.

by on (#48683)
Dwedit wrote:
Compression doesn't always require a sliding window. You can still do amazing stuff when constrained.

As I see it, the big problem is that you usually need random access to data, and reading from the middle of a compressed stream can be hard/slow depending on the type of compression.

A way to solve the problem is to break down large amounts of data (such as level maps) into smaller pieces and compress them individually. This will work well if you only need access to small parts of the whole thing at a time. You'd still need some sort of RAM buffer though, even if it wasn't very large.

Another option is to use compression schemes that result in data of constant sizes, much like how metatiles are used to compress tiles. That allows you to still have random access to the data because everything is at a predictable location.

I think that mapper-less games can only go so far. Games like SMB managed to have quite a few levels using so little ROM because it's level format was so compact, but that also resulted in somewhat dull-looking levels.

Game logic doesn't take that much space, it's the data that uses most of it. Levels in particular need a great deal of memory, because of their physical structure and because of the objects that populate them. So in order to have a large game (long and numerous levels) in a small cart, the most obvious thing to sacrifice is level detail.

by on (#48684)
I used to not think much of Dictionary compression, then I saw what it did for Zelda oracle of ages. Dictionary compression can be some nice stuff, especially when you have tons of ROM, and constrained RAM. You can do random access with no history, as long as you have a link to the correct address. You can get a correct address by creating pointers to arbitrary subdivisions of the data.

For example, the Dragon Warrior Games used RLE, which requires no history. In order to get random access, they used one pointer for each row, then sequentially access within the row.

by on (#48686)
tokumaru wrote:
I guess I'll change my mind only when a FME7-level mapper (I would hardly be interested on an MMC3 clone, because of it's crappy scanline IRQ) is available for everyone, but nobody that has the technical knowledge for that seems to be interested.


No love for Konami VRC mappers? They seems interesting to me except the fact that they were not intended for pal..

tepples wrote:
Celius wrote:
I actually am a really big fan of the MMC3's bankswitching capabilities.

You won't be once you finish a game and find that nobody has cloned MMC3 in a CPLD yet.


No offense intended but who are you fooling except yourself when thinking that you "may" have to reproduce in big quantity your game? How can you expect that from a dead console? How can you be sure that your game will be good enough for that? Thinking with that mentality just limit you on what things you can do on the nes until someone make a new mapper for homebrew.

People should focus first on having fun programming their nes as a hobby and forget about those small details. Making your game is more important than that. Using any approach to achieve your goal is fine. As long that it's works on an emulator and at the least a powerpak, that's good enough. People that really want it on a cart will find a way to make it themselves.

I don't expect my games to be great ones, never was: I just hope to have fun making them and, at last, to have been able to make "something" on that console.

And, by the way, if your game becomes that good, there is a good chance that those hong kong pirates will find a way to reproduce it and make money on your back anyway :lol:

by on (#48689)
UNROM could support 4 Mbytes of PRG, just by using an 8-bit latch instead of the 4-bit latch/counter Nintendo always used. UNROM logic is also nice because you can easily swap a chip to reverse the banking logic $8000-$BFFF fixed, $C000-$FFFF switchable). I used that setup for my Chipography NSF, because of the many DPCM samples.

With all that space, I'd conclude that better compression is even more useful, because you can fit even more stuff in it!!

I could really use some better CHR compression about now, as I'm using RLE on Garage Cart #2. So tokumaru, or anyone else that has an experimental NES CHR compression scheme, it'd be nice to have on-board the project. Even if the encoder is in BASIC or whatever, as long as it works, heheh.

by on (#48690)
Memblers wrote:
UNROM could support 4 Mbytes of PRG, just by using an 8-bit latch instead of the 4-bit latch/counter Nintendo always used.

I considered doing that in case I ever need more space. I was planning on stealing the 4-bit counter of another cart though, and using the 2 of them. I would probably only benefit from 1 extra address line though, since I only have 512KB chips. Installing 2 512KB chips and using the higher address line to select between the 2 to get 1MB is a bit too complex... And I don't expect to make a game that large.

Quote:
So tokumaru, or anyone else that has an experimental NES CHR compression scheme, it'd be nice to have on-board the project. Even if the encoder is in BASIC or whatever, as long as it works, heheh.

I decided to look at Codemasters' format exactly because I was trying to find a way to improve my own compression format, which we discussed before. I do have a few ideas which I plan to test really soon (I've been quite busy lately), and will release my results here (compressor in C, decompressor in 6502). Unless you need it right away, you should catch my release.

by on (#48723)
tokumaru wrote:
Memblers wrote:
UNROM could support 4 Mbytes of PRG, just by using an 8-bit latch instead of the 4-bit latch/counter Nintendo always used.

I considered doing that in case I ever need more space. I was planning on stealing the 4-bit counter of another cart though, and using the 2 of them.


Not a bad idea really, as the 8-bit latch (74HC374) I used on Squeedo was tricky to set up (it's clocked by a positive chip enable for 0x5000). That'd be annoying by itself, but it came free as the leftover pieces of a WRAM decoder circuit. :)


Quote:
I do have a few ideas which I plan to test really soon (I've been quite busy lately), and will release my results here (compressor in C, decompressor in 6502). Unless you need it right away, you should catch my release.


Awesome. It takes ages for me to finalize an ambitious project like this, so I'll be able to use it later, clear up some space for a few things at the end.


Also another point regarding compression, I've discovered that in Matrixz's NESnake 2, there is a really nice compressed music format but I've yet to test it out (the encoder uses Visual Basic, yet it's no-go with the free version of VB I downloaded from Microsoft, damn them). I'll figure it out though somehow. It actually uses NT2's NED/DAT format, reformatted and decompresses as it plays.

I totaled up the music data in NES Snake 2, and it's 22,398 bytes originally and 14,751 bytes after compression. Matrixz as well as Codemasters, are "Pure Genius!" :)