This page is a mirror of Tepples' nesdev forum mirror (URL TBD).
Last updated on Oct-18-2019 Download

About FCEUX's sound...

About FCEUX's sound...
by on (#128626)
Most of you probably know about FCEUX, one of the greatest NES emulators on the Internet. Anyway, I have a question about it and NES emulators in general: How do they exactly recreate the sound of the games? The sound in an NES game doesn't come from the software itself. Rather, it contains commands for the APU. But since there's no such hardware on modern platforms, the emulator software has to... well... emulate it. So, does it use any sound samples, like the music trackers do? There has to be something that has the square waves, triangle wave, etc. recorded, so that it can play it. Can anyone who has studied the source code of an emulator or two, tell me are if there are any samples used and where they are. I will use this knowledge for an interesting experiment. I decided, if the samples could be changed, to change the regular square waves and stuff with real instrument samples and hear how the games would sound like after that. It would be very interesting to hear... :)
Re: About FCEUX's sound...
by on (#128628)
Samples:
  • The triangle sample is FEDCBA98765432100123456789ABCDEF, generated by a 5-bit counter fed to a quad XOR gate.
  • The square wave sample is 01000000, 01100000, 01111000, or 10011111, generated by a 3-bit counter fed to some duty cycle lookup logic.
  • The noise generator is a 15-bit polynomial counter with a well-known formula for its 32,767 states.
These were reverse engineered by playing notes through the APU using NES programs and then emulated. Later they were confirmed by the Visual 2A03 project taking photomicrographs of the decapped APU.

So what you want in an emulator is something that watches for writes to APU registers and then approximates what it would sound like on a sampler unit. Compare the sound of original Super Mario Bros. series to the Super Mario All-Stars versions to hear how it might sound.
Re: About FCEUX's sound...
by on (#128630)
So, the short answer is: "You can't modify the samples to MIDI instrument samples, unless you make an external program." That's too bad. It would've been interesting to hear how a lot of games would sound like with these new sound samples.
Re: About FCEUX's sound...
by on (#128634)
AlienX wrote:
So, the short answer is: "You can't modify the samples to MIDI instrument samples, unless you make an external program." That's too bad. It would've been interesting to hear how a lot of games would sound like with these new sound samples.

[rant]

We've heard it before -- or at least I have, and likely all other folks who were around when "NES emulation" started -- a version of iNES (I think it was iNES, but maybe it was Pasowing or something?) offered a "MIDI output" option in its audio configuration. The result sounded like dogshit being shovelled through a plastic funnel: it was atrocious. Once you learn a little bit about how NES/FC audio works, you realise how the two things are really quite separate/unrelated and shouldn't be directly connected in this manner.

Every single aspect of NES "songs", all the way down to the sequencing/composing, is intended to be heard on the console itself. That's how the composers and sequencers intended. Changing any of that (e.g. making it use MIDI, doing weird/wonky post-effects on it, "stereo mods" nonsense) results in something that both sounds truly awful and was not what the composer intended. You have to understand that the composition methodologies were done with the NES hardware in mind; yes, many musicians would compose something on a keyboard/MIDI system as a baseline for what they wanted, but the effort didn't stop there. Those same musicians would quite often be the same people to make their music sound how they wanted it on the NES. Meaning: *they intended for it to sound the way it does*.

If you want evidence of this, check out the differences in how the songs sound (both audibly and their actual sequences/compositions) between games that are on NES carts vs. the same game but on the Famicom Disk System. Castlevania 2 is a perfect example (NES = Castlevania 2: Simon's Quest, FDS = Dracula 2: Noroi no Fuunin) -- they sound similar but the sequences are actually quite different because the composers had different limitations/ideas in mind when working on one system vs. the other. There are some exceptions to this comparison though -- specifically NES = Magic of Scheherazade vs. FDS = Arabian Dorimu Sherazado -- where the FDS version was composed by someone different. But I hope you understand what I'm getting at.

Then there are composers like Rob Hubbard and many others who literally would just sit down and bang out raw bytes that correlated with the music they wanted, through both experience and trial-and-error getting what they wanted. That's a form of composition in itself, but it's not the same as, say, entering notes via a MIDI keyboard. In turn, their songs sound very different compared to the aforementioned type (not to mention the obvious style differences).

If you want "MIDI-ised" versions of NES songs, there are tons of musicians/composers who make great MIDI mixes (or remixes) of those songs, do all the sequencing themselves, choose good patch sets, and go the extra mile to make it sound amazing (Jake Kaufman/Virt is one such individual). Likewise, orchestras that are hired to play classic game songs (example) take the time to get a composer and conductor who understands what's needed, but these sound absolutely nothing like what comes out of the NES. There are bands that try to merge the two "technologies" (for lack of better term), like Powerglove, Minibosses, Armcannon, etc., but again these are effectively "remixed" or "reperformed" much like an orchestra.

The point I'm trying to drive home here is that people need to just leave NES music/audio alone and be heard how it was originally intended. I am all for people doing remixes and coming up with new/tweaked compositions and all that (I'm a huge fan of all of that too!), but when it comes to things like emulators, the audio output should remain as true to the actual console as possible -- no exceptions.

[/rant]
Re: About FCEUX's sound...
by on (#128641)
koitsu wrote:
but when it comes to things like emulators, the audio output should remain as true to the actual console as possible

The same goes for video and the incredibly shitty HQ345xWhatever filters that create squiggly lines and color blotches.

I'm all for updating graphics, sound and controls in old games in the spirit of "what if", but this has to be done carefully in a case-by-case basis with human intervention to fill in the gaps if you really expect the results to be any good. Any automatic attempt to create detail that just isn't there in the original material will fail miserably.
Re: About FCEUX's sound...
by on (#128646)
You might get better results by playing different synth sounds in place of the 2A03 ones. For example, exchanging a saw wave for the triangle wave could sound interesting. The parameters and expected tonal qualities aren't so wildly different that a lot of sounds and effects would probably not sound so bad. An example of this in action (by accident) is some old versions of Nesticle - the 50% duty wave was pretty poorly recreated, and it sounded a lot more like a ~35% duty wave. Still, it was an interesting sound, as it wasn't so incompatible with what the sound designers had in mind.
Re: About FCEUX's sound...
by on (#128661)
tokumaru wrote:
The same goes for video and the incredibly shitty HQ345xWhatever filters that create squiggly lines and color blotches.

Funnily enough, it's the Hq3x filter, that gave me this idea. I tried it and the graphics looked like they've been vectorized. It didn't feel like an NES game, rather, some free indie remake. And the 8-bit sound didn't match the graphics at all. So I thought: "Wouldn't it be interesting, if the NES's APU used more complex sounds than just square and triangle waves?" I'm all for authenticity, but I would like to see this idea realized. Now, I know, that NES soundtracks, played through MIDI do not sound as good as the original(some are not bad, thou), so my idea was not to just have MIDI instruments slapped onto the sound channels, only the sound samples for the instruments themselves. No fading in or out, no echo or bulls**t like that(Arpeggios do not sound well, when you use the default violin instrument), just pure, non-stopping, raw, less-than-a-second long instrument sound samples. I think if these are used, it could work OK.
koitsu wrote:
*they intended for it to sound the way it does*

You do realize, that the idea of replacing the NES sounds with something else completely throws what the composers intended out of the window. This is not about having games sound, the way they're supposed to, rather, exactly the opposite. It's a twist on NES sound and for an EXPERIMENT, it's worth trying. Also, I know many people make remixes in MIDI, but they have some contributions from the remixers themselves, so it's not what I'm looking for.
FCEUX supports Lua scripts, which can make some graphic additions to the games, so maybe they could make some for sounds too.
Again, this is an EXPERIMENT. It's not meant to be a new awesome thing. It's just a curiosity, that could be explored, an that's what I'm trying to do.
Re: About FCEUX's sound...
by on (#128662)
I think HDNES is the only one that enhances with better sound capabilities than other emulators (Can replace and patch RAM to use OGG files), so I think the solution is HDNES for now.
Re: About FCEUX's sound...
by on (#128664)
Thanks, but the emulator doesn't work on my computer for some reason. :(
Oh, well...
BTW, I checked out G-NES, which is the old emulator with MIDI output and I can see what koitsu was talking about. Well, the MIDI square wave never sounded too well, so my idea is to add different samples for the different waves. Guitar for 0% square wave, Sawtooth for 25% square wave, String Ensemble for 50% square wave and Pan flute for triangle wave. Another huge problem in G-NES is the fact, that the Noise channel is uses one MIDI channel for all of it's notes. The MIDI percussion channels don't change their pitch, when a different note is played, so the sound effects sound non-existent.
Re: About FCEUX's sound...
by on (#128665)
To make up for the NES's limited timbre, games often use specific sequences of modifications of the pitch, volume, and duty cycle at a 60Hz spacing. (FamiTracker calls these "instrument sequences").
To really come up with a good conversion of audio to MIDI, I think you'd need to look into the future to see what the future sets of modifications will be, and then pick an appropriate MIDI patch.

This would require an emulator that can emulate the NES at 4-10x real time ... which is probably plenty doable nowadays.
Re: About FCEUX's sound...
by on (#128668)
You're correct that audio is far less causal than video. Perhaps a better course is to patch the games' music engines to use a generic sampler chip on the cart, such as a Namco 163's synthesizer or perhaps an audio module ripped out of a Super NES and mapped somewhere in $4xxx instead of $002140-$002143.

In any case, if you want to hear what music would sound like with the triangle waves replaced, listen to any Game Boy or Game Boy Color soundtrack. If you want to hear what music would sound like with both the square and triangle waves replaced, listen to any Namco 163 or TurboGrafx-16 soundtrack.
Re: About FCEUX's sound...
by on (#128696)
Good suggestion, but I want to hear NES soundtracks, that weren't made with an expansion chip in mind, that were not altered in any way and I want to hear them, while playing the games they come from. Seems like a lot to ask, but emulators, like HDNES this seems possible... unfortunately, it doesn't work. The program opens fine, but when I load a game and start it, the emulator crashes! No error message, nothing, it just tries to open the SDL window and it stops. What's wrong? Do I need Visual C++ or something? Is something wrong with OpenGL? And I'm not trying to open a game with some obscure mapper, I'm trying to open stuff like "Super Mario Bros." and "Castlevania II: Simons's Quest". Somebody, please, tell me if there's a solution to this problem, I want to get this emulator running. Besides sound, I can also try doing some graphic packs.
Re: About FCEUX's sound...
by on (#128702)
AlienX wrote:
Good suggestion, but I want to hear NES soundtracks, that weren't made with an expansion chip in mind, that were not altered in any way and I want to hear them, while playing the games they come from.


Um, isn't this what nearly every emulator already does? What emulators are "altering" the audio in some way? Yes some emulators might sound a bit different because they generate, mix, and do other things to the audio a bit different. But to the average person's ear, a good emulator is going to sound identical to the real system. Now if you have a more expert ear for audio properties you could probably tell them apart depending on which emulator is used.
Re: About FCEUX's sound...
by on (#128706)
AlienX wrote:
I want to hear NES soundtracks, that weren't made with an expansion chip in mind, that were not altered in any way and I want to hear them, while playing the games they come from.
You've either completely misunderstood me, or got horrifically confused by what tepples said.

Here's how sound on the NES works:

You have seven different waveforms ("sounds")⁽¹⁾ that it can make, plus some recorded samples. But that's really restrictive. So many games automatically add automatic changes to the various properties to make it sound (for example) more a wah pedal on an electric guitar, or the exponential decay in volume from keyboard and plucked instruments.

To simple take these seven waveforms, convert them to MIDI patches, and then apply the volume and pitch to it is completely wrong. Oftentimes, MIDI notes won't even finish their attack within the first 1/60th of a second (the update rate of these effects), and when the game changes these properties every 1/60th of a second, if you replace it with (e.g.) a piano either you're now double-counting the volume (natural from the MIDI patch, plus the effect from the NES), or you're entirely ignoring the volume effects from the NES.

So a useful conversion requires asking "what sequence of modifications is the software going to make so that I can usefully convert the concept the composer was trying to represent into a new sound".

What tepples suggested was: instead of emulating into the future to look ahead at what the sounds will be to figure out which MIDI patch to use, instead hook into the game's audio driver and look at the values in memory to determine which sequence of modifications the game is going to use before it starts playing the sound. The bit about "expansion audio" was a implementation detail, and not specifically relevant; he was merely suggesting that a way this could be achieved would be to provide a MIDI-like interface to the NES, and games could then be modified to use that interface.

¹ footnote: those seven waveforms are: 12.5% duty cycle pulse wave, 25% duty cycle pulse wave, 50% duty cycle square wave, 75% duty cycle pulse wave (which sounds identical to humans as the 25% pulse wave but the distinction may be relevant for conversion), a 32-sample triangle wave without volume control, a 32767-sample white noise sample, and a 93-sample tonal noise sample.
Quote:
HDNES [...] doesn't work.
You should ask in its thread.
Re: About FCEUX's sound...
by on (#128758)
lidnariq wrote:
AlienX wrote:
I want to hear NES soundtracks, that weren't made with an expansion chip in mind, that were not altered in any way and I want to hear them, while playing the games they come from.
You've either completely misunderstood me, or got horrifically confused by what tepples said.


I think by this he means that listening to a N163 or VRC6 game is not really achieving what he'd like to hear.
Re: About FCEUX's sound...
by on (#128760)
Three things can be done in the emulator. All three methods would require configuration for each individual game (or at least for each company's music engine). I'll try to explain each of them using simpler language and give each a name so we have a reference point for further discussion.
  1. Wavetable method: Replace the square and triangle samples with other short looping samples. The result would sound like a Namco 163 game.
  2. Non-causal method: Watch the writes to the 2A03 audio ports, and after several frames, look at the duty, volume, and pitch to figure out what instrument the game was trying to play. This would add lag between the input and audio while the emulator is mapping the duty and volume to a most likely instrument, making it useful for NSFs and for rendering input-log movies to video but not for real-time play.
  3. Expansion method: Patch the game at runtime to send note-on messages, such as MIDI notes through a Miracle piano, instead of playing the notes through the 2A03 audio ports. This is the same process as patching a game to use Famicom expansion audio.
Re: About FCEUX's sound...
by on (#128762)
tepples wrote:
Non-causal method: Watch the writes to the 2A03 audio ports, and after several frames, look at the duty, volume, and pitch to figure out what instrument the game was trying to play. This would add lag between the input and audio while the emulator is mapping the duty and volume to a most likely instrument, making it useful for NSFs and for rendering input-log movies to video but not for real-time play.
<facepalm> No, that's not necessary. Just because it's an anticausal technique doesn't mean you have to emulate it with latency: it just means you need to emulate the future (and assume input from the player) to figure out what sound to play in the present. Computers are perfectly capable of doing just that. It's even perfectly possible to look at the exceptions to what you expected will have been played and use it to change what's being now, to compensate for assuming future player input.

mikejmoffitt wrote:
I think by this he means that listening to a N163 or VRC6 game is not really achieving what he'd like to hear.
If his point was "I don't want to listen to something that's only been upgraded to N163 sound", I addressed that: tepples clearly meant "using expansion audio" as shorthand for "providing a MIDI-like abstraction to the NES".
Re: About FCEUX's sound...
by on (#128763)
lidnariq wrote:
it just means you need to emulate the future (and assume input from the player) to figure out what sound to play in the present.

Good point. But other than that keys held this frame will likely equal keys held last frame, you can't really predict input from the player. So expect a huge spike in CPU usage to re-emulate the whole NES several frames ahead whenever a player presses or releases a button.