If you're talking about an MSX game, then yes. Nothing about the carts are preserved in the ROM images, and you need a per-game database, or you'll need some kind of "cartridge type" list that the user has to choose from when booting the game.
If you're talking about the NES, sort of yes. People stick a fan-made "iNES" header at the top of the ROM data, but they made the format before all the details of the NES were known. So using that information is not enough for many titles, and so you need a database for those titles to work properly.
For the SNES and Genesis, the games have internal headers that tell you most of the details of the games, but not enough for complete emulation. SNES emulators in the past have been able to add bizarre tricks that just so happen to line up with the existing commercial games to not need a database, but it really turns out to be a mini database disguised by being obtuse. Example: "if the ROM is larger than 2MB, and has save RAM data, and has 32K bank granularity, then map the save RAM data to a smaller area of memory." Rather than "if the game is Fire Emblem, treat it differently than Ys 3."
For the Game Boy Advance, emulators and flash cart tools will scan the ROM looking for signatures from code libraries from Nintendo that handle save memory, like "FLASH512_V", "EEPROM_V", "SRAM_V", etc. Developers wised up and started including fake strings to fool them, so once again ... you end up needing a database.
There's very, very few cartridge-based systems where only the ROM dump is all you need. If I had to name one, I'd say the Neo Geo Pocket is one.
There are even cases like Mega Man X (Japan), where they made a mistake with the copy protection, and to fix it, resorted to wiring a resistor right onto the back of the PCB of every cartridge released to change the memory map.
And databases + heuristics fall apart when it comes to homebrew, fan translations, ROM hacks, etc.
True preservation requires games include a detailed description of the entire PCBs of these games, not just the ROM chips. But unfortunately, after trying for 15 years, I've not been able to come up with a stable, simple format for every system in the world that doesn't rely on emulator-specific design decisions.
I mean, how do you represent every possible circuit board configuration on the planet in a text file, that isn't going to make emulator authors for just the NES or just the Game Boy balk?
This hits on something I've wondered about for a long time. Since carts did not just include data, but also mappers and coprocessors, how do you write an emulator that can accurately play an arbitrary ROM dump?
Does your emulator have to have knowledge of every single game, and what mappers and/or "helper" chips each had, and emulate just the ones appropriate for a given game?
Something I've never been able to wrap my head around is how ROMs are dumped for emulators from cartridges? Dumping instructions and assets makes total sense to me, and packaging that up in a data file that can be interpreted by an emulator too, but how does an emulator model the hardware of every 'expansion' chip in a cartridge? How is that dumped from an original cartridge?
It's not just "the ROM in it", it's "the ROMs in it". Single-rom games don't benefit from containers. The whole point of treating all games as containers is to allow multi-rom games to be free of metadata. 30% of NES games, 1% of SNES games, and most arcade games are multi-rom. In order for the emulator to tell contents apart without metadata, you have to have a naming scheme. So, for example, Galaxian.fc.zip would contain "program.rom" and "character.rom". Currently, those pieces are blobbed together with a header in front. Thus you can't open a .NES file in your file browser and see if it has a character rom, what size it is, what sha256 it has. This causes massive tedium with NES dump verification since you can't assume anything at face value. All parts and values are obscured and need extracted.
The other reason metadata is evil is that it's a royal PITA to maintain. It's far easier to update your emulator than thousands of individual files. Do not listen to anyone who tells you that we need to drag licensed games into the metadata mud for the sake of unlicensed games. They are pulling your chain, an unlicensed db for nes is entirely feasible, or even the mapper number applied to the extension ala "Cheetahmen II.224.fc.zip".
MAME did this right with a database and NEVER writing inside the container, but only because their hand was forced into it. So many arcade games have 3 or more roms, so they were never tempted to take shortcuts and blob shit together to avoid zip containers. Then they screwed up their own good fortune with some truly bizarre stuff. Google non-merged vs merged and behold threads and threads of confusion. Now weigh that confusion against the supposed benefit.
People have assembled lists of all known cartridge configurations (as many games tended to share the same overall configuration), and assigned them IDs. The ROM specifies, in its header, the ID of the cartridge configuration it needs. (In NES jargon these are called "mappers" [1].) The emulator is expected to hard-code support for every single mapper; this is, of course, a huge nuisance and source of complexity, but there's little that can be done about it.
I thought something like the ROM/RAM mixing in cartridges must be the case. But then how on earth do people make files of those games that can then run on emulators?
"I have a one-of-a-kind setup in order to analyze and dump the entire memory maps of cartridges. All other methods to back up SNES cartridges cannot do this."
Now, I'm _really_ not an emulator expert, but as I recall both snes9x and zsnes will read rom dumps from zip files. So it seems to be possible to do this without too much cooperation.
If archivers started including useful metadata in addition to the rom dumps, I'm sure the emulators would start to use it as well.
The simplest way to deploy your homebrew ROMs to a real NES is to use something called a "flash cartridge"[1]. This is equivalent to a real NES cartridge except it is backed by rewritable flash memory instead of a permanent ROM. It contains all of the necessary glue logic[2] so the console can't tell the difference. You can rewrite the flash if you connect the cart to your computer via USB.
The magical thing to google if you want to do your own research is "NES flash cart".
The "EverDrive" that aji suggests is, indeed, an example of a flash cart. It may or may not be the best one for the NES - I honestly don't know, all of my experience is with the GameBoy.
dpflan's approach of erasing and reprogramming a REAL NES cart is also possible, if you enjoy pain and suffering in the name of being lavishly historically accurate. (I know I do sometimes.)
The conclusion that they downloaded the rom from the internet because it has an iNes header is kind of ridiculous to start with; if you build an NES emulator, you're going to need to have some sort of header to indicate what mapper (if any) was used on the cartridge and what stuff was present. Given that the community has already built a standard for that, why not use it?
If you go ahead and use the community header format; of course your rom image will be indistinguishable from a pirated rom -- the header and data should be the same. It's not clear if they downloaded the rom from the internet, read it off a cartridge in their archive (or obtained from employee's collections or the used market), or from a possible archive of all roms they've ever had made.
Given the long life and mass distribution of almost all their titles, there's not a danger of loss of the published material. Certainly, unpublished roms and the occasional title that was not widely produced are in danger of loss if they aren't archived. The source materials are also subject to loss if Nintendo doesn't archive them.
There's some indication that Nintendo may have an archival program after all; Star Fox 2 was recently released on the SNES Classic from a ready for release, but never released or leaked rom; different than the leaked prototype roms. They also add in-memory patches to some roms that run in virtual console and nes/snes classics; I haven't seen anything indicating if they do that based on disassembly of the roms or based on the original sources.
For example, The iNES format, for NES emulators, has a 16 byte header, not part of the original ROM, that identifies the mapper, screen mirroing if not controlled by the mapper, and whether or not battery backed RAM is included.
NES mapper numbers were standardized among emulator authors. Look for "NES mapper list". Some very early games didn't use a mapper (that's what mapper 0 is).
SNES ROMs have a checksum right in the header, but there were still bad dumps circulating for years. Presumably they escaped before the header was properly reverse engineered.
I'm not exactly sure why that checksum exists, the console doesn't check it, and in manufacturing you could just read and compare. But it's certainly convenient for verifying a good dump.
No. NES files as used today contain a few dozen bytes at the start which are used to tell emulators a little bit about the type of hardware that needs to be emulated. This header appears in the ROMs that Nintendo is selling. Nintendo would not have invented precisely the same header as is used by unofficial emulators by chance.
Note that no one is claiming that this is illegal on Nintendo's part.
That's true of probably 90% of ROM images though. To get most SNES ROMs to work you need to not only emulate the SNES console itself but also emulate the individual game's chipset. Most SNES games use one of the following expansion chipsets:
Many SNES games include one-off chipsets that common SNES emulators implement as well; for example just something simple like saving your game can't be done on an SNES and needs an expansion chip.
Not really, no. Most of what I use are FreeBSD tools I wrote. So it's mostly just connect cart, turn on SNES, run commands on the terminal, log the resulting SHA256 and cart serial#s into a database, move ROM to archive, rinse repeat.
I very much agree. My vision was a ZIP file that when selected in an emulator, could be extracted for instant access, and any save RAM, cheat codes, save states, etc could be stored alongside the game in said folder. It would be bundled with an extensible markup description of the board, and optional pack-in content like box art and user manuals. For games stored on flash memory that could write to themselves (BS Memory, Neo Geo Pocket), this would also prevent the archived, pristene copy from being damaged. The folder would then act like a physical cartridge, and all your game saves would move along with it.
It ... was not a popular idea, to put it mildly. I further shot myself in the foot by trying to force the idea on people. You will never win people over with an approach like that.
Let's avoid mentioning names for obvious reasons, but unfortunately those most capable of adopting and promoting such a format through their database effort are unwilling to use ZIP archives in this way. They will only allow for an uncompressed single-file-per-game approach.
As mentioned in my article, this turns into quite the problem. When you take the 8KB program ROM for Galaxian, and append a character ROM for graphics onto it, but your header can only tell you the size of the program ROM in 16KB increments, it means there's no way of specifying where the program ROM ends, and the character ROM begins. Any emulator that supports Galaxian does it via game-specific workarounds like checksum identification.
> Or are people finding roms, perhaps intended for use with emulators, and loading those onto physical cartridges and selling them without renumeration or permission?
I think they're talking about this case (based on the wording "rom hack cartridges"). Someone takes an original game ROM and modifies it in some way (translation, level mods, whatever). Someone else takes the modified code, burns it onto actual hardware, and sells the result.
> Sort of related: I know this requires some specialized equipment to make new cartridges, but do you know if they sell blank cartridges that can just be loaded with games?
The usual method that I know is to remove the ROM chips from a donor cartridge (a working game), and replace them with chips flashed with customized data. The donor cart would need to provide the same custom chips/memory mappers/whatever that the game code expects to be present.
If you're talking about an MSX game, then yes. Nothing about the carts are preserved in the ROM images, and you need a per-game database, or you'll need some kind of "cartridge type" list that the user has to choose from when booting the game.
If you're talking about the NES, sort of yes. People stick a fan-made "iNES" header at the top of the ROM data, but they made the format before all the details of the NES were known. So using that information is not enough for many titles, and so you need a database for those titles to work properly.
For the SNES and Genesis, the games have internal headers that tell you most of the details of the games, but not enough for complete emulation. SNES emulators in the past have been able to add bizarre tricks that just so happen to line up with the existing commercial games to not need a database, but it really turns out to be a mini database disguised by being obtuse. Example: "if the ROM is larger than 2MB, and has save RAM data, and has 32K bank granularity, then map the save RAM data to a smaller area of memory." Rather than "if the game is Fire Emblem, treat it differently than Ys 3."
For the Game Boy Advance, emulators and flash cart tools will scan the ROM looking for signatures from code libraries from Nintendo that handle save memory, like "FLASH512_V", "EEPROM_V", "SRAM_V", etc. Developers wised up and started including fake strings to fool them, so once again ... you end up needing a database.
There's very, very few cartridge-based systems where only the ROM dump is all you need. If I had to name one, I'd say the Neo Geo Pocket is one.
There are even cases like Mega Man X (Japan), where they made a mistake with the copy protection, and to fix it, resorted to wiring a resistor right onto the back of the PCB of every cartridge released to change the memory map.
And databases + heuristics fall apart when it comes to homebrew, fan translations, ROM hacks, etc.
True preservation requires games include a detailed description of the entire PCBs of these games, not just the ROM chips. But unfortunately, after trying for 15 years, I've not been able to come up with a stable, simple format for every system in the world that doesn't rely on emulator-specific design decisions.
I mean, how do you represent every possible circuit board configuration on the planet in a text file, that isn't going to make emulator authors for just the NES or just the Game Boy balk?
reply