Documenting the PS2 CSL sequenced formats (SQ/BD/HD), and comparisons to PS1 generic sequenced formats (SEQ/VH/VB) by baku-chan at 6:27 PM EST on November 10, 2023
Hello there! I’m baku-chan, and I’ve been making the rounds across the internet to ask about the inner workings of the PS2’s Component Sound Library, or CSL. Or specifically, about the three file formats that make up a single piece of music in CSL: SQ, BD, and HD.
From what I’ve seen, there seems to almost zero information available about these PS2 sequenced music formats compared to their PS1 generic equivalents (SEQ/VH/VB). My biggest dream would be for someone to completely reverse-engineer these formats so that they can be freely used, but I’m sure that would take a significant amount of work and research to pull off! As such, I would be content for at least some of the questions I have about these formats to be answered if such an endeavor isn’t feasible.
Meanwhile, I have two main motives for wanting some information about these formats, which I hope will contextualize what kind of answers would be helpful for me. The first motive is that I’m working to make homebrew sequenced music a reality on the PS2, and by extension on the PS1 whose sound chip the former is directly backwards-compatible with on a hardware level. In order for this to be possible, I need as much information on each of the relevant file formats as possible so that arbitrary data can be written to them in such a way that something resembling music can be made. As for the second motive, I would also like to have these formats documented just for the sake of them being documented, as there, again, seems to be almost no documentation available on them compared to its predecessor formats and I want to change that.
Some questions that I’m looking to have answered in relation to the PS2 and its SQ/BD/HD file formats include:
— How close is PS2 SQ to PS1 SEQ, and in turn to SMF format 0 MIDI? Based on what I’ve read from loveemu’s research on the matter in particular, the PS1 SEQ format is apparently very close to SMF format 0 MIDI, except with a few modifications. Given this, would it be accurate to say that the score data — that is, the actual key ons and key offs and such — is identical between PS1 SQ and SMF format 0 MIDI? This question is especially important to me because if it is identical, then this would greatly expedite our ability to reverse-engineer that part of the format and use that knowledge to write arbitrary score data and, in turn, create custom songs that can play on a PS2. Another thing that’s important in that context is how close PS1 SEQ is, in turn, to PS2 SQ, especially in regards to score data. Would it be accurate to say that they are very close to one another, if not possibly identical? And if they are indeed close or even identical, then how close would PS2 SQ be to SMF format 0 MIDI?
— How close is PS2 HD to PS1 VH? They seem to be somewhat different from each other based on a cursory comparison, but that could be about things being shuffled around from PS1 VH just as much as any actual meaningful changes being made from there. Given that SPU2 is essentially two PS1 SPUs attached together with little to no modification besides extra sound RAM, would it be accurate to say that settings for things like ADSR, portamento, pitch bend, and such work basically the same way with PS2 HD as they do with PS1 VH? And if they do, then how is the representation of such settings different on a hexadecimal level between the two?
— How close is PS2 BD to PS1 VB? From my understanding, PS1 VB is simply a collection of VAG files, and that the PS2 uses the exact same form of SPU-ADPCM that’s used on the PS1. Given that, would it be accurate to say that PS2 BD is also simply a collection of VAG files? And if so, is how they’re represented and organized exactly the same as it’s done with PS1 VB, or is it done differently? And is there any reliable way to tell exactly when a sample begins and ends?
— How do tracks work, exactly? On both PS1 SEQ and PS2 SQ, it appears that all of the tracks — in the MIDI sense of the word — are interleaved with one another, rather than being represented separately one after another like I’ve seen with other sequenced formats like Square’s AKAO and Konami’s KDT1. Given this, is there some sort of marker on, say, each key on and key off that determines which track is actually supposed to be keyed on or keyed off? Furthermore, is there some marker that determines which core (CORE0 or CORE1) a certain track is playing on? Also, is there a marker that determines the timing of a key on or key off of a track relative to the key on or key off of another track, given that tracks aren’t separated like they are in other sequenced formats?
— How do you set reverb? On the PS1, I know that there’s a flag for each instrument in the VH file that determines whether reverb is applied to said instrument, while whether or not reverb is enabled on SPU itself is set by sending certain NRPN values as defined in the SEQ file. However, on PS2, there are two reverb units split between the two individual cores that make up SPU2, which complicates things somewhat compared to PS1. So with that said, how does one set the reverb on either one or both of the cores on PS2? Do you send certain NRPN values like with PS1, or is it done another way? Furthermore, in a scenario where, say, CORE0’s reverb is set to STUDIO_C and CORE1’s reverb is set to HALL, how do I determine which of the two reverb settings a certain instrument uses, or alternatively which core an instrument should play on?
— How do attribute changes, loops, and markers work? I understand that on PS1 SEQ, it’s possible to change attributes of an instrument on runtime by sending certain NRPN values, just like with reverb. I also understand that other NRPN values can be sent to determine loop points, as well as to set markers within the song. Do things work similarly with PS2 SQ, or is everything done another way instead?
— What do program changes do, exactly? Perhaps my relative ignorance of how MIDI works is showing here, haha, but this has confused me especially looking at how they appear to exist in PS2 SQ. There are dozens of numbers attached to these, but I have no idea what these numbers mean or what they actually do, if anything. My initial guess was that perhaps they had something to do with reverb, but I highly suspect not, in retrospect. Simply put, how do program changes work, and what do they do? Furthermore, do program changes work differently between PS1 SEQ and PS2 SQ, or for that matter between both of those and SMF format 0 MIDI?
— What’s the numerical representation of silence in SPU-ADPCM? My guess is that it would just be zero; is this correct? Otherwise, what is it? Could you reliably replace every value in an individual sample file with said silence value and have the result be, well, a completely silent sample file?
Whoever responds to these questions, thank you in advance! I truly appreciate it.
I can't answer to most of your questions and people with the knowledge have mostly left. Instead I recommend looking at various tool's sources, and directly asking the people making those tools at discord (after researching).
vb/bd are a bunch of spu-adpcm samples pasted together and vh/hd are custom headers (different) that point to them plus other stuff.
There are various ways to represent silence in SPU-ADPCM but 0s is probably ok.
I can only partially answer your questions, specifically those that deal with ordinary MIDI data:
>> How do tracks work, exactly?
You don't really need separate tracks because every MIDI status byte (event) has a 4-bit channel number (0…15). The other 4 bits are the event type. This is the same concept of data transmission as in every other scenario where MIDI data is sent from i.e. an external MIDI keyboard to the PC. In this case there is also just ONE data stream and the channel assignment is made clear using the channel number in the MIDI events. The timing of all channels is done via delta-time values preceding every event. It does not matter on which channel a note is played because the delta time is based on whatever event comes next ANYWHERE in one of the 16 channels. If 4 notes play at the exact same time on 4 different channels, those 4 events all have a delta-value of 0 between them.
>> How do you set reverb?
By using controller-change events (hex B) which are 3 bytes long (status byte, controller number, value to set). But the actual controller number can vary. Could be 12 (hex C) or 91 (hex 5B) or any other of the not-so-clearly-defined controller numbers. Depends on the playback engine. The same event is used to set global channel volume (hex 7) or pan position (hex A) etc...
>> How do attribute changes, loops, and markers work?
Depends on what you mean by "attributes". You can use controller changes (see above) which are the most compact format or meta-events (hex F) or even SysEx messages for extremely specific stuff. Loops are generally implemented using a marker (meta event, hex FF 06) that specifies the loop start and at the end-of-track event (meta event, FF 2F 00) the playback engine jumps to that marker.
>> What do program changes do, exactly?
They change the . . . well . . . program (the instrument) that's currently assigned to one of the 16 channels. Program change is event type (hex) C followed by another byte specifying the actual program to use. Only 128 programs can be selected that way. If you need more then the program change must be preceded by a "bank select" controller change event (event B, controller 0).
I hope that helps a bit. My main focus is N64 music, not Playstation but some of the concepts apply here, too. N64's early "compact MIDI" format is also standard MIDI with very few alterations. Looking at an example SQ file from "dot Hack Infection" (PS2) shows that it uses indeed the standard MIDI data format.
There was actually documentation and C headers for these file formats provided by sony with the ps2 sdk (if you're not averse to looking at such things)
SQ is basically standard midi with some minor space saving tricks.
I started decompiling the IOP modules using these formats at https://github.com/Ziemas/csl_decomp/ but it's incomplete and I haven't touched it in a while.
Thanks for your responses, everyone! Much appreciated. It looks like diving into the wider world of MIDI is going to be what I need to do next in order to really understand how these formats actually work...
@Ziemas I'm not necessarily averse to the mere idea at looking at official documentation for this kind of thing, no. The Sega Saturn scene's understanding of a lot of SCSP stuff relies a lot on official SEGA documents that were leaked at some point as much as actual down-and-dirty reverse-engineering, so there's that. I'd consider such documents that have been floating around and referenced by others long enough to be basically common knowledge after a certain point. But that's just the thing with PS2-related stuff, though; any documentation regarding the SDK clearly haven't reached that level of saturation yet, let alone CSL-related stuff. I mean, certainly I wasn't able to just search "SQ/BD/HD documentation" and get what I needed, at least. Based on what you're telling me, though, it seems that such documentation *does* exist somewhere on the internet at this point, but the problem is just how deep I would have to dig, and perhaps most importantly *where* I would have to dig, that might be prove to be a problem.
In happier news, though, I'm excited by the fact that someone is actually going through and trying to decompile the actual CSL drivers now! Especially since the PS2SDK project was able to get a homebrew version of LIDSD.IRX up but neglected MODHYSN.IRX and MODMIDI.IRX for some reason. It makes it seem like no one really cares about sequenced music when I see things like that... but anyway! If you're able to complete this project one day, that would be wonderful, because perhaps we could use this to expand what we can do with CSL.
I wonder, though, which versions of these drivers are you working on specifically? I know that several different variants have shown up over the years (as I suspect you already know but for the benefit of those reading, there are strings near the end of the driver files that say which version they are).
@uyjulian By any chance, would librspu2 be the driver that runs during the PS2's startup sequence (or while running the main menu in general)? A list of functions in the PS2 BIOS that I found — written by you, it seems? — refers to it as OSDSND and calls it the so-called "tentative" driver.