support for TRAKINFO #1982

rfomin · 2024-11-06T13:10:09Z

Perhaps we need to autoload extras.wad

yyjson allows comments and has better error reporting.

fabiangreffrath · 2024-11-06T13:20:24Z

What's the advantage of yyjson over cjson?

rfomin · 2024-11-06T13:25:42Z

What's the advantage of yyjson over cjson?

yyjson allows comments and has better error reporting.

rfomin · 2024-11-06T13:55:54Z

@ceski-1 Remix from KEX port is very loud compared to MIDI music. Should we do something about it?

fabiangreffrath · 2024-11-06T14:08:07Z

So, TRAKINFO is a simple assignment from SHA1 of the original MIDI lump to a new music remix lump name? Is there a spec somewhere?

rfomin · 2024-11-06T14:18:26Z

So, TRAKINFO is a simple assignment from SHA1 of the original MIDI lump to a new music remix lump name?

Yes, it looks like this:

	//D_INTRO
	"0c0acce45130bab935d2f1e85664b29a3c724fcd" : 
	{
		"MIDI" : "O_INTRO",
		"Remixed" : "H_INTRO"
	},

O_* files are Roland SC-55 recodings, H_* files are excellent Andrew Hulshult remix.

Is there a spec somewhere?

No. TRAKINFO has comments and in a different format compared to the ID24 spec.

fabiangreffrath · 2024-11-06T14:37:20Z

So, we ignore the "MIDI" part and just render MIDI as MIDI instead of playing the pre-recordings?

rfomin · 2024-11-06T14:44:17Z

So, we ignore the "MIDI" part and just render MIDI as MIDI instead of playing the pre-recordings?

Yes, but maybe we should also use the Roland SC-55 recordings. People seem to like them.

The nice thing about these SHA1 hashes is that remixed music works with PWADs. For example, Scythe 1 can now use remixed tracks.

elf-alchemist · 2024-11-06T15:46:29Z

@ceski-1 Remix from KEX port is very loud compared to MIDI music. Should we do something about it?

Hi, sorry to barge in, just throwing my two cents in. Since Update 1 dropped for KEX Doom a volume property that does exactly what it implies, and was used to increase the volume of the relatively quiet SC55 recordings of the OST. They can be seen on the second TRAKINFO lump in the KEX extras.wad

// O_E1M1
"b292d36331c81e1f2ee4d7bd2938e7fedb39f172" : 
{
	"volume" : "2.0"
},

rfomin · 2024-11-06T16:16:50Z

Since Update 1 dropped for KEX Doom a volume property that does exactly what it implies, and was used to increase the volume of the relatively quiet SC55 recordings of the OST. They can be seen on the second TRAKINFO lump in the KEX extras.wad

Yes, I know about that. I was talking about volume relative to a standard MIDI player, e.g. "MS GS Wavetable". We have a "gain" setting in the advanced MIDI menu, maybe we should add gain options for non-MIDI music as well.

Perhaps the volume property should also be used, but I'm not sure yet.

elf-alchemist · 2024-11-06T17:05:45Z

Perhaps the volume property should also be used, but I'm not sure yet.

I was thinking of that, to use the volume feature on the H_* trackset to lower its volume.

// H_E1M1
"09bbd8f7bd85f835ddfe07d366ac4b6e0de96ea3" : 
{
	"volume": 0.5,
}

Another (worse) idea is to just chuck the volume property in the vanilla MUS lumps themselves.

// D_E1M1
"99767e32769229897f7722848fb1ceccc2314d09" : 
{
	"volume": 2.0,
}

Having a gain setting (beyond also the volume property) may honestly be the simplest way to fix this.

ceski-1 · 2024-11-06T19:06:36Z

@ceski-1 Remix from KEX port is very loud compared to MIDI music. Should we do something about it?

First, some general observations:

notes

The mastering is very poor for the "remix" tracks. They appear to have been recorded too loudly, clipping and distorting the audio, and then adjusted back down but the damage was already done (hence the clipped/flat peaks). The "original" tracks don't loop correctly (they forgot loop markers?) and are even quieter than the raw (uncompressed volume) SC-55 pack's tracks. There will inevitability be false bug reports about broken looping for the KEX "original" tracks, and volume issues in general.

There's a few solutions here. We can support the volume property. If we do that, we may have to assign a new volume value for every KEX track ourselves, maybe in a custom TRAKINFO lump provided in woof.pk3. Additionally, we could add a "sampled music" gain thermo (mp3/ogg/it/mod/xm/etc.). The midi options would have to be reorganized into a music options menu to fit this thermo. Another solution is auto-gain adjustments, e.g. a library that analyzes a song's overall volume and adjusts it to perceptually the same level.

Other notes:

The interaction/priority between SC-55 packs, autoloading, KEX tracks (remix and original), and sampled music provided by wads is now blurred and confusing
Should extras.wad be autoloaded if a GOG/Steam install is detected? Maybe just the music tracks?

rfomin · 2024-11-07T03:12:52Z

The "original" tracks don't loop correctly (they forgot loop markers?) and are even quieter than the raw (uncompressed volume) SC-55 pack's tracks.

They don't have LOOP_START/LOOP_END tags.

We can support the volume property. If we do that, we may have to assign a new volume value for every KEX track ourselves

They already set volume property for O_* files in second TRAKINFO file (it's always "2.0").

Another solution is auto-gain adjustments, e.g. a library that analyzes a song's overall volume and adjusts it to perceptually the same level.

Interesting, but it's not going to work for native MIDI, is it?

The interaction/priority between SC-55 packs, autoloading, KEX tracks (remix and original), and sampled music provided by wads is now blurred and confusing

We don't have any special rules for SC-55 packs or sampled music, it all works like regular WAD lumps. What this PR adds is SHA1 checking of the playing track and replacing it on the fly with the remixed track if the menu option is enabled.

I think we should ignore the SC-55 recordings from extras.wad, they are quiet and have no loops. Andrew Hulshult's remix is the most interesting part.

rfomin · 2024-11-07T03:36:46Z

It is interesting to note that the TNT and Sigil SC-55 recordings have loop tags.

ceski-1 · 2024-11-07T03:40:29Z

They already set volume property for O_* files in second TRAKINFO file (it's always "2.0").

Yes, but the remix H_* files would still need something less than 1.0.

I think we should ignore the SC-55 recordings from extras.wad, they are quiet and have no loops. Andrew Hulshult's remix is the most interesting part.

Agreed.

elf-alchemist · 2024-11-07T10:19:58Z

With the shift in focus towards Hulshut's remixes, will this mean then, no support for TRAKINFO in general? This feature can certainly be used for other WADs beyond just the IDKFA OST, one of the megawads I've playtested has a "B-side" set of 32 additional MIDIs and using the so-called "Original" option would fit it in perfectly. And even on KEX itself, I've used TRAKINFO on some of my Mod browser uploads to feature the Aubrey Hodges PSX OSTs as a "Remixed" option as well. This would be a bit of a missed opportunity, I feel.

rfomin · 2024-11-07T10:41:18Z

This feature can certainly be used for other WADs beyond just the IDKFA OST, one of the megawads I've playtested has a "B-side" set of 32 additional MIDIs and using the so-called "Original" option would fit it in perfectly. And even on KEX itself, I've used TRAKINFO on some of my Mod browser uploads to feature the Aubrey Hodges PSX OSTs as a "Remixed" option as well.

Wow, I didn't realise anyone was using it. TRAKINFO is clearly intended for internal use of the KEX port, it is not user friendly and has a different format than the rest of the ID24 specs. @fabiangreffrath , @ceski-1 should we bring back TRAKINFO?

fabiangreffrath · 2024-11-07T10:55:57Z

I'm with @elf-alchemist and believe that not implementing TRAKINFO support would be a missed opportunity. They already provided a valid use case, so the question if anyone else is using it outside the KEX port doesn't even arise anymore. Also, regarding user friendliness, you just added support for the most user-unfriendly HUD description format possible - and yet you are working on an editor and people will start using it in their own mods or configurations. 😆

rfomin · 2024-11-07T11:17:04Z

Also, regarding user friendliness, you just added support for the most user-unfriendly HUD description format possible - and yet you are working on an editor and people will start using it in their own mods or configurations. 😆

Well, at least it was thought out and designed for the editor from the beginning.

TRAKINFO is weird, we have to first calculate a SHA1 hash for the MIDI track, and then another SHA1 hash for the remixed track (which is a big OGG file) just for the volume property. Frankly, I'd like to avoid this.

fabiangreffrath · 2024-11-07T12:30:25Z

The sha1 code addition and cjson -> yyjson change still needs to get reflected in the README.md file, please.

rfomin · 2024-11-07T12:52:09Z

The sha1 code addition and cjson -> yyjson change still needs to get reflected in the README.md file, please.

Fixed in c09d881

elf-alchemist · 2024-11-07T13:28:44Z

Wow, I didn't realise anyone was using it. TRAKINFO is clearly intended for internal use of the KEX port, it is not user friendly and has a different format than the rest of the ID24 specs.

Agreed on how weird and KEX-bespoke the format is. Once I had learned of it being only a SHA1 for a given MIDI lump I started using it on (almost) every single upload I made on the Mod Browser, you can even see on the installation size of any mod I upload that it is always in the dozens of megabytes, because the default GUS MIDI synth sounds... polarizing... they don't have SF2 support, I rendered all tracks in SCC1 Florestan and used sha1sum to get everything together (helps that I'm very familiar with manual JSON editing).

Honestly, I do definitely believe in the potential for a more generic and better designed "layered OSTs" feature in the future, but for now, I personally believe plain TRAKINFO will suffice for most use-cases... for now.

rfomin · 2024-11-07T13:54:39Z

the default GUS MIDI synth sounds... polarizing... they don't have SF2 support, I rendered all tracks in SCC1 Florestan

But we do support SF2 and MS GS Wavetable, and it's possible to use Nuked SC-55. We also support the SNDFONT lump if you want to use a specific soundfont in your mod.

I'm curious about new WADs with alternative soundtracks, can you link one?

elf-alchemist · 2024-11-07T18:07:51Z

But we do support SF2 and MS GS Wavetable, and it's possible to use Nuked SC-55. We also support the SNDFONT lump if you want to use a specific soundfont in your mod.

Yes, I was just giving some additional background on me using the lump itself, mostly driven by KEX's shortcomings compared to source ports. I do believe the use-cases for Woof will be fewer, though no less genuine.

I'm curious about new WADs with alternative soundtracks, can you link one?

My compilation of Dr. Sleep's Inferno has it's normal MIDI selection, rendered "Original" tracks and Aubrey Hodges' PSX OST as "Remixed" tracks. And a friend of mine has uploaded Doom 64 for Doom II with it's default OST, the rendered version as "Original" and a tracker *.it rendition of the D64 OST by Immorpher as the "Remixed" option.

fabiangreffrath · 2024-11-08T09:09:46Z

third-party/CMakeLists.txt


 add_library(spng STATIC spng/spng.c)
 target_woof_settings(spng)
 target_compile_definitions(spng PRIVATE SPNG_USE_MINIZ INTERFACE SPNG_STATIC)
 target_include_directories(spng INTERFACE spng)
 target_link_libraries(spng miniz)
+
+add_library(yyjson STATIC yyjson/yyjson.c)


We have a libyyjson package in Debian. Could we please first try to build against the system package before our own vendored copy?

https://tracker.debian.org/pkg/yyjson

MSYS2 doesn't have it. We also need the latest version for better error reporting.

What will I have to change to prefer the system libyyjson for the Debian package?

What will I have to change to prefer the system libyyjson for the Debian package?

Quite a lot I think. We can choose other library, e.g. https://github.com/akheron/jansson

I'm fine with the library itself. I just don't want to receive bug reports against the Debian package, because I use a code copy and not the system library.

Eh, Debian is tough 😄 Okay, I'll try to add searching for system package.

Fixed in 4703897

ceski-1 · 2024-11-12T05:33:52Z

src/mn_setup.c

@@ -2502,6 +2504,10 @@ static void RestartMusic(void)
    S_RestartMusic();
 }

+static const char *extra_music_strings[] = {
+    "Off", "Remix", "Original"


Nit: "Off", "Original", "Remix" seems more appropriate for Woof.

This is the order from the menu in the KEX port. I think most users want "Remix", "Original" sounds a lot like "MS GS Wavetable" anyway.

ceski-1 · 2024-11-12T05:49:15Z

src/s_sound.c

+    BIND_NUM(extra_music, EXMUS_REMIX, EXMUS_NONE, EXMUS_ROLAND_SC55,
+             "Extra soundtrack (0 = Off; 1 = Remix; 2 = Roland SC-55");


Nit: Rename "Roland SC-55", EXMUS_ROLAND_SC55, and related vars/strings to "Original", etc.

Fixed in e5a555a

rfomin · 2024-11-12T08:46:48Z

src/d_main.c

@@ -1827,6 +1828,12 @@ void D_DoomMain(void)
  // [FG] emulate a specific version of Doom
  InitGameVersion();

+  char *extras = D_FindWADByName("extras.wad");


Do we need to add any checks here? The name is too simple, there was also a version for the Unity port.

Won't hurt to auto-load it even if it doesn't contain what we expect?

Won't hurt to auto-load it even if it doesn't contain what we expect?

Probably won't. But we need -noextras parameter just in case it annoys some users (for example, it contains DSSECRET sound or icons for weapon rolette).

rfomin added 2 commits November 6, 2024 20:02

switch to yyjson, add sha1 library from Chocolate Doom

4490f4f

yyjson allows comments and has better error reporting.

implement TRAKINFO

09f210c

fix MSYS2 build

04e8d45

convert to sha1_digest format during parsing

666f4b5

add support for MIDI synth recordings

a822ed7

rfomin force-pushed the trakinfo branch from 2a1f398 to a822ed7 Compare November 6, 2024 15:40

rfomin changed the title ~~support for TRAKINFO~~ Nov 7, 2024

rfomin force-pushed the trakinfo branch from 577fe79 to c0be7b6 Compare November 7, 2024 10:33

rfomin mentioned this pull request Nov 7, 2024

Heap overflow #1991

Closed

ceski-1 mentioned this pull request Nov 8, 2024

Use improved FluidSynth reverb/chorus defaults, add config settings #1993

Merged

fabiangreffrath reviewed Nov 8, 2024

View reviewed changes

rfomin added 5 commits November 12, 2024 08:35

add license information

9ae8b9c

simplify m_json interface

aaa018d

Merge branch 'master' into trakinfo

9afcc68

fix merge, cleanup

8146858

search for the system package yyjson

4703897

rfomin force-pushed the trakinfo branch from c09d881 to 4703897 Compare November 12, 2024 03:27

rfomin changed the title ~~support for remixed soundtrack (extras.wad)~~ Nov 12, 2024

rfomin added 2 commits November 12, 2024 10:38

fix memory leaks (clang-tidy warnings)

abe0172

update CI workflow

c7d649f

ceski-1 reviewed Nov 12, 2024

View reviewed changes

rfomin added 3 commits November 12, 2024 15:35

cosmetic fixes

e5a555a

search for extras.wad

36cbbab

fix build, set "Off" by default

c315ca6

rfomin commented Nov 12, 2024

View reviewed changes

rfomin added 2 commits November 12, 2024 18:15

add M_StringToDigest, -noextras parameter

a816e1c

add S_ACTION flag

e2105fe

rfomin merged commit 726abe6 into fabiangreffrath:master Nov 13, 2024
8 checks passed

rfomin deleted the trakinfo branch November 13, 2024 05:51

		BIND_NUM(extra_music, EXMUS_REMIX, EXMUS_NONE, EXMUS_ROLAND_SC55,
		"Extra soundtrack (0 = Off; 1 = Remix; 2 = Roland SC-55");

support for TRAKINFO #1982

support for TRAKINFO #1982

Conversation

rfomin commented Nov 6, 2024

fabiangreffrath commented Nov 6, 2024

rfomin commented Nov 6, 2024

rfomin commented Nov 6, 2024

fabiangreffrath commented Nov 6, 2024

rfomin commented Nov 6, 2024

fabiangreffrath commented Nov 6, 2024

rfomin commented Nov 6, 2024

elf-alchemist commented Nov 6, 2024

rfomin commented Nov 6, 2024

elf-alchemist commented Nov 6, 2024

ceski-1 commented Nov 6, 2024 • edited Loading

rfomin commented Nov 7, 2024

rfomin commented Nov 7, 2024

ceski-1 commented Nov 7, 2024

elf-alchemist commented Nov 7, 2024

rfomin commented Nov 7, 2024

fabiangreffrath commented Nov 7, 2024

rfomin commented Nov 7, 2024

fabiangreffrath commented Nov 7, 2024

rfomin commented Nov 7, 2024

elf-alchemist commented Nov 7, 2024

rfomin commented Nov 7, 2024

elf-alchemist commented Nov 7, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ceski-1 commented Nov 6, 2024 •

edited

Loading