Comparison of audio formats for games

When you want to make games, like I do, you may stumble upon the question which audio format to use. There is MP3, there is MIDI and much more!

This article ist mostly written for developers who create their own engine, such as in C or C++. Existing game engines, such as Unity, Unreal or GameMaker, may be more restricted in which formats you can use and how you can play back the audio.

Audio stream (MP3, WAV etc.)

This category of audio is probably the most well known one. Essentially, these files simply contains a single stream of raw audio data. The audio data may be given for a single (mono) channel, for two (stereo) channels or more. The bit depth may be 8-bit, 16-bit or more. The samplerate may be 44100 Hz, 48000 Hz or something else.

Since raw audio can be quite big in file size, most audio stream formats use some sort of compression, usually lossy by removing frequencies that are (possibly) not heard by the human ear and more compression magic.

The most well known audio formats I know about are:

• WAV (.wav)
• MP3 (.mp3)
• Ogg Vorbis (.ogg)
• Ogg Opus (.opus)
• FLAC (.flac)

WAV

The WAV format is probably the simplest audio format of all. The format is quite simple and if the audio is not compressed in any way (that's the usual case), you can write a WAV encoder and decoder easily yourself.

C/C++ integration

If you want to use a library for WAV encoding or decoding, I can suggest libsndfile. It also supports many other audio formats.

MP3

MP3 is probably the most known audio type in the world. But it also has the poorest quality of those formats I know. The only benefits of MP3 I can think of are:

• smaller file size than WAV and maybe FLAC
• everybody knows MP3, anybody can play MP3

But if you are working on your own game engine, you can decide yourself which audio format to choose. So I would strongly suggest to use Vorbis or Opus instead.

C/C++ integration

If you want to decode MP3 files, there are to libraries I can recommend:

• libsndfile (also supports many other formats)
• LAME

Ogg Vorbis and Opus

For me those two formats are similar, since they are both usually used in an "ogg" container and are both developed by ther Xiph.Org Foundation. Opus appeared after Vorbis and is meant to replace Vorbis.

Both Vorbis and Opus have better audio quality and smaller file size than MP3.

C/C++ integration

For Vorbis and Opus, there are easy to use official libraries.

If you intend to integrate also other audio formats, such as FLAC, MP3 or WAV, I recommend the use of libsndfile, which supports all of these formats with a uniform interface.

FLAC

FLAC stands for Free Lossless Audio Codec. Unlike most other formats, and like WAV, this format provides lossless audio. It uses a good compression algorithm to be smaller than usual WAV files, but it will be bigger than with Vorbis and Opus for example.

C/C++ integration

There is the official library by xiph.org.

If you intend to integrate also other audio formats, such as Opus or WAV, I recommend the use of libsndfile, which supports all of these formats with a uniform interface.

MIDI

This is probably the second well known audio format. Unlike sampled audio streams, like MP3 or WAV, this file format only contains music instructions. You can basically imagine its contents like this:

• play note C5 at 3 seconds on channel 1 using instrument 7
• stop note on channel 1 at 4 seconds
• and so on...

This this format does not include any audio samples, it is also the smallest of all file formats. However, in order to play a MIDI file, you will also need a so called SoundFont, which actually includes the instrument audio data that can be played.

If you have a MIDI file on your machine and play it with an audio player, it may sound different using different audio players or machines, since each player and each system may have a different default sound font they use for audio playback. If you want your MIDI file to sound the same for any player, you should provide your own sound font with your game. Luckily, you only need a single sound font (assuming it contains all instruments you need) and can have hundreds of little MIDI files in your game you can play.

C/C++ integration

For playing MIDI files, I suggest FluidSynth. You can load a sound font into it and then either play back an existing MIDI file or programmatically send MIDI commands to the library.

Module / Tracker music

These file formats are basically a combination of a MIDI file and a sound font (although the technical details are completely different).

A module file contains a list of audio commands to play and all the instrument (audio) data so a standalone module file can be played back. This makes a module file smaller than audio streams, but larger than MIDI files (if you don't count the sound font).

The four best known tracker formats are:

• MOD
• XM ("extended module" by Triton's FastTracker 2)
• S3M (ScreamTracker 3 Module)
• IT (Impulse Tracker)

C/C++ integration

If you want to play tracker music, I can recommend libopenmpt, or, if you need more control about the playback and event syncing, BASSMOD.

Multiple layers

If you want to use music in your game, you might want to work with several audio layers and adjusting a layer's volume depending on the area you are in the game. Let's say, you have the same music (tempo, length) for your overworld and underwater, but using different instruments.

If you work with audio streams such as Vorbis or Opus, you will need to have one file per layer and make sure that all layers are started playing exactly synchronously, but all layers except for one being muted. During the gameplay, you can adjust the layers' volumes dynamically.

MIDI and tracker files both work with multiple channels for different instruments. Depending on the options your library has, you can work with a single audio file containing all layers and adjust a channels volume dynamically.

Loops

Audio streams normally do not support loop points. Depending on the music, you may simply loop the whole music and it sounds well. But if you have a music with an intro that shall only be played once, you will need a dedicated solution. The best way to go is to write down the music's loop points somewhere (such as a meta file) and after decoding the last audio sample, instead of jumping back to sample 0 of the file, you jump back to the specified loop point.

MIDI normally also do not have loop points. However, it's possible to add for example "cue" points to a MIDI file. You can remember the cue point's location and jump back to that when the end of music is reached.

Tracker music basically all support loop points. They either include a "restart position" property in the file header or a "jump to" command within the music itself. Tracker libraries will make use of these loop points and you don't need to do anything here.