Game Coding [ PART I ]

406 Chapter 13 n Game Audio { return -1L; } return static_cast<long>(pVorbisData->dataRead); } You might notice that the method that fakes the fclose() doesn’t do anything. Ordinarily, you might free the memory in the buffer, but since the raw sound data is managed by the resource cache, nothing needs to be done. Here’s what the ParseOgg() method looks like: bool OggResourceLoader::ParseOgg(char *oggStream, size_t length, shared_ptr<ResHandle> handle) { shared_ptr<SoundResourceExtraData> extra = static_pointer_cast<SoundResourceExtraData>(handle->GetExtra()); OggVorbis_File vf; // for the vorbisfile interface ov_callbacks oggCallbacks; OggMemoryFile *vorbisMemoryFile = new OggMemoryFile; vorbisMemoryFile->dataRead = 0; vorbisMemoryFile->dataSize = length; vorbisMemoryFile->dataPtr = (unsigned char *)oggStream; oggCallbacks.read_func = VorbisRead; oggCallbacks.close_func = VorbisClose; oggCallbacks.seek_func = VorbisSeek; oggCallbacks.tell_func = VorbisTell; int ov_ret = ov_open_callbacks(vorbisMemoryFile, &vf, NULL, 0, oggCallbacks); assert(ov_ret>=0); // ok now the tricky part // the vorbis_info struct keeps the most of the interesting format info vorbis_info *vi = ov_info(&vf,-1); memset(&extra->m_WavFormatEx, 0, sizeof(extra->m_WavFormatEx)); extra->m_WavFormatEx.cbSize = sizeof(extra->m_WavFormatEx); extra->m_WavFormatEx.nChannels = vi->channels; // ogg vorbis is always 16 bit extra->m_WavFormatEx.wBitsPerSample = 16; extra->m_WavFormatEx.nSamplesPerSec = vi->rate;

Game Sound System Architecture 407 extra->m_WavFormatEx.nAvgBytesPerSec = extra->m_WavFormatEx.nSamplesPerSec* extra->m_WavFormatEx.nChannels*2; extra->m_WavFormatEx.nBlockAlign = 2* extra->m_WavFormatEx.nChannels; extra->m_WavFormatEx.wFormatTag = 1; DWORD size = 4096 * 16; DWORD pos = 0; int sec = 0; int ret = 1; // get the total number of PCM samples DWORD bytes = (DWORD)ov_pcm_total(&vf, -1); bytes *= 2 * vi->channels; if (handle->Size() != bytes) { GCC_ERROR(“The Ogg size does not match the memory buffer size”); ov_clear(&vf); SAFE_DELETE(vorbisMemoryFile); return false; } // now read in the bits while(ret && pos<bytes) { ret = ov_read(&vf, handle->WriteableBuffer()+pos, size, 0, 2, 1, &sec); pos += ret; if (bytes - pos < size) { size = bytes - pos; } } extra->m_LengthMilli = 1000.f * ov_time_total(&vf, -1); ov_clear(&vf); delete vorbisMemoryFile; return true; } This method shows you how to decompress an OGG memory buffer using the Vor- bis API. The method will decompress the OGG stream into a PCM buffer that is essentially identical to the results you saw earlier with the WaveResourceLoader. The first part of the method initializes the OggMemoryFile structure and sets up the callback functions for Vorbis. Then a structure called vorbis_info is used to initialize the members of the WAVEFORMATEX, stored with the resource handle.

408 Chapter 13 n Game Audio After the memory buffer is double-checked to be big enough to handle the decom- pressed OGG stream, the ov_read function is called in a loop to decompress it. If you feel sufficiently energetic one weekend, this is where you’ll want to play around if you’d like to implement decompression of the OGG stream in real time. Instead of decompressing the entire buffer, you’ll decompress a part of it, save the stream where you left off, and let DirectSound play the buffer. Before DirectSound finishes playing the buffer, you’ll run the decompression loop again into a different buffer. If your timing is right, DirectSound will be playing from one buffer while you are decompressing into another. If you think this is touchy work, you are right; it is for this reason that sound systems were typically fraught with weird bugs and insta- bility. Imagine what would happen if the source OGG stream were thrown out of the resource cache, causing a cache miss and a huge delay in providing DirectSound with the data it needs to create the illusion of a continuous sound from a single uncom- pressed stream. Always Show Something Moving Any time you have a while loop that might take some time, such as decompressing a large OGG file, it’s a good idea to create a callback function that your game can use to monitor the progress of the routine. This might be important for creating a progress bar or some other animation that will give your players something to look at other than a completely stalled screen. Console games are usually required to have on-screen animations during loads, but this is a good idea for PC games, too. If you are just lifting this OGG code into your game and ignoring the rest of this chapter, don’t forget to link the Vorbis libraries into your project. Since there’s no encoding going on here, you can just link the following libraries: libvorbisfile_static.lib, libvorbis_static.lib, and libogg_static.lib. If you are compiling your code under Visual Studio, you can add the following lines of code to one of your CPP files. In the GameCode4 source, they are in GameCode4.cpp, where all of the other #pragma comment() statements are. #pragma comment(lib, “libogg_static.lib”) #pragma comment(lib, “libvorbis_static.lib”) #pragma comment(lib, “libvorbisfile_static.lib”) To learn more about the OGG format, go to www.xiph.org/. The technology is open source, the sound is every bit as good as MP3, and you don’t have to worry about paying expensive license fees. In other words, unless you have money to burn, use OGG for sound data compression. Lots of audio tools support OGG, too. You can go to the Xiph website to find out which ones.

Game Sound System Architecture 409 IAudioBuffer Interface and AudioBuffer Class Now that you’ve got a sound in memory, it’s time to play it. IAudioBuffer exposes methods such as volume control, pausing, and monitoring individual sounds while they are in memory. IAudioBuffer, and a partial implementation AudioBuffer, are meant to be platform agnostic. You’ll see the DirectSound specific implementa- tion shortly. Here’s the interface class: class IAudioBuffer { public: virtual ~IAudioBuffer() { } virtual void *VGet()=0; virtual shared_ptr<ResHandle> const VGetResource()=0; virtual bool VRestore()=0; virtual bool VPlay(int volume, bool looping)=0; virtual bool VPause()=0; virtual bool VStop()=0; virtual bool VResume()=0; virtual bool VTogglePause()=0; virtual bool VIsPlaying()=0; virtual bool VIsLooping() const=0; virtual void VSetVolume(int volume)=0; virtual int VGetVolume() const=0; virtual float VGetProgress() const=0; }; The first method is a virtual destructor, which will be overloaded by classes that implement the interface. If this destructor weren’t virtual, it would be impossible to release audio resources grabbed for this sound effect. The next method, VGet(), is used to grab an implementation-specific handle to the allocated sound. When I say implementation-specific, I’m talking about the piece of data used by the audio system implementation to track sounds internally. In the case of a DirectSound implementation, this would be a LPDIRECTSOUNDBUFFER. This is for internal use only, for whatever class implements the IAudio interface to call. Your high-level game code will never call this method unless it knows what the implementation is and wants to do something really specific. The next method, VRestore(), is primarily for Windows games since it is possible for them to lose control of their sound buffers, requiring their restoration. The audio system will double-check to see if an audio buffer has been lost before it sends

410 Chapter 13 n Game Audio commands to the sound driver to play the sound. If it has been lost, it will call the VRestore() method, and everything will be back to normal. Hopefully, anyway. The next four methods can control the play status on an individual sound effect. VPlay() gets a volume from 0–100 and a Boolean looping, which you set to true if you want the sound to loop. VPause(), VStop(), VResume(), and VPause() let you control the progress of a sound. The volume methods do exactly what you’d think they do: set and retrieve the cur- rent volume of the sound. The method that sets the volume will do so instantly, or nearly so. If you want a gradual fade, on the other hand, you’ll have to use something a little higher level. Luckily, we’ll do exactly that later on in this chapter. The last method, VGetProgress(), returns a floating-point number between 0.0f and 1.0f and is meant to track the progress of a sound as it is being played. If the sound effect is one-fourth of the way through playing, this method will return 0.25f. All Things Go from 0.0 to 1.0 Measuring things like sound effects in terms of a coefficient ranging from 0.0 to 1.0 instead of a number of milliseconds is a nice trick. This abstraction gives you some flexibility if the actual length of the sound effect changes, especially if it is timed with animations, or animations are tied to sound, which is very frequently the case. If either the sound changes or the animation changes, it is easy to track one versus the other. With the interface defined, we can write a little platform-agnostic code and create the AudioBuffer class. The real meat of this class is the management of the smart pointer to a SoundResource. This guarantees that the memory for your sound effect can’t go out of scope while the sound effect is being played. class AudioBuffer : public IAudioBuffer { public: virtual shared_ptr<ResHandle> VGetResource() { return m_Resource; } virtual bool VIsLooping() const { return m_isLooping; } virtual int VGetVolume() const { return m_Volume; } protected: AudioBuffer(shared_ptr<ResHandle >resource) { m_Resource = resource; m_isPaused = false;

Game Sound System Architecture 411 m_isLooping = false; m_Volume = 0; } // disable public construction shared_ptr<ResHandle> m_Resource; // Is the sound paused bool m_isPaused; // Is the sound looping bool m_isLooping; //the volume int m_Volume; }; This class holds the precious smart pointer to your sound data managed by the resource cache and implements the IAudioBuffer interface. VIsLooping() and VGetVolume() tell you if your sound is a looping sound and the current volume setting. VGetResource() returns a smart pointer to the sound resource, which manages the sound data. We’re nearly to the point where you have to dig into DirectSound. Before that hap- pens, take a look at the classes that encapsulate the system that manages the list of active sounds: IAudio and Audio. IAudio Interface and Audio Class IAudio has three main purposes: create, manage, and release audio buffers. class IAudio { public: virtual bool VActive()=0; virtual IAudioBuffer *VInitAudioBuffer(shared_ptr<ResHandle> soundResource)=0; virtual void VReleaseAudioBuffer(IAudioBuffer* audioBuffer)=0; virtual void VStopAllSounds()=0; virtual void VPauseAllSounds()=0; virtual void VResumeAllSounds()=0; virtual bool VInitialize()=0; virtual void VShutdown()=0; };

412 Chapter 13 n Game Audio VActive() is something you can call to determine if the sound system is active. As rare as it may be, a sound card might be disabled or not installed. It is also likely that during initialization or game shutdown, you’ll want to know if the sound system has a heartbeat. The next two methods, VInitAudioBuffer() and VReleaseAudioBuffer(), are called when you want to launch a new sound or tell the audio system you are done with it and it can release audio resources back to the system. This is important, so read it twice. You’ll call these for each instance of a sound, even if it is exactly the same effect. You might want to play the same sound effect at two different volumes, such as when two players are firing the same type of weapon at each other, or you have multiple explosions going off at the same time in different places. You’ll notice that the only parameter to the initialize method is a shared pointer to a ResHandle object. This object contains the single copy of the actual decompressed PCM sound data. The result of the call, assuming it succeeds, is a pointer to an object that implements the IAudioBuffer interface. What this means is that the audio system is ready to play the sound. The next three methods are system-wide sound controls, mostly for doing things like pausing and resuming sounds when needed, such as when the player on a Windows game Alt-Tabs away from your game. It’s extremely annoying to have game sound effects continue in the background if you are trying to check email or convince your boss you aren’t playing a game. The last two methods, VInitialize() and VShutdown(), are used to create and tear down the sound system. Let’s take a look at a platform-agnostic partial imple- mentation of the IAudio interface: class Audio : public IAudio { public: Audio(); virtual void VStopAllSounds(); virtual void VPauseAllSounds(); virtual void VResumeAllSounds(); virtual void VShutdown(); static bool HasSoundCard(void); bool IsPaused() { return m_AllPaused; } protected: typedef std::list<IAudioBuffer *> AudioBufferList;

Game Sound System Architecture 413 AudioBufferList m_AllSamples; // List of all currently allocated buffers bool m_AllPaused; // Has the sound system been paused? bool m_Initialized; // Has the sound system been initialized? }; We’ll use STL to organize the active sounds in a linked list called m_AllSamples. This is probably good for almost any game because you’ll most likely have only a handful of sounds active at one time. Linked lists are great containers for a small number of objects. Since the sounds are all stored in the linked list, and each sound object implements the IAudioBuffer interface, you can define routines that per- form an action on every sound in the system. void Audio::VShutdown() { AudioBufferList::iterator i=m_AllSamples.begin(); while (i!=m_AllSamples.end()) { IAudioBuffer *audioBuffer = (*i); audioBuffer->VStop(); m_AllSamples.pop_front(); } } //Stop all active sounds, including music void Audio::VPauseAllSounds() { AudioBufferList::iterator i; AudioBufferList::iterator end; for(i=m_AllSamples.begin(), end=m_AllSamples.end(); i!=end; ++i) { IAudioBuffer *audioBuffer = (*i); audioBuffer->VPause(); } m_AllPaused=true; } void Audio::VResumeAllSounds() { AudioBufferList::iterator i; AudioBufferList::iterator end; for(i=m_AllSamples.begin(), end=m_AllSamples.end(); i!=end; ++i) { IAudioBuffer *audioBuffer = (*i); audioBuffer->VResume();

414 Chapter 13 n Game Audio } m_AllPaused=false; } void Audio::VStopAllSounds() { IAudioBuffer *audioBuffer = NULL; AudioBufferList::iterator i; AudioBufferList::iterator end; for(i=m_AllSamples.begin(), end=m_AllSamples.end(); i!=end; ++i) { audioBuffer = (*i); audioBuffer->VStop(); } m_AllPaused=false; } The code for each of these routines iterates the list of currently playing sounds and calls the appropriate stop, resume, or pause method of the IAudioBuffer object. DirectSound Implementations The Audio and AudioBuffer classes are useless on their own; we must still create the platform-specific code. Since DirectSound is completely free to use by anyone, we’ll create our platform-specific code around that technology. You’ll need to extend this code if you want to play MP3 or MIDI. Still, DirectSound can make a good foundation for a game’s audio system. Let’s take a look at the implementation for DirectSoundAudio first, which extends the Audio class we just discussed: class DirectSoundAudio : public Audio { public: DirectSoundAudio() { m_pDS = NULL; } virtual bool VActive() { return m_pDS != NULL; } virtual IAudioBuffer *VInitAudioBuffer( shared_ptr<ResHandle> soundResource); virtual void VReleaseAudioBuffer(IAudioBuffer* audioBuffer); virtual void VShutdown(); virtual bool VInitialize(HWND hWnd); protected: IDirectSound8* m_pDS;

Game Sound System Architecture 415 HRESULT SetPrimaryBufferFormat( DWORD dwPrimaryChannels, DWORD dwPrimaryFreq, DWORD dwPrimaryBitRate ); }; The only piece of data in this class is a pointer to an IDirectSound8 object, which is DirectSound’s gatekeeper, so to speak. Initialization, shutdown, and creating audio buffers are all done through this object. One way to look at this is that DirectSoundAudio is a C++ wrapper around IDirectSound8. Let’s look at initialization and shutdown first: bool DirectSoundAudio::VInitialize(HWND hWnd) { if ( m_Initialized ) return true; m_Initialized=false; m_AllSamples.clear(); SAFE_RELEASE( m_pDS ); HRESULT hr; // Create IDirectSound using the primary sound device if( FAILED( hr = DirectSoundCreate8( NULL, &m_pDS, NULL ) ) ) return false; // Set DirectSound coop level if( FAILED( hr = m_pDS->SetCooperativeLevel( hWnd, DSSCL_PRIORITY) ) ) return false; if( FAILED( hr = SetPrimaryBufferFormat( 8, 44100, 16 ) ) ) return false; m_Initialized = true; return true; } This code is essentially lifted straight from the DirectX sound samples, so it might look pretty familiar. When you set the cooperative level on the DirectSound object, you’re telling the sound driver you want more control over the sound system, specif- ically how the primary sound buffer is structured and how other applications run at the same time. The DSSCL_PRIORITY level is better than DSSCL_NORMAL because you can change the format of the output buffer. This is a good setting for games that still want to allow background applications like Microsoft Messenger or Outlook to be able to send something to the speakers.

416 Chapter 13 n Game Audio Why bother? If you don’t do this, and set the priority level to DSSCL_NORMAL, you’re basically informing the sound driver that you’re happy with whatever pri- mary sound buffer format is in place, which might not be the same sound format you need for your game audio. The problem is one of conversion. Games use tons of audio, and the last thing you need is for every sound to go through some conver- sion process so it can be mixed in the primary buffer. If you have 100,000 audio files and they are all stored in 44KHz, the last thing you want is to have each one be converted to 22KHz, because it’s a waste of time. Take control and use DSSCL_PRIORITY. The call to SetPrimaryBufferFormat() sets your primary buffer format to a fla- vor you want; most likely, it will be 44KHz, 16-bit, and some number of channels that you feel is a good trade-off between memory use and the number of simulta- neous sound effects you’ll have in your game. For the purposes of this class, I’m choosing eight channels, but in a commercial game you could have 32 channels or even more. The memory you’ll spend with more channels is dependent on your sound hardware, so be cautious about grabbing a high number of channels—you might find some audio cards won’t support it. HRESULT DirectSoundAudio::SetPrimaryBufferFormat( DWORD dwPrimaryChannels, DWORD dwPrimaryFreq, DWORD dwPrimaryBitRate ) { // !WARNING! - Setting the primary buffer format and then using this // for DirectMusic messes up DirectMusic! // // If you want your primary buffer format to be 22KHz stereo, 16-bit // call with these parameters: SetPrimaryBufferFormat(2, 22050, 16); HRESULT hr; LPDIRECTSOUNDBUFFER pDSBPrimary = NULL; if( ! m_pDS ) return CO_E_NOTINITIALIZED; // Get the primary buffer DSBUFFERDESC dsbd; ZeroMemory( &dsbd, sizeof(DSBUFFERDESC) ); dsbd.dwSize = sizeof(DSBUFFERDESC); dsbd.dwFlags = DSBCAPS_PRIMARYBUFFER; dsbd.dwBufferBytes = 0; dsbd.lpwfxFormat = NULL;

Game Sound System Architecture 417 if( FAILED( hr = m_pDS->CreateSoundBuffer( &dsbd, &pDSBPrimary, NULL ) ) ) return DXUT_ERR( L”CreateSoundBuffer”, hr ); WAVEFORMATEX wfx; ZeroMemory( &wfx, sizeof(WAVEFORMATEX) ); wfx.wFormatTag = (WORD) WAVE_FORMAT_PCM; wfx.nChannels = (WORD) dwPrimaryChannels; wfx.nSamplesPerSec = (DWORD) dwPrimaryFreq; wfx.wBitsPerSample = (WORD) dwPrimaryBitRate; wfx.nBlockAlign = (WORD) (wfx.wBitsPerSample / 8 * wfx.nChannels); wfx.nAvgBytesPerSec = (DWORD) (wfx.nSamplesPerSec * wfx.nBlockAlign); if( FAILED( hr = pDSBPrimary->SetFormat(&wfx) ) ) return DXUT_ERR( L”SetFormat”, hr ); SAFE_RELEASE( pDSBPrimary ); return S_OK; } You have to love DirectSound. This method essentially makes two method calls, and the rest of the code simply fills in parameters. The first call is to CreateSound- Buffer(), which actually returns a pointer to the primary sound buffer where all your sound effects are mixed into a single sound stream that is rendered by the sound card. The second call to SetFormat() tells the sound driver to change the primary buffer’s format to one that you specify. The shutdown method, by contrast, is extremely simple: void DirectSoundAudio::VShutdown() { if(m_Initialized) { Audio::VShutdown(); SAFE_RELEASE(m_pDS); m_Initialized = false; } } The base class’s VShutdown() is called to stop and release all the sounds still active. The SAFE_RELEASE on m_pDS will release the IDirectSound8 object and shut down the sound system completely. The last two methods of the DirectSoundAudio class allocate and release audio buffers. An audio buffer is the C++ representation of an active sound effect. In our

418 Chapter 13 n Game Audio platform-agnostic design, an audio buffer is created from a sound resource, presum- ably something loaded from a file or more likely a resource file. IAudioBuffer *DirectSoundAudio::VInitAudioBuffer(shared_ptr<ResHandle> resHandle) { shared_ptr<SoundResourceExtraData> extra = ::static_pointer_cast<SoundResourceExtraData>(resHandle->GetExtra()); if( ! m_pDS ) return NULL; switch(soundResource->GetSoundType()) { case SOUND_TYPE_OGG: case SOUND_TYPE_WAVE: // We support WAVs and OGGs break; case SOUND_TYPE_MP3: case SOUND_TYPE_MIDI: // If it’s a midi file, then do nothing at this time... // maybe we will support this in the future GCC_ERROR(“MP3s and MIDI are not supported”); return NULL; break; default: GCC_ERROR(“Unknown sound type”); return NULL; } LPDIRECTSOUNDBUFFER sampleHandle; // Create the direct sound buffer, and only request the flags needed // since each requires some overhead and limits if the buffer can // be hardware accelerated DSBUFFERDESC dsbd; ZeroMemory( &dsbd, sizeof(DSBUFFERDESC) ); dsbd.dwSize = sizeof(DSBUFFERDESC); dsbd.dwFlags = DSBCAPS_CTRLVOLUME; dsbd.dwBufferBytes = resHandle->Size(); dsbd.guid3DAlgorithm = GUID_NULL; dsbd.lpwfxFormat = const_cast<WAVEFORMATEX *>(extra->GetFormat()); HRESULT hr; if( FAILED( hr = m_pDS->CreateSoundBuffer( &dsbd, &sampleHandle, NULL ) ) )

Game Sound System Architecture 419 { return NULL; } // Add handle to the list IAudioBuffer *audioBuffer = (IAudioBuffer *)(new DirectSoundAudioBuffer(sampleHandle, resHandle)); m_AllSamples.push_front(audioBuffer); return audioBuffer; } Notice the switch statement at the beginning of this code? It branches on the sound type, which signifies what kind of sound resource is about to play: WAV, MP3, OGG, or MIDI. In our simple example, we’re only looking at WAV data or OGG data that has been decompressed, so if you want to extend this system to play other kinds of sound formats, you’ll hook that new code in right there. For now, those other for- mats are short circuited and will force a failure. The call to IDirectSound8::CreateSoundBuffer() is preceded by setting vari- ous values of a DSBUFFERDESC structure that informs DirectSound what kind of sound is being created. Take special note of the flags, since that member controls what can happen to the sound. An example is the DSBCAPS_CTRLVOLUME flag, which tells DirectSound that we want to be able to control the volume of this sound effect. Other examples include DSBCAPS_CTRL3D, which enables 3D sound, or DSBCAPS_CTRLPAN, which enables panning control. Take a look at the DirectSound docs to learn more about this important structure. After we’re sure we’re talking about a sound data format we support, there are two things to do. First, the sound data is passed onto DirectSound’s CreateSoundBuf- fer() method, which creates an IDirectSoundBuffer8 object. Next, the Direct- Sound sound buffer is handed to our C++ wrapper class, DirectSound AudioBuffer, and inserted into the master list of sound effects managed by Audio. Releasing an audio buffer is pretty trivial: void DirectSoundAudio::VReleaseAudioBuffer(IAudioBuffer *sampleHandle) { sampleHandle->VStop(); m_AllSamples.remove(sampleHandle); } The call to IAudioBuffer::VStop() stops the sound effect, and it is then removed from the list of active sounds.

420 Chapter 13 n Game Audio The second piece of this platform-dependent puzzle is the implementation of the DirectSoundAudioBuffer, which picks up and defines the remaining unimple- mented virtual functions from the IAudioBuffer interface. class DirectSoundAudioBuffer : public AudioBuffer { protected: LPDIRECTSOUNDBUFFER m_Sample; public: DirectSoundAudioBuffer( LPDIRECTSOUNDBUFFER sample, shared_ptr<ResHandle> resource); virtual void *VGet(); virtual bool VRestore(); virtual bool VPlay(int volume, bool looping); virtual bool VPause(); virtual bool VStop(); virtual bool VResume(); virtual bool VTogglePause(); virtual bool VIsPlaying(); virtual void VSetVolume(int volume); private: HRESULT FillBufferWithSound( ); HRESULT RestoreBuffer( BOOL* pbWasRestored ); }; The methods in this class are pretty easy C++ wrappers around IDirectSound Buffer8. The exceptions are FillBufferWithSound() and RestoreBuffer(). DirectSoundAudioBuffer::DirectSoundAudioBuffer( LPDIRECTSOUNDBUFFER sample, shared_ptr<CSoundResource> resource) : AudioBuffer(resource) { m_Sample = sample; FillBufferWithSound(); } void *DirectSoundAudioBuffer::VGet() { if (!VRestore()) return NULL;

Game Sound System Architecture 421 return m_Sample; } bool DirectSoundAudioBuffer::VPlay(int volume, bool looping) { if(!g_Audio->VActive()) return false; VStop(); m_Volume = volume; m_isLooping = looping; LPDIRECTSOUNDBUFFER pDSB = (LPDIRECTSOUNDBUFFER)VGet(); if (!pDSB) return false; pDSB->SetVolume( volume ); DWORD dwFlags = looping ? DSBPLAY_LOOPING : 0L; return (S_OK==pDSB->Play( 0, 0, dwFlags ) ); } bool DirectSoundAudioBuffer::VStop() { if(!g_Audio->VActive()) return false; LPDIRECTSOUNDBUFFER pDSB = (LPDIRECTSOUNDBUFFER)VGet(); if( pDSB ) return false; m_isPaused=true; pDSB->Stop(); return true; } bool DirectSoundAudioBuffer::VResume() { m_isPaused=false; return VPlay(VGetVolume(), VIsLooping()); } bool DirectSoundAudioBuffer::VTogglePause() { if(!g_Audio->VActive()) return false;

422 Chapter 13 n Game Audio if(m_isPaused) { VResume(); } else { VPause(); } return true; } bool DirectSoundAudioBuffer::VIsPlaying() { if(!g_Audio->VActive()) return false; DWORD dwStatus = 0; LPDIRECTSOUNDBUFFER pDSB = (LPDIRECTSOUNDBUFFER)VGet(); pDSB->GetStatus( &dwStatus ); bool bIsPlaying = ( ( dwStatus & DSBSTATUS_PLAYING ) != 0 ); return bIsPlaying; } void DirectSoundAudioBuffer::VSetVolume(int volume) { const int gccDSBVolumeMin = DSBVOLUME_MIN; if(!g_Audio->VActive()) return; LPDIRECTSOUNDBUFFER pDSB = (LPDIRECTSOUNDBUFFER)VGet(); assert(volume>=0 && volume<=100 && “Volume must be between 0 and 100”); // convert volume from 0-100 into range for DirectX // Don’t forget to use a logarithmic scale! float coeff = (float)volume / 100.0f; float logarithmicProportion = coeff >0.1f ? 1+log10(coeff) : 0; float range = (DSBVOLUME_MAX - gccDSBVolumeMin ); float fvolume = ( range * logarithmicProportion ) + gccDSBVolumeMin ; GCC_ASSERT(fvolume>=gccDSBVolumeMin && fvolume<=DSBVOLUME_MAX); HRESULT hr = pDSB->SetVolume( LONG(fvolume) ); GCC_ASSERT(hr==S_OK); }

Game Sound System Architecture 423 Most of the previous code has a similar structure and is a lightweight wrapper around IDirectSoundBuffer8. The first few lines check to see if the audio system is running, the audio buffer has been initialized, and parameters have reasonable values. Take note of the VSetVolume method; it has to renormalize the volume value from 0–100 to a range compatible with DirectSound, and it does so with a log- arithmic scale, since sound intensity is logarithmic in nature. The last three methods in this class are a little trickier, so I’ll give you a little more detail on them. The first, VRestore(), is called to restore sound buffers if they are ever lost. If that happens, you have to restore it with some DirectSound calls and then fill it with sound data again—it doesn’t get restored with its data intact. The VRestore() method calls RestoreBuffer() to restore the sound buffer, and if that is successful, it calls FillBufferWithSound() to put the sound data back where it belongs. bool DirectSoundAudioBuffer::VRestore() { HRESULT hr; BOOL bRestored; // Restore the buffer if it was lost if( FAILED( hr = RestoreBuffer( &bRestored ) ) ) return NULL; if( bRestored ) { // The buffer was restored, so we need to fill it with new data if( FAILED( hr = FillBufferWithSound( ) ) ) return NULL; } return true; } This implementation of RestoreBuffer() is pretty much lifted from the Direct- Sound samples. Hey, at least I admit to it! If you’re paying attention, you’ll notice an unfortunate bug in the code—see if you can find it: HRESULT DirectSoundAudioBuffer::RestoreBuffer( BOOL* pbWasRestored ) { HRESULT hr; if( m_Sample ) return CO_E_NOTINITIALIZED; if( pbWasRestored ) *pbWasRestored = FALSE;

424 Chapter 13 n Game Audio DWORD dwStatus; if( FAILED( hr = m_Sample->GetStatus( &dwStatus ) ) ) return DXUT_ERR( L”GetStatus”, hr ); if( dwStatus & DSBSTATUS_BUFFERLOST ) { // Since the app could have just been activated, then // DirectSound may not be giving us control yet, so // the restoring the buffer may fail. // If it does, sleep until DirectSound gives us control but fail if // if it goes on for more than 1 second int count = 0; do { hr = m_Sample->Restore(); if( hr == DSERR_BUFFERLOST ) Sleep( 10 ); } while( ( hr = m_Sample->Restore() ) == DSERR_BUFFERLOST && ++count < 100 ); if( pbWasRestored != NULL ) *pbWasRestored = TRUE; return S_OK; } else { return S_FALSE; } } The bug in the method is the termination condition of the do/while loop; it could try forever, assuming DirectSound was in some wacky state. This could hang your game and cause your players to curse your name and post all kinds of nasty things on the Internet. Making the code better depends on what you want to do when this kind of failure happens. You likely would throw up a dialog box and exit the game. It’s totally up to you. The lesson here is that just because you grab something directly from a DirectX sample doesn’t mean you should install it into your game unmodified! The next method is FillBufferWithSound(). Its job is to copy the sound data from a sound resource into a prepared and locked sound buffer. There’s also a bit of code to handle the special case where the sound resource has no data—in that case, the sound buffer gets filled with silence. Notice that “silence” isn’t necessarily a buffer with all zeros.

Game Sound System Architecture 425 HRESULT DirectSoundAudioBuffer::FillBufferWithSound( void ) { HRESULT hr; VOID *pDSLockedBuffer = NULL; // DirectSound buffer pointer DWORD dwDSLockedBufferSize = 0; // Size of DirectSound buffer DWORD dwWavDataRead = 0; // Data to read from the wav file if( m_Sample ) return CO_E_NOTINITIALIZED; // Make sure we have focus, and we didn’t just switch in from // an app which had a DirectSound device if( FAILED( hr = RestoreBuffer( NULL ) ) ) return DXUT_ERR( L”RestoreBuffer”, hr ); int pcmBufferSize = m_Resource->Size(); shared_ptr<SoundResourceExtraData> extra = static_pointer_cast<SoundResourceExtraData>(m_Resource->GetExtra()); // Lock the buffer down if( FAILED( hr = m_Sample->Lock( 0, pcmBufferSize, &pDSLockedBuffer, &dwDSLockedBufferSize, NULL, NULL, 0L ) ) ) return DXUT_ERR( L”Lock”, hr ); if( pcmBufferSize == 0 ) { // Wav is blank, so just fill with silence FillMemory( (BYTE*) pDSLockedBuffer, dwDSLockedBufferSize, (BYTE)(m_Resource->GetFormat()->wBitsPerSample == 8 ? 128 : 0 ) ); } else { CopyMemory(pDSLockedBuffer, m_Resource->GetPCMBuffer(), pcmBufferSize); if( pcmBufferSize < (int)dwDSLockedBufferSize ) { // If the buffer sizes are different fill in the rest with silence FillMemory( (BYTE*) pDSLockedBuffer + pcmBufferSize, dwDSLockedBufferSize - pcmBufferSize, (BYTE)(m_Resource->GetFormat()->wBitsPerSample == 8 ? 128 : 0 ) ); } }

426 Chapter 13 n Game Audio // Unlock the buffer, we don’t need it anymore. m_Sample->Unlock( pDSLockedBuffer, dwDSLockedBufferSize, NULL, 0 ); return S_OK; } There’s also some special case code that handles the case where the DirectSound buffer is longer than the sound data—any space left over is filled with silence. There’s one last method to implement in the IAudioBuffer interface, the VGetProgress() method: float DirectSoundAudioBuffer::VGetProgress() { LPDIRECTSOUNDBUFFER pDSB = (LPDIRECTSOUNDBUFFER)VGet(); DWORD progress = 0; pDSB->GetCurrentPosition(&progress, NULL); float length = (float)m_Resource->Size(); return (float)progress / length; } This useful little routine calculates the current progress of a sound buffer as it is being played. Sound plays at a constant rate, so things like music and speech will sound exactly as they were recorded. It’s up to you, the skilled programmer, to get your game to display everything exactly in sync with the sound. You do this by poll- ing the sound effect’s progress when your game is about to start or change an animation. Perhaps you have an animation of a window cracking and then shattering. You’d launch the sound effect and animation simultaneously, call VGetProgress() on your sound effect every frame, and set your animation progress accordingly. This is especially important because players can detect even tiny miscues between sound effects and animation. Sound Processes All of the classes you’ve seen so far form the bare bones of an audio system for a computer game. What’s missing is some way to launch and monitor a sound effect as it is playing, perhaps to coordinate it with an animation. If you paid some atten- tion in Chapter 7, you’ll remember the Process class. It turns out to be perfect for this job. class SoundProcess : public Process {

Game Sound System Architecture 427 public: SoundProcess( shared_ptr<ResHandle> soundResource, int typeOfSound=PROC_SOUNDFX, int volume=100, bool looping=false); virtual ~SoundProcess(); virtual void VOnUpdate(const int deltaMilliseconds); virtual void VOnInitialize(); virtual void VKill(); virtual void VTogglePause(); void Play(const int volume, const bool looping); void Stop(); void SetVolume(int volume); int GetVolume(); int GetLengthMilli(); bool IsSoundValid() { return m_SoundResource!=NULL; } bool IsPlaying(); bool IsLooping() { return m_AudioBuffer->VIsLooping(); } float GetProgress(); protected: SoundProcess(); //Disable Default Construction void InitializeVolume(); void Replay() { m_bInitialUpdate = true; }; shared_ptr<ResHandle> m_SoundResource; shared_ptr<IAudioBuffer> m_AudioBuffer; int m_Volume; bool m_isLooping; }; This class provides a single object that manages individual sounds. Many of the methods are re-implementations of some IAudioBuffer methods, and while this isn’t the best C++ design, it can make things a little easier in your code. As you might expect, the parameters to initialize this object are a ResHandle and initial sound settings. One parameter needs a little explanation, typeOfSound. Every process has a type, and sound processes use this to distinguish themselves

428 Chapter 13 n Game Audio into sound categories such as sound effects, music, ambient background effects, or speech. This creates an easy way for a game to turn off or change the volume level of a particular type of sound, which most gamers will expect. If players want to turn down the music level so they can hear speech better, it’s a good idea to let them. SoundProcess::SoundProcess( shared_ptr<ResHandle> soundResource, int typeOfSound, int volume, bool looping) : CProcess(typeOfSound, 0), m_SoundResource(soundResource), m_Volume(volume), m_isLooping(looping) { InitializeVolume(); } SoundProcess::~SoundProcess() { if (m_AudioBuffer) { g_Audio->VReleaseAudioBuffer(m_AudioBuffer.get()); } } The meat of the code in SoundProcess is in the next few methods. One important concept to understand about sounds is that the code might create a sound process long before the sound should be played or even loaded. Since sound effects tend to require a lot of data, it’s a good idea to be careful about when you instantiate sound effects. After all, you don’t want your game to stutter or suffer wacky pauses. The code shown next assumes the simplest case, where you want the sound to begin play- ing immediately, but it’s good to know that you don’t have to do it this way. void SoundProcess::VOnInitialize() { if ( m_handle == NULL || m_handle->GetExtra() == NULL) return; //This sound will manage its own handle in the other thread IAudioBuffer *buffer = g_Audio->VInitAudioBuffer(m_handle); if (!buffer) { VKill(); return; }

Game Sound System Architecture 429 m_AudioBuffer.reset(buffer); Play(m_Volume, m_isLooping); } The VOnUpdate method monitors the sound effect as it’s being played. Once it is finished, it kills the process and releases the audio buffer. If the sound is looping, it will play until some external call kills the process. Again, you don’t have to do it this way in your game. Perhaps you’d rather have the process hang out until you kill it explicitly: void SoundProcess::VOnUpdate(const int deltaMilliseconds) { // Call base Process::VOnUpdate(deltaMilliseconds); if ( IsDead() && IsLooping() ) { Replay(); } } This class overloads the VKill() method to coordinate with the audio system. If the process is going to die, so should the sound effect. void SoundProcess::VKill() { if ( IsPlaying() ) Stop(); Process::VKill(); } Notice that the base class’s VKill() is called at the end of the method, rather than the beginning. You can look at VKill() similar to a destructor, which means this calling order is a safer way to organize the code. As advertised, the remaining methods do nothing more than pass calls into the IAudioBuffer object. bool SoundProcess::IsPlaying() { if ( ! m_handle || ! m_AudioBuffer ) return false; return m_AudioBuffer->VIsPlaying(); }

430 Chapter 13 n Game Audio int SoundProcess::GetLengthMilli() { if ( m_handle && handle->GetExtra() ) { shared_ptr<SoundResourceExtraData> extra = static_pointer_cast<SoundResourceExtraData>(m_handle->GetExtra()); return extra->GetLengthMilli(); } else return 0; } void SoundProcess::SetVolume(int volume) { if(m_AudioBuffer==NULL) return; GCC_ASSERT(volume>=0 && volume<=100 && “Volume must be a number between 0 and 100”); m_Volume = volume; m_AudioBuffer->VSetVolume(volume); } int SoundProcess::GetVolume() { if(m_AudioBuffer==NULL) return 0; m_Volume = m_AudioBuffer->VGetVolume(); return m_Volume; } void SoundProcess::TogglePause() { if(m_AudioBuffer) m_AudioBuffer->VTogglePause(); } void SoundProcess::Play(const int volume, const bool looping) { GCC_ASSERT(volume>=0 && volume<=100 && “Volume must be a number between 0 and 100”);

Game Sound System Architecture 431 if(!m_AudioBuffer) return; m_AudioBuffer->VPlay(volume, looping); } void SoundProcess::Stop() { if(m_AudioBuffer) m_AudioBuffer->VStop(); } float SoundProcess::GetProgress() { if (m_AudioBuffer) return m_AudioBuffer->VGetProgress(); return 0.0f; } Launching Sound Effects The only thing you need to see now is how to tie all this together to launch and monitor a sound effect in your game. It may seem a little anticlimactic, but here it is: Resource resource(“SpaceGod7-Level2.ogg”); shared_ptr<ResHandle> rh = g_pApp->m_ResCache->GetHandle(&resource); shared_ptr<SoundProcess> sfx(new SoundProcess(srh, PROC_MUSIC, 100, true)); m_pProcessManager->Attach(sfx); There’s clearly an awful lot of work going on in the background, all of which you now know how to do. Launching a sound effect ties together much of the code you’ve seen in this book: a cooperative multitasker, the resource cache, and a bit of DirectX, which launches an extra thread to manage the problem of getting data to the sound card at exactly the right speed. Still, it’s nice to know that all that functionality can be accessed with three lines of code. If you want to launch three sound effects based on the same data, one playing as soon as the other is complete, here’s how you do it. Each one plays at a lower volume level than the one before it. Resource resource(“blip.wav”); shared_ptr<SoundProcess> sfx1(new SoundProcess(srh, PROC_SOUNDFX, 100, false)); shared_ptr<SoundProcess> sfx2(new SoundProcess(srh, PROC_SOUNDFX, 60, false)); shared_ptr<SoundProcess> sfx3(new SoundProcess(srh, PROC_SOUNDFX, 40, false));

432 Chapter 13 n Game Audio m_pView->m_pProcessManager->Attach(sfx1); sfx1->SetNext(sfx2); sfx2->SetNext(sfx3); Other Technical Hurdles There are a few more wires to connect, in code and in your brain, before you’re ready to install a working sound system in your game. Most sounds are tied directly to game objects or events. Even music is tied to the intensity of the game or, even bet- ter, the impending intensity of the game! Tying sounds to game objects and synchro- nization are critical problems in any game sound system. If you have multiple effects at one time, you’ll also have to worry about mixing issues. Sounds and Game Objects Imagine the following game situation: A wacky machine in a lab is active and makes some kind of “wub-wub-wub” sound tied to an animation. Your hero, armed with his favorite plasma grenade, tosses one over to the machine and stands back to watch the fun. The grenade explodes, taking the wacky machine and the “wub-wub-wub” noise with it. What’s really going on in the background? Your game has some grand data structure of game actors, one of which is the doomed machine. When the grenade goes off, there’s likely a bit of code or script that searches the local area for actors that can be damaged. Each actor in the blast radius will get some damage, and the code that applies damage will notice the machine is a gonner. What happens next is a core technical problem in computer games: When the machine is destroyed, related game actors or systems should be notified. This can include things like sound effects or animation processes. Most games solve this with the trigger/event system, and this is no exception. For purposes of clarity, the audio system code presented in this chapter has no such hook, but the event system you read about in Chapter 11, “Game Event Manage- ment,” is exactly what is needed to solve this problem. You’ll see how these two sys- tems are tied together in Chapter 21, “A Game of Teapot Wars.” Timing and Synchronization Imagine the following problem: You have a great explosion graphics effect that has a secondary and tertiary explosion after the initial blast. How could you synchronize the graphics to each explosion? The pacing of the sound is pretty much constant, so the graphics effect should monitor the progress of the sound and react accordingly. We can use the Process class to make this work:

Other Technical Hurdles 433 class ExplosionProcess : public Process { public: ExplosionProcess() : Process(PROC_GAMESPECIFIC) { m_Stage=0; } protected: int m_Stage; shared_ptr<SoundProcess> m_Sound; virtual void VOnUpdate(unsigned long deltaMs); virtual void VOnInit(); }; void ExplosionProcess::VOnInit() { Process::VOnInit(); Resource resource(“explosion.wav”); shared_ptr<ResHandle> srh = g_pApp->m_ResCache->GetHandle(&resource); m_Sound.reset(GCC_NEW SoundProcess(srh)); // Imagine cool explosion graphics setup code here!!!! // // // } void ExplosionProcess::VOnUpdate(unsigned long deltaMs) { float progress = m_Sound->GetProgress(); switch (m_Stage) { case 0: if (progress>0.55f) { ++m_Stage; // Imagine secondary explosion effect launch right here! } break; case 1: if (progress>0.75f) { ++m_Stage; // Imagine tertiary explosion effect launch right here! } break;

434 Chapter 13 n Game Audio default: break; } } The ExplosionProcess owns a sound effect and drives the imaginary animation code. The sound effect is initialized during the VOnInit() call, and VOnUpdate() handles the rest as you’ve seen before. There’s a trivial state machine that switches state as the sound progresses past some constant points, 55 percent and 75 percent of the way through. Do you notice the hidden problem with this code? This is a common gotcha in com- puter game design. What happens if the audio designer decides to change the sound effect and bring the secondary explosion closer to the beginning of the sound? It’s equally likely an artist will change the timing of an animated texture, which could have exactly the same effect. Either way, the explosion effect looks wrong, and it’s anyone’s guess who will get the bug: programmer, artist, or audio engineer. The Butterfly Effect Code, animations, and sound data are tightly coupled and mutually dependent entities. You can’t easily change one without changing the others, and you can make your life hell by relating all three with hard-coded constants. There’s no silver bullet for this problem, but there are preventative measures. It might seem like more work, but you could consider factoring the ExplosionClass into three distinct entities, each with its own animation and sound data. Either way, make sure that you have some asset tracking so you can tell when someone changes anything in your game: code, sounds, or animations. When something breaks unexpectedly, the first thing you check is changes to the files. Mixing Issues Sound in your game will include sound effects, music, and speech. Depending on what’s going on, you might have to change the volume of one or more of these ele- ments to accentuate it. A good example of this is speech, and I’m not talking about the random barks of a drunken guard. Games will introduce clues and objectives with AI dialogue, and it’s important that the player be able to hear it. If you played Thief: Deadly Shadows, there’s a good example of this in the Seaside Mansion mission about halfway through the game. The thunder and lightning outside were so loud it was difficult to hear the AI dialogue. That’s one of the reasons there is this notion of SoundType in our audio system. It gives you a bit of data to hang on to if you want to globally change the volume of

Other Technical Hurdles 435 certain sounds. In the case of Thief, it would have been a good idea to cut the volume of the storm effects so the game clues in the AI dialogue would be crystal clear. Don’t Depend on Dialogue Dialogue is great to immerse players in game fiction, but you can’t depend on it 100 percent. If you give critical clues and objectives via dialogue, make sure that you have some secondary way to record and represent the objectives, such as a special screen where the player can read a synopsis. It’s too easy for a player to miss something. If your game enables subtitles, you can provide a screen that shows the last conversation. While we’re talking about mixing, you’ve got to take some care when changing the levels of sound effects. Any discrete jump in volume is jarring. Solve this problem with a simple fade mechanism: FadeProcess::FadeProcess( shared_ptr<SoundProcess> sound, int fadeTime, int endVolume) : Process(PROC_INTERPOLATOR) { m_Sound = sound; m_TotalFadeTime = fadeTime; m_StartVolume = sound->GetVolume(); m_EndVolume = endVolume; m_ElapsedTime = 0; VOnUpdate(0); } void FadeProcess::VOnUpdate(unsigned long deltaMs) { if (!m_bInitialUpdate) m_ElapsedTime += deltaMilliseconds; Process::VOnUpdate(deltaMilliseconds); if (m_Sound->IsDead()) Succeed(); float cooef = (float)m_ElapsedTime / m_TotalFadeTime; if (cooef>1.0f) cooef = 1.0f;

436 Chapter 13 n Game Audio if (cooef<0.0f) cooef = 0.0f; int newVolume = m_StartVolume + ( float(m_EndVolume - m_StartVolume) * cooef); if (m_ElapsedTime >= m_TotalFadeTime) { newVolume = m_EndVolume; Succeed(); } m_Sound->SetVolume(newVolume); } This class can change the volume of a sound effect over time, either up or down. It assumes the initial volume of the sound effect has already been set properly, and all the times are given in milliseconds. Here’s how you would create some background music and fade it in over 10 seconds: Resource resource(“SpaceGod7-Level2.ogg”); shared_ptr<ResHandle> rh = g_pApp->m_ResCache->GetHandle(&resource); shared_ptr<SoundProcess> music(GCC_NEW SoundProcess(rh, PROC_MUSIC, 0, true)); m_pProcessManager->Attach(music); shared_ptr<FadeProcess> fadeProc(new FadeProcess(music, 10000, 100)); m_pProcessManager->AttachProcess(fadeProc); The fade process grabs a smart pointer reference to the sound it is modifying, and once the volume has been interpolated to the final value, the process kills itself. Note that the original sound is created with a zero volume, and the fade process brings it up to 100. Simlish Sims have their own language of gibberish called “Simlish,” which is meant to sound like a real language. While there are a few words and phrases that come to have actual translations, it’s mostly just designed to sound appropriate and go with the tone of the conversation. Even the music has to use this fake language, which presents an interesting challenge to the musicians who create the songs you can hear when your sim turns on its radio. How was Simlish born? Well, there is no possible way to record all the speech and dialogue required for each interaction without it sounding completely stale after the first few times. By having Simlish, we are more easily able to create that illusion without shipping a terabyte of speech.

Some Random Notes 437 Some Random Notes In the last 50 pages or so, you’ve read about sound resources, audio buffers, audio systems, and sound processes. This is where most books would stop. Neither my editor nor my readers will find any surprise in me continuing on a bit about sound. Why? I haven’t told you a thing about how to actually use sound in your game. Data-Driven Sound Settings Sometimes I think this book is equally good at showing you how not to code game technology as it is at showing you how to code correctly. The observant programmer would notice that all my sound examples in the previous pages all had hard-coded constants for things like volume, fade-in times, or animation points. From day one, most programmers learn that hard-coded constants are a bad thing, and they can become a complete nightmare in computer game programming. The reason that I use so many of them in this book is because they make the code easier to read and understand. Real computer games would load all of this data at runtime from a data file. If the data changes, you don’t have to recompile. If the game can reload the data at runtime with a cheat key, you can test and tweak this kind of data and get instant feedback. With this kind of data-driven solution, programmers don’t even have to be in the building when the audio guys are making the sounds. Believe it or not, they actually work later than programmers. This leaves programmers doing what they do best— programming game technology! A bit of volume data can also be tweaked more eas- ily than the original sound file can be releveled, so your audio engineer doesn’t have to be in the building, either. So who’s left to set the level on new sound effects on a Saturday? It’s so easy that even a producer could do it. For those of you outside the game industry, I could as well have said, “It’s so easy, your boss could do it!” Record All Audio at Full Volume Every sound effect should be recorded at full volume, even if it is way too loud for the game. This gives the sound the highest degree of waveform accuracy before it is attenuated and mixed with the other sounds in the primary sound buffer.

438 Chapter 13 n Game Audio Background Ambient Sounds and Music Most games have a music track and an ambient noise track along with the incident sounds you hear throughout the game. These additional tracks are usually long because they’ll most likely loop back to their beginning until there’s some environ- mental reason to change them. An example of an ambient noise track could be the sounds of a factory, crowd noises, traffic, or some other noise. Sounds like these have the effect of placing the players in the environment and give the impression that there’s a lot more going on than what they see. This technique was used brilliantly in Thief: Deadly Shadows in the city sections. You could clearly hear city dwellers, small animals, carts, and other such noise, and it made you feel as if you were in the middle of a medieval city. But be warned—if the background ambient has recognizable elements, such as some- one saying, “Good morning,” players will recognize the loop quickly and get annoyed. Keep any ambient background close to “noise” instead of easily discernible sounds. Music adds to this environment by communicating gut-level intensity. It helps inform the player and add a final polish to the entire experience. Perhaps my favorite example of music used in a game is the original Halo. Instead of only reacting to the current events in the game, the music segues just before a huge battle, telling the player he’d better reload his shotgun and wipe the sweat off his controller. Live Music Rocks—from Professional Musicians On the Microsoft Casino project, I thought it would be a great idea to record live music. It would be classy and add a unique feel to the game. I’d never produced live music for a game, so I was a little nervous about it. The music was composed by an Austin composer, Paul Baker, who also conducted the band. I got to play the part of the big-time producer guy behind the glass in the recording studio. I thought it was really cool until I heard the band run through the music the first time. It was horrible! I thought I’d be fired and wondered quickly how I could salvage this disaster. My problem was that I’d never seen how professional musicians work. They arrived that day having never seen the music before, and the first time they played it they were all sight-reading. After a break, they played it one more time, and it was nearly flawless. I thought I’d just heard a miracle, but that’s just my own naiveté. There was one errant horn note, blurted a little off time, and they cleared everyone out of the room except for the one horn player. He said, “gimme two measures,” and they ran the tape back. At exactly the right moment, he played the single note he screwed up, and it was mixed flawlessly into the recording. I was so impressed by the live performance that I’ll never be afraid of doing it in other games.

Some Random Notes 439 The CPU budget for all this sound decompression is definitely a factor. The longish music and ambient noise tracks will certainly be streamed from a compressed source rather than loaded into memory in uncompressed PCM format. Different platforms and decompression algorithms will take different and sometimes surprising amounts of your CPU time, so you should keep track of this. Do some benchmarks on your own and set your budgets accordingly. Speech In-game character speech is a great design technique to immerse the player and add dimension to the AI characters. Games use character speech to communicate story, provide clues, and show alert levels in patrolling guards. Players expect a great script, smooth integration of the speech effects with the graphics, and most importantly, the ability to ignore it if they want to. Random Barks A bark is another way of saying “filler speech.” It usually describes any bit of speech that is not part of a scripted sequence. Good examples of this are AI characters talking to themselves or reactions to game events like throwing a grenade around a corner. Some of my favorite random barks came from the drunk guard in Thief: Deadly Sha- dows. It was perfect comic relief in what is normally a dark game with long stretches of tension. If you hid in a nearby shadow, you’d hear this inebriated and somewhat mentally challenged guard talk to himself for a really long time. In the background, a piece of code selects a random speech file at random intervals. The code must keep track of the barks so it doesn’t play the same thing three times in a row, and it should also make sure that the random barks don’t overlap any scripted speech the AI character might have, either. Something that works well is a queue for each AI character regarding speech. Here a high-priority scripted bark will wipe out any random barks in the queue, and you can keep already barked elements in the queue to record which ones have played in recent history. Too Much of a Good Thing Is True Back on Microsoft Casino, there was this “blue-haired old lady” character that was our first AI character installed in the game. She only had one random bark: “Have you even been to Hoover Dam?,” she would say, waiting for her cards. We sent the build to Microsoft QA, and we waited patiently for the phone call, which came way too quickly for it to be news of acceptance. The lead QA on the phone told us that after just one hour of

440 Chapter 13 n Game Audio testing, no one in QA was “ever likely to ever visit the @%$#& Hoover Dam” and could we please remove the bark technology until more barks were recorded. The bark was so reviled, we had to remove it entirely from the game. Game Fiction Characters talking among themselves or straight to the player are an excellent way to give objectives or clues. If you do this, you must beware of a few gotchas. First, you shouldn’t force the player to listen to the speech again if he’s heard it before. Second, you should record a summary of the clue or objective in a form that can be refer- enced later—it’s too easy to miss something in a speech-only clue. The third gotcha involves localization. One game that comes to mind that solved the first gotcha in an interesting way was Munch’s Odyssey, an Xbox launch title. Throughout the game, this funny mystic character appears and tells you exactly what you need to do to beat the level. If you’ve heard the spiel before, you can hit a button on the controller and your char- acter, Abe or Munch, will interrupt and say something like, “I’ve heard all this before,” and the mystic guy will bark, “Whatever…” and disappear. Very effective. The second gotcha is usually solved with some kind of in-game notebook or objec- tives page. It’s too easy to miss something in scripted speech, especially when it tends to be more colorful and therefore a little easier to miss the point entirely. Almost every mission-based game has this design conceit—it keeps the players on track and keeps them from getting lost. The last gotcha has to do with language translation. What if you record your speech in English and want to sell your game in Japan? Clearly, the solution involves some kind of subtitle to go along with the scripted speech. Re-recording the audio can be prohibitively expensive if you have a huge script and a small budget. Lip Synching Synching the speech files to in-game character animations is challenging, from both a technology and tools standpoint. This is yet another one of those topics that could use an entire book, so let me at least convince you that the problem is a big one and not one to be taken lightly or without experience. It turns out that human speech has a relatively small number of unique sounds and sound combinations. You can create convincing speech with only a few dozen sounds. Since each sound is made with a particular position of the mouth and ton- gue, it follows that an artist can create these positions in a character ahead of time.

The Last Dance 441 While the speech is being played, a data stream is fed into the animation system that tells which phoneme to show on the character at each point in time. The animation system interpolates smoothly between each sound extreme, and you get the illusion that the character is actually talking. Cheap Hacks for Lip Synching There are ways of doing lip synching on the cheap, depending on your game. Interstate 76 solved the lip synching problem by removing all the lips; none of their characters had mouths at all! Another clever solution is to have the characters animate to a particular phrase like “peas and carrots, peas and carrots.” Wing Commander II, a game published by Origin Systems in the mid-1990s, had all its characters lip-synched to a single phrase: “Play more games like Wing Commander II.” Recording Speech Most games will use at most a few hundred sound effects. This relatively small num- ber of sound files is trivial to manage compared to speech files in some games. Thief: Deadly Shadows had somewhere around 10,000 lines of speech. You can’t just throw these files into a single directory. You’ve got to be organized. The first thing you need is a script—one for each character. As part of the script, the character should be described in detail, even to the point of including a rendering of the character’s face. Voice actors use the character description to get “into character” and create a great performance. The recording session will most likely have the voice actor read the entire script from top to bottom, perhaps repeating a line or phrase a few times to get it right. It’s criti- cal to keep accurate records about which one of the lines you intend to keep and which you’ll throw away. A few days later, you could find it difficult to remember. You’ll record to DAT tape or some other high-fidelity media and later split the ses- sion into individual, uncompressed files. Here’s where your organization will come into key importance: You should have a database that links each file with exactly what is said. This will help foreign language translators record localized speech or create localized subtitles. It is, after all, quite likely that the actor may ad lib a bit and say something a little differently than the original script. The Last Dance The one thing I hope you get from this chapter besides a bit of technology advice is that sound is a critically important part of your game. This book has certainly spent

442 Chapter 13 n Game Audio enough pages on it. Most programmers and designers tend to wait until the very end of the production cycle before bringing in the sound engineers and composers. By that time, it’s usually too late to create a cohesive sound design in your game, and the whole thing will be horribly rushed. As much as sound designers have resigned themselves to this fate, it is still something you should try to avoid. Get organized from the very beginning, and ask yourself whether each task in your game schedule needs an audio component. You’ll be surprised how many objects need sound effects, how many levels need their own background ambient tracks, and how much speech you need to record. Basically, if it moves at all, it needs sound. Sound technology will also stress more of your core game components than any other system, including resource cache, streaming, memory, main loop, and more. Once your sound system is working flawlessly, you can feel confident about your lower-level systems. That’s another good reason to get sound in your game as early as possible. Most importantly, don’t forget that sound sets the emotional stage for players. They will get critical player direction, feel more immersed, and react more strongly to everything if you’ve done a good job with your sound system and the sounds that it makes possible.