[SDL] Audio capture support and new Mac/iOS audio target...

[Ryan C. Gordon]

Heads up: SDL now supports audio capture! With the latest in revision control, you can record audio from a microphone. This will go into 2.0.5.

To use: you can enumerate capture devices the same way as you normally would; that previously-unused "iscapture" parameter should be set to 1.

You open audio devices for capture as before, but with iscapture=1. The difference is your callback will fire as usual, but you are expected to read data from it--the audio from your microphone--instead of writing data to be played.

If you hate callbacks, I've added SDL_DequeueAudio(), which works like 2.0.4's SDL_QueueAudio() but pulls data that is collecting from the mic.

This is implemented for darn near everything: Windows, Mac, Linux, iOS, Android, Emscripten.

You can try the Enscripten version in your web browser here:
https://icculus.org/~icculus/emscripten/capture/testaudiocapture.html

(This works in Firefox and Chrome on the desktop, Chrome on Android, and not at all on Safari or iOS at the moment. WebKit does not currently offer audio capture support.)

Feedback welcome!

Also worth noting: the Mac/iOS audio code has changed from AudioUnits to AudioQueues. This fixed several problems and was more pleasant to work with.

If you have a Mac or iOS SDL game, please test the new code, even if you don't care about recording audio.

--ryan.

[Ryan C. Gordon]

Wondering why they even bother with those snapshots when
https://hg.libsdl.org/SDL/archive/tip.tar.bz2
will always get you the latest and (arguably) greatest version.

To be clear, the Mercurial "tip" tag is the latest commit to the
repository, regardless of branch, which at this very moment in time will
get you the bleeding edge of SDL 1.2.

(default.tar.bz2 might work better? I haven't tried.)

--ryan.

[Kai Sterker]

Post by Ryan C. Gordon
Also worth noting: the Mac/iOS audio code has changed from AudioUnits to

AudioQueues. This fixed

Post by Ryan C. Gordon
several problems and was more pleasant to work with.
If you have a Mac or iOS SDL game, please test the new code, even if you

don't care about recording

Post by Ryan C. Gordon
audio.

Compiled current SDL hg with DEPLOYMENT_TARGET 10.6 on 10.11 and tested on
both 10.11 and 10,6. Seems to work for me.

Btw., since we're talking about audio stuff here: any plans of having Ogg
Opus support added to SDL_mixer?

Kai

[Alex Barry]

Mostly an issue of stability, even if there are unaddressed bugs.

Also, looks cool Ryan!

Smile]

Wondering why they even bother with those snapshots when
https://hg.libsdl.org/SDL/archive/tip.tar.bz2
will always get you the latest and (arguably) greatest version.
Of course, after a simple "brew install mercurial" on your Mac, there's no
longer any need to bother about those deceptive links.
Kai
_______________________________________________
SDL mailing list
http://lists.libsdl.org/listinfo.cgi/sdl-libsdl.org

[Sanette]

Post by Ryan C. Gordon
Heads up: SDL now supports audio capture! With the latest in revision control, you can record audio from a microphone. This will go into 2.0.5.

Thanks so much ! this was long awaited !

[Fejwin]

Here the last part of the error messages before the make routine aborts
Loading Image...

[Fejwin]

Just in case if anyone runs into the same kind of issue as I described above.
I fixed the problem by commenting out all calls to the function
Code:
WIN_SetErrorFromHRESULT

in the file SDL_render_d3d11.c, as well as some calls to the function
Code:
SDL_SetError

Somehow their syntax caused errors during the build process - but since their purpose is only to generate error descriptions when some routine fails, I thought removing them won't hurt functionality. With this the code built successfully.

[Fejwin]

Question:
Does SDL_OpenAudioDevice open a recording device in exclusive mode, so that no other application can access and record from this device in parallel?

[Fejwin]

Hi Ryan C. Gordon,

This is wonderful news! I am a bit of a newbe and have two questions:
Is there any bare-bone tutorial on how to use sound capture in SDL?
Also, are there functions built in to get let's say the power of the recorded sound signal?

Keep up the great work!
Cheers,
Fejwin

[Ryan C. Gordon]

On 09/07/2016 08:25 AM, Fejwin wrote:

(Whoops, I missed this email before, sorry for the late reply!)

Post by Fejwin
Is there any bare-bone tutorial on how to use sound capture in SDL?

There is not, but you can see the sample program here:

https://hg.libsdl.org/SDL/raw-file/e9c3e64fdc84/test/testaudiocapture.c

The relevant parts are:

If enumerating audio devices, you use SDL_TRUE to get a list of capture
devices instead of playback devices. Likewise for
SDL_GetAudioDeviceName()...

https://hg.libsdl.org/SDL/file/e9c3e64fdc84/test/testaudiocapture.c#l115

You open the audio device with SDL_TRUE for the second parameter (you
want an audio capture device instead of the usual playback device)...

https://hg.libsdl.org/SDL/file/e9c3e64fdc84/test/testaudiocapture.c#l143

Now you would get a callback that fires regularly, just like you would
for a playback device. Instead of writing data to a buffer in that
callback, you read data from it. The callback fires when enough data has
been read from the microphone.

If you don't like doing that, you can use a NULL callback, like that
example code does:

https://hg.libsdl.org/SDL/file/e9c3e64fdc84/test/testaudiocapture.c#l125

In which case, you can read from the mic with SDL_DequeueAudio():

https://hg.libsdl.org/SDL/file/e9c3e64fdc84/test/testaudiocapture.c#l76

And it'll hand you whatever data is available at the moment, continually
buffering more as it comes in. If you do this, you have to either call
SDL_DequeueAudio() regularly, or make sure to SDL_PauseAudioDevice()
when you don't care about recording, or SDL will keep buffering up audio
until it runs out of memory. Naturally, the callback avoids this
problem. Just be careful about times when you're busy doing something
that blocks, like maybe loading a new game level, that you might be
consuming memory and not realize it (and then have a lot of
needlessly-buffered audio when you come back to it later).

When you're done recording and want to clean up, you close capture
devices the same way you always would with SDL_CloseAudioDevice().
Depending on your needs, SDL_Quit() cleans up any open ones for you when
shutting everything else down, too, and that might be more simple.

Basically, that's all there is to it. It's a handful of function calls
from start to finish.

Post by Fejwin
Also, are there functions built in to get let's say the power of the recorded sound signal?

We only provide raw PCM data, but you can calculate this from that. The
dirt simple way is what ioquake3 does for their VoIP power meter:

https://github.com/ioquake/ioq3/blob/master/code/client/cl_main.c#L513

Which is strictly speaking not _correct_ but works pretty well anyhow.

Post by Fejwin
Also, what version of SDL should I get to play around with these new
features?

The latest in revision control, which will become an official 2.0.5
release in a few weeks, at most.

(I hope.)

--ryan.