From 938d60d0f4589a3c6982838176475f077df640a4 Mon Sep 17 00:00:00 2001 From: Jesse Talavera-Greenberg Date: Tue, 6 Jun 2023 15:55:06 -0400 Subject: [PATCH] Add microphone support via a new driver (#14731) * Some slight fixes * Update libretro.h * Log calls to RETRO_ENVIRONMENT_GET_MICROPHONE_INTERFACE * Finish proof-of-concept for mic support - It works, but doesn't support floating-point audio yet - It may need to be resampled, too * Add macros that aren't available in SDL 2 * Comment out a variable definition for now - For C89 compliance * Add some comments for clarity * Let ALSA tolerate a null new_rate * Partial ALSA microphone support - Not yet tested - Mic is created and destroyed - Mic can also be paused or unpaused - Mic is paused or unpaused with the rest of the driver - Microphone is not yet read * Install error logging in the ALSA driver - It defers to RARCH_ERR * Free the ALSA microphone in alsa_free * Fix an indent * First draft of alsa_read_microphone * Deinitialize SDL Audio in sdl_audio_free * Save and restore the ALSA error logger - You should always practice safe global state * Add newlines to some RARCH_ERRs * Add some logging * Check for the mic being active via settings instead of via flags * Adjusted a log entry to be less misleading - A frequency of 0Hz looks weird to the uninformed - In reality, it means the driver used the requested frequency * Fix an incorrect format string * Tidy up logging in alsa.c * Rename audio_enable_microphone to audio_enable_input * Rename microphone_device to audio_input_device * Add audio_input_latency and audio_input_block_frames settings * Add all mic-related settings to the options menu * Adjust logging for alsa.c - Log the ALSA library version - Add errno details * Refer to the microphone in logs by name * Use %u instead of %d for some log items * Add input_samples_buf * Remove an inaccurate comment * Change type of input_samples_buf * Clean up audio_driver_flush_microphone_input * Comment convert_float_to_s16 - It helped me understand what it's doing - Turns out it'll work just fine on mono audio * Don't use the resampler for mic input * Fix crash in the ALSA driver when reading from a mic * Update some logging messages * ALSA support now works for mics * Reuse some common functions in alsa.c * Add alsa_thread_microphone_t * Refactor alsa.c - Introduce alsa_init_pcm to init any PCM that we're using - Vastly simplifies the implementation of alsa_init and alsa_init_microphone - Will be used for the read-based versions next * Make ALSA logging a little more consistent * Clean up the mic with alsa_free_microphone if alsa_init_microphone fails * Remove an unused function * Move some cleanup in alsa.c to a common function * First crack at mic support for alsathread - Refactor some duplicate code into functions - Use functions introduced in alsa.c - Create and destroy the mic * Slight cleanups for clarity * Implement alsa_thread_set/get_microphone_state * More work on alsathread - No more crashing, but the mic just returns silence * Slight cleanups for clarity * Add alsa_set_mic_enabled_internal - For setting the state of a microphone while considering its current state * Use alsa_set_mic_enabled_internal * Log a little more info * Log when the audio driver is started/stopped * Move base microphone driver code into a new directory - Add microphone_driver.c to Makefile.common - Rename functions as needed * Initialize and deinitialize the microphone driver * Implement sdl_microphone.c * Un-const an argument - In case the driver context needs to do any locking * Revise comments for microphone_driver.h * Remove an unimplemented function * Remove some functions from the mic driver * Remove mic functions from audio_thread_wrapper * Remove mic functions from sdl_audio * Fix microphone_null * Split the mic code for the alsa audio drivers into microphone drivers * Fix an extra struct member * Add a setting for the mic driver * Add a command to reinitialize the microphone driver * Rename mic-related settings * Add DRIVER_MICROPHONE_MASK to DRIVERS_CMD_ALL * Rename audio_enable_input to microphone_enable * Remove some labels from qt_options * Search for microphone_driver within find_driver_nonempty * Clean up some mic driver code * Pending mics now return silence * Adjust some logging and comments * Some cleanup in the microphone driver * Invert a flag check - Oops * Fix a log message * Fix the wrong flags being checked * Slight refactor of wasapi_init_device - Add a data_flow parameter - Declare it in a header - In preparation for WASAPI mic support * Add some WASAPI macros for _IAudioCaptureClient * Move some common WASAPI functions to audio/common/wasapi.c - They'll be used by the mic and the audio drivers * Add wasapi_log_hr * Generalize mmdevice_list_new to look for capture devices, too * Fix a function declaration * Move driver-specific device_list_new functions into their respective files * Clean up some declarations * First draft of wasapi microphone driver * Add wasapi_microphone_device_list_free * Change function parameter names to be consistent with microphone_driver * Partially implement wasapi_microphone_read - Mostly copied from the audio driver so far - It doesn't compile yet - But it'll be beautiful when I'm done with it * Refactor the mic driver's functions - Rename get_mic_active to mic_alive - Split set_mic_active into start_mic and stop_mic - Refactor the SDL mic driver accordingly * Edit some WASAPI functions for logging and clarity * Implement more of the WASAPI mic driver * Rename write_event to read_event * Pass the WASAPI driver context to the various read functions * Mostly implement the read function for the WASAPI mic driver * Fix a crash in microphone_driver - Forgot to move the position of the name of null_driver * Reduce some logging in wasapi common functions - Only log the chosen audio client format, not all attempted ones * Add some macro wrappers for IAudioClient methods * Update mic driver configuration - Make the mic driver configurable in the menu - Add config items for WASAPI-related options similar to the audio driver * Fix a menu entry scrolling through audio devices instead of mic devices * Add some utility functions * Expose the new utility functions in wasapi.h * Add extra logging in the WASAPI common functions * Add sharemode_name * Use _IAudioClient_Initialize macro in some places * Pass channels to wasapi_init_client - Remember, mics are in mono * Use _IAudioClient_Initialize macro some more * Forgot to pass channels in some places * Add some utility functions * Forgot an #include * Add wasapi_select_device_format * Simplify the format selection logic in wasapi_init_client_sh * Unset the microphone in wasapi_microphone_close_mic - Ought to prevent a potential segfault * Simplify some logging * Fix incorrect value being passed to _IAudioCaptureClient_ReleaseBuffer * Remove some unneeded logging * Add some values to hresult_name * Polish up wasapi_select_device_format - Test for formats manually when Windows can't - Add some debug logging - Check for channels * Compute the fields of WAVEFORMATEXTENSIBLE correctly - As per the doc's stated requirements * Simplify logic for WASAPI client creation * Fix a potential hang in wasapi_microphone_read_shared_buffered * Stop the microphone if the driver is stopped * Don't name the microphone event * Ensure that wasapi_init_client reports the correct format and rate * Implement exclusive microphone read access for WASAPI * Add _IAudioCaptureClient_GetNextPacketSize macro * Organize cases in hresult_name * Clear some extra fields if wasapi_set_format is setting a Pcm format * Adjust some logs * Adjust some logs * Remove unneeded local vars * Add a log * Update wasapi.c * Update wasapi.c * Fix shared-mode mic support in WASAPI producing broken input - Turns out it had nothing to do with shared mode * Reuse a common function - Remove wasapi_microphone_read_shared_buffered - Rename wasapi_microphone_read_exclusive to wasapi_microphone_read_buffered * Remove some code I was using for test purposes * Clarify some language * Double the default shared-mode mic buffer length * Split getting a device's name into a separate function, then use it * Fix the ALSA mic drivers - To comply with changes I previously made to the mic driver interface * Remove unused synchronization primitives from the SDL microphone driver * Add sdl_microphone_mic_use_float * Document audio_driver_state_flags - I needed to understand these to see if similar flags were required for the mic driver * Remove an unused function in wasapi.c * Add and document flags in microphone_driver.h * Remove driver-specific mic start/stop functions - The mic driver itself doesn't do much processing - That honor goes to individual mics * Remove some unused fields in microphone_driver.h * Add CMD_EVENT_MICROPHONE_STOP/START * Remove unused functions from microphone_null * Change how the mic driver state is referenced in some places * Simplify the SDL microphone driver - The driver backend no longer keeps a reference to the mic (the frontend does that) - Remove functions that are no longer needed - Don't track paused state, just query the mic itself * Simplify the WASAPI microphone driver - Don't track the driver running state or the microphone handle, the frontend does that now - Remove support for unbuffered input (hunterk suggested that it wasn't necessary) * Make microphone_wasapi_sh_buffer_length a uint, not an int - It won't be negative anymore - 0 now represents the default value * Make the microphone frontend more robust - Improve documentation for how various functions should be implemented - Closes all microphones before freeing the driver (so backends don't have to) - Tracks the enabled state of each microphone, so backends don't have to (but they still can) * Stop the mic driver in core_unload_game * Ensure mic support is compatible with the revised menu code * Move alsa.h into audio/common * Remove RETRO_ENVIRONMENT_GET_MICROPHONE_ENABLED - It was never really needed * Refactor the ALSA microphone driver - Move common ALSA functions to audio/common/alsa.c - Replace alsa_set_mic_enabled_internal with alsa_start/stop_pcm - Don't track the microphone handle in the ALSA driver context - Remove unneeded fields * Move some common alsathread code into audio/common/alsathread.c * Change return type of mic_driver_open_mic_internal to bool * First crack at resampling mic input * Remove an extraneous check - I think something distracted me when I was writing this line * Add stereo/mono conversion functions * Make alsa_start_pcm and alsa_stop_pcm more robust - They now return success if the stream is already running and stopped, respectively * Revise some mic-related comments in libretro.h * First crack at resampling mic input * Simplify an expression * Simplify an expression * Fix a log tag * Allow mic resampler to be configured separately from audio resampler * Add some comments * Set the source ratio to something sensible * Stop deadlock in `alsathread` mic driver * Allow mics to be initialized even when core is loaded from CLI - When loading content from CLI, the drivers are initialized a little differently - That threw off the mic initialization code * Rename the functions in retro_microphone_interface * Revise some mic-related comments in libretro.h * Update retro_microphone_interface - Add get_mic_rate - Add a parameter to open_mic - The modifications don't do anything yet * Use parameter objects in the microphone handle * Replace get_mic_rate with get_params * Add a microphone interface version * Remove part of a comment * Set the effective params in mic_driver_microphone_handle_init * Drop a stray newline * Change where the mic interface is zeroed - I was accidentally throwing out the version that the core was asking for * Reduce logspam for wasapi_set_nonblock_state - Now it only logs when the sync mode is changed * Change DEFAULT_WASAPI_SH_BUFFER_LENGTH to 0 - -16 is no longer a valid value * Set the new_rate in wasapi_init * Change description of microphone sample rate in the settings * First attempt at resampling configured mic input * Forgot a section * Fix some input samples being skipped * Rename a variable for clarity * Add microphone.outgoing_samples * Update the mic driver - Processed samples are now buffered - The resampler is skipped if the ratio is (very close to) 1 * Remove part of a comment * Update some comments in audio_resampler.h * Slightly refactor the SDL microphone driver - Move SDL_AudioSpec to a field of sdl_microphone_handle_t - Allow SDL to change the requested format and sample rate - Request floating-point input - Implement sdl_microphone_mic_use_float * Fix a non-C89-compliant declaration * Add new files to griffin.c * Remove a C++-style comment * Add two more files to griffin.c * Remove some unneeded declarations in microphone_driver.h * Remove a stray comma in configuration.c - For C89 compliance * Fix compilation on some platforms * Change some function signatures * Make the ALSA drivers always set the audio rate * Fix the alsathread mic driver * Make state_manager_frame_is_reversed return false if HAVE_REWIND isn't defined * Mute the microphone if the core is running in fast-forward, slow-mo, or rewind * Clarify a comment * Clarify a comment * Add a comment * Don't allocate memory for slowmo samples in the mic driver - We're not supporting slowmo for mics, so it's not needed * Fix a { * Add my name to AUTHORS.h * Add driver_lifetime_flags - For drivers that have special setup/teardown needs * Ensure that resetting the mic driver maintains active mic handles - Prevents fullscreen toggle from stopping all mic input * Update CHANGES.md * Move some default microphone settings to a new part of the config file * Ensure that RetroArch can use the audio format that Windows suggests * Remove references to mic support in the SDL audio driver * Remove unused WASAPI functions * Return failure if RetroArch couldn't select a WASAPI format * Ensure that Windows uses the WASAPI mic driver by default * Treat disabled mic support as a warning, not an error * Clarify some WASAPI-related microphone settings * Remove some unused variables * Add or revise microphone-related comments * Rearrange doc comments for microphone types in libretro.h * Remove a space * Remove some unused flags * Remove ALSA error logger - It was never used anyway * Remove unneeded microphone-related arguments * Document a parameter * Remove a logging call * Add a constant for the microphone's shared buffer length for WASAPI * Fix stylistic inconsistencies * Make mic_driver_get_sample_size a macro instead of a function * Move the microphone implementation to the audio directory * Make microphone support optional (but enabled by default) * Fix the griffin build --- AUTHORS.h | 1 + CHANGES.md | 8 +- Makefile.common | 35 +- audio/audio_defines.h | 72 +- audio/audio_driver.c | 104 ++- audio/audio_driver.h | 49 +- audio/audio_thread_wrapper.c | 40 +- audio/common/alsa.c | 483 ++++++++++ audio/common/alsa.h | 56 ++ audio/common/alsathread.c | 45 + audio/common/alsathread.h | 40 + audio/common/mmdevice_common.c | 61 +- audio/common/mmdevice_common.h | 9 +- audio/common/mmdevice_common_inline.h | 20 + audio/common/wasapi.c | 745 +++++++++++++++ audio/common/wasapi.h | 35 + audio/drivers/alsa.c | 208 +---- audio/drivers/alsathread.c | 287 ++---- audio/drivers/sdl_audio.c | 153 +++- audio/drivers/wasapi.c | 453 +-------- audio/drivers_microphone/alsa.c | 297 ++++++ audio/drivers_microphone/alsathread.c | 412 +++++++++ audio/drivers_microphone/sdl_microphone.c | 400 ++++++++ audio/drivers_microphone/wasapi.c | 565 ++++++++++++ audio/microphone_driver.c | 861 ++++++++++++++++++ audio/microphone_driver.h | 667 ++++++++++++++ command.h | 9 +- config.def.h | 15 + configuration.c | 109 ++- configuration.h | 34 + defaults.h | 1 + driver.h | 43 +- gfx/video_driver.c | 4 +- griffin/griffin.c | 20 + intl/msg_hash_chs.c | 16 + intl/msg_hash_lbl.h | 60 ++ intl/msg_hash_us.c | 16 + intl/msg_hash_us.h | 98 +- .../audio/conversion/float_to_s16.c | 23 +- .../audio/conversion/mono_to_stereo_float.c | 44 + .../audio/conversion/stereo_to_mono_float.c | 45 + .../include/audio/audio_resampler.h | 29 + .../include/audio/conversion/dual_mono.h | 69 ++ libretro-common/include/audio/dsp_filter.h | 3 + libretro-common/include/libretro.h | 224 ++++- list_special.h | 3 + menu/cbs/menu_cbs_deferred_push.c | 15 + menu/cbs/menu_cbs_get_value.c | 4 + menu/cbs/menu_cbs_ok.c | 55 ++ menu/cbs/menu_cbs_sublabel.c | 55 ++ menu/cbs/menu_cbs_title.c | 9 + menu/drivers/materialui.c | 9 +- menu/menu_cbs.h | 6 + menu/menu_displaylist.c | 203 ++++- menu/menu_displaylist.h | 6 + menu/menu_driver.c | 25 +- menu/menu_driver.h | 3 + menu/menu_setting.c | 415 ++++++++- msg_hash.h | 27 + qb/config.params.sh | 1 + retroarch.c | 95 +- retroarch.cfg | 8 + retroarch.h | 12 + runloop.c | 83 +- 64 files changed, 7001 insertions(+), 1001 deletions(-) create mode 100644 audio/common/alsa.c create mode 100644 audio/common/alsa.h create mode 100644 audio/common/alsathread.c create mode 100644 audio/common/alsathread.h create mode 100644 audio/common/wasapi.c create mode 100644 audio/common/wasapi.h create mode 100644 audio/drivers_microphone/alsa.c create mode 100644 audio/drivers_microphone/alsathread.c create mode 100644 audio/drivers_microphone/sdl_microphone.c create mode 100644 audio/drivers_microphone/wasapi.c create mode 100644 audio/microphone_driver.c create mode 100644 audio/microphone_driver.h create mode 100644 libretro-common/audio/conversion/mono_to_stereo_float.c create mode 100644 libretro-common/audio/conversion/stereo_to_mono_float.c create mode 100644 libretro-common/include/audio/conversion/dual_mono.h diff --git a/AUTHORS.h b/AUTHORS.h index 02d2df62f0..c7f18ccc72 100644 --- a/AUTHORS.h +++ b/AUTHORS.h @@ -231,6 +231,7 @@ Jean-Sébastien Guay (Skylark13) Jeff (jeffbdavenport) Jeff Sousa (LordeIlluminati) jess (winneon) +Jesse Talavera-Greenberg (JesseTG) Joan Coll Cerdán (johanbcn) Job Adrian Salinas Gonzalez (efylan) Joe Osborn (JoeOsborn) diff --git a/CHANGES.md b/CHANGES.md index 42ec015a8e..321fd9c64b 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,4 +1,9 @@ # Future +- LIBRETRO/MICROPHONE: Add new API for microphone support. +- MICROPHONE: Add support for microphones. +- MICROPHONE/ALSA: Add `alsa` and `alsathread` microphone drivers. +- MICROPHONE/SDL: Add `sdl2` microphone driver. +- MICROPHONE/WASAPI: Add `wasapi` microphone driver. # 1.15.0 - AI SERVICE: Fix NVDA switching to Powershell on speak @@ -116,8 +121,9 @@ as well. Without this, sound files can not be opened from file browser with core - MENU/CHEATS: Fixed label capitalization in cheats (Add New After/Before This) - MENU/SOUNDS: Add scrolling sounds for RGUI, XMB, MaterialUI and Ozone. - MENU/SOUNDS: Better scrolling sound implementation, add new 'notice back' sound -- MENU/SOUNDS: Scroll sound fixes. Correctly get list size in xmb.c for playing scrolling sound when switching categories, play the scrolling sound when pressing cancel in ozone, play the sound when scrolling with ZL and ZR, play the correct sound when scrolling with L- MIYOO: L3/R3 support for Dingux Gamepad controller device. +- MENU/SOUNDS: Scroll sound fixes. Correctly get list size in xmb.c for playing scrolling sound when switching categories, play the scrolling sound when pressing cancel in ozone, play the sound when scrolling with ZL and ZR, play the correct sound when scrolling with L - MENU/WIDGETS: Show square sized widget on volume mute. Volume widget is currently fixed size always, and thus showing a lot of empty space when muting, therefore shorten the box to icon size only when muting. +- MIYOO: L3/R3 support for Dingux Gamepad controller device. - NETWORKING: Call ssl_socket_close for SSL sockets - NETWORKING/CHEEVOS: net_http - Temporary fix for cheevos crash. Don't use new timeout/poll code for cheevos HTTP requests. - NETWORKING/MENU: Network information cleanup: diff --git a/Makefile.common b/Makefile.common index 201c1b7d78..1286908851 100644 --- a/Makefile.common +++ b/Makefile.common @@ -327,6 +327,11 @@ OBJ += \ cores/dynamic_dummy.o \ $(LIBRETRO_COMM_DIR)/queues/message_queue.o +ifeq ($(HAVE_MICROPHONE), 1) + DEFINES += -DHAVE_MICROPHONE + OBJ += audio/microphone_driver.o +endif + ifeq ($(HAVE_REWIND), 1) DEFINES += -DHAVE_REWIND OBJ += state_manager.o @@ -820,7 +825,12 @@ endif endif ifeq ($(HAVE_ALSA), 1) - OBJ += audio/drivers/alsa.o + OBJ += audio/drivers/alsa.o \ + audio/common/alsa.o + + ifeq ($(HAVE_MICROPHONE), 1) + OBJ += audio/drivers_microphone/alsa.o + endif ifneq ($(HAVE_HAKCHI), 1) ifneq ($(HAVE_SEGAM), 1) @@ -832,7 +842,12 @@ ifeq ($(HAVE_ALSA), 1) ifneq ($(MIYOO), 1) ifeq ($(HAVE_THREADS), 1) - OBJ += audio/drivers/alsathread.o + OBJ += audio/drivers/alsathread.o \ + audio/common/alsathread.o + + ifeq ($(HAVE_MICROPHONE), 1) + OBJ += audio/drivers_microphone/alsathread.o + endif endif endif @@ -895,9 +910,14 @@ endif ifeq ($(HAVE_WASAPI), 1) HAVE_MMDEVAPI = 1 - OBJ += audio/drivers/wasapi.o + OBJ += audio/drivers/wasapi.o \ + audio/common/wasapi.o DEFINES += -DHAVE_WASAPI LIBS += -lole32 -lksuser + + ifeq ($(HAVE_MICROPHONE), 1) + OBJ += audio/drivers_microphone/wasapi.o + endif endif ifeq ($(HAVE_XAUDIO), 1) @@ -928,7 +948,9 @@ ifeq ($(HAVE_NEON),1) endif OBJ += $(LIBRETRO_COMM_DIR)/audio/conversion/s16_to_float.o \ - $(LIBRETRO_COMM_DIR)/audio/conversion/float_to_s16.o + $(LIBRETRO_COMM_DIR)/audio/conversion/float_to_s16.o \ + $(LIBRETRO_COMM_DIR)/audio/conversion/mono_to_stereo_float.o \ + $(LIBRETRO_COMM_DIR)/audio/conversion/stereo_to_mono_float.o \ ifeq ($(HAVE_RWAV), 1) DEFINES += -DHAVE_RWAV @@ -1540,6 +1562,11 @@ ifeq ($(HAVE_SDL2), 1) gfx/common/sdl2_common.o DEF_FLAGS += $(SDL2_CFLAGS) LIBS += $(SDL2_LIBS) + + ifeq ($(HAVE_MICROPHONE), 1) + OBJ += audio/drivers_microphone/sdl_microphone.o + # Microphones are not supported by SDL 1.x + endif else ifeq ($(HAVE_SDL), 1) HAVE_SDL_COMMON = 1 OBJ += gfx/drivers/sdl_gfx.o diff --git a/audio/audio_defines.h b/audio/audio_defines.h index 450b8cf3cf..18de5f0338 100644 --- a/audio/audio_defines.h +++ b/audio/audio_defines.h @@ -83,17 +83,87 @@ enum audio_mixer_state AUDIO_STREAM_STATE_PLAYING_SEQUENTIAL }; +/** + * Bit flags that describe the current state of the audio driver. + */ enum audio_driver_state_flags { + /** + * Indicates that the driver was successfully created + * and is currently valid. + * You may submit samples for output at any time. + * + * This flag does \em not mean that the player will hear anything; + * the driver might be suspended. + * + * @see AUDIO_FLAG_SUSPENDED + */ AUDIO_FLAG_ACTIVE = (1 << 0), + + /** + * Indicates that the audio driver outputs floating-point samples, + * as opposed to integer samples. + * + * All audio is sent through the resampler, + * which operates on floating-point samples. + * + * If this flag is set, then the resampled output doesn't need + * to be converted back to \c int16_t format. + * + * This won't affect the audio that the core writes; + * either way, it's supposed to output \c int16_t samples. + * + * This flag won't be set if the selected audio driver + * doesn't support (or is configured to not use) \c float samples. + * + * @see audio_driver_t::use_float + */ AUDIO_FLAG_USE_FLOAT = (1 << 1), + + /** + * Indicates that the audio driver is not currently rendering samples, + * although it's valid and can be resumed. + * + * Usually set when RetroArch needs to simulate audio output + * without actually rendering samples (e.g. runahead), + * or when reinitializing the driver. + * + * Samples will still be accepted, but they will be silently dropped. + */ AUDIO_FLAG_SUSPENDED = (1 << 2), + + /** + * Indicates that the audio mixer is available + * and can mix one or more audio streams. + * + * Will not be set if RetroArch was built without \c HAVE_AUDIOMIXER. + */ AUDIO_FLAG_MIXER_ACTIVE = (1 << 3), + + /** + * Indicates that the frontend will never need audio from the core, + * usually when runahead is active. + * + * When set, any audio received by the core will not be processed. + * + * Will not be set if RetroArch was built without \c HAVE_RUNAHEAD. + * + * @see RETRO_ENVIRONMENT_GET_AUDIO_VIDEO_ENABLE + */ AUDIO_FLAG_HARD_DISABLE = (1 << 4), + + /** + * Indicates that audio rate control is enabled. + * This means that the audio system will adjust the rate at which + * it sends samples to the driver, + * minimizing the occurrences of buffer overrun or underrun. + * + * @see audio_driver_t::write_avail + * @see audio_driver_t::buffer_size + */ AUDIO_FLAG_CONTROL = (1 << 5) }; - typedef struct audio_statistics { unsigned samples; diff --git a/audio/audio_driver.c b/audio/audio_driver.c index ccd9229ee8..7140751bbd 100644 --- a/audio/audio_driver.c +++ b/audio/audio_driver.c @@ -76,7 +76,7 @@ audio_driver_t audio_null = { NULL, NULL, NULL, /* write_avail */ - NULL + NULL /* buffer_size */ }; audio_driver_t *audio_drivers[] = { @@ -192,6 +192,12 @@ const char *config_get_audio_driver_options(void) return char_list_new_special(STRING_LIST_AUDIO_DRIVERS, NULL); } +unsigned audio_driver_get_sample_size(void) +{ + audio_driver_state_t *audio_st = &audio_driver_st; + return (audio_st->flags & AUDIO_FLAG_USE_FLOAT) ? sizeof(float) : sizeof(int16_t); +} + #ifdef HAVE_TRANSLATE /* TODO/FIXME - Doesn't currently work. Fix this. */ bool audio_driver_is_ai_service_speech_running(void) @@ -339,7 +345,6 @@ bool audio_driver_deinit(void) audio_driver_mixer_deinit(); #endif audio_driver_free_devices_list(); - return audio_driver_deinit_internal( settings->bools.audio_enable); } @@ -385,12 +390,17 @@ bool audio_driver_find_driver( } /** - * audio_driver_flush: - * @data : pointer to audio buffer. - * @right : amount of samples to write. + * Writes audio samples to audio driver's output. + * Will first perform DSP processing (if enabled) and resampling. * - * Writes audio samples to audio driver. Will first - * perform DSP processing (if enabled) and resampling. + * @param audio_st The overall state of the audio driver. + * @param slowmotion_ratio The factor by which slow motion extends the core's runtime + * (e.g. a value of 2 means the core is running at half speed). + * @param audio_fastforward_mute True if no audio should be output while the game is in fast-forward. + * @param data Audio output data that was most recently provided by the core. + * @param samples The size of \c data, in samples. + * @param is_slowmotion True if the player is currently running the game in slow motion. + * @param is_fastmotion True if the player is currently running the game in fast-forward. **/ static void audio_driver_flush( audio_driver_state_t *audio_st, @@ -407,37 +417,46 @@ static void audio_driver_flush( src_data.data_out = NULL; src_data.output_frames = 0; + /* We'll assign a proper output to the resampler later in this function */ convert_s16_to_float(audio_st->input_data, data, samples, audio_volume_gain); + /* The resampler operates on floating-point frames, + * so we gotta convert the input first */ src_data.data_in = audio_st->input_data; src_data.input_frames = samples >> 1; + /* Remember, we allocated buffers that are twice as big as needed. + * (see audio_driver_init) */ #ifdef HAVE_DSP_FILTER if (audio_st->dsp) - { + { /* If we want to process our audio for reasons besides resampling... */ struct retro_dsp_data dsp_data; - dsp_data.input = NULL; - dsp_data.input_frames = 0; - dsp_data.output = NULL; - dsp_data.output_frames = 0; - dsp_data.input = audio_st->input_data; dsp_data.input_frames = (unsigned)(samples >> 1); + dsp_data.output = NULL; + dsp_data.output_frames = 0; + /* Initialize the DSP input/output. + * Our DSP implementations generally operate directly on the input buffer, + * so the output/output_frames attributes here are zero; + * the DSP filter will set them to useful values, + * most likely to be the same as the inputs. */ retro_dsp_filter_process(audio_st->dsp, &dsp_data); if (dsp_data.output) - { + { /* If the DSP filter succeeded... */ src_data.data_in = dsp_data.output; src_data.input_frames = dsp_data.output_frames; + /* Then let's pass the DSP's output to the resampler's input */ } } #endif src_data.data_out = audio_st->output_samples_buf; + /* Now the resampler will write to the driver state's scratch buffer */ if (audio_st->flags & AUDIO_FLAG_CONTROL) { @@ -530,19 +549,21 @@ static void audio_driver_flush( } #endif + /* Now we write our processed audio output to the driver. + * It may not be played immediately, depending on the driver implementation. */ { const void *output_data = audio_st->output_samples_buf; - unsigned output_frames = (unsigned)src_data.output_frames; + unsigned output_frames = (unsigned)src_data.output_frames; /* Unit: frames */ if (audio_st->flags & AUDIO_FLAG_USE_FLOAT) - output_frames *= sizeof(float); + output_frames *= sizeof(float); /* Unit: bytes */ else { convert_float_to_s16(audio_st->output_samples_conv_buf, (const float*)output_data, output_frames * 2); output_data = audio_st->output_samples_conv_buf; - output_frames *= sizeof(int16_t); + output_frames *= sizeof(int16_t); /* Unit: bytes */ } audio_st->current_audio->write(audio_st->context_audio_data, @@ -574,7 +595,7 @@ bool audio_driver_init_internal( bool audio_cb_inited) { unsigned new_rate = 0; - float *samples_buf = NULL; + float *out_samples_buf = NULL; settings_t *settings = (settings_t*)settings_data; size_t max_bufsamples = AUDIO_CHUNK_SIZE_NONBLOCKING * 2; bool audio_enable = settings->bools.audio_enable; @@ -590,23 +611,26 @@ bool audio_driver_init_internal( #endif /* Accomodate rewind since at some point we might have two full buffers. */ size_t outsamples_max = AUDIO_CHUNK_SIZE_NONBLOCKING * 2 * AUDIO_MAX_RATIO * slowmotion_ratio; - int16_t *conv_buf = (int16_t*)memalign_alloc(64, outsamples_max * sizeof(int16_t)); - float *audio_buf = (float*)memalign_alloc(64, AUDIO_CHUNK_SIZE_NONBLOCKING * 2 * sizeof(float)); + int16_t *out_conv_buf = (int16_t*)memalign_alloc(64, outsamples_max * sizeof(int16_t)); + size_t audio_buf_length = AUDIO_CHUNK_SIZE_NONBLOCKING * 2 * sizeof(float); + float *audio_buf = (float*)memalign_alloc(64, audio_buf_length); bool verbosity_enabled = verbosity_is_enabled(); convert_s16_to_float_init_simd(); convert_float_to_s16_init_simd(); - if (!conv_buf || !audio_buf) + if (!out_conv_buf || !audio_buf) goto error; memset(audio_buf, 0, AUDIO_CHUNK_SIZE_NONBLOCKING * 2 * sizeof(float)); - audio_driver_st.input_data = audio_buf; - audio_driver_st.output_samples_conv_buf = conv_buf; - audio_driver_st.chunk_block_size = AUDIO_CHUNK_SIZE_BLOCKING; - audio_driver_st.chunk_nonblock_size = AUDIO_CHUNK_SIZE_NONBLOCKING; - audio_driver_st.chunk_size = audio_driver_st.chunk_block_size; + audio_driver_st.input_data = audio_buf; + audio_driver_st.input_data_length = audio_buf_length; + audio_driver_st.output_samples_conv_buf = out_conv_buf; + audio_driver_st.output_samples_conv_buf_length = outsamples_max * sizeof(int16_t); + audio_driver_st.chunk_block_size = AUDIO_CHUNK_SIZE_BLOCKING; + audio_driver_st.chunk_nonblock_size = AUDIO_CHUNK_SIZE_NONBLOCKING; + audio_driver_st.chunk_size = audio_driver_st.chunk_block_size; #ifdef HAVE_REWIND /* Needs to be able to hold full content of a full max_bufsamples @@ -668,6 +692,7 @@ bool audio_driver_init_internal( audio_latency, settings->uints.audio_block_frames, &new_rate); + RARCH_LOG("[Audio]: Started synchronous audio driver\n"); } if (new_rate != 0) @@ -735,11 +760,14 @@ bool audio_driver_init_internal( audio_driver_st.data_ptr = 0; - if (!(samples_buf = (float*)memalign_alloc(64, outsamples_max * sizeof(float)))) + out_samples_buf = (float*)memalign_alloc(64, outsamples_max * sizeof(float)); + + if (!out_samples_buf) goto error; - audio_driver_st.output_samples_buf = (float*)samples_buf; - audio_driver_st.flags &= ~AUDIO_FLAG_CONTROL; + audio_driver_st.output_samples_buf = (float*)out_samples_buf; + audio_driver_st.output_samples_buf_length = outsamples_max * sizeof(float); + audio_driver_st.flags &= ~AUDIO_FLAG_CONTROL; if ( !audio_cb_inited @@ -1425,7 +1453,7 @@ void audio_driver_load_system_sounds(void) task_push_audio_mixer_load(path_bgm, audio_driver_load_menu_bgm_callback, NULL, true, AUDIO_MIXER_SLOT_SELECTION_MANUAL, AUDIO_MIXER_SYSTEM_SLOT_BGM); if (path_cheevo_unlock && audio_enable_cheevo_unlock) task_push_audio_mixer_load(path_cheevo_unlock, NULL, NULL, true, AUDIO_MIXER_SLOT_SELECTION_MANUAL, AUDIO_MIXER_SYSTEM_SLOT_ACHIEVEMENT_UNLOCK); - if (audio_enable_menu_scroll) + if (audio_enable_menu_scroll) { if (path_up) task_push_audio_mixer_load(path_up, NULL, NULL, true, AUDIO_MIXER_SLOT_SELECTION_MANUAL, AUDIO_MIXER_SYSTEM_SLOT_UP); @@ -1458,13 +1486,13 @@ void audio_driver_mixer_play_menu_sound(unsigned i) audio_driver_mixer_play_stream_internal(i, AUDIO_STREAM_STATE_PLAYING); } -void audio_driver_mixer_play_scroll_sound(bool direction_up) +void audio_driver_mixer_play_scroll_sound(bool direction_up) { settings_t *settings = config_get_ptr(); bool audio_enable_menu = settings->bools.audio_enable_menu; bool audio_enable_menu_scroll = settings->bools.audio_enable_menu_scroll; if (audio_enable_menu && audio_enable_menu_scroll) - audio_driver_mixer_play_menu_sound(direction_up ? AUDIO_MIXER_SYSTEM_SLOT_UP : AUDIO_MIXER_SYSTEM_SLOT_DOWN); + audio_driver_mixer_play_menu_sound(direction_up ? AUDIO_MIXER_SYSTEM_SLOT_UP : AUDIO_MIXER_SYSTEM_SLOT_DOWN); } void audio_driver_mixer_play_stream_looped(unsigned i) @@ -1646,6 +1674,10 @@ bool audio_driver_start(bool is_shutdown) audio_st->context_audio_data, is_shutdown)) goto error; + RARCH_DBG("[Audio]: Started audio driver \"%s\" (is_shutdown=%s)\n", + audio_st->current_audio->ident, + is_shutdown ? "true" : "false"); + return true; error: @@ -1657,14 +1689,20 @@ error: bool audio_driver_stop(void) { + bool stopped; if ( !audio_driver_st.current_audio || !audio_driver_st.current_audio->stop || !audio_driver_st.context_audio_data || !audio_driver_alive() ) return false; - return audio_driver_st.current_audio->stop( + stopped = audio_driver_st.current_audio->stop( audio_driver_st.context_audio_data); + + if (stopped) + RARCH_DBG("[Audio]: Stopped audio driver \"%s\"\n", audio_driver_st.current_audio->ident); + + return stopped; } #ifdef HAVE_REWIND diff --git a/audio/audio_driver.h b/audio/audio_driver.h index 59000d950d..f23e9a30bf 100644 --- a/audio/audio_driver.h +++ b/audio/audio_driver.h @@ -112,10 +112,19 @@ typedef struct audio_driver */ ssize_t (*write)(void *data, const void *buf, size_t size); - /* Temporarily pauses the audio driver. */ + /** + * Temporarily pauses the audio driver. + * + * @param data Opaque handle to the audio driver context + * that was returned by \c init. + * @return \c true if the audio driver was successfully paused, + * \c false if there was an error. + **/ bool (*stop)(void *data); - /* Resumes audio driver from the paused state. */ + /** + * Resumes audio driver from the paused state. + **/ bool (*start)(void *data, bool is_shutdown); /* Is the audio driver currently running? */ @@ -130,7 +139,7 @@ typedef struct audio_driver * */ void (*set_nonblock_state)(void *data, bool toggle); - /* Stops and frees driver data. */ + /* Stops and frees driver. */ void (*free)(void *data); /* Defines if driver will take standard floating point samples, @@ -166,20 +175,43 @@ typedef struct uint64_t free_samples_count; struct string_list *devices_list; + + /** + * A scratch buffer for audio output to be processed, + * up to (but excluding) the point where it's converted to 16-bit audio + * to give to the driver. + */ float *output_samples_buf; + size_t output_samples_buf_length; #ifdef HAVE_REWIND int16_t *rewind_buf; #endif + + /** + * A scratch buffer for processed audio output to be converted to 16-bit, + * so that it can be sent to the driver. + */ int16_t *output_samples_conv_buf; + size_t output_samples_conv_buf_length; #ifdef HAVE_DSP_FILTER retro_dsp_filter_t *dsp; #endif const retro_resampler_t *resampler; void *resampler_data; + + /** + * The current audio driver. + */ const audio_driver_t *current_audio; + void *context_audio_data; + + /** + * Scratch buffer for preparing data for the resampler + */ float *input_data; + size_t input_data_length; #ifdef HAVE_AUDIOMIXER struct audio_mixer_stream mixer_streams[AUDIO_MIXER_MAX_SYSTEM_STREAMS]; @@ -291,6 +323,17 @@ bool audio_driver_start(bool is_shutdown); bool audio_driver_stop(void); +/** + * If you need to query the size of audio samples, + * use this function instead of checking the flags directly. + * + * @return The size of a single audio sample in bytes, + * as determined by the presence of the \c AUDIO_FLAG_USE_FLOAT flag. + * Will currently return either 2 (for \c uint16_t) or 4 (for \c float), + * although this may change if we add support for more sample types. + */ +unsigned audio_driver_get_sample_size(void); + #ifdef HAVE_TRANSLATE /* TODO/FIXME - Doesn't currently work. Fix this. */ bool audio_driver_is_ai_service_speech_running(void); diff --git a/audio/audio_thread_wrapper.c b/audio/audio_thread_wrapper.c index 964ddd29b9..f52feefc7b 100644 --- a/audio/audio_thread_wrapper.c +++ b/audio/audio_thread_wrapper.c @@ -51,6 +51,10 @@ typedef struct audio_thread } audio_thread_t; +/** + * The thread that manages the life of the audio driver. + * The wrapped audio driver lives and dies with this function. + */ static void audio_thread_loop(void *data) { audio_thread_t *thr = (audio_thread_t*)data; @@ -113,6 +117,10 @@ static void audio_thread_loop(void *data) thr->driver->free(thr->driver_data); } +/** + * Lets the audio thread finish what it's doing, + * then stops it from doing further work. + */ static void audio_thread_block(audio_thread_t *thr) { if (!thr) @@ -133,15 +141,19 @@ static void audio_thread_block(audio_thread_t *thr) slock_unlock(thr->lock); } +/** + * Resumes the audio thread. + * This function is called from the main thread. + */ static void audio_thread_unblock(audio_thread_t *thr) { if (!thr) return; - slock_lock(thr->lock); - thr->stopped = false; - scond_signal(thr->cond); - slock_unlock(thr->lock); + slock_lock(thr->lock); /* Prevent the audio thread from touching this flag... */ + thr->stopped = false; /* ...so that the main thread can do it. */ + scond_signal(thr->cond); /* Then let the audio thread know that it's okay to resume. */ + slock_unlock(thr->lock); /* "As you were." */ } static void audio_thread_free(void *data) @@ -153,13 +165,15 @@ static void audio_thread_free(void *data) if (thr->thread) { - slock_lock(thr->lock); - thr->stopped = false; + slock_lock(thr->lock); /* Let the audio thread finish what it's doing... */ + thr->stopped = false; /* Then stop it. "You're fired." */ thr->alive = false; - scond_signal(thr->cond); - slock_unlock(thr->lock); + scond_signal(thr->cond); /* Let the thread know it's okay to continue */ + slock_unlock(thr->lock); /* At this point, it will exit its loop. */ sthread_join(thr->thread); + /* Wait for the audio thread to exit, ensure that it's really dead. + * (It will call the wrapped driver's free() function.) */ } if (thr->lock) @@ -167,6 +181,7 @@ static void audio_thread_free(void *data) if (thr->cond) scond_free(thr->cond); free(thr); + /* The audio driver is done, clean up the thread itself. */ } static bool audio_thread_alive(void *data) @@ -191,6 +206,9 @@ static bool audio_thread_stop(void *data) if (!thr) return false; + /* Don't immediately call stop on the driver; + * let the audio thread finish its current loop iteration. + * It will call stop then. */ audio_thread_block(thr); thr->is_paused = true; @@ -219,6 +237,8 @@ static void audio_thread_set_nonblock_state(void *data, bool state) { (void)data; (void)state; + /* Ignored, because blocking state is irrelevant + * when audio is running on a separate thread. */ } static bool audio_thread_use_float(void *data) @@ -251,7 +271,7 @@ static ssize_t audio_thread_write(void *data, const void *buf, size_t size) } static const audio_driver_t audio_thread = { - NULL, + NULL, /* No need to wrap init, it's called at the start of the thread loop */ audio_thread_write, audio_thread_stop, audio_thread_start, @@ -262,6 +282,8 @@ static const audio_driver_t audio_thread = { "audio-thread", NULL, /* No point in using rate control with threaded audio. */ NULL, + NULL, + NULL }; /** diff --git a/audio/common/alsa.c b/audio/common/alsa.c new file mode 100644 index 0000000000..e564e9897f --- /dev/null +++ b/audio/common/alsa.c @@ -0,0 +1,483 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2011-2017 Daniel De Matteis + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ +#include +#include + +#include +#include + +#include "../audio_driver.h" +#include "../common/alsa.h" +#include "../../verbosity.h" + +int alsa_init_pcm(snd_pcm_t **pcm, + const char* device, + snd_pcm_stream_t stream, + unsigned rate, + unsigned latency, + unsigned channels, + alsa_stream_info_t *stream_info, + unsigned *new_rate, + int mode) +{ + snd_pcm_format_t format; + snd_pcm_uframes_t buffer_size; + snd_pcm_hw_params_t *params = NULL; + snd_pcm_sw_params_t *sw_params = NULL; + unsigned latency_usec = latency * 1000; + unsigned periods = 4; + unsigned orig_rate = rate; + const char *alsa_dev = device ? device : "default"; + int errnum = 0; + + RARCH_DBG("[ALSA]: Requesting device \"%s\" for %s stream\n", alsa_dev, snd_pcm_stream_name(stream)); + + if ((errnum = snd_pcm_open(pcm, alsa_dev, stream, mode)) < 0) + { + RARCH_ERR("[ALSA]: Failed to open %s stream on device \"%s\": %s\n", + snd_pcm_stream_name(stream), + alsa_dev, + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_hw_params_malloc(¶ms)) < 0) + { + RARCH_ERR("[ALSA]: Failed to allocate hardware parameters: %s\n", + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_hw_params_any(*pcm, params)) < 0) + { + RARCH_ERR("[ALSA]: Failed to query hardware parameters from %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + format = (snd_pcm_hw_params_test_format(*pcm, params, SND_PCM_FORMAT_FLOAT) == 0) + ? SND_PCM_FORMAT_FLOAT : SND_PCM_FORMAT_S16; + stream_info->has_float = (format == SND_PCM_FORMAT_FLOAT); + + RARCH_LOG("[ALSA]: Using %s sample format for %s device \"%s\"\n", + snd_pcm_format_name(format), + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm) + ); + + if ((errnum = snd_pcm_hw_params_set_access(*pcm, params, SND_PCM_ACCESS_RW_INTERLEAVED)) < 0) + { + RARCH_ERR("[ALSA]: Failed to set %s access for %s device \"%s\": %s\n", + snd_pcm_access_name(SND_PCM_ACCESS_RW_INTERLEAVED), + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + stream_info->frame_bits = snd_pcm_format_physical_width(format) * channels; + + if ((errnum = snd_pcm_hw_params_set_format(*pcm, params, format)) < 0) + { + RARCH_ERR("[ALSA]: Failed to set %s format for %s device \"%s\": %s\n", + snd_pcm_format_name(format), + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_hw_params_set_channels(*pcm, params, channels)) < 0) + { + RARCH_ERR("[ALSA]: Failed to set %u-channel audio for %s device \"%s\": %s\n", + channels, + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + /* Don't allow rate resampling when probing for the default rate (but ignore if this call fails) */ + if ((errnum = snd_pcm_hw_params_set_rate_resample(*pcm, params, false)) < 0) + { + RARCH_WARN("[ALSA]: Failed to request a default unsampled rate for %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + } + + if ((errnum = snd_pcm_hw_params_set_rate_near(*pcm, params, &rate, 0)) < 0) + { + RARCH_ERR("[ALSA]: Failed to request a rate near %uHz for %s device \"%s\": %s\n", + rate, + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + if (new_rate) + *new_rate = rate; + + if ((snd_pcm_hw_params_set_buffer_time_near(*pcm, params, &latency_usec, NULL)) < 0) + { + RARCH_ERR("[ALSA]: Failed to request a buffer time near %uus for %s device \"%s\": %s\n", + latency_usec, + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + + } + + if ((errnum = snd_pcm_hw_params_set_periods_near(*pcm, params, &periods, NULL)) < 0) + { + RARCH_ERR("[ALSA]: Failed to request %u periods per buffer for %s device \"%s\": %s\n", + periods, + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_hw_params(*pcm, params)) < 0) + { /* This calls snd_pcm_prepare() under the hood */ + RARCH_ERR("[ALSA]: Failed to install hardware parameters for %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + /* Shouldn't have to bother with this, + * but some drivers are apparently broken. */ + if ((errnum = snd_pcm_hw_params_get_period_size(params, &stream_info->period_frames, NULL)) < 0) + { + RARCH_WARN("[ALSA]: Failed to get an exact period size from %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + RARCH_WARN("[ALSA]: Trying the minimum period size instead\n"); + + if ((errnum = snd_pcm_hw_params_get_period_size_min(params, &stream_info->period_frames, NULL)) < 0) + { + RARCH_ERR("[ALSA]: Failed to get min period size from %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + goto error; + } + } + + stream_info->period_size = snd_pcm_frames_to_bytes(*pcm, stream_info->period_frames); + if (stream_info->period_size < 0) + { + RARCH_ERR("[ALSA]: Failed to convert a period size of %lu frames to bytes: %s\n", + stream_info->period_frames, + snd_strerror(stream_info->period_frames)); + goto error; + } + + RARCH_LOG("[ALSA]: Period: %u periods per buffer (%lu frames, %lu bytes)\n", + periods, + stream_info->period_frames, + stream_info->period_size); + + if ((errnum = snd_pcm_hw_params_get_buffer_size(params, &buffer_size)) < 0) + { + RARCH_WARN("[ALSA]: Failed to get exact buffer size from %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + RARCH_WARN("[ALSA]: Trying the maximum buffer size instead\n"); + + if ((errnum = snd_pcm_hw_params_get_buffer_size_max(params, &buffer_size)) < 0) + { + RARCH_ERR("[ALSA]: Failed to get max buffer size from %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + goto error; + } + } + + + stream_info->buffer_size = snd_pcm_frames_to_bytes(*pcm, buffer_size); + if (stream_info->buffer_size < 0) + { + RARCH_ERR("[ALSA]: Failed to convert a buffer size of %lu frames to bytes: %s\n", + buffer_size, + snd_strerror(buffer_size)); + goto error; + } + RARCH_LOG("[ALSA]: Buffer size: %lu frames (%lu bytes)\n", buffer_size, stream_info->buffer_size); + + stream_info->can_pause = snd_pcm_hw_params_can_pause(params); + + RARCH_LOG("[ALSA]: Can pause: %s.\n", stream_info->can_pause ? "yes" : "no"); + + if ((errnum = snd_pcm_sw_params_malloc(&sw_params)) < 0) + { + RARCH_ERR("[ALSA]: Failed to allocate software parameters: %s\n", + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_sw_params_current(*pcm, sw_params)) < 0) + { + RARCH_ERR("[ALSA]: Failed to query current software parameters for %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_sw_params_set_start_threshold(*pcm, sw_params, buffer_size / 2)) < 0) + { + RARCH_ERR("[ALSA]: Failed to set start %lu-frame threshold for %s device \"%s\": %s\n", + buffer_size / 2, + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + if ((errnum = snd_pcm_sw_params(*pcm, sw_params)) < 0) + { + RARCH_ERR("[ALSA]: Failed to install software parameters for %s device \"%s\": %s\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm), + snd_strerror(errnum)); + + goto error; + } + + snd_pcm_hw_params_free(params); + snd_pcm_sw_params_free(sw_params); + + RARCH_LOG("[ALSA]: Initialized %s device \"%s\"\n", + snd_pcm_stream_name(stream), + snd_pcm_name(*pcm)); + + return 0; +error: + if (params) + snd_pcm_hw_params_free(params); + + if (sw_params) + snd_pcm_sw_params_free(sw_params); + + if (*pcm) + { + alsa_free_pcm(*pcm); + *pcm = NULL; + } + + return errnum; +} + +void alsa_free_pcm(snd_pcm_t *pcm) +{ + if (pcm) + { + int errnum = 0; + + if ((errnum = snd_pcm_drop(pcm)) < 0) + { + RARCH_WARN("[ALSA]: Failed to drop remaining samples in %s stream \"%s\": %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_strerror(errnum)); + } + + if ((errnum = snd_pcm_close(pcm)) < 0) + { + RARCH_WARN("[ALSA]: Failed to close %s stream \"%s\": %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_strerror(errnum)); + } + } +} + +bool alsa_start_pcm(snd_pcm_t *pcm) +{ + int errnum = 0; + snd_pcm_state_t pcm_state; + + if (!pcm) + return false; + + pcm_state = snd_pcm_state(pcm); + switch (pcm_state) + { + case SND_PCM_STATE_PAUSED: /* If we're unpausing a valid (but paused) stream... */ + if ((errnum = snd_pcm_pause(pcm, false)) < 0) /* ...but we failed... */ + goto error; + + break; + case SND_PCM_STATE_PREPARED: + /* If we're starting this stream for the first time... */ + if ((errnum = snd_pcm_start(pcm)) < 0) /* ..but we failed... */ + goto error; + + break; + case SND_PCM_STATE_RUNNING: + RARCH_DBG("[ALSA]: %s stream \"%s\" is already running, no action needed.\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm)); + return true; + default: + RARCH_ERR("[ALSA]: Failed to start %s stream \"%s\" in unexpected state %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state)); + return false; + } + + RARCH_DBG("[ALSA]: Started %s stream \"%s\", transitioning from %s to %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state), + snd_pcm_state_name(snd_pcm_state(pcm))); + + return true; + +error: + RARCH_ERR("[ALSA]: Failed to start %s stream \"%s\" in state %s: %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state), + snd_strerror(errnum)); + + return false; +} + +bool alsa_stop_pcm(snd_pcm_t *pcm) +{ + int errnum = 0; + snd_pcm_state_t pcm_state; + + if (!pcm) + return false; + + pcm_state = snd_pcm_state(pcm); + switch (pcm_state) + { + case SND_PCM_STATE_PAUSED: + RARCH_DBG("[ALSA]: %s stream \"%s\" is already paused, no action needed.\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm)); + return true; + case SND_PCM_STATE_PREPARED: + RARCH_DBG("[ALSA]: %s stream \"%s\" is prepared but not running, no action needed.\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm)); + return true; + case SND_PCM_STATE_RUNNING: + /* If we're pausing an active stream... */ + if ((errnum = snd_pcm_pause(pcm, true)) < 0) /* ...but we failed... */ + goto error; + + break; + default: + RARCH_ERR("[ALSA]: Failed to stop %s stream \"%s\" in unexpected state %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state)); + + return false; + } + + RARCH_DBG("[ALSA]: Stopped %s stream \"%s\", transitioning from %s to %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state), + snd_pcm_state_name(snd_pcm_state(pcm))); + + return true; + +error: + RARCH_ERR("[ALSA]: Failed to stop %s stream \"%s\" in state %s: %s\n", + snd_pcm_stream_name(snd_pcm_stream(pcm)), + snd_pcm_name(pcm), + snd_pcm_state_name(pcm_state), + snd_strerror(errnum)); + + return false; +} + +struct string_list *alsa_device_list_type_new(const char* type) +{ + void **hints, **n; + union string_list_elem_attr attr; + struct string_list *s = string_list_new(); + + if (!s) + return NULL; + + attr.i = 0; + + if (snd_device_name_hint(-1, "pcm", &hints) != 0) + goto error; + + n = hints; + + while (*n) + { + char *name = snd_device_name_get_hint(*n, "NAME"); + char *io = snd_device_name_get_hint(*n, "IOID"); + char *desc = snd_device_name_get_hint(*n, "DESC"); + + /* description of device IOID - input / output identifcation + * ("Input" or "Output"), NULL means both) */ + + if (!io || (string_is_equal(io, type))) + string_list_append(s, name, attr); + + if (name) + free(name); + if (io) + free(io); + if (desc) + free(desc); + + n++; + } + + /* free hint buffer too */ + snd_device_name_free_hint(hints); + + return s; + + error: + string_list_free(s); + return NULL; +} diff --git a/audio/common/alsa.h b/audio/common/alsa.h new file mode 100644 index 0000000000..d459c21c74 --- /dev/null +++ b/audio/common/alsa.h @@ -0,0 +1,56 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 The RetroArch team + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + + +#ifndef _RETROARCH_ALSA +#define _RETROARCH_ALSA + +#include +#include "queues/fifo_queue.h" +#include "rthreads/rthreads.h" +/* Header file for common functions that are used by alsa and alsathread. */ + +/** + * Used for info that's common to all pcm devices + * that's relevant for our purposes. + */ +typedef struct alsa_stream_info +{ + size_t buffer_size; + size_t period_size; + snd_pcm_uframes_t period_frames; + unsigned int frame_bits; + bool has_float; + bool can_pause; +} alsa_stream_info_t; + +int alsa_init_pcm(snd_pcm_t **pcm, + const char* device, + snd_pcm_stream_t stream, + unsigned rate, + unsigned latency, + unsigned channels, + alsa_stream_info_t *stream_info, + unsigned *new_rate, + int mode); +void alsa_free_pcm(snd_pcm_t *pcm); +void *alsa_device_list_new(void *data); +struct string_list *alsa_device_list_type_new(const char* type); +void alsa_device_list_free(void *data, void *array_list_data); + +bool alsa_start_pcm(snd_pcm_t *pcm); +bool alsa_stop_pcm(snd_pcm_t *pcm); + +#endif /* _RETROARCH_ALSA */ diff --git a/audio/common/alsathread.c b/audio/common/alsathread.c new file mode 100644 index 0000000000..48a771f304 --- /dev/null +++ b/audio/common/alsathread.c @@ -0,0 +1,45 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2011-2017 Daniel De Matteis + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include "alsathread.h" + +void alsa_thread_free_info_members(alsa_thread_info_t *info) +{ + if (info) + { + if (info->worker_thread) + { + slock_lock(info->cond_lock); + info->thread_dead = true; + slock_unlock(info->cond_lock); + sthread_join(info->worker_thread); + } + if (info->buffer) + fifo_free(info->buffer); + if (info->cond) + scond_free(info->cond); + if (info->fifo_lock) + slock_free(info->fifo_lock); + if (info->cond_lock) + slock_free(info->cond_lock); + if (info->pcm) + { + alsa_free_pcm(info->pcm); + } + } + /* Do NOT free() info itself; it's embedded within another struct + * that will be freed. */ +} \ No newline at end of file diff --git a/audio/common/alsathread.h b/audio/common/alsathread.h new file mode 100644 index 0000000000..dafa5be8e0 --- /dev/null +++ b/audio/common/alsathread.h @@ -0,0 +1,40 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2011-2017 Daniel De Matteis + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#ifndef RETROARCH_ALSATHREAD_H +#define RETROARCH_ALSATHREAD_H + +#include +#include +#include "queues/fifo_queue.h" +#include "rthreads/rthreads.h" +#include "./alsa.h" + +typedef struct alsa_thread_info +{ + snd_pcm_t *pcm; + fifo_buffer_t *buffer; + sthread_t *worker_thread; + slock_t *fifo_lock; + scond_t *cond; + slock_t *cond_lock; + alsa_stream_info_t stream_info; + volatile bool thread_dead; +} alsa_thread_info_t; + +void alsa_thread_free_info_members(alsa_thread_info_t *info); + +#endif diff --git a/audio/common/mmdevice_common.c b/audio/common/mmdevice_common.c index 6a09aa583a..6801693b90 100644 --- a/audio/common/mmdevice_common.c +++ b/audio/common/mmdevice_common.c @@ -21,20 +21,51 @@ #include "mmdevice_common.h" #include "mmdevice_common_inline.h" -void *mmdevice_list_new(void *u) + +char* mmdevice_name(IMMDevice *device) +{ + HRESULT hr; + IPropertyStore *prop_store = NULL; + PROPVARIANT prop_var; + bool prop_var_init = false; + char* result = NULL; + + if (!device) + return NULL; + + hr = _IMMDevice_OpenPropertyStore(device, STGM_READ, &prop_store); + + if (FAILED(hr)) + return NULL; + + PropVariantInit(&prop_var); + prop_var_init = true; + hr = _IPropertyStore_GetValue(prop_store, PKEY_Device_FriendlyName, &prop_var); + if (FAILED(hr)) + goto done; + + result = utf16_to_utf8_string_alloc(prop_var.pwszVal); + +done: + if (prop_var_init) + PropVariantClear(&prop_var); + + IFACE_RELEASE(prop_store); + + return result; +} + +void *mmdevice_list_new(const void *u, EDataFlow data_flow) { HRESULT hr; UINT i; - PROPVARIANT prop_var; union string_list_elem_attr attr; IMMDeviceEnumerator *enumerator = NULL; IMMDeviceCollection *collection = NULL; UINT dev_count = 0; IMMDevice *device = NULL; LPWSTR dev_id_wstr = NULL; - IPropertyStore *prop_store = NULL; bool br = false; - bool prop_var_init = false; char *dev_id_str = NULL; char *dev_name_str = NULL; struct string_list *sl = string_list_new(); @@ -54,7 +85,7 @@ void *mmdevice_list_new(void *u) goto error; hr = _IMMDeviceEnumerator_EnumAudioEndpoints(enumerator, - eRender, DEVICE_STATE_ACTIVE, &collection); + data_flow, DEVICE_STATE_ACTIVE, &collection); if (FAILED(hr)) goto error; @@ -75,19 +106,7 @@ void *mmdevice_list_new(void *u) if (!(dev_id_str = utf16_to_utf8_string_alloc(dev_id_wstr))) goto error; - hr = _IMMDevice_OpenPropertyStore(device, STGM_READ, &prop_store); - if (FAILED(hr)) - goto error; - - PropVariantInit(&prop_var); - prop_var_init = true; - hr = _IPropertyStore_GetValue( - prop_store, PKEY_Device_FriendlyName, - &prop_var); - if (FAILED(hr)) - goto error; - - if (!(dev_name_str = utf16_to_utf8_string_alloc(prop_var.pwszVal))) + if (!(dev_name_str = mmdevice_name(device))) goto error; br = string_list_append(sl, dev_name_str, attr); @@ -96,15 +115,12 @@ void *mmdevice_list_new(void *u) if (dev_id_str) sl->elems[sl->size-1].userdata = dev_id_str; - PropVariantClear(&prop_var); - prop_var_init = false; if (dev_id_wstr) CoTaskMemFree(dev_id_wstr); if (dev_name_str) free(dev_name_str); dev_name_str = NULL; dev_id_wstr = NULL; - IFACE_RELEASE(prop_store); IFACE_RELEASE(device); } @@ -120,9 +136,6 @@ error: free(dev_name_str); dev_id_str = NULL; dev_name_str = NULL; - if (prop_var_init) - PropVariantClear(&prop_var); - IFACE_RELEASE(prop_store); if (dev_id_wstr) CoTaskMemFree(dev_id_wstr); dev_id_wstr = NULL; diff --git a/audio/common/mmdevice_common.h b/audio/common/mmdevice_common.h index 647f36cd38..ea839c2898 100644 --- a/audio/common/mmdevice_common.h +++ b/audio/common/mmdevice_common.h @@ -17,10 +17,17 @@ #define _MMDEVICE_COMMON_H #include +#include "mmdevice_common_inline.h" RETRO_BEGIN_DECLS -void *mmdevice_list_new(void *u); +void *mmdevice_list_new(const void *u, EDataFlow data_flow); + +/** + * Gets the friendly name of the provided IMMDevice. + * The string must be freed with free(). + */ +char* mmdevice_name(IMMDevice *device); RETRO_END_DECLS diff --git a/audio/common/mmdevice_common_inline.h b/audio/common/mmdevice_common_inline.h index df0f219d6f..80a7405ff1 100644 --- a/audio/common/mmdevice_common_inline.h +++ b/audio/common/mmdevice_common_inline.h @@ -60,6 +60,10 @@ DEFINE_PROPERTYKEY(PKEY_Device_FriendlyName, 0xa45c254e, 0xdf1c, 0x4efd, 0x80, 0 #define _IAudioClient_GetBufferSize(This,pNumBufferFrames) ( (This)->GetBufferSize(pNumBufferFrames) ) #define _IAudioClient_GetStreamLatency(This,phnsLatency) ( (This)->GetStreamLatency(phnsLatency) ) #define _IAudioClient_GetDevicePeriod(This,phnsDefaultDevicePeriod,phnsMinimumDevicePeriod) ( (This)->GetDevicePeriod(phnsDefaultDevicePeriod,phnsMinimumDevicePeriod) ) +#define _IAudioClient_Initialize(This,ShareMode,StreamFlags,hnsBufferDuration,hnsPeriodicity,pFormat,AudioSessionGuid) \ + ( (This)->Initialize(ShareMode,StreamFlags,hnsBufferDuration,hnsPeriodicity,pFormat,AudioSessionGuid)) +#define _IAudioClient_IsFormatSupported(This,ShareMode,pFormat,ppClosestMatch) \ + ( (This)->IsFormatSupported(ShareMode,pFormat,ppClosestMatch)) #define _IMMDevice_Activate(This,iid,dwClsCtx,pActivationParams,ppv) ((This)->Activate(iid,(dwClsCtx),pActivationParams,ppv)) #define _IMMDeviceEnumerator_EnumAudioEndpoints(This,dataFlow,dwStateMask,ppDevices) (This)->EnumAudioEndpoints(dataFlow,dwStateMask,ppDevices) #define _IMMDeviceEnumerator_GetDefaultAudioEndpoint(This,dataFlow,role,ppEndpoint) (This)->GetDefaultAudioEndpoint(dataFlow,role,ppEndpoint) @@ -67,6 +71,12 @@ DEFINE_PROPERTYKEY(PKEY_Device_FriendlyName, 0xa45c254e, 0xdf1c, 0x4efd, 0x80, 0 #define _IMMDevice_GetId(This,ppstrId) ((This)->GetId(ppstrId)) #define _IPropertyStore_GetValue(This,key,pv) ( (This)->GetValue(key,pv) ) #define _IMMDeviceCollection_GetCount(This,cProps) ( (This)->GetCount(cProps) ) +#define _IAudioCaptureClient_GetBuffer(This,ppData,pNumFramesToRead,pdwFlags,pu64DevicePosition,pu64QPCPosition) \ + ( (This) -> GetBuffer(ppData,pNumFramesToRead,pdwFlags,pu64DevicePosition,pu64QPCPosition) ) +#define _IAudioCaptureClient_ReleaseBuffer(This,NumFramesRead) \ + ( (This) -> ReleaseBuffer(NumFramesRead) ) +#define _IAudioCaptureClient_GetNextPacketSize(This,pNumFramesInNextPacket) \ + ( (This) -> GetNextPacketSize(pNumFramesInNextPacket) ) #else #define _IMMDeviceCollection_Item(This,nDevice,ppdevice) (This)->lpVtbl->Item(This,nDevice,ppdevice) #define _IAudioClient_Start(This) ( (This)->lpVtbl -> Start(This) ) @@ -82,6 +92,10 @@ DEFINE_PROPERTYKEY(PKEY_Device_FriendlyName, 0xa45c254e, 0xdf1c, 0x4efd, 0x80, 0 #define _IAudioClient_GetBufferSize(This,pNumBufferFrames) ( (This)->lpVtbl -> GetBufferSize(This,pNumBufferFrames) ) #define _IAudioClient_GetStreamLatency(This,phnsLatency) ( (This)->lpVtbl -> GetStreamLatency(This,phnsLatency) ) #define _IAudioClient_GetDevicePeriod(This,phnsDefaultDevicePeriod,phnsMinimumDevicePeriod) ( (This)->lpVtbl -> GetDevicePeriod(This,phnsDefaultDevicePeriod,phnsMinimumDevicePeriod) ) +#define _IAudioClient_Initialize(This,ShareMode,StreamFlags,hnsBufferDuration,hnsPeriodicity,pFormat,AudioSessionGuid) \ + ( (This)->lpVtbl->Initialize(This,ShareMode,StreamFlags,hnsBufferDuration,hnsPeriodicity,pFormat,AudioSessionGuid)) +#define _IAudioClient_IsFormatSupported(This,ShareMode,pFormat,ppClosestMatch) \ + ( (This)->lpVtbl->IsFormatSupported(This,ShareMode,pFormat,ppClosestMatch)) #define _IMMDevice_Activate(This,iid,dwClsCtx,pActivationParams,ppv) ((This)->lpVtbl->Activate(This,&(iid),dwClsCtx,pActivationParams,ppv)) #define _IMMDeviceEnumerator_EnumAudioEndpoints(This,dataFlow,dwStateMask,ppDevices) (This)->lpVtbl->EnumAudioEndpoints(This,dataFlow,dwStateMask,ppDevices) #define _IMMDeviceEnumerator_GetDefaultAudioEndpoint(This,dataFlow,role,ppEndpoint) (This)->lpVtbl->GetDefaultAudioEndpoint(This,dataFlow,role,ppEndpoint) @@ -89,6 +103,12 @@ DEFINE_PROPERTYKEY(PKEY_Device_FriendlyName, 0xa45c254e, 0xdf1c, 0x4efd, 0x80, 0 #define _IMMDevice_GetId(This,ppstrId) (This)->lpVtbl->GetId(This,ppstrId) #define _IPropertyStore_GetValue(This,key,pv) ( (This)->lpVtbl -> GetValue(This,&(key),pv) ) #define _IMMDeviceCollection_GetCount(This,cProps) ( (This)->lpVtbl -> GetCount(This,cProps) ) +#define _IAudioCaptureClient_GetBuffer(This,ppData,pNumFramesToRead,pdwFlags,pu64DevicePosition,pu64QPCPosition) \ + ( (This)->lpVtbl -> GetBuffer(This,ppData,pNumFramesToRead,pdwFlags,pu64DevicePosition,pu64QPCPosition) ) +#define _IAudioCaptureClient_ReleaseBuffer(This,NumFramesRead) \ + ( (This)->lpVtbl -> ReleaseBuffer(This,NumFramesRead) ) +#define _IAudioCaptureClient_GetNextPacketSize(This,pNumFramesInNextPacket) \ + ( (This)-> lpVtbl -> GetNextPacketSize(This,pNumFramesInNextPacket) ) #endif #ifdef __cplusplus diff --git a/audio/common/wasapi.c b/audio/common/wasapi.c new file mode 100644 index 0000000000..1439f94734 --- /dev/null +++ b/audio/common/wasapi.c @@ -0,0 +1,745 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2011-2017 Daniel De Matteis + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include "wasapi.h" +#include +#ifdef HAVE_MICROPHONE +#include "audio/microphone_driver.h" +#endif +#include "queues/fifo_queue.h" +#include "lists/string_list.h" +#include "configuration.h" +#include "verbosity.h" +#include "string/stdstring.h" +#include "mmdevice_common.h" + +const char *hresult_name(HRESULT hr) +{ + switch (hr) + { + /* Standard error codes */ + case E_INVALIDARG: + return "E_INVALIDARG"; + case E_NOINTERFACE: + return "E_NOINTERFACE"; + case E_OUTOFMEMORY: + return "E_OUTOFMEMORY"; + case E_POINTER: + return "E_POINTER"; + /* Standard success codes */ + case S_FALSE: + return "S_FALSE"; + case S_OK: + return "S_OK"; + /* AUDCLNT error codes */ + case AUDCLNT_E_ALREADY_INITIALIZED: + return "AUDCLNT_E_ALREADY_INITIALIZED"; + case AUDCLNT_E_BUFDURATION_PERIOD_NOT_EQUAL: + return "AUDCLNT_E_BUFDURATION_PERIOD_NOT_EQUAL"; + case AUDCLNT_E_BUFFER_ERROR: + return "AUDCLNT_E_BUFFER_ERROR"; + case AUDCLNT_E_BUFFER_OPERATION_PENDING: + return "AUDCLNT_E_BUFFER_OPERATION_PENDING"; + case AUDCLNT_E_BUFFER_SIZE_NOT_ALIGNED: + return "AUDCLNT_E_BUFFER_SIZE_NOT_ALIGNED"; + case AUDCLNT_E_BUFFER_SIZE_ERROR: + return "AUDCLNT_E_BUFFER_SIZE_ERROR"; + case AUDCLNT_E_CPUUSAGE_EXCEEDED: + return "AUDCLNT_E_CPUUSAGE_EXCEEDED"; + case AUDCLNT_E_DEVICE_IN_USE: + return "AUDCLNT_E_DEVICE_IN_USE"; + case AUDCLNT_E_DEVICE_INVALIDATED: + return "AUDCLNT_E_DEVICE_INVALIDATED"; + case AUDCLNT_E_ENDPOINT_CREATE_FAILED: + return "AUDCLNT_E_ENDPOINT_CREATE_FAILED"; + case AUDCLNT_E_EXCLUSIVE_MODE_NOT_ALLOWED: + return "AUDCLNT_E_EXCLUSIVE_MODE_NOT_ALLOWED"; + case AUDCLNT_E_INVALID_DEVICE_PERIOD: + return "AUDCLNT_E_INVALID_DEVICE_PERIOD"; + case AUDCLNT_E_INVALID_SIZE: + return "AUDCLNT_E_INVALID_SIZE"; + case AUDCLNT_E_NOT_INITIALIZED: + return "AUDCLNT_E_NOT_INITIALIZED"; + case AUDCLNT_E_OUT_OF_ORDER: + return "AUDCLNT_E_OUT_OF_ORDER"; + case AUDCLNT_E_SERVICE_NOT_RUNNING: + return "AUDCLNT_E_SERVICE_NOT_RUNNING"; + case AUDCLNT_E_UNSUPPORTED_FORMAT: + return "AUDCLNT_E_UNSUPPORTED_FORMAT"; + case AUDCLNT_E_WRONG_ENDPOINT_TYPE: + return "AUDCLNT_E_WRONG_ENDPOINT_TYPE"; + /* AUDCLNT success codes */ + case AUDCLNT_S_BUFFER_EMPTY: + return "AUDCLNT_S_BUFFER_EMPTY"; + /* Something else; probably from an API that we started using + * after mic support was implemented */ + default: + return ""; + } +} + +const char *wave_subtype_name(const GUID *guid) +{ + if (IsEqualGUID(guid, &KSDATAFORMAT_SUBTYPE_IEEE_FLOAT)) + { + return "KSDATAFORMAT_SUBTYPE_IEEE_FLOAT"; + } + + return ""; +} + +const char *wave_format_name(const WAVEFORMATEXTENSIBLE *format) +{ + switch (format->Format.wFormatTag) + { + case WAVE_FORMAT_PCM: + return "WAVE_FORMAT_PCM"; + case WAVE_FORMAT_EXTENSIBLE: + return wave_subtype_name(&format->SubFormat); + default: + return ""; + } +} + +const char* wasapi_error(DWORD error) +{ + static char error_message[256]; + + FormatMessage( + FORMAT_MESSAGE_IGNORE_INSERTS | + FORMAT_MESSAGE_FROM_SYSTEM, + NULL, + error, + MAKELANGID(LANG_ENGLISH, SUBLANG_DEFAULT), + error_message, + sizeof(error_message) - 1, + NULL); + + return error_message; +} + +static const char* wasapi_data_flow_name(EDataFlow data_flow) +{ + switch (data_flow) + { + case eCapture: + return "eCapture"; + case eRender: + return "eRender"; + case eAll: + return "eAll"; + default: + break; + } + + return ""; +} + +static void wasapi_set_format(WAVEFORMATEXTENSIBLE *wf, + bool float_fmt, unsigned rate, unsigned channels); + +/** + * @param[in] format The format to check. + * @return \c true if \c format is suitable for RetroArch. + */ +static bool wasapi_is_format_suitable(const WAVEFORMATEXTENSIBLE *format) +{ + if (!format) + return false; + + if (format->Format.nChannels == 0 || format->Format.nChannels > 2) + /* RetroArch only supports mono mic input and stereo speaker output */ + return false; + + switch (format->Format.wFormatTag) + { + case WAVE_FORMAT_PCM: + if (format->Format.wBitsPerSample != 16) + /* Integer samples must be 16-bit */ + return false; + break; + case WAVE_FORMAT_EXTENSIBLE: + if (!IsEqualGUID(&format->SubFormat, &KSDATAFORMAT_SUBTYPE_IEEE_FLOAT)) + /* RetroArch doesn't support any other subformat */ + return false; + + if (format->Format.wBitsPerSample != 32) + /* floating-point samples must be 32-bit */ + return false; + break; + default: + /* Other formats are unsupported */ + return false; + } + + return true; +} +/** + * Selects a sample format suitable for the given device. + * @param[in,out] format The place where the chosen format will be written, + * as well as the first format checked. + * @param[in] client The audio client (i.e. device handle) for which a format will be selected. + * @param[in] mode The device mode (shared or exclusive) that \c client will use. + * @param[in] channels The number of channels that will constitute one audio frame. + * @return \c true if successful, \c false if a suitable format wasn't found or there was an error. + * If \c true, the selected format will be written to \c format. + * If \c false, the value referred by \c format will be unchanged. + */ +static bool wasapi_select_device_format(WAVEFORMATEXTENSIBLE *format, IAudioClient *client, AUDCLNT_SHAREMODE mode, unsigned channels) +{ + static const unsigned preferred_rates[] = { 48000, 44100, 96000, 192000, 32000 }; + const bool preferred_formats[] = {format->Format.wFormatTag == WAVE_FORMAT_EXTENSIBLE, format->Format.wFormatTag != WAVE_FORMAT_EXTENSIBLE}; + /* Try the requested sample format first, then try the other one */ + WAVEFORMATEXTENSIBLE *suggested_format = NULL; + bool result = false; + HRESULT hr = _IAudioClient_IsFormatSupported(client, mode, + (const WAVEFORMATEX *) format, (WAVEFORMATEX **) &suggested_format); + /* The Windows docs say that casting these arguments to WAVEFORMATEX* is okay. */ + + switch (hr) + { + case S_OK: + /* The requested format is okay without any changes */ + RARCH_DBG("[WASAPI]: Desired format (%s, %u-channel, %uHz) can be used as-is.\n", + wave_format_name(format), format->Format.nChannels, format->Format.nSamplesPerSec); + result = true; + break; + case S_FALSE: + /* The requested format is unsupported, but Windows has suggested a similar one. */ + RARCH_DBG("[WASAPI]: Windows suggests a format of (%s, %u-channel, %uHz).\n", + wave_format_name(suggested_format), suggested_format->Format.nChannels, suggested_format->Format.nSamplesPerSec); + if (wasapi_is_format_suitable(suggested_format)) + { + *format = *suggested_format; + result = true; + } + else + { + result = false; + RARCH_ERR("[WASAPI]: Windows suggested a format, but RetroArch can't use it.\n"); + } + + break; + case AUDCLNT_E_UNSUPPORTED_FORMAT: + { /* The requested format is unsupported + * and Windows was unable to suggest another. + * Usually happens with exclusive mode. + * RetroArch will try selecting a format. */ + int i, j; + WAVEFORMATEXTENSIBLE possible_format; + HRESULT format_check_hr; + RARCH_WARN("[WASAPI]: Requested format not supported, and Windows could not suggest one. RetroArch will do so.\n"); + for (i = 0; i < ARRAY_SIZE(preferred_formats); ++i) + { + for (j = 0; j < ARRAY_SIZE(preferred_rates); ++j) + { + wasapi_set_format(&possible_format, preferred_formats[i], preferred_rates[j], channels); + format_check_hr = _IAudioClient_IsFormatSupported(client, mode, (const WAVEFORMATEX *) &possible_format, NULL); + if (SUCCEEDED(format_check_hr)) + { + *format = possible_format; + result = true; + RARCH_DBG("[WASAPI]: RetroArch suggests a format of (%s, %u-channel, %uHz).\n", + wave_format_name(format), format->Format.nChannels, format->Format.nSamplesPerSec); + goto done; + } + } + } + RARCH_ERR("[WASAPI]: Failed to select client format: No suitable format available\n"); + result = false; + break; + } + default: + /* Something else went wrong. */ + RARCH_ERR("[WASAPI]: Failed to select client format: %s\n", hresult_name(hr)); + result = false; + break; + } +done: + if (suggested_format) + { /* IAudioClient::IsFormatSupported allocates a format object */ + CoTaskMemFree(suggested_format); + } + + return result; +} + +static void wasapi_set_format(WAVEFORMATEXTENSIBLE *wf, + bool float_fmt, unsigned rate, unsigned channels) +{ + WORD wBitsPerSample = float_fmt ? 32 : 16; + WORD nBlockAlign = (channels * wBitsPerSample) / 8; + DWORD nAvgBytesPerSec = rate * nBlockAlign; + + wf->Format.nChannels = channels; + wf->Format.nSamplesPerSec = rate; + wf->Format.nAvgBytesPerSec = nAvgBytesPerSec; + wf->Format.nBlockAlign = nBlockAlign; + wf->Format.wBitsPerSample = wBitsPerSample; + + if (float_fmt) + { + wf->Format.wFormatTag = WAVE_FORMAT_EXTENSIBLE; + wf->Format.cbSize = sizeof(WORD) + sizeof(DWORD) + sizeof(GUID); + wf->Samples.wValidBitsPerSample = wBitsPerSample; + wf->dwChannelMask = channels == 1 ? KSAUDIO_SPEAKER_MONO : KSAUDIO_SPEAKER_STEREO; + wf->SubFormat = KSDATAFORMAT_SUBTYPE_IEEE_FLOAT; + } + else + { + wf->Format.wFormatTag = WAVE_FORMAT_PCM; + wf->Format.cbSize = 0; + wf->Samples.wValidBitsPerSample = 0; + wf->dwChannelMask = 0; + memset(&wf->SubFormat, 0, sizeof(wf->SubFormat)); + } +} + +static IAudioClient *wasapi_init_client_ex(IMMDevice *device, + bool *float_fmt, unsigned *rate, unsigned latency, unsigned channels) +{ + WAVEFORMATEXTENSIBLE wf; + IAudioClient *client = NULL; + REFERENCE_TIME minimum_period = 0; + REFERENCE_TIME buffer_duration = 0; + UINT32 buffer_length = 0; + HRESULT hr = _IMMDevice_Activate(device, + IID_IAudioClient, + CLSCTX_ALL, NULL, (void**)&client); + + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: IMMDevice::Activate failed: %s\n", hresult_name(hr)); + return NULL; + } + + hr = _IAudioClient_GetDevicePeriod(client, NULL, &minimum_period); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get device period of exclusive-mode client: %s\n", hresult_name(hr)); + goto error; + } + + /* buffer_duration is in 100ns units */ + buffer_duration = latency * 10000.0; + if (buffer_duration < minimum_period) + buffer_duration = minimum_period; + + wasapi_set_format(&wf, *float_fmt, *rate, channels); + RARCH_LOG("[WASAPI]: Requesting format: %u-bit %u-channel client with %s samples at %uHz\n", + wf.Format.wBitsPerSample, + wf.Format.nChannels, wave_format_name(&wf), wf.Format.nSamplesPerSec); + + if (wasapi_select_device_format(&wf, client, AUDCLNT_SHAREMODE_EXCLUSIVE, channels)) + { + RARCH_LOG("[WASAPI]: Using format: %u-bit %u-channel client with %s samples at %uHz\n", + wf.Format.wBitsPerSample, + wf.Format.nChannels, wave_format_name(&wf), wf.Format.nSamplesPerSec); + } + else + { + RARCH_ERR("[WASAPI]: Failed to select a suitable device format\n"); + goto error; + } + + hr = _IAudioClient_Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, + AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, + buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); + if (hr == AUDCLNT_E_BUFFER_SIZE_NOT_ALIGNED) + { + RARCH_WARN("[WASAPI] Unaligned buffer size: %s\n", hresult_name(hr)); + hr = _IAudioClient_GetBufferSize(client, &buffer_length); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI] Failed to get buffer size of client: %s\n", hresult_name(hr)); + goto error; + } + + IFACE_RELEASE(client); + hr = _IMMDevice_Activate(device, + IID_IAudioClient, + CLSCTX_ALL, NULL, (void**)&client); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI] IMMDevice::Activate failed: %s\n", hresult_name(hr)); + return NULL; + } + + buffer_duration = 10000.0 * 1000.0 / (*rate) * buffer_length + 0.5; + hr = _IAudioClient_Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, + AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, + buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); + } + if (hr == AUDCLNT_E_ALREADY_INITIALIZED) + { + IFACE_RELEASE(client); + hr = _IMMDevice_Activate(device, + IID_IAudioClient, + CLSCTX_ALL, NULL, (void**)&client); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI] IMMDevice::Activate failed: %s\n", hresult_name(hr)); + return NULL; + } + + hr = _IAudioClient_Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, + AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, + buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); + } + if (hr != AUDCLNT_E_UNSUPPORTED_FORMAT) + { + if (hr == AUDCLNT_E_DEVICE_IN_USE) + goto error; + + if (hr == AUDCLNT_E_EXCLUSIVE_MODE_NOT_ALLOWED) + goto error; + } + + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to create exclusive-mode client: %s\n", hresult_name(hr)); + goto error; + } + + *float_fmt = wf.Format.wFormatTag != WAVE_FORMAT_PCM; + *rate = wf.Format.nSamplesPerSec; + + RARCH_LOG("[WASAPI]: Initialized exclusive %s client at %uHz, latency %ums\n", + *float_fmt ? "float" : "pcm", *rate, latency); + + return client; + +error: + IFACE_RELEASE(client); + + return NULL; +} + +static IAudioClient *wasapi_init_client_sh(IMMDevice *device, + bool *float_fmt, unsigned *rate, unsigned latency, unsigned channels) +{ + WAVEFORMATEXTENSIBLE wf; + IAudioClient *client = NULL; + bool float_fmt_res = *float_fmt; + unsigned rate_res = *rate; + HRESULT hr = _IMMDevice_Activate(device, + IID_IAudioClient, + CLSCTX_ALL, NULL, (void**)&client); + + if (FAILED(hr)) + { /* If we couldn't create the IAudioClient... */ + RARCH_ERR("[WASAPI]: Failed to create %s IAudioClient: %s\n", hresult_name(hr)); + return NULL; + } + + wasapi_set_format(&wf, float_fmt_res, rate_res, channels); + + if (wasapi_select_device_format(&wf, client, AUDCLNT_SHAREMODE_SHARED, channels)) + { + RARCH_LOG("[WASAPI]: Requesting %u-channel shared-mode client with %s samples at %uHz\n", + wf.Format.nChannels, wave_format_name(&wf), wf.Format.nSamplesPerSec); + } + else + { + RARCH_ERR("[WASAPI]: Failed to select a suitable device format\n"); + goto error; + } + + hr = _IAudioClient_Initialize(client, AUDCLNT_SHAREMODE_SHARED, + AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, + 0, 0, (WAVEFORMATEX*)&wf, NULL); + + if (hr == AUDCLNT_E_ALREADY_INITIALIZED) + { + IFACE_RELEASE(client); + hr = _IMMDevice_Activate(device, + IID_IAudioClient, + CLSCTX_ALL, NULL, (void**)&client); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: IMMDevice::Activate failed: %s\n", hresult_name(hr)); + return NULL; + } + + hr = _IAudioClient_Initialize(client, AUDCLNT_SHAREMODE_SHARED, + AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, + 0, 0, (WAVEFORMATEX*)&wf, NULL); + } + + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: IAudioClient::Initialize failed: %s\n", hresult_name(hr)); + goto error; + } + + *float_fmt = wf.Format.wFormatTag != WAVE_FORMAT_PCM; + *rate = wf.Format.nSamplesPerSec; + + RARCH_LOG("[WASAPI]: Initialized shared %s client at %uHz\n", + wave_format_name(&wf), *rate); + + return client; + +error: + IFACE_RELEASE(client); + + return NULL; +} + +IMMDevice *wasapi_init_device(const char *id, EDataFlow data_flow) +{ + HRESULT hr; + UINT32 dev_count, i; + IMMDeviceEnumerator *enumerator = NULL; + IMMDevice *device = NULL; + IMMDeviceCollection *collection = NULL; + const char *data_flow_name = wasapi_data_flow_name(data_flow); + + if (id) + { + RARCH_LOG("[WASAPI]: Initializing %s device \"%s\" ...\n", data_flow_name, id); + } + else + { + RARCH_LOG("[WASAPI]: Initializing default %s device.. \n", data_flow_name); + } + +#ifdef __cplusplus + hr = CoCreateInstance(CLSID_MMDeviceEnumerator, NULL, CLSCTX_ALL, + IID_IMMDeviceEnumerator, (void **)&enumerator); +#else + hr = CoCreateInstance(&CLSID_MMDeviceEnumerator, NULL, CLSCTX_ALL, + &IID_IMMDeviceEnumerator, (void **)&enumerator); +#endif + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to create device enumerator: %s\n", hresult_name(hr)); + goto error; + } + + if (id) + { /* If a specific device was requested... */ + int32_t idx_found = -1; + struct string_list *list = (struct string_list*)mmdevice_list_new(NULL, data_flow); + + if (!list) + { + RARCH_ERR("[WASAPI]: Failed to allocate %s device list\n", data_flow_name); + goto error; + } + + if (list->elems) + { /* If any devices were found... */ + unsigned d; + for (d = 0; d < list->size; d++) + { + RARCH_LOG("[WASAPI]: %u : %s\n", d, list->elems[d].data); + if (string_is_equal(id, list->elems[d].data)) + { + idx_found = d; + break; + } + } + /* Index was not found yet based on name string, + * just assume id is a one-character number index. */ + + if (idx_found == -1 && isdigit(id[0])) + { + idx_found = strtoul(id, NULL, 0); + RARCH_LOG("[WASAPI]: Fallback, %s device index is a single number index instead: %u.\n", data_flow_name, idx_found); + } + } + string_list_free(list); + + if (idx_found == -1) + idx_found = 0; + + hr = _IMMDeviceEnumerator_EnumAudioEndpoints(enumerator, + data_flow, DEVICE_STATE_ACTIVE, &collection); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to enumerate audio endpoints: %s\n", hresult_name(hr)); + goto error; + } + + hr = _IMMDeviceCollection_GetCount(collection, &dev_count); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to count IMMDevices: %s\n", hresult_name(hr)); + goto error; + } + + for (i = 0; i < dev_count; ++i) + { + hr = _IMMDeviceCollection_Item(collection, i, &device); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get IMMDevice #%d: %s\n", i, hresult_name(hr)); + goto error; + } + + if (i == idx_found) + break; + + IFACE_RELEASE(device); + } + } + else + { + hr = _IMMDeviceEnumerator_GetDefaultAudioEndpoint( + enumerator, data_flow, eConsole, &device); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get default audio endpoint: %s\n", hresult_name(hr)); + goto error; + } + } + + if (!device) + goto error; + + IFACE_RELEASE(collection); + IFACE_RELEASE(enumerator); + + return device; + +error: + IFACE_RELEASE(collection); + IFACE_RELEASE(enumerator); + + if (id) + { + RARCH_WARN("[WASAPI]: Failed to initialize %s device \"%s\"\n", data_flow_name, id); + } + else + { + RARCH_ERR("[WASAPI]: Failed to initialize default %s device\n", data_flow_name); + } + + return NULL; +} + +IAudioClient *wasapi_init_client(IMMDevice *device, bool *exclusive, + bool *float_fmt, unsigned *rate, unsigned latency, unsigned channels) +{ + HRESULT hr; + IAudioClient *client; + double latency_res; + REFERENCE_TIME device_period = 0; + REFERENCE_TIME stream_latency = 0; + UINT32 buffer_length = 0; + + RARCH_DBG("[WASAPI]: Requesting %s %s client (rate=%uHz, latency=%ums).\n", + *exclusive ? "exclusive" : "shared", + *float_fmt ? "float" : "pcm", *rate, latency); + + if (*exclusive) + { + client = wasapi_init_client_ex(device, float_fmt, rate, latency, channels); + if (!client) + { + RARCH_WARN("[WASAPI] Failed to initialize exclusive client, attempting shared client.\n"); + client = wasapi_init_client_sh(device, float_fmt, rate, latency, channels); + if (client) + *exclusive = false; + } + } + else + { + client = wasapi_init_client_sh(device, float_fmt, rate, latency, channels); + if (!client) + { + RARCH_WARN("[WASAPI] Failed to initialize shared client, attempting exclusive client.\n"); + client = wasapi_init_client_ex(device, float_fmt, rate, latency, channels); + if (client) + *exclusive = true; + } + } + + if (!client) + return NULL; + + /* next calls are allowed to fail (we losing info only) */ + + if (*exclusive) + { + hr = _IAudioClient_GetDevicePeriod(client, NULL, &device_period); + if (SUCCEEDED(hr)) + { + RARCH_LOG("[WASAPI]: Minimum exclusive-mode device period is %uns (= %.1fms)\n", + device_period * 100, (double)device_period * 100 / 1e6); + } + /* device_period is in 100ns units */ + } + else + { + hr = _IAudioClient_GetDevicePeriod(client, &device_period, NULL); + if (SUCCEEDED(hr)) + { + RARCH_LOG("[WASAPI]: Default shared-mode device period is %uns (= %.1fms)\n", + device_period * 100, (double)device_period * 100 / 1e6); + } + } + + if (FAILED(hr)) + { + RARCH_WARN("[WASAPI]: IAudioClient::GetDevicePeriod failed: %s\n", hresult_name(hr)); + } + + if (!*exclusive) + { + hr = _IAudioClient_GetStreamLatency(client, &stream_latency); + if (SUCCEEDED(hr)) + { + RARCH_LOG("[WASAPI]: Shared stream latency is %uns (= %.1fms)\n", + stream_latency * 100, (double)stream_latency * 100 / 1e6); + } + else + { + RARCH_WARN("[WASAPI]: IAudioClient::GetStreamLatency failed: %s\n", hresult_name(hr)); + } + } + + hr = _IAudioClient_GetBufferSize(client, &buffer_length); + if (SUCCEEDED(hr)) + { + size_t num_samples = buffer_length * channels; + size_t num_bytes = num_samples * (*float_fmt ? sizeof(float) : sizeof(int16_t)); + RARCH_LOG("[WASAPI]: Endpoint buffer size is %u audio frames (= %u samples, = %u bytes)\n", + buffer_length, num_samples, num_bytes); + } + else + { + RARCH_WARN("[WASAPI]: IAudioClient::GetBufferSize failed: %s.\n", hresult_name(hr)); + } + + if (*exclusive) + latency_res = (double)buffer_length * 1000.0 / (*rate); + else + latency_res = (double)(stream_latency + device_period) / 10000.0; + + RARCH_LOG("[WASAPI]: Client initialized (%s, %s, %uHz, %.1fms).\n", + *exclusive ? "exclusive" : "shared", + *float_fmt ? "float" : "pcm", *rate, latency_res); + + RARCH_LOG("[WASAPI]: Client's buffer length is %u frames (%.1fms).\n", + buffer_length, (double)buffer_length * 1000.0 / (*rate)); + + RARCH_LOG("[WASAPI]: Device period is %.1fms (%lld frames).\n", + (double)device_period / 10000.0, device_period * (*rate) / 10000000); + + return client; +} \ No newline at end of file diff --git a/audio/common/wasapi.h b/audio/common/wasapi.h new file mode 100644 index 0000000000..ac4be783b2 --- /dev/null +++ b/audio/common/wasapi.h @@ -0,0 +1,35 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2011-2017 Daniel De Matteis + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +/** + * Contains WASAPI-specific support functions that are used + * by the WASAPI audio and microphone drivers. + * + */ + +#ifndef RETROARCH_COMMON_WASAPI_H +#define RETROARCH_COMMON_WASAPI_H + +#include "../common/mmdevice_common_inline.h" +#include "boolean.h" + +const char *hresult_name(HRESULT hr); +const char* wasapi_error(DWORD error); +IMMDevice *wasapi_init_device(const char *id, EDataFlow data_flow); +IAudioClient *wasapi_init_client(IMMDevice *device, bool *exclusive, + bool *float_fmt, unsigned *rate, unsigned latency, unsigned channels); + +#endif /* RETROARCH_COMMON_WASAPI_H */ \ No newline at end of file diff --git a/audio/drivers/alsa.c b/audio/drivers/alsa.c index 8a60432484..900bbcd520 100644 --- a/audio/drivers/alsa.c +++ b/audio/drivers/alsa.c @@ -20,160 +20,53 @@ #include #include +#include #include "../audio_driver.h" +#include "../common/alsa.h" #include "../../verbosity.h" typedef struct alsa { snd_pcm_t *pcm; - size_t buffer_size; - unsigned int frame_bits; + alsa_stream_info_t stream_info; bool nonblock; - bool has_float; - bool can_pause; bool is_paused; } alsa_t; static bool alsa_use_float(void *data) { alsa_t *alsa = (alsa_t*)data; - return alsa->has_float; -} - -static bool find_float_format(snd_pcm_t *pcm, void *data) -{ - snd_pcm_hw_params_t *params = (snd_pcm_hw_params_t*)data; - - if (snd_pcm_hw_params_test_format(pcm, params, SND_PCM_FORMAT_FLOAT) == 0) - { - RARCH_LOG("[ALSA]: Using floating point format.\n"); - return true; - } - - RARCH_LOG("[ALSA]: Using signed 16-bit format.\n"); - return false; + return alsa->stream_info.has_float; } +static void alsa_free(void *data); static void *alsa_init(const char *device, unsigned rate, unsigned latency, unsigned block_frames, unsigned *new_rate) { - snd_pcm_format_t format; - snd_pcm_uframes_t buffer_size; - snd_pcm_hw_params_t *params = NULL; - snd_pcm_sw_params_t *sw_params = NULL; - unsigned latency_usec = latency * 1000; - unsigned channels = 2; - unsigned periods = 4; - unsigned orig_rate = rate; - const char *alsa_dev = "default"; - alsa_t *alsa = (alsa_t*)calloc(1, sizeof(alsa_t)); + alsa_t *alsa = (alsa_t*)calloc(1, sizeof(alsa_t)); if (!alsa) + { + RARCH_ERR("[ALSA]: Failed to allocate driver context\n"); return NULL; + } - if (device) - alsa_dev = device; + RARCH_LOG("[ALSA]: Using ALSA version %s\n", snd_asoundlib_version()); - if (snd_pcm_open( - &alsa->pcm, alsa_dev, SND_PCM_STREAM_PLAYBACK, SND_PCM_NONBLOCK) < 0) + if (alsa_init_pcm(&alsa->pcm, device, SND_PCM_STREAM_PLAYBACK, rate, latency, 2, &alsa->stream_info, new_rate, SND_PCM_NONBLOCK) < 0) + { goto error; - - if (snd_pcm_hw_params_malloc(¶ms) < 0) - goto error; - - if (snd_pcm_hw_params_any(alsa->pcm, params) < 0) - goto error; - - alsa->has_float = find_float_format(alsa->pcm, params); - format = alsa->has_float ? SND_PCM_FORMAT_FLOAT : SND_PCM_FORMAT_S16; - - if (snd_pcm_hw_params_set_access( - alsa->pcm, params, SND_PCM_ACCESS_RW_INTERLEAVED) < 0) - goto error; - - /* channels hardcoded to 2 for now */ - alsa->frame_bits = snd_pcm_format_physical_width(format) * 2; - - if (snd_pcm_hw_params_set_format(alsa->pcm, params, format) < 0) - goto error; - - if (snd_pcm_hw_params_set_channels(alsa->pcm, params, channels) < 0) - goto error; - - /* Don't allow rate resampling when probing for the default rate (but ignore if this call fails) */ - snd_pcm_hw_params_set_rate_resample(alsa->pcm, params, 0 ); - if (snd_pcm_hw_params_set_rate_near(alsa->pcm, params, &rate, 0) < 0) - goto error; - - if (rate != orig_rate) - *new_rate = rate; - - if (snd_pcm_hw_params_set_buffer_time_near( - alsa->pcm, params, &latency_usec, NULL) < 0) - goto error; - - if (snd_pcm_hw_params_set_periods_near( - alsa->pcm, params, &periods, NULL) < 0) - goto error; - - if (snd_pcm_hw_params(alsa->pcm, params) < 0) - goto error; - - /* Shouldn't have to bother with this, - * but some drivers are apparently broken. */ - if (snd_pcm_hw_params_get_period_size(params, &buffer_size, NULL)) - snd_pcm_hw_params_get_period_size_min(params, &buffer_size, NULL); - - RARCH_LOG("[ALSA]: Period size: %d frames\n", (int)buffer_size); - - if (snd_pcm_hw_params_get_buffer_size(params, &buffer_size)) - snd_pcm_hw_params_get_buffer_size_max(params, &buffer_size); - - RARCH_LOG("[ALSA]: Buffer size: %d frames\n", (int)buffer_size); - - alsa->buffer_size = snd_pcm_frames_to_bytes(alsa->pcm, buffer_size); - alsa->can_pause = snd_pcm_hw_params_can_pause(params); - - RARCH_LOG("[ALSA]: Can pause: %s.\n", alsa->can_pause ? "yes" : "no"); - - if (snd_pcm_sw_params_malloc(&sw_params) < 0) - goto error; - - if (snd_pcm_sw_params_current(alsa->pcm, sw_params) < 0) - goto error; - - if (snd_pcm_sw_params_set_start_threshold( - alsa->pcm, sw_params, buffer_size / 2) < 0) - goto error; - - if (snd_pcm_sw_params(alsa->pcm, sw_params) < 0) - goto error; - - snd_pcm_hw_params_free(params); - snd_pcm_sw_params_free(sw_params); + } return alsa; error: RARCH_ERR("[ALSA]: Failed to initialize...\n"); - if (params) - snd_pcm_hw_params_free(params); - if (sw_params) - snd_pcm_sw_params_free(sw_params); + alsa_free(alsa); - if (alsa) - { - if (alsa->pcm) - { - snd_pcm_close(alsa->pcm); - snd_config_update_free_global(); - } - - free(alsa); - } return NULL; } @@ -186,8 +79,8 @@ static ssize_t alsa_write(void *data, const void *buf_, size_t size_) alsa_t *alsa = (alsa_t*)data; const uint8_t *buf = (const uint8_t*)buf_; snd_pcm_sframes_t written = 0; - snd_pcm_sframes_t size = BYTES_TO_FRAMES(size_, alsa->frame_bits); - size_t frames_size = alsa->has_float ? sizeof(float) : sizeof(int16_t); + snd_pcm_sframes_t size = BYTES_TO_FRAMES(size_, alsa->stream_info.frame_bits); + size_t frames_size = alsa->stream_info.has_float ? sizeof(float) : sizeof(int16_t); /* Workaround buggy menu code. * If a write happens while we're paused, we might never progress. */ @@ -279,7 +172,7 @@ static bool alsa_stop(void *data) if (alsa->is_paused) return true; - if (alsa->can_pause + if (alsa->stream_info.can_pause && !alsa->is_paused) { int ret = snd_pcm_pause(alsa->pcm, 1); @@ -303,9 +196,9 @@ static bool alsa_start(void *data, bool is_shutdown) { alsa_t *alsa = (alsa_t*)data; if (!alsa->is_paused) - return true; + return true; - if (alsa->can_pause + if (alsa->stream_info.can_pause && alsa->is_paused) { int ret = snd_pcm_pause(alsa->pcm, 0); @@ -328,13 +221,9 @@ static void alsa_free(void *data) if (alsa) { - if (alsa->pcm) - { - snd_pcm_drop(alsa->pcm); - snd_pcm_close(alsa->pcm); - snd_config_update_free_global(); - } + alsa_free_pcm(alsa->pcm); + snd_config_update_free_global(); free(alsa); } } @@ -345,66 +234,23 @@ static size_t alsa_write_avail(void *data) snd_pcm_sframes_t avail = snd_pcm_avail(alsa->pcm); if (avail < 0) - return alsa->buffer_size; + return alsa->stream_info.buffer_size; - return FRAMES_TO_BYTES(avail, alsa->frame_bits); + return FRAMES_TO_BYTES(avail, alsa->stream_info.frame_bits); } static size_t alsa_buffer_size(void *data) { alsa_t *alsa = (alsa_t*)data; - return alsa->buffer_size; + return alsa->stream_info.buffer_size; } -static void *alsa_device_list_new(void *data) +void *alsa_device_list_new(void *data) { - void **hints, **n; - union string_list_elem_attr attr; - struct string_list *s = string_list_new(); - - if (!s) - return NULL; - - attr.i = 0; - - if (snd_device_name_hint(-1, "pcm", &hints) != 0) - goto error; - - n = hints; - - while (*n) - { - char *name = snd_device_name_get_hint(*n, "NAME"); - char *io = snd_device_name_get_hint(*n, "IOID"); - char *desc = snd_device_name_get_hint(*n, "DESC"); - - /* description of device IOID - input / output identifcation - * ("Input" or "Output"), NULL means both) */ - - if (!io || (string_is_equal(io, "Output"))) - string_list_append(s, name, attr); - - if (name) - free(name); - if (io) - free(io); - if (desc) - free(desc); - - n++; - } - - /* free hint buffer too */ - snd_device_name_free_hint(hints); - - return s; - -error: - string_list_free(s); - return NULL; + return alsa_device_list_type_new("Output"); } -static void alsa_device_list_free(void *data, void *array_list_data) +void alsa_device_list_free(void *data, void *array_list_data) { struct string_list *s = (struct string_list*)array_list_data; diff --git a/audio/drivers/alsathread.c b/audio/drivers/alsathread.c index 1e239ef119..030f23dad0 100644 --- a/audio/drivers/alsathread.c +++ b/audio/drivers/alsathread.c @@ -16,72 +16,63 @@ #include -#include - #include #include #include #include +#include #include "../audio_driver.h" +#include "../common/alsa.h" /* For some common functions/types */ +#include "../common/alsathread.h" #include "../../verbosity.h" -#define TRY_ALSA(x) if (x < 0) \ - goto error; - typedef struct alsa_thread { - snd_pcm_t *pcm; - fifo_buffer_t *buffer; - sthread_t *worker_thread; - slock_t *fifo_lock; - scond_t *cond; - slock_t *cond_lock; - size_t buffer_size; - size_t period_size; - snd_pcm_uframes_t period_frames; + alsa_thread_info_t info; bool nonblock; bool is_paused; - bool has_float; - volatile bool thread_dead; } alsa_thread_t; static void alsa_worker_thread(void *data) { alsa_thread_t *alsa = (alsa_thread_t*)data; - uint8_t *buf = (uint8_t *)calloc(1, alsa->period_size); + uint8_t *buf = (uint8_t *)calloc(1, alsa->info.stream_info.period_size); + uintptr_t thread_id = sthread_get_current_thread_id(); if (!buf) { - RARCH_ERR("failed to allocate audio buffer"); + RARCH_ERR("[ALSA] [playback thread %u]: Failed to allocate audio buffer\n", thread_id); goto end; } - while (!alsa->thread_dead) + RARCH_DBG("[ALSA] [playback thread %p]: Beginning playback worker thread\n", thread_id); + while (!alsa->info.thread_dead) { size_t avail; size_t fifo_size; snd_pcm_sframes_t frames; - slock_lock(alsa->fifo_lock); - avail = FIFO_READ_AVAIL(alsa->buffer); - fifo_size = MIN(alsa->period_size, avail); - fifo_read(alsa->buffer, buf, fifo_size); - scond_signal(alsa->cond); - slock_unlock(alsa->fifo_lock); + slock_lock(alsa->info.fifo_lock); + avail = FIFO_READ_AVAIL(alsa->info.buffer); + fifo_size = MIN(alsa->info.stream_info.period_size, avail); + fifo_read(alsa->info.buffer, buf, fifo_size); + scond_signal(alsa->info.cond); + slock_unlock(alsa->info.fifo_lock); /* If underrun, fill rest with silence. */ - memset(buf + fifo_size, 0, alsa->period_size - fifo_size); + memset(buf + fifo_size, 0, alsa->info.stream_info.period_size - fifo_size); - frames = snd_pcm_writei(alsa->pcm, buf, alsa->period_frames); + frames = snd_pcm_writei(alsa->info.pcm, buf, alsa->info.stream_info.period_frames); if (frames == -EPIPE || frames == -EINTR || frames == -ESTRPIPE) { - if (snd_pcm_recover(alsa->pcm, frames, 1) < 0) + if (snd_pcm_recover(alsa->info.pcm, frames, false) < 0) { - RARCH_ERR("[ALSA]: (#2) Failed to recover from error (%s)\n", - snd_strerror(frames)); + RARCH_ERR("[ALSA] [playback thread %u]: Failed to recover from error: %s\n", + thread_id, + snd_strerror(frames)); break; } @@ -89,36 +80,26 @@ static void alsa_worker_thread(void *data) } else if (frames < 0) { - RARCH_ERR("[ALSA]: Unknown error occurred (%s).\n", - snd_strerror(frames)); + RARCH_ERR("[ALSA] [playback thread %u]: Error writing audio to device: %s\n", + thread_id, + snd_strerror(frames)); break; } } end: - slock_lock(alsa->cond_lock); - alsa->thread_dead = true; - scond_signal(alsa->cond); - slock_unlock(alsa->cond_lock); + slock_lock(alsa->info.cond_lock); + alsa->info.thread_dead = true; + scond_signal(alsa->info.cond); + slock_unlock(alsa->info.cond_lock); free(buf); + RARCH_DBG("[ALSA] [playback thread %p]: Ending playback worker thread\n", thread_id); } static bool alsa_thread_use_float(void *data) { alsa_thread_t *alsa = (alsa_thread_t*)data; - return alsa->has_float; -} - -static bool alsathread_find_float_format(snd_pcm_t *pcm, - snd_pcm_hw_params_t *params) -{ - if (snd_pcm_hw_params_test_format(pcm, params, SND_PCM_FORMAT_FLOAT) == 0) - { - RARCH_LOG("ALSA: Using floating point format.\n"); - return true; - } - RARCH_LOG("ALSA: Using signed 16-bit format.\n"); - return false; + return alsa->info.stream_info.has_float; } static void alsa_thread_free(void *data) @@ -127,26 +108,7 @@ static void alsa_thread_free(void *data) if (alsa) { - if (alsa->worker_thread) - { - slock_lock(alsa->cond_lock); - alsa->thread_dead = true; - slock_unlock(alsa->cond_lock); - sthread_join(alsa->worker_thread); - } - if (alsa->buffer) - fifo_free(alsa->buffer); - if (alsa->cond) - scond_free(alsa->cond); - if (alsa->fifo_lock) - slock_free(alsa->fifo_lock); - if (alsa->cond_lock) - slock_free(alsa->cond_lock); - if (alsa->pcm) - { - snd_pcm_drop(alsa->pcm); - snd_pcm_close(alsa->pcm); - } + alsa_thread_free_info_members(&alsa->info); free(alsa); } } @@ -156,86 +118,39 @@ static void *alsa_thread_init(const char *device, unsigned block_frames, unsigned *new_rate) { - snd_pcm_uframes_t buffer_size; - snd_pcm_format_t format; - snd_pcm_hw_params_t *params = NULL; - snd_pcm_sw_params_t *sw_params = NULL; - const char *alsa_dev = device ? device : "default"; - unsigned latency_usec = latency * 1000 / 2; - unsigned channels = 2; - unsigned periods = 4; - alsa_thread_t *alsa = (alsa_thread_t*) - calloc(1, sizeof(alsa_thread_t)); + alsa_thread_t *alsa = (alsa_thread_t*)calloc(1, sizeof(alsa_thread_t)); if (!alsa) + { + RARCH_ERR("[ALSA] Failed to allocate driver context\n"); return NULL; + } - TRY_ALSA(snd_pcm_open(&alsa->pcm, alsa_dev, SND_PCM_STREAM_PLAYBACK, 0)); + RARCH_LOG("[ALSA] Using ALSA version %s\n", snd_asoundlib_version()); - TRY_ALSA(snd_pcm_hw_params_malloc(¶ms)); - TRY_ALSA(snd_pcm_hw_params_any(alsa->pcm, params)); + if (alsa_init_pcm(&alsa->info.pcm, device, SND_PCM_STREAM_PLAYBACK, rate, latency, 2, &alsa->info.stream_info, new_rate, 0) < 0) + { + goto error; + } - alsa->has_float = alsathread_find_float_format(alsa->pcm, params); - format = alsa->has_float ? SND_PCM_FORMAT_FLOAT : SND_PCM_FORMAT_S16; - - TRY_ALSA(snd_pcm_hw_params_set_access( - alsa->pcm, params, SND_PCM_ACCESS_RW_INTERLEAVED)); - TRY_ALSA(snd_pcm_hw_params_set_format(alsa->pcm, params, format)); - TRY_ALSA(snd_pcm_hw_params_set_channels(alsa->pcm, params, channels)); - TRY_ALSA(snd_pcm_hw_params_set_rate(alsa->pcm, params, rate, 0)); - - TRY_ALSA(snd_pcm_hw_params_set_buffer_time_near( - alsa->pcm, params, &latency_usec, NULL)); - TRY_ALSA(snd_pcm_hw_params_set_periods_near( - alsa->pcm, params, &periods, NULL)); - - TRY_ALSA(snd_pcm_hw_params(alsa->pcm, params)); - - /* Shouldn't have to bother with this, - * but some drivers are apparently broken. */ - if (snd_pcm_hw_params_get_period_size(params, &alsa->period_frames, NULL)) - snd_pcm_hw_params_get_period_size_min( - params, &alsa->period_frames, NULL); - RARCH_LOG("ALSA: Period size: %d frames\n", (int)alsa->period_frames); - if (snd_pcm_hw_params_get_buffer_size(params, &buffer_size)) - snd_pcm_hw_params_get_buffer_size_max(params, &buffer_size); - RARCH_LOG("ALSA: Buffer size: %d frames\n", (int)buffer_size); - - alsa->buffer_size = snd_pcm_frames_to_bytes(alsa->pcm, buffer_size); - alsa->period_size = snd_pcm_frames_to_bytes(alsa->pcm, alsa->period_frames); - - TRY_ALSA(snd_pcm_sw_params_malloc(&sw_params)); - TRY_ALSA(snd_pcm_sw_params_current(alsa->pcm, sw_params)); - TRY_ALSA(snd_pcm_sw_params_set_start_threshold( - alsa->pcm, sw_params, buffer_size / 2)); - TRY_ALSA(snd_pcm_sw_params(alsa->pcm, sw_params)); - - snd_pcm_hw_params_free(params); - snd_pcm_sw_params_free(sw_params); - - alsa->fifo_lock = slock_new(); - alsa->cond_lock = slock_new(); - alsa->cond = scond_new(); - alsa->buffer = fifo_new(alsa->buffer_size); - if (!alsa->fifo_lock || !alsa->cond_lock || !alsa->cond || !alsa->buffer) + alsa->info.fifo_lock = slock_new(); + alsa->info.cond_lock = slock_new(); + alsa->info.cond = scond_new(); + alsa->info.buffer = fifo_new(alsa->info.stream_info.buffer_size); + if (!alsa->info.fifo_lock || !alsa->info.cond_lock || !alsa->info.cond || !alsa->info.buffer) goto error; - alsa->worker_thread = sthread_create(alsa_worker_thread, alsa); - if (!alsa->worker_thread) + alsa->info.worker_thread = sthread_create(alsa_worker_thread, alsa); + if (!alsa->info.worker_thread) { - RARCH_ERR("error initializing worker thread"); + RARCH_ERR("[ALSA]: Failed to initialize worker thread\n"); goto error; } return alsa; error: - RARCH_ERR("ALSA: Failed to initialize...\n"); - if (params) - snd_pcm_hw_params_free(params); - - if (sw_params) - snd_pcm_sw_params_free(sw_params); + RARCH_ERR("[ALSA]: Failed to initialize...\n"); alsa_thread_free(alsa); @@ -246,7 +161,7 @@ static ssize_t alsa_thread_write(void *data, const void *buf, size_t size) { alsa_thread_t *alsa = (alsa_thread_t*)data; - if (alsa->thread_dead) + if (alsa->info.thread_dead) return -1; if (alsa->nonblock) @@ -254,38 +169,38 @@ static ssize_t alsa_thread_write(void *data, const void *buf, size_t size) size_t avail; size_t write_amt; - slock_lock(alsa->fifo_lock); - avail = FIFO_WRITE_AVAIL(alsa->buffer); + slock_lock(alsa->info.fifo_lock); + avail = FIFO_WRITE_AVAIL(alsa->info.buffer); write_amt = MIN(avail, size); - fifo_write(alsa->buffer, buf, write_amt); - slock_unlock(alsa->fifo_lock); + fifo_write(alsa->info.buffer, buf, write_amt); + slock_unlock(alsa->info.fifo_lock); return write_amt; } else { size_t written = 0; - while (written < size && !alsa->thread_dead) + while (written < size && !alsa->info.thread_dead) { size_t avail; - slock_lock(alsa->fifo_lock); - avail = FIFO_WRITE_AVAIL(alsa->buffer); + slock_lock(alsa->info.fifo_lock); + avail = FIFO_WRITE_AVAIL(alsa->info.buffer); if (avail == 0) { - slock_unlock(alsa->fifo_lock); - slock_lock(alsa->cond_lock); - if (!alsa->thread_dead) - scond_wait(alsa->cond, alsa->cond_lock); - slock_unlock(alsa->cond_lock); + slock_unlock(alsa->info.fifo_lock); + slock_lock(alsa->info.cond_lock); + if (!alsa->info.thread_dead) + scond_wait(alsa->info.cond, alsa->info.cond_lock); + slock_unlock(alsa->info.cond_lock); } else { size_t write_amt = MIN(size - written, avail); - fifo_write(alsa->buffer, + fifo_write(alsa->info.buffer, (const char*)buf + written, write_amt); - slock_unlock(alsa->fifo_lock); + slock_unlock(alsa->info.fifo_lock); written += write_amt; } } @@ -330,76 +245,18 @@ static size_t alsa_thread_write_avail(void *data) alsa_thread_t *alsa = (alsa_thread_t*)data; size_t val; - if (alsa->thread_dead) + if (alsa->info.thread_dead) return 0; - slock_lock(alsa->fifo_lock); - val = FIFO_WRITE_AVAIL(alsa->buffer); - slock_unlock(alsa->fifo_lock); + slock_lock(alsa->info.fifo_lock); + val = FIFO_WRITE_AVAIL(alsa->info.buffer); + slock_unlock(alsa->info.fifo_lock); return val; } static size_t alsa_thread_buffer_size(void *data) { alsa_thread_t *alsa = (alsa_thread_t*)data; - return alsa->buffer_size; -} - -static void *alsa_thread_device_list_new(void *data) -{ - void **hints, **n; - union string_list_elem_attr attr; - struct string_list *s = string_list_new(); - - if (!s) - return NULL; - - attr.i = 0; - - if (snd_device_name_hint(-1, "pcm", &hints) != 0) - goto error; - - n = hints; - - while (*n) - { - char *name = snd_device_name_get_hint(*n, "NAME"); - char *io = snd_device_name_get_hint(*n, "IOID"); - char *desc = snd_device_name_get_hint(*n, "DESC"); - - /* description of device IOID - input / output identifcation - * ("Input" or "Output"), NULL means both) */ - - if (!io || string_is_equal(io,"Output")) - string_list_append(s, name, attr); - - if (name) - free(name); - if (io) - free(io); - if (desc) - free(desc); - - n++; - } - - /* free hint buffer too */ - snd_device_name_free_hint(hints); - - return s; - -error: - string_list_free(s); - return NULL; -} - -static void alsa_thread_device_list_free(void *data, void *array_list_data) -{ - struct string_list *s = (struct string_list*)array_list_data; - - if (!s) - return; - - string_list_free(s); + return alsa->info.stream_info.buffer_size; } audio_driver_t audio_alsathread = { @@ -412,8 +269,8 @@ audio_driver_t audio_alsathread = { alsa_thread_free, alsa_thread_use_float, "alsathread", - alsa_thread_device_list_new, - alsa_thread_device_list_free, + alsa_device_list_new, /* Reusing these functions from alsa.c */ + alsa_device_list_free, /* because they don't use the driver context */ alsa_thread_write_avail, - alsa_thread_buffer_size, + alsa_thread_buffer_size }; diff --git a/audio/drivers/sdl_audio.c b/audio/drivers/sdl_audio.c index 56554083f5..241e410a5d 100644 --- a/audio/drivers/sdl_audio.c +++ b/audio/drivers/sdl_audio.c @@ -30,6 +30,36 @@ #include "../audio_driver.h" #include "../../verbosity.h" +#include "retro_assert.h" + +#ifndef HAVE_SDL2 +typedef Uint32 SDL_AudioDeviceID; + +/** Compatibility stub that defers to SDL_PauseAudio. */ +#define SDL_PauseAudioDevice(dev, pause_on) SDL_PauseAudio(pause_on) + +/** Compatibility stub that defers to SDL_LockAudio. */ +#define SDL_LockAudioDevice(dev) SDL_LockAudio() + +/** Compatibility stub that defers to SDL_UnlockAudio. */ +#define SDL_UnlockAudioDevice(dev) SDL_UnlockAudio() + +/** Compatibility stub that defers to SDL_CloseAudio. */ +#define SDL_CloseAudioDevice(dev) SDL_CloseAudio() + +/* Macros for checking audio format bits that were introduced in SDL 2 */ +#define SDL_AUDIO_MASK_BITSIZE (0xFF) +#define SDL_AUDIO_MASK_DATATYPE (1<<8) +#define SDL_AUDIO_MASK_ENDIAN (1<<12) +#define SDL_AUDIO_MASK_SIGNED (1<<15) +#define SDL_AUDIO_BITSIZE(x) (x & SDL_AUDIO_MASK_BITSIZE) +#define SDL_AUDIO_ISFLOAT(x) (x & SDL_AUDIO_MASK_DATATYPE) +#define SDL_AUDIO_ISBIGENDIAN(x) (x & SDL_AUDIO_MASK_ENDIAN) +#define SDL_AUDIO_ISSIGNED(x) (x & SDL_AUDIO_MASK_SIGNED) +#define SDL_AUDIO_ISINT(x) (!SDL_AUDIO_ISFLOAT(x)) +#define SDL_AUDIO_ISLITTLEENDIAN(x) (!SDL_AUDIO_ISBIGENDIAN(x)) +#define SDL_AUDIO_ISUNSIGNED(x) (!SDL_AUDIO_ISSIGNED(x)) +#endif typedef struct sdl_audio { @@ -37,18 +67,24 @@ typedef struct sdl_audio slock_t *lock; scond_t *cond; #endif - fifo_buffer_t *buffer; + /** + * The queue used to store outgoing samples to be played by the driver. + * Audio from the core ultimately makes its way here, + * the last stop before the driver plays it. + */ + fifo_buffer_t *speaker_buffer; bool nonblock; bool is_paused; + SDL_AudioDeviceID speaker_device; } sdl_audio_t; -static void sdl_audio_cb(void *data, Uint8 *stream, int len) +static void sdl_audio_playback_cb(void *data, Uint8 *stream, int len) { sdl_audio_t *sdl = (sdl_audio_t*)data; - size_t avail = FIFO_READ_AVAIL(sdl->buffer); + size_t avail = FIFO_READ_AVAIL(sdl->speaker_buffer); size_t write_size = len > (int)avail ? avail : len; - fifo_read(sdl->buffer, stream, write_size); + fifo_read(sdl->speaker_buffer, stream, write_size); #ifdef HAVE_THREADS scond_signal(sdl->cond); #endif @@ -103,41 +139,75 @@ static void *sdl_audio_init(const char *device, * SDL double buffers audio and we do as well. */ frames = find_num_frames(rate, latency / 4); + /* First, let's initialize the output device. */ spec.freq = rate; spec.format = AUDIO_S16SYS; spec.channels = 2; spec.samples = frames; /* This is in audio frames, not samples ... :( */ - spec.callback = sdl_audio_cb; + spec.callback = sdl_audio_playback_cb; spec.userdata = sdl; - if (SDL_OpenAudio(&spec, &out) < 0) + /* No compatibility stub for SDL_OpenAudioDevice because its return value + * is different from that of SDL_OpenAudio. */ +#ifdef HAVE_SDL2 + sdl->speaker_device = SDL_OpenAudioDevice(NULL, false, &spec, &out, 0); + + if (sdl->speaker_device == 0) +#else + sdl->speaker_device = SDL_OpenAudio(&spec, &out); + + if (sdl->speaker_device < 0) +#endif { - RARCH_ERR("[SDL audio]: Failed to open SDL audio: %s\n", SDL_GetError()); + RARCH_ERR("[SDL audio]: Failed to open SDL audio output device: %s\n", SDL_GetError()); goto error; } *new_rate = out.freq; + RARCH_DBG("[SDL audio]: Opened SDL audio out device with ID %u\n", + sdl->speaker_device); + RARCH_DBG("[SDL audio]: Requested a speaker frequency of %u Hz, got %u Hz\n", + spec.freq, out.freq); + RARCH_DBG("[SDL audio]: Requested %u channels for speaker, got %u\n", + spec.channels, out.channels); + RARCH_DBG("[SDL audio]: Requested a %u-frame speaker buffer, got %u frames (%u bytes)\n", + frames, out.samples, out.size); + RARCH_DBG("[SDL audio]: Got a speaker silence value of %u\n", out.silence); + RARCH_DBG("[SDL audio]: Requested speaker audio format: %u-bit %s %s %s endian\n", + SDL_AUDIO_BITSIZE(spec.format), + SDL_AUDIO_ISSIGNED(spec.format) ? "signed" : "unsigned", + SDL_AUDIO_ISFLOAT(spec.format) ? "floating-point" : "integer", + SDL_AUDIO_ISBIGENDIAN(spec.format) ? "big" : "little"); + + RARCH_DBG("[SDL audio]: Received speaker audio format: %u-bit %s %s %s endian\n", + SDL_AUDIO_BITSIZE(spec.format), + SDL_AUDIO_ISSIGNED(spec.format) ? "signed" : "unsigned", + SDL_AUDIO_ISFLOAT(spec.format) ? "floating-point" : "integer", + SDL_AUDIO_ISBIGENDIAN(spec.format) ? "big" : "little"); #ifdef HAVE_THREADS sdl->lock = slock_new(); sdl->cond = scond_new(); #endif - RARCH_LOG("[SDL audio]: Requested %u ms latency, got %d ms\n", + RARCH_LOG("[SDL audio]: Requested %u ms latency for output device, got %d ms\n", latency, (int)(out.samples * 4 * 1000 / (*new_rate))); /* Create a buffer twice as big as needed and prefill the buffer. */ - bufsize = out.samples * 4 * sizeof(int16_t); - tmp = calloc(1, bufsize); - sdl->buffer = fifo_new(bufsize); + bufsize = out.samples * 4 * sizeof(int16_t); + tmp = calloc(1, bufsize); + sdl->speaker_buffer = fifo_new(bufsize); if (tmp) { - fifo_write(sdl->buffer, tmp, bufsize); + fifo_write(sdl->speaker_buffer, tmp, bufsize); free(tmp); } - SDL_PauseAudio(0); + RARCH_DBG("[SDL audio]: Initialized speaker sample queue with %u bytes\n", bufsize); + + SDL_PauseAudioDevice(sdl->speaker_device, false); + return sdl; error: @@ -151,41 +221,49 @@ static ssize_t sdl_audio_write(void *data, const void *buf, size_t size) sdl_audio_t *sdl = (sdl_audio_t*)data; if (sdl->nonblock) - { + { /* If we shouldn't wait for space in a full outgoing sample queue... */ size_t avail, write_amt; - SDL_LockAudio(); - avail = FIFO_WRITE_AVAIL(sdl->buffer); - write_amt = avail > size ? size : avail; - fifo_write(sdl->buffer, buf, write_amt); - SDL_UnlockAudio(); - ret = write_amt; + SDL_LockAudioDevice(sdl->speaker_device); /* Stop the SDL speaker thread from running */ + avail = FIFO_WRITE_AVAIL(sdl->speaker_buffer); + write_amt = avail > size ? size : avail; /* Enqueue as much data as we can */ + fifo_write(sdl->speaker_buffer, buf, write_amt); + SDL_UnlockAudioDevice(sdl->speaker_device); /* Let the speaker thread run again */ + ret = write_amt; /* If the queue was full...well, too bad. */ } else { size_t written = 0; while (written < size) - { + { /* Until we've written all the sample data we have available... */ size_t avail; - SDL_LockAudio(); - avail = FIFO_WRITE_AVAIL(sdl->buffer); + SDL_LockAudioDevice(sdl->speaker_device); /* Stop the SDL speaker thread from running */ + avail = FIFO_WRITE_AVAIL(sdl->speaker_buffer); if (avail == 0) - { - SDL_UnlockAudio(); + { /* If the outgoing sample queue is full... */ + SDL_UnlockAudioDevice(sdl->speaker_device); + /* Let the SDL speaker thread run so it can play the enqueued samples, + * which will free up space for us to write new ones. */ #ifdef HAVE_THREADS slock_lock(sdl->lock); + /* Let *only* the SDL speaker thread touch the outgoing sample queue */ + scond_wait(sdl->cond, sdl->lock); + /* Block until SDL tells us that it's made room for new samples */ + slock_unlock(sdl->lock); + /* Now let this thread use the outgoing sample queue (which we'll do next iteration) */ #endif } else { size_t write_amt = size - written > avail ? avail : size - written; - fifo_write(sdl->buffer, (const char*)buf + written, write_amt); - SDL_UnlockAudio(); + fifo_write(sdl->speaker_buffer, (const char*)buf + written, write_amt); + /* Enqueue as many samples as we have available without overflowing the queue */ + SDL_UnlockAudioDevice(sdl->speaker_device); /* Let the SDL speaker thread run again */ written += write_amt; } } @@ -199,7 +277,8 @@ static bool sdl_audio_stop(void *data) { sdl_audio_t *sdl = (sdl_audio_t*)data; sdl->is_paused = true; - SDL_PauseAudio(1); + SDL_PauseAudioDevice(sdl->speaker_device, true); + return true; } @@ -216,7 +295,8 @@ static bool sdl_audio_start(void *data, bool is_shutdown) sdl_audio_t *sdl = (sdl_audio_t*)data; sdl->is_paused = false; - SDL_PauseAudio(0); + SDL_PauseAudioDevice(sdl->speaker_device, false); + return true; } @@ -231,15 +311,24 @@ static void sdl_audio_free(void *data) { sdl_audio_t *sdl = (sdl_audio_t*)data; - SDL_CloseAudio(); - if (sdl) { - fifo_free(sdl->buffer); + if (sdl->speaker_device > 0) + { + SDL_CloseAudioDevice(sdl->speaker_device); + } + + if (sdl->speaker_buffer) + { + fifo_free(sdl->speaker_buffer); + } + #ifdef HAVE_THREADS slock_free(sdl->lock); scond_free(sdl->cond); #endif + + SDL_QuitSubSystem(SDL_INIT_AUDIO); } free(sdl); } diff --git a/audio/drivers/wasapi.c b/audio/drivers/wasapi.c index 981113a2e6..3634fdf280 100644 --- a/audio/drivers/wasapi.c +++ b/audio/drivers/wasapi.c @@ -21,6 +21,7 @@ #include "../common/mmdevice_common.h" #include "../common/mmdevice_common_inline.h" +#include "../common/wasapi.h" #include "../audio_driver.h" #include "../../verbosity.h" @@ -43,439 +44,8 @@ typedef struct bool running; } wasapi_t; -static IMMDevice *wasapi_init_device(const char *id) -{ - HRESULT hr; - UINT32 dev_count, i; - IMMDeviceEnumerator *enumerator = NULL; - IMMDevice *device = NULL; - IMMDeviceCollection *collection = NULL; - - if (id) - RARCH_LOG("[WASAPI]: Initializing device: \"%s\"..\n", id); - else - RARCH_LOG("[WASAPI]: Initializing default device..\n"); - -#ifdef __cplusplus - hr = CoCreateInstance(CLSID_MMDeviceEnumerator, NULL, CLSCTX_ALL, - IID_IMMDeviceEnumerator, (void **)&enumerator); -#else - hr = CoCreateInstance(&CLSID_MMDeviceEnumerator, NULL, CLSCTX_ALL, - &IID_IMMDeviceEnumerator, (void **)&enumerator); -#endif - if (FAILED(hr)) - goto error; - - if (id) - { - int32_t idx_found = -1; - struct string_list *list = (struct string_list*)mmdevice_list_new(NULL); - - /* Search for device name first */ - if (list) - { - if (list->elems) - { - unsigned i; - for (i = 0; i < list->size; i++) - { - if (string_is_equal(id, list->elems[i].data)) - { - RARCH_DBG("[WASAPI]: Found device #%d: \"%s\"\n", i, list->elems[i].data); - idx_found = i; - break; - } - } - - /* Index was not found yet based on name string, - * just assume id is a one-character number index. */ - if (idx_found == -1 && isdigit(id[0])) - { - idx_found = strtoul(id, NULL, 0); - RARCH_LOG("[WASAPI]: Fallback, device index is a single number index instead: %d.\n", idx_found); - - } - } - string_list_free(list); - } - - if (idx_found == -1) - idx_found = 0; - - hr = _IMMDeviceEnumerator_EnumAudioEndpoints(enumerator, - eRender, DEVICE_STATE_ACTIVE, &collection); - if (FAILED(hr)) - goto error; - - hr = _IMMDeviceCollection_GetCount(collection, &dev_count); - if (FAILED(hr)) - goto error; - - for (i = 0; i < dev_count; ++i) - { - hr = _IMMDeviceCollection_Item(collection, i, &device); - if (FAILED(hr)) - goto error; - - if (i == idx_found) - break; - - IFACE_RELEASE(device); - } - } - else - { - hr = _IMMDeviceEnumerator_GetDefaultAudioEndpoint( - enumerator, eRender, eConsole, &device); - if (FAILED(hr)) - goto error; - } - - if (!device) - goto error; - - IFACE_RELEASE(collection); - IFACE_RELEASE(enumerator); - - return device; - -error: - IFACE_RELEASE(collection); - IFACE_RELEASE(enumerator); - - if (id) - RARCH_ERR("[WASAPI]: Failed to initialize device: \"%s\".\n", id); - else - RARCH_ERR("[WASAPI]: Failed to initialize default device.\n"); - - return NULL; -} - -static unsigned wasapi_pref_rate(unsigned i) -{ - const unsigned r[] = { 48000, 44100, 96000, 192000, 32000 }; - - if (i >= sizeof(r) / sizeof(unsigned)) - return 0; - - return r[i]; -} - -static void wasapi_set_format(WAVEFORMATEXTENSIBLE *wf, - bool float_fmt, unsigned rate) -{ - wf->Format.nChannels = 2; - wf->Format.nSamplesPerSec = rate; - - if (float_fmt) - { - wf->Format.wFormatTag = WAVE_FORMAT_EXTENSIBLE; - wf->Format.nAvgBytesPerSec = rate * 8; - wf->Format.nBlockAlign = 8; - wf->Format.wBitsPerSample = 32; - wf->Format.cbSize = sizeof(WORD) + sizeof(DWORD) + sizeof(GUID); - wf->Samples.wValidBitsPerSample = 32; - wf->dwChannelMask = KSAUDIO_SPEAKER_STEREO; - wf->SubFormat = KSDATAFORMAT_SUBTYPE_IEEE_FLOAT; - } - else - { - wf->Format.wFormatTag = WAVE_FORMAT_PCM; - wf->Format.nAvgBytesPerSec = rate * 4; - wf->Format.nBlockAlign = 4; - wf->Format.wBitsPerSample = 16; - wf->Format.cbSize = 0; - } -} - -static IAudioClient *wasapi_init_client_sh(IMMDevice *device, - bool *float_fmt, unsigned *rate, unsigned latency) -{ - WAVEFORMATEXTENSIBLE wf; - int i, j; - IAudioClient *client = NULL; - bool float_fmt_res = *float_fmt; - unsigned rate_res = *rate; - HRESULT hr = _IMMDevice_Activate(device, - IID_IAudioClient, - CLSCTX_ALL, NULL, (void**)&client); - if (FAILED(hr)) - return NULL; - - /* once for float, once for pcm (requested first) */ - for (i = 0; i < 2; ++i) - { - rate_res = *rate; - if (i == 1) - float_fmt_res = !float_fmt_res; - - /* for requested rate (first) and all preferred rates */ - for (j = 0; rate_res; ++j) - { - RARCH_LOG("[WASAPI]: Initializing client (shared, %s, %uHz, %.1fms)..\n", - float_fmt_res ? "float" : "pcm", rate_res, (float)latency); - - wasapi_set_format(&wf, float_fmt_res, rate_res); -#ifdef __cplusplus - hr = client->Initialize(AUDCLNT_SHAREMODE_SHARED, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK, - 0, 0, (WAVEFORMATEX*)&wf, NULL); -#else - hr = client->lpVtbl->Initialize(client, AUDCLNT_SHAREMODE_SHARED, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK, - 0, 0, (WAVEFORMATEX*)&wf, NULL); -#endif - if (hr == AUDCLNT_E_ALREADY_INITIALIZED) - { - HRESULT hr; - IFACE_RELEASE(client); - hr = _IMMDevice_Activate(device, - IID_IAudioClient, - CLSCTX_ALL, NULL, (void**)&client); - if (FAILED(hr)) - return NULL; - -#ifdef __cplusplus - hr = client->Initialize(AUDCLNT_SHAREMODE_SHARED, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK, - 0, 0, (WAVEFORMATEX*)&wf, NULL); -#else - hr = client->lpVtbl->Initialize(client, AUDCLNT_SHAREMODE_SHARED, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK, - 0, 0, (WAVEFORMATEX*)&wf, NULL); -#endif - } - if (hr != AUDCLNT_E_UNSUPPORTED_FORMAT) - { - i = 2; /* break from outer loop too */ - break; - } - - RARCH_WARN("[WASAPI]: Unsupported format.\n"); - rate_res = wasapi_pref_rate(j); - if (rate_res == *rate) /* requested rate is already tested */ - rate_res = wasapi_pref_rate(++j); /* skip it */ - } - } - - if (FAILED(hr)) - goto error; - - *float_fmt = float_fmt_res; - *rate = rate_res; - - return client; - -error: - IFACE_RELEASE(client); - - return NULL; -} - -static IAudioClient *wasapi_init_client_ex(IMMDevice *device, - bool *float_fmt, unsigned *rate, unsigned latency) -{ - WAVEFORMATEXTENSIBLE wf; - int i, j; - IAudioClient *client = NULL; - bool float_fmt_res = *float_fmt; - unsigned rate_res = *rate; - REFERENCE_TIME minimum_period = 0; - REFERENCE_TIME buffer_duration = 0; - UINT32 buffer_length = 0; - HRESULT hr = _IMMDevice_Activate(device, - IID_IAudioClient, - CLSCTX_ALL, NULL, (void**)&client); - if (FAILED(hr)) - return NULL; - - hr = _IAudioClient_GetDevicePeriod(client, NULL, &minimum_period); - if (FAILED(hr)) - goto error; - - /* buffer_duration is in 100ns units */ - buffer_duration = latency * 10000.0; - if (buffer_duration < minimum_period) - buffer_duration = minimum_period; - - /* once for float, once for pcm (requested first) */ - for (i = 0; i < 2; ++i) - { - rate_res = *rate; - if (i == 1) - float_fmt_res = !float_fmt_res; - - /* for requested rate (first) and all preferred rates */ - for (j = 0; rate_res; ++j) - { - RARCH_LOG("[WASAPI]: Initializing client (exclusive, %s, %uHz, %.1fms)..\n", - float_fmt_res ? "float" : "pcm", rate_res, (float)latency); - - wasapi_set_format(&wf, float_fmt_res, rate_res); -#ifdef __cplusplus - hr = client->Initialize(AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#else - hr = client->lpVtbl->Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#endif - if (hr == AUDCLNT_E_BUFFER_SIZE_NOT_ALIGNED) - { - hr = _IAudioClient_GetBufferSize(client, &buffer_length); - if (FAILED(hr)) - goto error; - - IFACE_RELEASE(client); - hr = _IMMDevice_Activate(device, - IID_IAudioClient, - CLSCTX_ALL, NULL, (void**)&client); - if (FAILED(hr)) - return NULL; - - buffer_duration = 10000.0 * 1000.0 / rate_res * buffer_length + 0.5; - -#ifdef __cplusplus - hr = client->Initialize(AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#else - hr = client->lpVtbl->Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#endif - } - if (hr == AUDCLNT_E_ALREADY_INITIALIZED) - { - IFACE_RELEASE(client); - hr = _IMMDevice_Activate(device, - IID_IAudioClient, - CLSCTX_ALL, NULL, (void**)&client); - if (FAILED(hr)) - return NULL; - -#ifdef __cplusplus - hr = client->Initialize(AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#else - hr = client->lpVtbl->Initialize(client, AUDCLNT_SHAREMODE_EXCLUSIVE, - AUDCLNT_STREAMFLAGS_EVENTCALLBACK | AUDCLNT_STREAMFLAGS_NOPERSIST, - buffer_duration, buffer_duration, (WAVEFORMATEX*)&wf, NULL); -#endif - } - if (hr != AUDCLNT_E_UNSUPPORTED_FORMAT) - { - if (hr == AUDCLNT_E_DEVICE_IN_USE) - goto error; - - if (hr == AUDCLNT_E_EXCLUSIVE_MODE_NOT_ALLOWED) - goto error; - - i = 2; /* break from outer loop too */ - break; - } - - RARCH_WARN("[WASAPI]: Unsupported format.\n"); - rate_res = wasapi_pref_rate(j); - if (rate_res == *rate) /* requested rate is already tested */ - rate_res = wasapi_pref_rate(++j); /* skip it */ - } - } - - if (FAILED(hr)) - goto error; - - *float_fmt = float_fmt_res; - *rate = rate_res; - - return client; - -error: - IFACE_RELEASE(client); - - return NULL; -} - -static IAudioClient *wasapi_init_client(IMMDevice *device, bool *exclusive, - bool *float_fmt, unsigned *rate, unsigned latency) -{ - HRESULT hr; - IAudioClient *client; - double latency_res; - REFERENCE_TIME device_period = 0; - REFERENCE_TIME stream_latency = 0; - UINT32 buffer_length = 0; - - if (*exclusive) - { - client = wasapi_init_client_ex(device, float_fmt, rate, latency); - if (!client) - { - client = wasapi_init_client_sh(device, float_fmt, rate, latency); - if (client) - *exclusive = false; - } - } - else - { - client = wasapi_init_client_sh(device, float_fmt, rate, latency); - if (!client) - { - client = wasapi_init_client_ex(device, float_fmt, rate, latency); - if (client) - *exclusive = true; - } - } - - if (!client) - return NULL; - - /* next calls are allowed to fail (we losing info only) */ - - if (*exclusive) - hr = _IAudioClient_GetDevicePeriod(client, NULL, &device_period); - else - hr = _IAudioClient_GetDevicePeriod(client, &device_period, NULL); - - if (FAILED(hr)) - RARCH_WARN("[WASAPI]: IAudioClient::GetDevicePeriod failed with error 0x%.8X.\n", hr); - - if (!*exclusive) - { - hr = _IAudioClient_GetStreamLatency(client, &stream_latency); - if (FAILED(hr)) - RARCH_WARN("[WASAPI]: IAudioClient::GetStreamLatency failed with error 0x%.8X.\n", hr); - } - - hr = _IAudioClient_GetBufferSize(client, &buffer_length); - if (FAILED(hr)) - RARCH_WARN("[WASAPI]: IAudioClient::GetBufferSize failed with error 0x%.8X.\n", hr); - - if (*exclusive) - latency_res = (double)buffer_length * 1000.0 / (*rate); - else - latency_res = (double)(stream_latency + device_period) / 10000.0; - - RARCH_LOG("[WASAPI]: Client initialized (%s, %s, %uHz, %.1fms).\n", - *exclusive ? "exclusive" : "shared", - *float_fmt ? "float" : "pcm", - *rate, latency_res); - - RARCH_LOG("[WASAPI]: Client buffer length is %u frames (%.1fms).\n", - buffer_length, - (double)buffer_length * 1000.0 / (*rate)); - - RARCH_LOG("[WASAPI]: Device period is %.1fms (%lld frames).\n", - (double)device_period / 10000.0, - device_period * (*rate) / 10000000); - - return client; -} - static void *wasapi_init(const char *dev_id, unsigned rate, unsigned latency, - unsigned u1, unsigned *u2) + unsigned u1, unsigned *new_rate) { HRESULT hr; UINT32 frame_count = 0; @@ -491,14 +61,14 @@ static void *wasapi_init(const char *dev_id, unsigned rate, unsigned latency, return NULL; w->exclusive = exclusive_mode; - w->device = wasapi_init_device(dev_id); + w->device = wasapi_init_device(dev_id, eRender); if (!w->device && dev_id) - w->device = wasapi_init_device(NULL); + w->device = wasapi_init_device(NULL, eRender); if (!w->device) goto error; w->client = wasapi_init_client(w->device, - &w->exclusive, &float_format, &rate, latency); + &w->exclusive, &float_format, &rate, latency, 2); if (!w->client) goto error; @@ -576,6 +146,9 @@ static void *wasapi_init(const char *dev_id, unsigned rate, unsigned latency, w->running = true; w->nonblock = !settings->bools.audio_sync; + if (new_rate) + *new_rate = rate; + return w; error: @@ -836,7 +409,8 @@ static void wasapi_set_nonblock_state(void *wh, bool nonblock) { wasapi_t *w = (wasapi_t*)wh; - RARCH_DBG("[WASAPI]: Sync %s.\n", nonblock ? "off" : "on"); + if (w->nonblock != nonblock) + RARCH_DBG("[WASAPI]: Sync %s.\n", nonblock ? "off" : "on"); w->nonblock = nonblock; } @@ -906,6 +480,11 @@ static size_t wasapi_buffer_size(void *wh) return w->engine_buffer_size; } +static void *wasapi_device_list_new(void *u) +{ + return mmdevice_list_new(u, eRender); +} + audio_driver_t audio_wasapi = { wasapi_init, wasapi_write, @@ -916,7 +495,7 @@ audio_driver_t audio_wasapi = { wasapi_free, wasapi_use_float, "wasapi", - mmdevice_list_new, + wasapi_device_list_new, wasapi_device_list_free, wasapi_write_avail, wasapi_buffer_size diff --git a/audio/drivers_microphone/alsa.c b/audio/drivers_microphone/alsa.c new file mode 100644 index 0000000000..85de4ebe0e --- /dev/null +++ b/audio/drivers_microphone/alsa.c @@ -0,0 +1,297 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include +#include +#include + +#include "audio/common/alsa.h" /* For some common functions/types */ +#include "audio/microphone_driver.h" +#include "verbosity.h" + + +#define BYTES_TO_FRAMES(bytes, frame_bits) ((bytes) * 8 / frame_bits) +#define FRAMES_TO_BYTES(frames, frame_bits) ((frames) * frame_bits / 8) + +typedef struct alsa_microphone_handle +{ + snd_pcm_t *pcm; + alsa_stream_info_t stream_info; +} alsa_microphone_handle_t; + +typedef struct alsa +{ + bool nonblock; +} alsa_microphone_t; + +static void *alsa_microphone_init(void) +{ + alsa_microphone_t *alsa = (alsa_microphone_t*)calloc(1, sizeof(alsa_microphone_t)); + + if (!alsa) + { + RARCH_ERR("[ALSA]: Failed to allocate driver context\n"); + return NULL; + } + + RARCH_LOG("[ALSA]: Using ALSA version %s\n", snd_asoundlib_version()); + + return alsa; +} + +static void alsa_microphone_close_mic(void *driver_context, void *microphone_context); +static void alsa_microphone_free(void *driver_context) +{ + alsa_microphone_t *alsa = (alsa_microphone_t*)driver_context; + /* The mic frontend should've closed all mics before calling free(). */ + + if (alsa) + { + snd_config_update_free_global(); + free(alsa); + } +} + +static bool alsa_microphone_start_mic(void *driver_context, void *microphone_context); +static int alsa_microphone_read(void *driver_context, void *microphone_context, void *buf_, size_t size_) +{ + alsa_microphone_t *alsa = (alsa_microphone_t*)driver_context; + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + uint8_t *buf = (uint8_t*)buf_; + snd_pcm_sframes_t read = 0; + int errnum = 0; + snd_pcm_sframes_t size; + size_t frames_size; + snd_pcm_state_t state; + + if (!alsa || !microphone || !buf) + return -1; + + size = BYTES_TO_FRAMES(size_, microphone->stream_info.frame_bits); + frames_size = microphone->stream_info.has_float ? sizeof(float) : sizeof(int16_t); + + state = snd_pcm_state(microphone->pcm); + if (state != SND_PCM_STATE_RUNNING) + { + RARCH_WARN("[ALSA]: Expected microphone \"%s\" to be in state RUNNING, was in state %s\n", + snd_pcm_name(microphone->pcm), + snd_pcm_state_name(state)); + + errnum = snd_pcm_start(microphone->pcm); + if (errnum < 0) + { + RARCH_ERR("[ALSA]: Failed to start microphone \"%s\": %s\n", + snd_pcm_name(microphone->pcm), + snd_strerror(errnum)); + + return -1; + } + } + + if (alsa->nonblock) + { + while (size) + { + snd_pcm_sframes_t frames = snd_pcm_readi(microphone->pcm, buf, size); + + if (frames == -EPIPE || frames == -EINTR || frames == -ESTRPIPE) + { + errnum = snd_pcm_recover(microphone->pcm, frames, 0); + if (errnum < 0) + { + RARCH_ERR("[ALSA]: Failed to read from microphone: %s\n", snd_strerror(frames)); + RARCH_ERR("[ALSA]: Additionally, recovery failed with: %s\n", snd_strerror(errnum)); + return -1; + } + + break; + } + else if (frames == -EAGAIN) + break; + else if (frames < 0) + return -1; + + read += frames; + buf += frames_size; + size -= frames; + } + } + else + { + bool eagain_retry = true; + + while (size) + { + snd_pcm_sframes_t frames; + int rc = snd_pcm_wait(microphone->pcm, -1); + + if (rc == -EPIPE || rc == -ESTRPIPE || rc == -EINTR) + { + if (snd_pcm_recover(microphone->pcm, rc, 1) < 0) + return -1; + continue; + } + + frames = snd_pcm_readi(microphone->pcm, buf, size); + + if (frames == -EPIPE || frames == -EINTR || frames == -ESTRPIPE) + { + if (snd_pcm_recover(microphone->pcm, frames, 1) < 0) + return -1; + + break; + } + else if (frames == -EAGAIN) + { + /* Definitely not supposed to happen. */ + if (eagain_retry) + { + eagain_retry = false; + continue; + } + break; + } + else if (frames < 0) + return -1; + + read += frames; + buf += frames_size; + size -= frames; + } + } + + return FRAMES_TO_BYTES(read, microphone->stream_info.frame_bits); +} + +static bool alsa_microphone_mic_alive(const void *driver_context, const void *microphone_context) +{ + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return snd_pcm_state(microphone->pcm) == SND_PCM_STATE_RUNNING; +} + +static void alsa_microphone_set_nonblock_state(void *driver_context, bool nonblock) +{ + alsa_microphone_t *alsa = (alsa_microphone_t*)driver_context; + alsa->nonblock = nonblock; +} + +static struct string_list *alsa_microphone_device_list_new(const void *data) +{ + (void)data; + return alsa_device_list_type_new("Input"); +} + +static void alsa_microphone_device_list_free(const void *driver_context, struct string_list *devices) +{ + (void)driver_context; + string_list_free(devices); + /* Does nothing if devices is NULL */ +} + +static void *alsa_microphone_open_mic(void *driver_context, + const char *device, + unsigned rate, + unsigned latency, + unsigned *new_rate) +{ + alsa_microphone_t *alsa = (alsa_microphone_t*)driver_context; + alsa_microphone_handle_t *microphone = NULL; + + if (!alsa) /* If we weren't given a valid ALSA context... */ + return NULL; + + microphone = calloc(1, sizeof(alsa_microphone_handle_t)); + + if (!microphone) /* If the microphone context couldn't be allocated... */ + return NULL; + + /* channels hardcoded to 1, because we only support mono mic input */ + if (alsa_init_pcm(µphone->pcm, device, SND_PCM_STREAM_CAPTURE, rate, latency, 1, µphone->stream_info, new_rate, SND_PCM_NONBLOCK) < 0) + { + goto error; + } + + return microphone; + +error: + RARCH_ERR("[ALSA]: Failed to initialize microphone...\n"); + + alsa_microphone_close_mic(alsa, microphone); + + return NULL; + +} +static void alsa_microphone_close_mic(void *driver_context, void *microphone_context) +{ + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (microphone) + { + alsa_free_pcm(microphone->pcm); + free(microphone); + } +} + +static bool alsa_microphone_start_mic(void *driver_context, void *microphone_context) +{ + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return alsa_start_pcm(microphone->pcm); +} + +static bool alsa_microphone_stop_mic(void *driver_context, void *microphone_context) +{ + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return alsa_stop_pcm(microphone->pcm); +} + +static bool alsa_microphone_mic_use_float(const void *driver_context, const void *microphone_context) +{ + alsa_microphone_handle_t *microphone = (alsa_microphone_handle_t*)microphone_context; + (void)driver_context; + + return microphone->stream_info.has_float; +} + +microphone_driver_t microphone_alsa = { + alsa_microphone_init, + alsa_microphone_free, + alsa_microphone_read, + alsa_microphone_set_nonblock_state, + "alsa", + alsa_microphone_device_list_new, + alsa_microphone_device_list_free, + alsa_microphone_open_mic, + alsa_microphone_close_mic, + alsa_microphone_mic_alive, + alsa_microphone_start_mic, + alsa_microphone_stop_mic, + alsa_microphone_mic_use_float +}; diff --git a/audio/drivers_microphone/alsathread.c b/audio/drivers_microphone/alsathread.c new file mode 100644 index 0000000000..772b5f1fd2 --- /dev/null +++ b/audio/drivers_microphone/alsathread.c @@ -0,0 +1,412 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include +#include +#include + +#include "audio/common/alsa.h" +#include "audio/common/alsathread.h" +#include "audio/microphone_driver.h" +#include "verbosity.h" +#include "retro_assert.h" + + +typedef struct alsa_thread_microphone_handle +{ + alsa_thread_info_t info; +} alsa_thread_microphone_handle_t; + +typedef struct alsa_thread +{ + bool nonblock; +} alsa_thread_microphone_t; + +static void *alsa_thread_microphone_init(void) +{ + alsa_thread_microphone_t *alsa = (alsa_thread_microphone_t*)calloc(1, sizeof(alsa_thread_microphone_t)); + + if (!alsa) + { + RARCH_ERR("[ALSA] Failed to allocate driver context\n"); + return NULL; + } + + RARCH_LOG("[ALSA] Using ALSA version %s\n", snd_asoundlib_version()); + + return alsa; +} + +static void alsa_thread_microphone_close_mic(void *driver_context, void *microphone_context); +static void alsa_thread_microphone_free(void *driver_context) +{ + alsa_thread_microphone_t *alsa = (alsa_thread_microphone_t*)driver_context; + + if (alsa) + { + free(alsa); + } +} + +/** @see alsa_thread_read_microphone() */ +static void alsa_microphone_worker_thread(void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + uint8_t *buf = NULL; + uintptr_t thread_id = sthread_get_current_thread_id(); + + retro_assert(microphone != NULL); + buf = (uint8_t *)calloc(1, microphone->info.stream_info.period_size); + if (!buf) + { + RARCH_ERR("[ALSA] [capture thread %p]: Failed to allocate audio buffer\n", thread_id); + goto end; + } + + RARCH_DBG("[ALSA] [capture thread %p]: Beginning microphone worker thread\n", thread_id); + RARCH_DBG("[ALSA] [capture thread %p]: Microphone \"%s\" is in state %s\n", + thread_id, + snd_pcm_name(microphone->info.pcm), + snd_pcm_state_name(snd_pcm_state(microphone->info.pcm))); + + while (!microphone->info.thread_dead) + { /* Until we're told to stop... */ + size_t avail; + size_t fifo_size; + snd_pcm_sframes_t frames; + int errnum = 0; + + /* Lock the incoming sample queue (the main thread may block) */ + slock_lock(microphone->info.fifo_lock); + + /* Fill the incoming sample queue with whatever we recently read */ + avail = FIFO_WRITE_AVAIL(microphone->info.buffer); + fifo_size = MIN(microphone->info.stream_info.period_size, avail); + fifo_write(microphone->info.buffer, buf, fifo_size); + + /* Tell the main thread that it's okay to query the mic again */ + scond_signal(microphone->info.cond); + + /* Unlock the incoming sample queue (the main thread may resume) */ + slock_unlock(microphone->info.fifo_lock); + + /* If underrun, fill rest with silence. */ + memset(buf + fifo_size, 0, microphone->info.stream_info.period_size - fifo_size); + + errnum = snd_pcm_wait(microphone->info.pcm, 33); + + if (errnum == 0) + { + RARCH_DBG("[ALSA] [capture thread %p]: Timeout after 33ms waiting for input\n", thread_id); + continue; + } + else if (errnum == -EPIPE || errnum == -ESTRPIPE || errnum == -EINTR) + { + RARCH_WARN("[ALSA] [capture thread %p]: Wait error: %s\n", + thread_id, + snd_strerror(errnum)); + + if ((errnum = snd_pcm_recover(microphone->info.pcm, errnum, false)) < 0) + { + RARCH_ERR("[ALSA] [capture thread %p]: Failed to recover from prior wait error: %s\n", + thread_id, + snd_strerror(errnum)); + + break; + } + + continue; + } + + frames = snd_pcm_readi(microphone->info.pcm, buf, microphone->info.stream_info.period_frames); + + if (frames == -EPIPE || frames == -EINTR || frames == -ESTRPIPE) + { + RARCH_WARN("[ALSA] [capture thread %p]: Read error: %s\n", + thread_id, + snd_strerror(frames)); + + if ((errnum = snd_pcm_recover(microphone->info.pcm, frames, false)) < 0) + { + RARCH_ERR("[ALSA] [capture thread %p]: Failed to recover from prior read error: %s\n", + thread_id, + snd_strerror(errnum)); + break; + } + + continue; + } + else if (frames < 0) + { + RARCH_ERR("[ALSA] [capture thread %p]: Read error: %s.\n", + thread_id, + snd_strerror(frames)); + break; + } + } + +end: + slock_lock(microphone->info.cond_lock); + microphone->info.thread_dead = true; + scond_signal(microphone->info.cond); + slock_unlock(microphone->info.cond_lock); + free(buf); + RARCH_DBG("[ALSA] [capture thread %p]: Ending microphone worker thread\n", thread_id); +} + +static int alsa_thread_microphone_read(void *driver_context, void *microphone_context, void *buf, size_t size) +{ + alsa_thread_microphone_t *alsa = (alsa_thread_microphone_t*)driver_context; + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + snd_pcm_state_t state; + + if (!alsa || !microphone || !buf) /* If any of the parameters were invalid... */ + return -1; + + if (microphone->info.thread_dead) /* If the mic thread is shutting down... */ + return -1; + + state = snd_pcm_state(microphone->info.pcm); + if (state != SND_PCM_STATE_RUNNING) + { + int errnum; + RARCH_WARN("[ALSA]: Expected microphone \"%s\" to be in state RUNNING, was in state %s\n", + snd_pcm_name(microphone->info.pcm), + snd_pcm_state_name(state)); + + errnum = snd_pcm_start(microphone->info.pcm); + if (errnum < 0) + { + RARCH_ERR("[ALSA]: Failed to start microphone \"%s\": %s\n", + snd_pcm_name(microphone->info.pcm), + snd_strerror(errnum)); + + return -1; + } + } + + if (alsa->nonblock) + { /* If driver interactions shouldn't block... */ + size_t avail; + size_t write_amt; + + /* "Hey, I'm gonna borrow the queue." */ + slock_lock(microphone->info.fifo_lock); + + avail = FIFO_READ_AVAIL(microphone->info.buffer); + write_amt = MIN(avail, size); + + /* "It's okay if you don't have any new samples, I'll just check in on you later." */ + fifo_read(microphone->info.buffer, buf, write_amt); + + /* "Here, take this queue back." */ + slock_unlock(microphone->info.fifo_lock); + + return (int)write_amt; + } + else + { + size_t read = 0; + while (read < size && !microphone->info.thread_dead) + { /* Until we've read all requested samples (or we're told to stop)... */ + size_t avail; + + /* "Hey, I'm gonna borrow the queue." */ + slock_lock(microphone->info.fifo_lock); + + avail = FIFO_READ_AVAIL(microphone->info.buffer); + + if (avail == 0) + { /* "Oh, wait, it's empty." */ + + /* "Here, take it back..." */ + slock_unlock(microphone->info.fifo_lock); + + /* "...I'll just wait right here." */ + slock_lock(microphone->info.cond_lock); + + /* "Unless we're closing up shop..." */ + if (!microphone->info.thread_dead) + /* "...let me know when you've produced some samples." */ + scond_wait(microphone->info.cond, microphone->info.cond_lock); + + /* "Oh, you're ready? Okay, I'm gonna continue." */ + slock_unlock(microphone->info.cond_lock); + } + else + { + size_t read_amt = MIN(size - read, avail); + + /* "I'll just go ahead and consume all these samples..." + * (As many as will fit in buf, or as many as are available.) */ + fifo_read(microphone->info.buffer,buf + read, read_amt); + + /* "I'm done, you can take the queue back now." */ + slock_unlock(microphone->info.fifo_lock); + read += read_amt; + } + + /* "I'll be right back..." */ + } + return (int)read; + } +} + +static bool alsa_thread_microphone_mic_alive(const void *driver_context, const void *microphone_context); + +static void *alsa_thread_microphone_open_mic(void *driver_context, + const char *device, + unsigned rate, + unsigned latency, + unsigned *new_rate) +{ + alsa_thread_microphone_t *alsa = (alsa_thread_microphone_t*)driver_context; + alsa_thread_microphone_handle_t *microphone = NULL; + + if (!alsa) /* If we weren't given a valid ALSA context... */ + return NULL; + + microphone = calloc(1, sizeof(alsa_thread_microphone_handle_t)); + + if (!microphone) + { /* If the microphone context couldn't be allocated... */ + RARCH_ERR("[ALSA] Failed to allocate microphone context\n"); + return NULL; + } + + if (alsa_init_pcm(µphone->info.pcm, device, SND_PCM_STREAM_CAPTURE, rate, latency, 1, µphone->info.stream_info, new_rate, 0) < 0) + { + goto error; + } + + microphone->info.fifo_lock = slock_new(); + microphone->info.cond_lock = slock_new(); + microphone->info.cond = scond_new(); + microphone->info.buffer = fifo_new(microphone->info.stream_info.buffer_size); + if (!microphone->info.fifo_lock || !microphone->info.cond_lock || !microphone->info.cond || !microphone->info.buffer || !microphone->info.pcm) + goto error; + + microphone->info.worker_thread = sthread_create(alsa_microphone_worker_thread, microphone); + if (!microphone->info.worker_thread) + { + RARCH_ERR("[ALSA]: Failed to initialize microphone worker thread\n"); + goto error; + } + RARCH_DBG("[ALSA]: Initialized microphone worker thread\n"); + + return microphone; + +error: + RARCH_ERR("[ALSA]: Failed to initialize microphone...\n"); + + if (microphone) + { + if (microphone->info.pcm) + { + snd_pcm_close(microphone->info.pcm); + } + + alsa_thread_microphone_close_mic(alsa, microphone); + } + + return NULL; +} + +static void alsa_thread_microphone_close_mic(void *driver_context, void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (microphone) + { + alsa_thread_free_info_members(µphone->info); + free(microphone); + } +} + +static bool alsa_thread_microphone_mic_alive(const void *driver_context, const void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t *)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return snd_pcm_state(microphone->info.pcm) == SND_PCM_STATE_RUNNING; +} + +static void alsa_thread_microphone_set_nonblock_state(void *driver_context, bool state) +{ + alsa_thread_microphone_t *alsa = (alsa_thread_microphone_t*)driver_context; + alsa->nonblock = state; +} + +static struct string_list *alsa_thread_microphone_device_list_new(const void *data) +{ + (void)data; + return alsa_device_list_type_new("Input"); +} + +static void alsa_thread_microphone_device_list_free(const void *driver_context, struct string_list *devices) +{ + (void)driver_context; + string_list_free(devices); + /* Does nothing if devices is NULL */ +} + +static bool alsa_thread_microphone_start_mic(void *driver_context, void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return alsa_start_pcm(microphone->info.pcm); +} + +static bool alsa_thread_microphone_stop_mic(void *driver_context, void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return alsa_stop_pcm(microphone->info.pcm); +} + +static bool alsa_thread_microphone_mic_use_float(const void *driver_context, const void *microphone_context) +{ + alsa_thread_microphone_handle_t *microphone = (alsa_thread_microphone_handle_t*)microphone_context; + + return microphone->info.stream_info.has_float; +} + +microphone_driver_t microphone_alsathread = { + alsa_thread_microphone_init, + alsa_thread_microphone_free, + alsa_thread_microphone_read, + alsa_thread_microphone_set_nonblock_state, + "alsathread", + alsa_thread_microphone_device_list_new, + alsa_thread_microphone_device_list_free, + alsa_thread_microphone_open_mic, + alsa_thread_microphone_close_mic, + alsa_thread_microphone_mic_alive, + alsa_thread_microphone_start_mic, + alsa_thread_microphone_stop_mic, + alsa_thread_microphone_mic_use_float +}; diff --git a/audio/drivers_microphone/sdl_microphone.c b/audio/drivers_microphone/sdl_microphone.c new file mode 100644 index 0000000000..df163664ff --- /dev/null +++ b/audio/drivers_microphone/sdl_microphone.c @@ -0,0 +1,400 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include "SDL.h" +#include "SDL_audio.h" +#include "verbosity.h" +#include "retro_assert.h" +#include "retro_math.h" +#include "audio/microphone_driver.h" +#include +#include + +typedef struct sdl_microphone_handle +{ +#ifdef HAVE_THREADS + slock_t *lock; + scond_t *cond; +#endif + + /** + * The queue used to store incoming samples from the driver. + */ + fifo_buffer_t *sample_buffer; + SDL_AudioDeviceID device_id; + SDL_AudioSpec device_spec; +} sdl_microphone_handle_t; + +typedef struct sdl_microphone +{ + bool nonblock; +} sdl_microphone_t; + +static INLINE int find_num_frames(int rate, int latency) +{ + int frames = (rate * latency) / 1000; + + /* SDL only likes 2^n sized buffers. */ + + return next_pow2(frames); +} + +static void *sdl_microphone_init(void) +{ + sdl_microphone_t *sdl = NULL; + uint32_t sdl_subsystem_flags = SDL_WasInit(0); + + /* Initialise audio subsystem, if required */ + if (sdl_subsystem_flags == 0) + { + if (SDL_Init(SDL_INIT_AUDIO) < 0) + return NULL; + } + else if ((sdl_subsystem_flags & SDL_INIT_AUDIO) == 0) + { + if (SDL_InitSubSystem(SDL_INIT_AUDIO) < 0) + return NULL; + } + + sdl = (sdl_microphone_t*)calloc(1, sizeof(*sdl)); + if (!sdl) + return NULL; + + return sdl; + +error: + free(sdl); + return NULL; +} + +static void sdl_microphone_close_mic(void *driver_context, void *microphone_context); + +static void sdl_microphone_free(void *data) +{ + sdl_microphone_t *sdl = (sdl_microphone_t*)data; + + if (sdl) + { + SDL_QuitSubSystem(SDL_INIT_AUDIO); + } + free(sdl); + /* NOTE: The microphone frontend should've closed the mics by now */ +} + +static void sdl_audio_record_cb(void *data, Uint8 *stream, int len) +{ + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t*)data; + size_t avail = FIFO_WRITE_AVAIL(microphone->sample_buffer); + size_t read_size = MIN(len, (int)avail); + /* If the sample buffer is almost full, just write as much as we can into it*/ + + fifo_write(microphone->sample_buffer, stream, read_size); +#ifdef HAVE_THREADS + scond_signal(microphone->cond); +#endif +} + +static void *sdl_microphone_open_mic(void *driver_context, + const char *device, + unsigned rate, + unsigned latency, + unsigned *new_rate) +{ + int frames; + size_t bufsize; + sdl_microphone_handle_t *microphone = NULL; + SDL_AudioSpec desired_spec = {0}; + void *tmp = NULL; + + (void)driver_context; + + if (!SDL_WasInit(SDL_INIT_AUDIO)) + { /* If the audio driver wasn't initialized yet... */ + RARCH_ERR("[SDL mic]: Attempted to initialize input device before initializing the audio subsystem\n"); + return NULL; + } + + microphone = (sdl_microphone_handle_t *)calloc(1, sizeof(sdl_microphone_handle_t)); + if (!microphone) + return NULL; + + if (verbosity_is_enabled()) + { /* Only print SDL audio devices if verbose logging is enabled */ + int i; + int num_available_microphones = SDL_GetNumAudioDevices(true); + RARCH_DBG("[SDL mic]: %d audio capture devices found:\n", num_available_microphones); + for (i = 0; i < num_available_microphones; ++i) { + RARCH_DBG("[SDL mic]: - %s\n", SDL_GetAudioDeviceName(i, true)); + } + } + + /* We have to buffer up some data ourselves, so we let SDL + * carry approximately half of the latency. + * + * SDL double buffers audio and we do as well. */ + frames = find_num_frames(rate, latency / 4); + + desired_spec.freq = rate; + desired_spec.format = AUDIO_F32SYS; + desired_spec.channels = 1; /* Microphones only usually provide input in mono */ + desired_spec.samples = frames; + desired_spec.userdata = microphone; + desired_spec.callback = sdl_audio_record_cb; + + microphone->device_id = SDL_OpenAudioDevice( + NULL, + true, + &desired_spec, + µphone->device_spec, + SDL_AUDIO_ALLOW_FREQUENCY_CHANGE | SDL_AUDIO_ALLOW_FORMAT_CHANGE); + + if (microphone->device_id == 0) + { + RARCH_ERR("[SDL mic]: Failed to open SDL audio input device: %s\n", SDL_GetError()); + goto error; + } + RARCH_DBG("[SDL mic]: Opened SDL audio input device with ID %u\n", + microphone->device_id); + RARCH_DBG("[SDL mic]: Requested a microphone frequency of %u Hz, got %u Hz\n", + desired_spec.freq, microphone->device_spec.freq); + RARCH_DBG("[SDL mic]: Requested %u channels for microphone, got %u\n", + desired_spec.channels, microphone->device_spec.channels); + RARCH_DBG("[SDL mic]: Requested a %u-sample microphone buffer, got %u samples (%u bytes)\n", + frames, microphone->device_spec.samples, microphone->device_spec.size); + RARCH_DBG("[SDL mic]: Got a microphone silence value of %u\n", microphone->device_spec.silence); + RARCH_DBG("[SDL mic]: Requested microphone audio format: %u-bit %s %s %s endian\n", + SDL_AUDIO_BITSIZE(desired_spec.format), + SDL_AUDIO_ISSIGNED(desired_spec.format) ? "signed" : "unsigned", + SDL_AUDIO_ISFLOAT(desired_spec.format) ? "floating-point" : "integer", + SDL_AUDIO_ISBIGENDIAN(desired_spec.format) ? "big" : "little"); + + RARCH_DBG("[SDL mic]: Received microphone audio format: %u-bit %s %s %s endian\n", + SDL_AUDIO_BITSIZE(desired_spec.format), + SDL_AUDIO_ISSIGNED(desired_spec.format) ? "signed" : "unsigned", + SDL_AUDIO_ISFLOAT(desired_spec.format) ? "floating-point" : "integer", + SDL_AUDIO_ISBIGENDIAN(desired_spec.format) ? "big" : "little"); + + if (new_rate) + *new_rate = microphone->device_spec.freq; + +#ifdef HAVE_THREADS + microphone->lock = slock_new(); + microphone->cond = scond_new(); +#endif + + RARCH_LOG("[SDL audio]: Requested %u ms latency for input device, got %d ms\n", + latency, (int)(microphone->device_spec.samples * 4 * 1000 / microphone->device_spec.freq)); + + /* Create a buffer twice as big as needed and prefill the buffer. */ + bufsize = microphone->device_spec.samples * 2 * (SDL_AUDIO_BITSIZE(microphone->device_spec.format) / 8); + tmp = calloc(1, bufsize); + microphone->sample_buffer = fifo_new(bufsize); + + RARCH_DBG("[SDL audio]: Initialized microphone sample queue with %u bytes\n", bufsize); + + if (tmp) + { + fifo_write(microphone->sample_buffer, tmp, bufsize); + free(tmp); + } + + RARCH_LOG("[SDL audio]: Initialized microphone with device ID %u\n", microphone->device_id); + return microphone; + +error: + free(microphone); + return NULL; +} + +static void sdl_microphone_close_mic(void *driver_context, void *microphone_context) +{ + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t *)microphone_context; + (void)driver_context; + + if (microphone) + { + if (microphone->device_id > 0) + { /* If the microphone was originally initialized successfully... */ + SDL_CloseAudioDevice(microphone->device_id); + } + + fifo_free(microphone->sample_buffer); + +#ifdef HAVE_THREADS + slock_free(microphone->lock); + scond_free(microphone->cond); +#endif + + RARCH_LOG("[SDL audio]: Freed microphone with former device ID %u\n", microphone->device_id); + free(microphone); + } +} + +static bool sdl_microphone_mic_alive(const void *data, const void *microphone_context) +{ + const sdl_microphone_handle_t *microphone = (const sdl_microphone_handle_t*)microphone_context; + (void)data; + + if (!microphone) + return false; + /* Both params must be non-null */ + + return SDL_GetAudioDeviceStatus(microphone->device_id) == SDL_AUDIO_PLAYING; +} + +static bool sdl_microphone_start_mic(void *driver_context, void *microphone_context) +{ + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t*)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + SDL_PauseAudioDevice(microphone->device_id, false); + + if (SDL_GetAudioDeviceStatus(microphone->device_id) != SDL_AUDIO_PLAYING) + { + RARCH_ERR("[SDL mic]: Failed to start microphone %u: %s\n", microphone->device_id, SDL_GetError()); + return false; + } + + RARCH_DBG("[SDL mic]: Started microphone %u\n", microphone->device_id); + return true; +} + +static bool sdl_microphone_stop_mic(void *driver_context, void *microphone_context) +{ + sdl_microphone_t *sdl = (sdl_microphone_t*)driver_context; + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t*)microphone_context; + + if (!sdl || !microphone) + return false; + + SDL_PauseAudioDevice(microphone->device_id, true); + + switch (SDL_GetAudioDeviceStatus(microphone->device_id)) + { + case SDL_AUDIO_PAUSED: + return true; + case SDL_AUDIO_PLAYING: + RARCH_ERR("[SDL mic]: Microphone %u failed to pause\n", microphone->device_id); + return false; + case SDL_AUDIO_STOPPED: + RARCH_WARN("[SDL mic]: Microphone %u is in state STOPPED; it may not start again\n", microphone->device_id); + return true; + default: + RARCH_ERR("[SDL mic]: Microphone %u is in unknown state\n", microphone->device_id); + return false; + } +} + +static void sdl_microphone_set_nonblock_state(void *driver_context, bool state) +{ + sdl_microphone_t *sdl = (sdl_microphone_t*)driver_context; + if (sdl) + sdl->nonblock = state; +} + +static int sdl_microphone_read(void *driver_context, void *microphone_context, void *buf, size_t size) +{ + int ret = 0; + sdl_microphone_t *sdl = (sdl_microphone_t*)driver_context; + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t*)microphone_context; + + if (!sdl || !microphone || !buf) + return -1; + + if (sdl->nonblock) + { /* If we shouldn't block on an empty queue... */ + size_t avail, read_amt; + + SDL_LockAudioDevice(microphone->device_id); /* Stop the SDL mic thread */ + avail = FIFO_READ_AVAIL(microphone->sample_buffer); + read_amt = avail > size ? size : avail; + if (read_amt > 0) + { /* If the incoming queue isn't empty... */ + fifo_read(microphone->sample_buffer, buf, read_amt); + /* ...then read as much data as will fit in buf */ + } + SDL_UnlockAudioDevice(microphone->device_id); /* Let the mic thread run again */ + ret = (int)read_amt; + } + else + { + size_t read = 0; + + while (read < size) + { /* Until we've given the caller as much data as they've asked for... */ + size_t avail; + + SDL_LockAudioDevice(microphone->device_id); + /* Stop the SDL microphone thread from running */ + avail = FIFO_READ_AVAIL(microphone->sample_buffer); + + if (avail == 0) + { /* If the incoming sample queue is empty... */ + SDL_UnlockAudioDevice(microphone->device_id); + /* Let the SDL microphone thread run so it can push some incoming samples */ +#ifdef HAVE_THREADS + slock_lock(microphone->lock); + /* Let *only* the SDL microphone thread access the incoming sample queue. */ + + scond_wait(microphone->cond, microphone->lock); + /* Wait until the SDL microphone thread tells us it's added some samples. */ + + slock_unlock(microphone->lock); + /* Allow this thread to access the incoming sample queue, which we'll do next iteration */ +#endif + } + else + { + size_t read_amt = MIN(size - read, avail); + fifo_read(microphone->sample_buffer, buf + read, read_amt); + /* Read as many samples as we have available without underflowing the queue */ + + SDL_UnlockAudioDevice(microphone->device_id); + /* Let the SDL microphone thread run again */ + read += read_amt; + } + } + ret = (int)read; + } + + return ret; +} + +static bool sdl_microphone_mic_use_float(const void *driver_context, const void *microphone_context) +{ + sdl_microphone_handle_t *microphone = (sdl_microphone_handle_t*)microphone_context; + (void)driver_context; + + return SDL_AUDIO_ISFLOAT(microphone->device_spec.format); +} + +microphone_driver_t microphone_sdl = { + sdl_microphone_init, + sdl_microphone_free, + sdl_microphone_read, + sdl_microphone_set_nonblock_state, + "sdl2", + NULL, + NULL, + sdl_microphone_open_mic, + sdl_microphone_close_mic, + sdl_microphone_mic_alive, + sdl_microphone_start_mic, + sdl_microphone_stop_mic, + sdl_microphone_mic_use_float, +}; \ No newline at end of file diff --git a/audio/drivers_microphone/wasapi.c b/audio/drivers_microphone/wasapi.c new file mode 100644 index 0000000000..1612c7bd5f --- /dev/null +++ b/audio/drivers_microphone/wasapi.c @@ -0,0 +1,565 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include +#include "audio/common/wasapi.h" +#include "audio/microphone_driver.h" +#include "queues/fifo_queue.h" +#include "configuration.h" +#include "verbosity.h" +#include "audio/common/mmdevice_common.h" + +typedef struct +{ + HANDLE read_event; + IMMDevice *device; + char *device_name; + IAudioClient *client; + IAudioCaptureClient *capture; + + /** + * The buffer in which samples from the microphone will be read and stored + * until the frontend fetches them. + */ + fifo_buffer_t *buffer; + + /** + * The size of an audio frame, in bytes. + * Mic input is in one channel with either 16-bit ints or 32-bit floats, + * so this will be 2 or 4. + */ + size_t frame_size; + size_t engine_buffer_size; + bool exclusive; + bool running; +} wasapi_microphone_handle_t; + +typedef struct wasapi_microphone +{ + bool nonblock; +} wasapi_microphone_t; + + +static void wasapi_microphone_close_mic(void *driver_context, void *microphone_context); + +static void *wasapi_microphone_init(void) +{ + settings_t *settings = config_get_ptr(); + wasapi_microphone_t *wasapi = (wasapi_microphone_t*)calloc(1, sizeof(wasapi_microphone_t)); + + if (!wasapi) + { + RARCH_ERR("[WASAPI mic]: Failed to allocate microphone driver context\n"); + return NULL; + } + + wasapi->nonblock = !settings->bools.audio_sync; + RARCH_DBG("[WASAPI mic]: Initialized microphone driver context\n"); + + return wasapi; +} + +static void wasapi_microphone_free(void *driver_context) +{ + wasapi_microphone_t *wasapi = (wasapi_microphone_t*)driver_context; + + if (!wasapi) + return; + + free(wasapi); +} + +/** + * Flushes microphone's most recent input to the provided context's FIFO queue. + * WASAPI requires that fetched input be consumed in its entirety, + * so the returned value may be less than the queue's size + * if the next packet won't fit in it. + * @param microphone Pointer to the microphone context. + * @return The number of bytes in the queue after fetching input, + * or -1 if there was an error. + */ +static int wasapi_microphone_fetch_fifo(wasapi_microphone_handle_t *microphone) +{ + UINT32 next_packet_size = 0; + /* Shared-mode capture streams split their input buffer into multiple packets, + * while exclusive-mode capture streams just use the one. + * + * The following loop will run at least once; + * for exclusive-mode streams, that's all that we'll need. + */ + + do + { + BYTE *mic_input = NULL; + UINT32 frames_read = 0; + UINT32 bytes_read = 0; + DWORD buffer_status_flags = 0; + HRESULT hr; + + hr = _IAudioCaptureClient_GetBuffer(microphone->capture, &mic_input, &frames_read, &buffer_status_flags, NULL, NULL); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get capture device \"%s\"'s buffer: %s\n", + microphone->device_name, + hresult_name(hr)); + return -1; + } + bytes_read = frames_read * microphone->frame_size; + + if (FIFO_WRITE_AVAIL(microphone->buffer) >= bytes_read && bytes_read > 0) + { /* If the queue has room for the packets we just got... */ + fifo_write(microphone->buffer, mic_input, bytes_read); + /* ...then enqueue the bytes directly from the mic's buffer */ + } + else + { /* Not enough space for new frames, so we can't consume this packet right now */ + frames_read = 0; + } + /* If there's insufficient room in the queue, then we can't read the packet. + * In that case, we leave the packet for next time. */ + + hr = _IAudioCaptureClient_ReleaseBuffer(microphone->capture, frames_read); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to release capture device \"%s\"'s buffer after consuming %u frames: %s\n", + microphone->device_name, + frames_read, + hresult_name(hr)); + return -1; + } + + if (!microphone->exclusive && frames_read > 0) + { /* If this is a shared-mode stream and we didn't run out of room in the sample queue... */ + hr = _IAudioCaptureClient_GetNextPacketSize(microphone->capture, &next_packet_size); + if (FAILED(hr)) + { /* Get the number of frames that the mic has for us. */ + RARCH_ERR("[WASAPI]: Failed to get capture device \"%s\"'s next packet size: %s\n", + microphone->device_name, hresult_name(hr)); + return -1; + } + } + else + { /* Exclusive-mode streams only deliver one packet at a time, though it's bigger. */ + next_packet_size = 0; + } + } + while (next_packet_size != 0); + + return FIFO_READ_AVAIL(microphone->buffer); +} + +/** + * Blocks until the provided microphone's capture event is signalled. + * + * @param microphone The microphone to wait on. + * @param timeout The amount of time to wait, in milliseconds. + * @return \c true if the event was signalled, + * \c false if it timed out or there was an error. + */ +static bool wasapi_microphone_wait_for_capture_event(wasapi_microphone_handle_t *microphone, DWORD timeout) +{ + switch (WaitForSingleObject(microphone->read_event, timeout)) + { /*...then let's wait for the mic to tell us that samples are ready. */ + case WAIT_OBJECT_0: + /* Okay, there's data available. */ + return true; + case WAIT_TIMEOUT: + /* Time out; there's nothing here for us. */ + RARCH_ERR("[WASAPI]: Failed to wait for capture device \"%s\" event: Timeout after %ums\n", microphone->device_name, timeout); + return false; + default: + RARCH_ERR("[WASAPI]: Failed to wait for capture device \"%s\" event: %s\n", microphone->device_name, wasapi_error(GetLastError())); + return false; + } +} + +/** + * Reads samples from a microphone, + * fetching more from it if necessary. + * Works for exclusive and shared-mode streams. + * + * @param microphone Pointer to the context of the microphone + * from which samples will be read. + * @param buffer The buffer in which the fetched samples will be stored. + * @param buffer_size The size of buffer, in bytes. + * @param timeout Timeout for new samples, in milliseconds. + * 0 means that this function won't wait for new samples, + * \c INFINITE means that this function will wait indefinitely. + * @return The number of samples that were retrieved, + * or -1 if there was an error (including timeout). + */ +static int wasapi_microphone_read_buffered( + wasapi_microphone_handle_t *microphone, + void *buffer, + size_t buffer_size, + DWORD timeout) +{ + int bytes_read = 0; /* Number of bytes sent to the core */ + int bytes_available = FIFO_READ_AVAIL(microphone->buffer); + + if (!bytes_available) + { /* If we don't have any queued samples to give to the core... */ + if (!wasapi_microphone_wait_for_capture_event(microphone, timeout)) + { /* If we couldn't wait for the microphone to signal a capture event... */ + return -1; + } + + bytes_available = wasapi_microphone_fetch_fifo(microphone); + if (bytes_available < 0) + { /* If we couldn't fetch samples from the microphone... */ + return -1; + } + } + + /* Now that we have samples available, let's give them to the core */ + + bytes_read = MIN(buffer_size, bytes_available); + fifo_read(microphone->buffer, buffer, bytes_read); + /* Read data from the sample queue and store it in the provided buffer */ + + return bytes_read; +} + +static int wasapi_microphone_read(void *driver_context, void *mic_context, void *buffer, size_t buffer_size) +{ + int bytes_read = 0; + wasapi_microphone_t *wasapi = (wasapi_microphone_t *)driver_context; + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t*)mic_context; + + if (!wasapi || !microphone || !buffer) + return -1; + + if (wasapi->nonblock) + { /* If microphones shouldn't block... */ + return wasapi_microphone_read_buffered(microphone, buffer, buffer_size, 0); + } + + if (microphone->exclusive) + { + int read; + for (read = -1; bytes_read < buffer_size; bytes_read += read) + { + read = wasapi_microphone_read_buffered(microphone, (char *) buffer + bytes_read, buffer_size - bytes_read, + INFINITE); + if (read == -1) + return -1; + } + } + else + { + int read; + + for (read = -1; bytes_read < buffer_size; bytes_read += read) + { + read = wasapi_microphone_read_buffered(microphone, (char *) buffer + bytes_read, buffer_size - bytes_read, + INFINITE); + if (read == -1) + return -1; + } + } + + return bytes_read; +} + +static void wasapi_microphone_set_nonblock_state(void *driver_context, bool nonblock) +{ + wasapi_microphone_t *wasapi = (wasapi_microphone_t*)driver_context; + + RARCH_LOG("[WASAPI mic]: Sync %s.\n", nonblock ? "off" : "on"); + + wasapi->nonblock = nonblock; +} + +static void *wasapi_microphone_open_mic(void *driver_context, const char *device, unsigned rate, + unsigned latency, unsigned *new_rate) +{ + settings_t *settings = config_get_ptr(); + HRESULT hr; + DWORD flags = 0; + UINT32 frame_count = 0; + REFERENCE_TIME dev_period = 0; + BYTE *dest = NULL; + bool float_format = settings->bools.microphone_wasapi_float_format; + bool exclusive_mode = settings->bools.microphone_wasapi_exclusive_mode; + unsigned sh_buffer_length = settings->uints.microphone_wasapi_sh_buffer_length; + wasapi_microphone_handle_t *microphone = calloc(1, sizeof(wasapi_microphone_handle_t)); + (void)driver_context; + + if (!microphone) + return NULL; + + microphone->exclusive = exclusive_mode; + microphone->device = wasapi_init_device(device, eCapture); + if (device && !microphone->device) + { /* If we requested a particular capture device, but couldn't open it... */ + RARCH_WARN("[WASAPI]: Failed to open requested capture device \"%s\", attempting to open default device\n", device); + microphone->device = wasapi_init_device(NULL, eCapture); + } + + if (!microphone->device) + { + RARCH_ERR("[WASAPI]: Failed to open capture device\n"); + goto error; + } + + microphone->device_name = mmdevice_name(microphone->device); + if (!microphone->device_name) + { + RARCH_ERR("[WASAPI]: Failed to get friendly name of capture device\n"); + goto error; + } + + microphone->client = wasapi_init_client(microphone->device, + µphone->exclusive, &float_format, &rate, latency, 1); + if (!microphone->client) + { + RARCH_ERR("[WASAPI]: Failed to open client for capture device \"%s\"\n", microphone->device_name); + goto error; + } + + hr = _IAudioClient_GetBufferSize(microphone->client, &frame_count); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get buffer size of IAudioClient for capture device \"%s\": %s\n", + microphone->device_name, hresult_name(hr)); + goto error; + } + + microphone->frame_size = float_format ? sizeof(float) : sizeof(int16_t); + microphone->engine_buffer_size = frame_count * microphone->frame_size; + + if (microphone->exclusive) + { /* If this mic should be used *exclusively* by RetroArch... */ + microphone->buffer = fifo_new(microphone->engine_buffer_size); + if (!microphone->buffer) + { + RARCH_ERR("[WASAPI]: Failed to initialize FIFO queue for capture device.\n"); + goto error; + } + + RARCH_LOG("[WASAPI]: Intermediate exclusive-mode capture buffer length is %u frames (%.1fms, %u bytes).\n", + frame_count, (double)frame_count * 1000.0 / rate, microphone->engine_buffer_size); + } + else + { + if (sh_buffer_length <= 0) + { /* If the user selected the "default" shared buffer length... */ + hr = _IAudioClient_GetDevicePeriod(microphone->client, &dev_period, NULL); + if (FAILED(hr)) + goto error; + + sh_buffer_length = (dev_period * rate / 10000000) * 2; + /* Default buffer seems to be too small, resulting in slowdown. + * Doubling it seems to work okay. Dunno why. */ + } + + microphone->buffer = fifo_new(sh_buffer_length * microphone->frame_size); + if (!microphone->buffer) + goto error; + + RARCH_LOG("[WASAPI]: Intermediate shared-mode capture buffer length is %u frames (%.1fms, %u bytes).\n", + sh_buffer_length, (double)sh_buffer_length * 1000.0 / rate, sh_buffer_length * microphone->frame_size); + } + + microphone->read_event = CreateEventA(NULL, FALSE, FALSE, NULL); + if (!microphone->read_event) + { + RARCH_ERR("[WASAPI]: Failed to allocate capture device's event handle\n"); + goto error; + } + + hr = _IAudioClient_SetEventHandle(microphone->client, microphone->read_event); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to set capture device's event handle: %s\n", hresult_name(hr)); + goto error; + } + + hr = _IAudioClient_GetService(microphone->client, + IID_IAudioCaptureClient, (void**)µphone->capture); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get capture device's IAudioCaptureClient service: %s\n", hresult_name(hr)); + goto error; + } + + /* Get and release the buffer, just to ensure that we can. */ + hr = _IAudioCaptureClient_GetBuffer(microphone->capture, &dest, &frame_count, &flags, NULL, NULL); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to get capture client buffer: %s\n", hresult_name(hr)); + goto error; + } + + hr = _IAudioCaptureClient_ReleaseBuffer(microphone->capture, 0); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI]: Failed to release capture client buffer: %s\n", hresult_name(hr)); + goto error; + } + + if (new_rate) + { /* The rate was (possibly) modified when we initialized the client */ + *new_rate = rate; + } + return microphone; + +error: + IFACE_RELEASE(microphone->capture); + IFACE_RELEASE(microphone->client); + IFACE_RELEASE(microphone->device); + if (microphone->read_event) + CloseHandle(microphone->read_event); + if (microphone->buffer) + fifo_free(microphone->buffer); + if (microphone->device_name) + free(microphone->device_name); + free(microphone); + + return NULL; +} + +static void wasapi_microphone_close_mic(void *driver_context, void *microphone_context) +{ + DWORD ir; + wasapi_microphone_t *wasapi = (wasapi_microphone_t*)driver_context; + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t*)microphone_context; + HANDLE write_event; + + if (!wasapi || !microphone) + return; + + write_event = microphone->read_event; + + IFACE_RELEASE(microphone->capture); + if (microphone->client) + _IAudioClient_Stop(microphone->client); + IFACE_RELEASE(microphone->client); + IFACE_RELEASE(microphone->device); + if (microphone->buffer) + fifo_free(microphone->buffer); + if (microphone->device_name) + free(microphone->device_name); + free(microphone); + + ir = WaitForSingleObject(write_event, 20); + if (ir == WAIT_FAILED) + { + RARCH_ERR("[WASAPI mic]: WaitForSingleObject failed: %s\n", wasapi_error(GetLastError())); + } + + /* If event isn't signaled log and leak */ + if (ir != WAIT_OBJECT_0) + return; + + CloseHandle(write_event); +} + +static bool wasapi_microphone_start_mic(void *driver_context, void *microphone_context) +{ + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t*)microphone_context; + HRESULT hr; + (void)driver_context; + + if (!microphone) + return false; + + hr = _IAudioClient_Start(microphone->client); + + if (SUCCEEDED(hr) || hr == AUDCLNT_E_NOT_STOPPED) + { /* Starting an already-active microphone is not an error */ + microphone->running = true; + } + else + { + RARCH_ERR("[WASAPI mic]: Failed to start capture device \"%s\"'s IAudioClient: %s\n", + microphone->device_name, hresult_name(hr)); + microphone->running = false; + } + + return microphone->running; +} + +static bool wasapi_microphone_stop_mic(void *driver_context, void *microphone_context) +{ + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t*)microphone_context; + HRESULT hr; + (void)driver_context; + + if (!microphone) + return false; + + hr = _IAudioClient_Stop(microphone->client); + if (FAILED(hr)) + { + RARCH_ERR("[WASAPI mic]: Failed to stop capture device \"%s\"'s IAudioClient: %s\n", + microphone->device_name, hresult_name(hr)); + return false; + } + + RARCH_LOG("[WASAPI mic]: Stopped capture device \"%s\"\n", microphone->device_name); + + microphone->running = false; + + return true; +} + +static bool wasapi_microphone_mic_alive(const void *driver_context, const void *mic_context) +{ + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t *)mic_context; + (void)driver_context; + + return microphone && microphone->running; +} + +static struct string_list *wasapi_microphone_device_list_new(const void *driver_context) +{ + return mmdevice_list_new(driver_context, eCapture); +} + +static void wasapi_microphone_device_list_free(const void *driver_context, struct string_list *devices) +{ + struct string_list *sl = (struct string_list*)devices; + + if (sl) + string_list_free(sl); +} + +static bool wasapi_microphone_use_float(const void *driver_context, const void *microphone_context) +{ + wasapi_microphone_handle_t *microphone = (wasapi_microphone_handle_t *)microphone_context; + (void)driver_context; + + if (!microphone) + return false; + + return microphone->frame_size == sizeof(float); +} + +microphone_driver_t microphone_wasapi = { + wasapi_microphone_init, + wasapi_microphone_free, + wasapi_microphone_read, + wasapi_microphone_set_nonblock_state, + "wasapi", + wasapi_microphone_device_list_new, + wasapi_microphone_device_list_free, + wasapi_microphone_open_mic, + wasapi_microphone_close_mic, + wasapi_microphone_mic_alive, + wasapi_microphone_start_mic, + wasapi_microphone_stop_mic, + wasapi_microphone_use_float +}; \ No newline at end of file diff --git a/audio/microphone_driver.c b/audio/microphone_driver.c new file mode 100644 index 0000000000..e054075b61 --- /dev/null +++ b/audio/microphone_driver.c @@ -0,0 +1,861 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#include +#include "microphone_driver.h" +#include "../configuration.h" +#include "../driver.h" +#include "../verbosity.h" +#include "../runloop.h" +#include "memalign.h" +#include "audio/conversion/s16_to_float.h" +#include "audio/conversion/float_to_s16.h" +#include "../list_special.h" +#include "retro_assert.h" +#include "string/stdstring.h" +#include "audio/conversion/dual_mono.h" + +static microphone_driver_state_t mic_driver_st; + +microphone_driver_t microphone_null = { + NULL, + NULL, + NULL, + NULL, + "null", + NULL, + NULL, + NULL, + NULL +}; + +microphone_driver_t *microphone_drivers[] = { +#ifdef HAVE_ALSA + µphone_alsa, +#if !defined(__QNX__) && !defined(MIYOO) && defined(HAVE_THREADS) + µphone_alsathread, +#endif +#endif +#ifdef HAVE_WASAPI + µphone_wasapi, +#endif +#ifdef HAVE_SDL2 + µphone_sdl, /* Microphones are not supported in SDL 1 */ +#endif + µphone_null, + NULL, +}; + +microphone_driver_state_t *microphone_state_get_ptr(void) +{ + return &mic_driver_st; +} + +#define mic_driver_get_sample_size(microphone) \ + (((microphone)->flags & MICROPHONE_FLAG_USE_FLOAT) ? sizeof(float) : sizeof(int16_t)) + +static bool mic_driver_open_mic_internal(retro_microphone_t* microphone); +bool microphone_driver_start(void) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + retro_microphone_t *microphone = &mic_st->microphone; + + if (microphone->flags & MICROPHONE_FLAG_ACTIVE) + { /* If there's an opened microphone that the core turned on... */ + + if (microphone->flags & MICROPHONE_FLAG_PENDING) + { /* If this microphone was requested before the driver was ready...*/ + retro_assert(microphone->microphone_context == NULL); + /* The microphone context shouldn't have been created yet */ + + /* Now that the driver and driver context are ready, let's initialize the mic */ + if (mic_driver_open_mic_internal(microphone)) + { + /* open_mic_internal will start the microphone if it's enabled */ + RARCH_DBG("[Microphone]: Initialized a previously-pending microphone\n"); + } + else + { + RARCH_ERR("[Microphone]: Failed to initialize a previously pending microphone; microphone will not be used\n"); + + microphone_driver_close_mic(microphone); + /* Not returning false because a mic failure shouldn't take down the driver; + * what if the player just unplugged their mic? */ + } + } + else + { /* The mic was already created, so let's just unpause it */ + microphone_driver_set_mic_state(microphone, true); + + RARCH_DBG("[Microphone]: Started a microphone that was enabled when the driver was last stopped\n"); + } + } + + return true; +} + +bool microphone_driver_stop(void) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + retro_microphone_t *microphone = &mic_st->microphone; + bool result = true; + + if ((microphone->flags & MICROPHONE_FLAG_ACTIVE) + && (microphone->flags & MICROPHONE_FLAG_ENABLED) + && !(microphone->flags & MICROPHONE_FLAG_PENDING)) + { /* If there's an opened microphone that the core turned on and received... */ + + result = mic_st->driver->stop_mic(mic_st->driver_context, microphone->microphone_context); + } + /* If the mic is pending, then we don't need to do anything. */ + + return result; +} + +/** + * config_get_microphone_driver_options: + * + * Get an enumerated list of all microphone driver names, separated by '|'. + * + * Returns: string listing of all microphone driver names, separated by '|'. + **/ +const char *config_get_microphone_driver_options(void) +{ + return char_list_new_special(STRING_LIST_MICROPHONE_DRIVERS, NULL); +} + +bool microphone_driver_find_driver( + void *settings_data, + const char *prefix, + bool verbosity_enabled) +{ + settings_t *settings = (settings_t*)settings_data; + int i = (int)driver_find_index( + "microphone_driver", + settings->arrays.microphone_driver); + + if (i >= 0) + mic_driver_st.driver = (const microphone_driver_t *) + microphone_drivers[i]; + else + { + const microphone_driver_t *tmp = NULL; + if (verbosity_enabled) + { + unsigned d; + RARCH_ERR("Couldn't find any %s named \"%s\"\n", prefix, + settings->arrays.microphone_driver); + + RARCH_LOG_OUTPUT("Available %ss are:\n", prefix); + for (d = 0; microphone_drivers[d]; d++) + { + if (microphone_drivers[d]) + RARCH_LOG_OUTPUT("\t%s\n", microphone_drivers[d]->ident); + } + RARCH_WARN("Going to default to first %s...\n", prefix); + } + + tmp = (const microphone_driver_t *)microphone_drivers[0]; + + if (!tmp) + return false; + mic_driver_st.driver = tmp; + } + + return true; +} + +static void mic_driver_microphone_handle_init(retro_microphone_t *microphone, const retro_microphone_params_t *params) +{ + if (microphone) + { + const settings_t *settings = config_get_ptr(); + microphone->microphone_context = NULL; + microphone->flags = MICROPHONE_FLAG_ACTIVE; + microphone->sample_buffer = NULL; + microphone->sample_buffer_length = 0; + + microphone->requested_params.rate = params ? params->rate : settings->uints.microphone_sample_rate; + microphone->actual_params.rate = 0; + /* We don't set the actual parameters until we actually open the mic. + * (Remember, the core can request one before the driver is ready.) */ + microphone->effective_params.rate = params ? params->rate : settings->uints.microphone_sample_rate; + /* We set the effective parameters because + * the frontend has to do what it can + * to give the core what it asks for. */ + } +} + +static void mic_driver_microphone_handle_free(retro_microphone_t *microphone, bool is_reset) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + const microphone_driver_t *mic_driver = mic_st->driver; + void *driver_context = mic_st->driver_context; + + if (!microphone) + return; + + if (!driver_context) + RARCH_WARN("[Microphone]: Attempted to free a microphone without an active driver context\n"); + + if (microphone->microphone_context) + { + mic_driver->close_mic(driver_context, microphone->microphone_context); + microphone->microphone_context = NULL; + } + + if (microphone->sample_buffer) + { + memalign_free(microphone->sample_buffer); + microphone->sample_buffer = NULL; + microphone->sample_buffer_length = 0; + } + + if (microphone->outgoing_samples) + { + fifo_free(microphone->outgoing_samples); + microphone->outgoing_samples = NULL; + } + + if (microphone->resampler && microphone->resampler->free && microphone->resampler_data) + microphone->resampler->free(microphone->resampler_data); + + microphone->resampler = NULL; + microphone->resampler_data = NULL; + + if ((microphone->flags & MICROPHONE_FLAG_ACTIVE) && is_reset) + { /* If the mic driver is being reset and the microphone was already valid... */ + + microphone->flags |= MICROPHONE_FLAG_PENDING; + /* ...then we need to keep the handle itself valid + * so it can be reinitialized. + * Otherwise the core will lose mic input. */ + } + else + { + memset(microphone, 0, sizeof(*microphone)); + } + /* Do NOT free the microphone handle itself! It's allocated statically! */ +} + +static enum resampler_quality microphone_driver_get_resampler_quality( + settings_t *settings) +{ + if (settings) + return (enum resampler_quality)settings->uints.microphone_resampler_quality; + return RESAMPLER_QUALITY_DONTCARE; +} + +bool microphone_driver_init_internal(void *settings_data) +{ + settings_t *settings = (settings_t*)settings_data; + microphone_driver_state_t *mic_st = &mic_driver_st; + bool verbosity_enabled = verbosity_is_enabled(); + size_t max_frames = AUDIO_CHUNK_SIZE_NONBLOCKING * AUDIO_MAX_RATIO; + + if (!settings->bools.microphone_enable) + { /* If the user has mic support turned off... */ + mic_st->flags &= ~MICROPHONE_DRIVER_FLAG_ACTIVE; + RARCH_WARN("[Microphone]: Refused to initialize microphone driver because it's disabled in the settings\n"); + return false; + } + + convert_s16_to_float_init_simd(); + convert_float_to_s16_init_simd(); + + if (!(microphone_driver_find_driver(settings, + "microphone driver", verbosity_enabled))) + { + RARCH_ERR("[Microphone]: Failed to initialize microphone driver. Will continue without mic input.\n"); + goto error; + } + + mic_st->input_frames_length = max_frames * sizeof(float); + mic_st->input_frames = (float*)memalign_alloc(64, mic_st->input_frames_length); + if (!mic_st->input_frames) + goto error; + + mic_st->converted_input_frames_length = max_frames * sizeof(float); + mic_st->converted_input_frames = (float*)memalign_alloc(64, mic_st->converted_input_frames_length); + if (!mic_st->converted_input_frames) + goto error; + + /* Need room for dual-mono frames */ + mic_st->dual_mono_frames_length = max_frames * sizeof(float) * 2; + mic_st->dual_mono_frames = (float*)memalign_alloc(64, mic_st->dual_mono_frames_length); + if (!mic_st->dual_mono_frames) + goto error; + + mic_st->resampled_frames_length = max_frames * sizeof(float) * 2; + mic_st->resampled_frames = (float*) memalign_alloc(64, mic_st->resampled_frames_length); + if (!mic_st->resampled_frames) + goto error; + + mic_st->resampled_mono_frames_length = max_frames * sizeof(float); + mic_st->resampled_mono_frames = (float*) memalign_alloc(64, mic_st->resampled_mono_frames_length); + if (!mic_st->resampled_mono_frames) + goto error; + + mic_st->final_frames_length = max_frames * sizeof(int16_t); + mic_st->final_frames = (int16_t*) memalign_alloc(64, mic_st->final_frames_length); + if (!mic_st->final_frames) + goto error; + + if (!mic_st->driver || !mic_st->driver->init) + goto error; + + mic_st->driver_context = mic_st->driver->init(); + if (!mic_st->driver_context) + goto error; + + if (!string_is_empty(settings->arrays.microphone_resampler)) + strlcpy(mic_st->resampler_ident, + settings->arrays.microphone_resampler, + sizeof(mic_st->resampler_ident)); + else + mic_st->resampler_ident[0] = '\0'; + + mic_st->resampler_quality = microphone_driver_get_resampler_quality(settings); + + RARCH_LOG("[Microphone]: Initialized microphone driver\n"); + + /* The mic driver was initialized, now we're ready to open mics */ + mic_st->flags |= MICROPHONE_DRIVER_FLAG_ACTIVE; + + if (!microphone_driver_start()) + goto error; + + return true; + +error: + RARCH_ERR("[Microphone]: Failed to start microphone driver. Will continue without audio input.\n"); + mic_st->flags &= ~MICROPHONE_DRIVER_FLAG_ACTIVE; + return microphone_driver_deinit(false); +} + +/** + * + * @param microphone Handle to the microphone to init with a context + */ +static bool mic_driver_open_mic_internal(retro_microphone_t* microphone) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + settings_t *settings = config_get_ptr(); + const microphone_driver_t *mic_driver = mic_st->driver; + void *driver_context = mic_st->driver_context; + unsigned runloop_audio_latency = runloop_state_get_ptr()->audio_latency; + unsigned setting_audio_latency = settings->uints.microphone_latency; + unsigned audio_latency = MAX(runloop_audio_latency, setting_audio_latency); + size_t max_samples = AUDIO_CHUNK_SIZE_NONBLOCKING * 1 * AUDIO_MAX_RATIO; + + if (!microphone || !mic_driver || !(mic_st->flags & MICROPHONE_DRIVER_FLAG_ACTIVE)) + return false; + + microphone->sample_buffer_length = max_samples * sizeof(int16_t); + microphone->sample_buffer = + (int16_t*)memalign_alloc(64, microphone->sample_buffer_length); + + if (!microphone->sample_buffer) + goto error; + + microphone->outgoing_samples = fifo_new(max_samples * sizeof(int16_t)); + if (!microphone->outgoing_samples) + goto error; + + microphone->microphone_context = mic_driver->open_mic(driver_context, + *settings->arrays.microphone_device ? settings->arrays.microphone_device : NULL, + microphone->requested_params.rate, + audio_latency, + µphone->actual_params.rate); + + if (!microphone->microphone_context) + goto error; + + microphone_driver_set_mic_state(microphone, microphone->flags & MICROPHONE_FLAG_ENABLED); + + RARCH_LOG("[Microphone]: Requested microphone sample rate of %uHz, got %uHz.\n", + microphone->requested_params.rate, + microphone->actual_params.rate + ); + + if (mic_driver->mic_use_float && mic_driver->mic_use_float(mic_st->driver_context, microphone->microphone_context)) + { + microphone->flags |= MICROPHONE_FLAG_USE_FLOAT; + } + + microphone->original_ratio = (double)microphone->effective_params.rate / microphone->actual_params.rate; + + if (!retro_resampler_realloc( + µphone->resampler_data, + µphone->resampler, + mic_st->resampler_ident, + mic_st->resampler_quality, + microphone->original_ratio)) + { + RARCH_ERR("[Microphone]: Failed to initialize resampler \"%s\".\n", mic_st->resampler_ident); + goto error; + } + + microphone->flags &= ~MICROPHONE_FLAG_PENDING; + RARCH_LOG("[Microphone]: Initialized microphone\n"); + return true; +error: + mic_driver_microphone_handle_free(microphone, false); + RARCH_ERR("[Microphone]: Driver attempted to initialize the microphone but failed\n"); + return false; +} + +static void microphone_driver_close_mic_internal(retro_microphone_t *microphone, bool is_reset) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + const microphone_driver_t *mic_driver = mic_st->driver; + void *driver_context = mic_st->driver_context; + + if ( microphone && + driver_context && + mic_driver && + mic_driver->close_mic) + { + mic_driver_microphone_handle_free(microphone, is_reset); + } +} + +void microphone_driver_close_mic(retro_microphone_t *microphone) +{ + mic_driver_microphone_handle_free(microphone, false); +} + +bool microphone_driver_set_mic_state(retro_microphone_t *microphone, bool state) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + const microphone_driver_t *mic_driver = mic_st->driver; + void *driver_context = mic_st->driver_context; + + if (!microphone + || !(microphone->flags & MICROPHONE_FLAG_ACTIVE) + || !mic_driver + || !mic_driver->start_mic + || !mic_driver->stop_mic) + return false; + /* If the provided microphone was null or invalid, or the driver is incomplete, stop. */ + + if (driver_context && microphone->microphone_context) + { /* If the driver is initialized... */ + bool success; + if (state) + { /* If we want to enable this mic... */ + success = mic_driver->start_mic(driver_context, microphone->microphone_context); + /* Enable the mic. (Enabling an active mic is a successful noop.) */ + + if (success) + { + microphone->flags |= MICROPHONE_FLAG_ENABLED; + RARCH_LOG("[Microphone]: Enabled microphone\n"); + } + else + { + RARCH_ERR("[Microphone]: Failed to enable microphone\n"); + } + } + else + { /* If we want to pause this mic... */ + success = mic_driver->stop_mic(driver_context, microphone->microphone_context); + /* Disable the mic. (If the mic is already stopped, disabling it should still be successful.) */ + + if (success) + { + microphone->flags &= ~MICROPHONE_FLAG_ENABLED; + RARCH_LOG("[Microphone]: Disabled microphone\n"); + } + else + { + RARCH_ERR("[Microphone]: Failed to disable microphone\n"); + } + } + + return success; + } + else + { /* The driver's not ready yet, so we'll make a note + * of what the mic's state should be */ + if (state) + { + microphone->flags |= MICROPHONE_FLAG_ENABLED; + } + else + { + microphone->flags &= ~MICROPHONE_FLAG_ENABLED; + } + + RARCH_DBG("[Microphone]: Set pending mic state to %s\n", + state ? "enabled" : "disabled"); + return true; + /* This isn't an error */ + } +} + +bool microphone_driver_get_mic_state(const retro_microphone_t *microphone) +{ + if (!microphone || !(microphone->flags & MICROPHONE_FLAG_ACTIVE)) + return false; + + return microphone->flags & MICROPHONE_FLAG_ENABLED; +} + +/** + * Pull queued microphone samples from the driver + * and copy them to the provided buffer(s). + * + * Note that microphone samples are provided in mono, + * so a "sample" and a "frame" are equivalent here. + * + * @param mic_st The overall state of the audio driver. + * @param[out] frames The buffer in which the core will receive microphone samples. + * @param num_frames The size of \c frames, in samples. + */ +static size_t microphone_driver_flush( + microphone_driver_state_t *mic_st, + retro_microphone_t *microphone, + size_t num_frames) +{ + struct resampler_data resampler_data; + unsigned sample_size = mic_driver_get_sample_size(microphone); + size_t bytes_to_read = MIN(mic_st->input_frames_length, num_frames * sample_size); + size_t frames_to_enqueue; + int bytes_read = mic_st->driver->read( + mic_st->driver_context, + microphone->microphone_context, + mic_st->input_frames, + bytes_to_read); + /* First, get the most recent mic data */ + + if (bytes_read <= 0) + return 0; + + resampler_data.input_frames = bytes_read / sample_size; + /* This is in frames, not samples or bytes; + * we're up-channeling the audio to stereo, + * so this number still applies. */ + + resampler_data.output_frames = 0; + /* The resampler sets the value of output_frames */ + + resampler_data.data_in = mic_st->dual_mono_frames; + resampler_data.data_out = mic_st->resampled_frames; + /* The buffers that will be used for the resampler's input and output */ + + resampler_data.ratio = (double)microphone->effective_params.rate / (double)microphone->actual_params.rate; + + if (fabs(resampler_data.ratio - 1.0f) < 1e-8) + { /* If the mic's native rate is practically the same as the requested one... */ + + /* ...then skip the resampler, since it'll produce (more or less) identical results. */ + frames_to_enqueue = MIN(FIFO_WRITE_AVAIL(microphone->outgoing_samples), resampler_data.input_frames); + if (microphone->flags & MICROPHONE_FLAG_USE_FLOAT) + { /* If this mic provides floating-point samples... */ + convert_float_to_s16(mic_st->final_frames, mic_st->input_frames, resampler_data.input_frames); + fifo_write(microphone->outgoing_samples, mic_st->final_frames, frames_to_enqueue * sizeof(int16_t)); + } + else + { + fifo_write(microphone->outgoing_samples, mic_st->input_frames, frames_to_enqueue * sizeof(int16_t)); + } + + return resampler_data.input_frames; + } + /* Couldn't take the fast path, so let's resample the mic input */ + + /* First we need to format the input for the resampler. */ + if (microphone->flags & MICROPHONE_FLAG_USE_FLOAT) + {/* If this mic provides floating-point samples... */ + + /* Samples are already in floating-point, so we just need to up-channel them. */ + convert_to_dual_mono_float(mic_st->dual_mono_frames, mic_st->input_frames, resampler_data.input_frames); + } + else + { + /* Samples are 16-bit, so we need to convert them first. */ + convert_s16_to_float(mic_st->converted_input_frames, mic_st->input_frames, resampler_data.input_frames, 1.0f); + convert_to_dual_mono_float(mic_st->dual_mono_frames, mic_st->converted_input_frames, resampler_data.input_frames); + } + + /* Now we resample the mic data. */ + microphone->resampler->process(microphone->resampler_data, &resampler_data); + + /* Next, we convert the resampled data back to mono... */ + convert_to_mono_float_left(mic_st->resampled_mono_frames, mic_st->resampled_frames, resampler_data.output_frames); + /* Why the left channel? No particular reason. + * Left and right channels are the same in this case anyway. */ + + /* Finally, we convert the audio back to 16-bit ints, as the mic interface requires. */ + convert_float_to_s16(mic_st->final_frames, mic_st->resampled_mono_frames, resampler_data.output_frames); + + frames_to_enqueue = MIN(FIFO_WRITE_AVAIL(microphone->outgoing_samples), resampler_data.output_frames); + fifo_write(microphone->outgoing_samples, mic_st->final_frames, frames_to_enqueue * sizeof(int16_t)); + return resampler_data.output_frames; +} + +int microphone_driver_read(retro_microphone_t *microphone, int16_t* frames, size_t num_frames) +{ + uint32_t runloop_flags = runloop_get_flags(); + size_t frames_remaining = num_frames; + microphone_driver_state_t *mic_st = &mic_driver_st; + const microphone_driver_t *driver = mic_st->driver; + bool core_paused = runloop_flags & RUNLOOP_FLAG_PAUSED; + bool is_fastforward = runloop_flags & RUNLOOP_FLAG_FASTMOTION; + bool is_slowmo = runloop_flags & RUNLOOP_FLAG_SLOWMOTION; + bool is_rewind = state_manager_frame_is_reversed(); + bool driver_active = mic_st->flags & MICROPHONE_DRIVER_FLAG_ACTIVE; + + if (!frames || !microphone) + /* If the provided arguments aren't valid... */ + return -1; + + if (!driver_active || !(microphone->flags & MICROPHONE_FLAG_ACTIVE)) + /* If the microphone or driver aren't active... */ + return -1; + + if (!driver || !driver->read || !driver->mic_alive) + /* If the driver is invalid or doesn't have the functions it needs... */ + return -1; + + if (num_frames == 0) + /* If the core didn't actually ask for any frames... */ + return 0; + + if ((microphone->flags & MICROPHONE_FLAG_PENDING) + || (microphone->flags & MICROPHONE_FLAG_SUSPENDED) + || !(microphone->flags & MICROPHONE_FLAG_ENABLED) + || is_fastforward + || is_slowmo + || is_rewind + ) + { /* If the microphone is pending, suspended, or disabled... + ...or if the core is in fast-forward, slow-mo, or rewind...*/ + memset(frames, 0, num_frames * sizeof(*frames)); + return (int)num_frames; + /* ...then copy silence to the provided buffer. Not an error if the mic is pending, + * because the user might have requested a microphone + * before the driver could provide it. */ + } + + /* Why mute the mic when the core isn't running at standard speed? + * Because I couldn't think of anything useful for the mic to do. + * If you can, send a PR! */ + + if (!mic_st->driver_context || !microphone->microphone_context) + /* If the driver or microphone's state haven't been allocated... */ + return -1; + + if (!driver->mic_alive(mic_st->driver_context, microphone->microphone_context)) + { /* If the mic isn't active like it should be at this point... */ + RARCH_ERR("[Microphone]: Mic frontend has the mic enabled, but the backend has it disabled.\n"); + return -1; + } + + if (num_frames > microphone->outgoing_samples->size) + /* If the core asked for more frames than we can fit... */ + return -1; + + retro_assert(mic_st->input_frames != NULL); + + while (FIFO_READ_AVAIL(microphone->outgoing_samples) < num_frames * sizeof(int16_t)) + { /* Until we can give the core the frames it asked for... */ + size_t frames_to_read = MIN(AUDIO_CHUNK_SIZE_NONBLOCKING, frames_remaining); + size_t frames_read = 0; + if (!core_paused) + /* If the game is running and the mic driver is active... */ + frames_read = microphone_driver_flush(mic_st, microphone, frames_to_read); + + /* Otherwise, advance the counters. We're not gonna get new data, + * but we still need to finish this loop */ + frames_remaining -= frames_read; + } /* If the queue already has enough samples to give, the loop will be skipped */ + + fifo_read(microphone->outgoing_samples, frames, num_frames * sizeof(int16_t)); + return (int)num_frames; +} + +bool microphone_driver_get_effective_params(const retro_microphone_t *microphone, retro_microphone_params_t *params) +{ + if (!microphone || !params) + /* If the arguments are null... */ + return false; + + if (!(microphone->flags & MICROPHONE_FLAG_ACTIVE)) + /* If this isn't an opened microphone... */ + return false; + + *params = microphone->effective_params; + + return true; +} + +/* NOTE: The core may request a microphone before the driver is ready. + * A pending handle will be provided in that case, and the frontend will + * initialize the microphone when the time is right; + * do not call this function twice on the same mic. */ +retro_microphone_t *microphone_driver_open_mic(const retro_microphone_params_t *params) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + const settings_t *settings = config_get_ptr(); + const microphone_driver_t *mic_driver = mic_st->driver; + void *driver_context = mic_st->driver_context; + + if (!settings) + { + RARCH_ERR("[Microphone]: Failed to open microphone due to uninitialized config\n"); + return NULL; + } + + if (!settings->bools.microphone_enable) + { /* Not checking mic_st->flags because they might not be set yet; + * don't forget, the core can ask for a mic + * before the audio driver is ready to create one. */ + RARCH_WARN("[Microphone]: Refused to open microphone because it's disabled in the settings\n"); + return NULL; + } + + if (mic_driver == µphone_null) + { + RARCH_WARN("[Microphone]: Cannot open microphone, null driver is configured.\n"); + return NULL; + } + + if (!mic_driver && + (string_is_equal(settings->arrays.microphone_driver, "null") + || string_is_empty(settings->arrays.microphone_driver))) + { /* If the mic driver hasn't been initialized, but it's not going to be... */ + RARCH_ERR("[Microphone]: Cannot open microphone as the driver won't be initialized\n"); + return NULL; + } + + if (mic_st->microphone.flags & MICROPHONE_FLAG_ACTIVE) + { /* If the core has requested a second microphone... */ + RARCH_ERR("[Microphone]: Failed to open a second microphone, frontend only supports one at a time right now\n"); + if (mic_st->microphone.flags & MICROPHONE_FLAG_PENDING) + /* If that mic is pending... */ + RARCH_ERR("[Microphone]: A microphone is pending initialization\n"); + else + /* That mic is initialized */ + RARCH_ERR("[Microphone]: An initialized microphone exists\n"); + + return NULL; + } + + /* Cores might ask for a microphone before the audio driver is ready to provide them; + * if that happens, we have to initialize the microphones later. + * But the user still wants a handle, so we'll give them one. + */ + mic_driver_microphone_handle_init(&mic_st->microphone, params); + /* If driver_context is NULL, the handle won't have a valid microphone context (but we'll create one later) */ + + if (driver_context) + { /* If the microphone driver is ready to open a microphone... */ + if (mic_driver_open_mic_internal(&mic_st->microphone)) /* If the microphone was successfully initialized... */ + RARCH_LOG("[Microphone]: Opened the requested microphone successfully\n"); + else + goto error; + } + else + { /* If the driver isn't ready to create a microphone... */ + mic_st->microphone.flags |= MICROPHONE_FLAG_PENDING; + RARCH_LOG("[Microphone]: Microphone requested before driver context was ready; deferring initialization\n"); + } + + return &mic_st->microphone; +error: + mic_driver_microphone_handle_free(&mic_st->microphone, false); + /* This function cleans up any resources and unsets all flags */ + + return NULL; +} + +static bool microphone_driver_free_devices_list(void) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + if ( + !mic_st->driver + || !mic_st->driver->device_list_free + || !mic_st->driver_context + || !mic_st->devices_list) + return false; + + mic_st->driver->device_list_free(mic_st->driver_context, mic_st->devices_list); + mic_st->devices_list = NULL; + return true; +} + +bool microphone_driver_deinit(bool is_reset) +{ + microphone_driver_state_t *mic_st = &mic_driver_st; + const microphone_driver_t *driver = mic_st->driver; + + microphone_driver_free_devices_list(); + microphone_driver_close_mic_internal(&mic_st->microphone, is_reset); + + if (driver && driver->free) + { + if (mic_st->driver_context) + driver->free(mic_st->driver_context); + + mic_st->driver_context = NULL; + } + + if (mic_st->input_frames) + memalign_free(mic_st->input_frames); + mic_st->input_frames = NULL; + mic_st->input_frames_length = 0; + + if (mic_st->converted_input_frames) + memalign_free(mic_st->converted_input_frames); + mic_st->converted_input_frames = NULL; + mic_st->converted_input_frames_length = 0; + + if (mic_st->dual_mono_frames) + memalign_free(mic_st->dual_mono_frames); + mic_st->dual_mono_frames = NULL; + mic_st->dual_mono_frames_length = 0; + + if (mic_st->resampled_frames) + memalign_free(mic_st->resampled_frames); + mic_st->resampled_frames = NULL; + mic_st->resampled_frames_length = 0; + + if (mic_st->resampled_mono_frames) + memalign_free(mic_st->resampled_mono_frames); + mic_st->resampled_mono_frames = NULL; + mic_st->resampled_mono_frames_length = 0; + + if (mic_st->final_frames) + memalign_free(mic_st->final_frames); + mic_st->final_frames = NULL; + mic_st->final_frames_length = 0; + + mic_st->resampler_quality = RESAMPLER_QUALITY_DONTCARE; + mic_st->flags &= ~MICROPHONE_DRIVER_FLAG_ACTIVE; + memset(mic_st->resampler_ident, '\0', sizeof(mic_st->resampler_ident)); + + return true; +} + +bool microphone_driver_get_devices_list(void **data) +{ + struct string_list**ptr = (struct string_list**)data; + if (!ptr) + return false; + *ptr = mic_driver_st.devices_list; + return true; +} \ No newline at end of file diff --git a/audio/microphone_driver.h b/audio/microphone_driver.h new file mode 100644 index 0000000000..328848712f --- /dev/null +++ b/audio/microphone_driver.h @@ -0,0 +1,667 @@ +/* RetroArch - A frontend for libretro. + * Copyright (C) 2023 Jesse Talavera-Greenberg + * + * RetroArch is free software: you can redistribute it and/or modify it under the terms + * of the GNU General Public License as published by the Free Software Found- + * ation, either version 3 of the License, or (at your option) any later version. + * + * RetroArch is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; + * without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR + * PURPOSE. See the GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along with RetroArch. + * If not, see . + */ + +#ifndef RETROARCH_MICROPHONE_DRIVER_H +#define RETROARCH_MICROPHONE_DRIVER_H + +#include +#include +#include +#include +#include "audio/audio_resampler.h" +#include "queues/fifo_queue.h" + +/** + * Flags that indicate the current state of the microphone driver. + */ +enum microphone_driver_state_flags +{ + /** + * Indicates that the driver was successfully created + * and is currently valid. + * You may open microphones and query them for samples at any time. + * + * This flag does \em not mean that the core will receive audio; + * the driver might be suspended. + */ + MICROPHONE_DRIVER_FLAG_ACTIVE = (1 << 0) +}; + +/** + * Flags that indicate the current state of a particular microphone. + */ +enum microphone_state_flags +{ + /** + * Indicates that the microphone was successfully created + * and is currently valid. + * You may query it for samples at any time. + * + * This flag does \em not mean that the core will receive anything, + * as there are several situations where a mic will return silence. + * + * If this flag is not set, then the others are meaningless. + * In that case, reads from this microphone will return an error. + */ + MICROPHONE_FLAG_ACTIVE = (1 << 0), + + /** + * Indicates that the core considers this microphone "on" + * and ready to retrieve audio. + * + * Even if a microphone is opened, the user might not want it running constantly; + * they might prefer to hold a button to use it. + * + * If this flag is not set, the microphone will not process input. + * Reads from it will return an error. + */ + MICROPHONE_FLAG_ENABLED = (1 << 1), + + /** + * Indicates that this microphone was requested + * before the microphone driver was initialized, + * so the driver will need to create this microphone + * when it's ready. + * + * This flag is also used to reinitialize microphones + * that were closed as part of a driver reinit. + * + * If this flag is set, reads from this microphone return silence + * of the requested length. + */ + MICROPHONE_FLAG_PENDING = (1 << 2), + + /** + * Indicates that the microphone provides floating-point samples, + * as opposed to integer samples. + * + * All audio is sent through the resampler, + * which operates on floating-point samples. + * + * If this flag is set, then the resampled output doesn't need + * to be converted back to \c int16_t format. + * + * This won't significantly affect the audio that the core receives; + * either way, it's supposed to receive \c int16_t samples. + * + * This flag won't be set if the selected microphone driver + * doesn't support (or is configured to not use) \c float samples. + * + * @see microphone_driver_t::mic_use_float + */ + MICROPHONE_FLAG_USE_FLOAT = (1 << 3), + + /** + * Indicates that the microphone driver is not currently retrieving samples, + * although it's valid and can be resumed. + * + * Usually set when RetroArch needs to simulate audio input + * without actually rendering samples (e.g. runahead), + * or when reinitializing the driver. + * + * If this flag is set, reads from this microphone return silence + * of the requested length. + * + * This is different from \c MICROPHONE_FLAG_ACTIVE and \c MICROPHONE_FLAG_ENABLED; + * \c MICROPHONE_FLAG_ACTIVE indicates that the microphone is valid, + * and \c MICROPHONE_FLAG_ENABLED indicates that the core has the microphone turned on. + */ + MICROPHONE_FLAG_SUSPENDED = (1 << 4) +}; + +/** + * Driver object that tracks a microphone's state. + * Pointers to this object are provided to cores + * for use as opaque handles by \c retro_microphone_interface_t. + */ +struct retro_microphone +{ + /** + * Pointer to the context object created by the underlying driver. + * It will contain data that's specific to each driver, + * such as device IDs or sample queues. + */ + void *microphone_context; + + /** + * Pointer to the data that will be copied to cores. + */ + int16_t* sample_buffer; + + /** + * Length of \c sample_buffer in bytes, \em not samples. + */ + size_t sample_buffer_length; + + /** + * Bit flags that describe the state of this microphone. + * + * @see microphone_state_flags + */ + enum microphone_state_flags flags; + + /** + * Samples that will be sent to the core. + */ + fifo_buffer_t *outgoing_samples; + + /** + * The requested microphone parameters, + * taken from the core's open_mic call. + */ + retro_microphone_params_t requested_params; + + /** + * The parameters of the microphone as it was provided. + */ + retro_microphone_params_t actual_params; + + /** + * The parameters of the microphone after any resampling + * or other changes. + */ + retro_microphone_params_t effective_params; + + /** + * Pointer to the configured resampler for microphones. + * May be different than the audio driver's resampler. + */ + const retro_resampler_t *resampler; + + /** + * Pointer to the resampler-specific context. + * Not shared with the audio driver's resampler. + */ + void *resampler_data; + + /** + * The ratio of the core-requested sample rate to the device's opened sample rate. + * If this is (almost) equal to 1, then resampling will be skipped. + */ + double original_ratio; +}; + +/** + * Defines the implementation of a microphone driver. + * All functions are mandatory unless otherwise noted. + */ +typedef struct microphone_driver +{ + /** + * Initializes the microphone driver. + * This function should not open any actual microphones; + * instead, it should set up any prerequisites necessary + * to create a microphone. + * + * After this function is called, + * microphones can be opened with \c open_mic. + * + * @returns A handle to the microphone driver context, + * or \c NULL if there was an error. + * + * @see microphone_driver_init_internal + **/ + void *(*init)(void); + + + /** + * Frees the driver context. + * There is no need to close the microphones in this function, + * the microphone system will do that before calling this. + * Does nothing if \c driver_context is \c NULL. + * + * @param driver_context Pointer to the microphone driver context. + * Provide the pointer that was returned by \c ::init(), + * \em not one of the handles returned by \c ::mic_open(). + * + * @see microphone_driver_deinit + */ + void (*free)(void *driver_context); + + /** + * Read samples from the microphone into the provided buffer. + * + * Samples are provided in mono. + * Since samples are in mono, a "frame" and a "sample" mean the same thing + * in the context of microphone input. + * + * If \c ::mic_use_float returns \c true, + * samples will be in 32-bit \c float format with a range of [-1.0, 1.0]. + * Otherwise, samples will be in signed 16-bit integer format. + * Data will be in native byte order either way. + * + * All reads should block until all requested frames are provided, + * unless set otherwise with set_nonblock_state(). + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] mic_context Pointer to the microphone context. + * Will be a value that was returned by \c ::open_mic(). + * @param[out] buffer Pointer to the buffer in which the read samples should be stored. + * @param buffer_size The available length of \c buffer in bytes, + * \em not samples or frames. + * @return The number of bytes that were successfully read, + * or \c -1 if there was an error. + * May be less than \c buffer_size if this microphone is non-blocking. + * If this microphone is in non-blocking mode and no new data is available, + * the driver should return 0 rather than -1. + * + * @note Do not apply resampling or up-channeling; + * the microphone frontend will do so. + * @note Do not return silence if unable to read samples; + * instead, return an error. + * The frontend will provide silence to the core in + * non-erroneous situations where microphone input is unsupported + * (such as in fast-forward or rewind). + * + * @see microphone_driver_read + */ + int (*read)(void *driver_context, void *mic_context, void *buffer, size_t buffer_size); + + /** + * Sets the nonblocking state of the driver. + * If the driver is in blocking mode (the default), + * \c ::read() will block the current thread + * until all requested samples are provided. + * Otherwise, \c ::read() will return as many samples as it can (which may be none) + * and return without waiting. + * + * If a driver does not support nonblocking mode, + * leave this function pointer as \c NULL. + * + * @param driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] nonblock \c true if the driver should be in nonblocking mode, + * \c false if it should be in blocking mode. + * */ + void (*set_nonblock_state)(void *driver_context, bool nonblock); + + /** + * A human-readable name for this driver. + * Shown to the user in the driver selection menu. + */ + const char *ident; + + /** + * Returns a list of all microphones (aka "capture devices") + * that are currently available. + * The user can select from these devices on the options menu. + * + * Optional, but must be implemented if \c device_list_free is implemented. + * The list returned by this function must be freed with \c device_list_free. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @return Pointer to a list of available capture devices, + * an empty list if no capture devices are available, + * or \c NULL if there was an error. + **/ + struct string_list *(*device_list_new)(const void *driver_context); + + /** + * Frees the microphone list that was returned by \c device_list_new. + * Optional, but must be provided if \c device_list_new is implemented. + * Will do nothing if any parameter is \c NULL. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init. + * @param[in] devices Pointer to the device list + * that was returned by \c device_list_new. + */ + void (*device_list_free)(const void *driver_context, struct string_list *devices); + + /** + * Initializes a microphone. + * Cores that use microphone functionality will call this via + * \c retro_microphone_interface_t::open_mic. + * + * The core may request a microphone before the driver is fully initialized, + * but driver implementations do not need to concern themselves with that; + * when the driver is ready, it will call this function. + * + * Opened microphones must \em not be activated, + * i.e. \c mic_alive on a newly-opened microphone should return \c false. + * + * @param data Handle to the driver context + * that was originally returned by \c init. + * @param device A specific device name (or other options) + * to create the microphone with. + * Each microphone driver interprets this differently, + * and some may ignore it. + * @param rate The requested sampling rate of the new microphone in Hz. + * @param latency The desired latency of the new microphone, in milliseconds. + * @param new_rate Pointer to the actual sample frequency, + * if the microphone couldn't be initialized with the value given by rate. + * If NULL, then the value will not be reported to the caller; + * this is not an error. + * @return An opaque handle to the newly-initialized microphone + * if it was successfully created, + * or \c NULL if there was an error. + * May be \c NULL if no microphone is available, + * or if the maximum number of microphones has been created. + * The returned handle should be provided to the \c microphone_context + * parameter of all other microphone functions. + * + * @note Your driver should keep track of the mic context + */ + void *(*open_mic)(void *driver_context, const char *device, unsigned rate, + unsigned latency, unsigned *new_rate); + + /** + * Releases the resources used by a particular microphone + * and stops its activity. + * Will do nothing if either \p driver_context or \p microphone_context is \c NULL. + * + * @param driver_context Opaque handle to the audio driver context + * that was used to create the provided microphone. + * Implementations may use this to help in cleaning up the microphone, + * but the driver context itself must \em not be released. + * @param microphone_context Opaque handle to the microphone that will be freed. + * Implementations should stop any recording activity before freeing resources. + * + * @post \p driver_context will still be valid, + * while \p microphone_context will not. + */ + void (*close_mic)(void *driver_context, void *microphone_context); + + /** + * Returns the active state of the provided microphone. + * This is the state of the device itself, + * not the user's desired on/off state. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] mic_context Pointer to a particular microphone's context. + * Will be a value that was returned by \c ::open_mic(). + * @return \c true if the provided microphone is active, + * \c false if not or if there was an error. + */ + bool (*mic_alive)(const void *driver_context, const void *mic_context); + + /** + * Begins capture activity on the provided microphone, if necessary. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] mic_context Pointer to a particular microphone's context. + * Will be a value that was returned by \c ::open_mic(). + * @return \c true if the microphone was successfully started + * or if it was already running. \c false if there was an error. + */ + bool (*start_mic)(void *driver_context, void *microphone_context); + + /** + * Pauses capture activity on the provided microphone, if necessary. + * This function must not deallocate the microphone. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] mic_context Pointer to a particular microphone's context. + * Will be a value that was returned by \c ::open_mic(). + * @return \c true if the microphone was successfully paused + * or if it was already stopped. \c false if there was an error. + */ + bool (*stop_mic)(void *driver_context, void *microphone_context); + + /** + * Queries whether this microphone captures floating-point samples, + * as opposed to 16-bit integer samples. + * + * Optional; if not provided, then \c int16_t samples are assumed. + * + * @param[in] driver_context Pointer to the driver context. + * Will be the value that was returned by \c ::init(). + * @param[in] mic_context Pointer to a particular microphone's context. + * Will be a value that was returned by \c ::open_mic(). + * @return \c true if this microphone provides floating-point samples. + */ + bool (*mic_use_float)(const void *driver_context, const void *microphone_context); +} microphone_driver_t; + +typedef struct microphone_driver_state +{ + /** + * The buffer that receives samples from the microphone backend, + * before they're processed. + */ + void *input_frames; + + /** + * The length of \c input_frames in bytes. + */ + size_t input_frames_length; + + /** + * The buffer that receives samples that have been + * converted to floating-point format, if necessary. + */ + float *converted_input_frames; + + /** + * The length of \c converted_input_frames in bytes. + */ + size_t converted_input_frames_length; + + /** + * The buffer that stores microphone samples + * after they've been converted to floating-point format + * and up-channeled to dual-mono. + */ + float *dual_mono_frames; + + /** + * The length of \c dual_mono_frames in bytes. + */ + size_t dual_mono_frames_length; + + /** + * The buffer that stores microphone samples + * after they've been converted to float, + * up-channeled to dual-mono, + * and resampled. + */ + float *resampled_frames; + + /** + * The length of \c resampled_frames in bytes. + */ + size_t resampled_frames_length; + + + /** + * The buffer that stores microphone samples + * after they've been resampled + * and converted to mono. + */ + float *resampled_mono_frames; + + /** + * The length of \c resampled_mono_frames in bytes. + */ + size_t resampled_mono_frames_length; + + /** + * The buffer that contains the microphone input + * after it's been totally processed and converted. + * The contents of this buffer will be provided to the core. + */ + int16_t *final_frames; + + /** + * The length of \c final_frames in bytes. + */ + size_t final_frames_length; + + /** + * The current microphone driver. + * Will be a pointer to one of the elements of \c microphone_drivers. + */ + const microphone_driver_t *driver; + + struct string_list *devices_list; + + /** + * Opaque handle to the driver-specific context. + */ + void *driver_context; + + /** + * The handle to the created microphone, if any. + * The libretro API is designed to expose multiple microphones, + * but RetroArch only supports one at a time for now. + * PRs welcome! + */ + retro_microphone_t microphone; + + enum microphone_driver_state_flags flags; + + enum resampler_quality resampler_quality; + + char resampler_ident[64]; +} microphone_driver_state_t; + +/** + * Starts all enabled microphones, + * and opens all pending microphones. + * It is not an error to call this function + * if the mic driver is already running. + * + * @return \c true if the configured driver was started + * and pending microphones opened, + * \c false if there was an error. + */ +bool microphone_driver_start(void); + +/** + * Stops all enabled microphones. + * It is not an error to call this function + * if the mic driver is already stopped, + * or if there are no open microphones. + * + * Microphones will not receive any input + * until \c microphone_driver_start is called again. + * + * @return \c true if the driver was stopped, + * \c false if there was an error. + */ +bool microphone_driver_stop(void); + +/** + * Driver function for opening a microphone. + * Provided to retro_microphone_interface::init_microphone(). + * @param[in] params Parameters for the newly-opened microphone + * that the core requested. + * May be \c NULL, in which case defaults will be selected. + * @return Pointer to the newly-opened microphone, + * or \c NULL if there was an error. + */ +retro_microphone_t *microphone_driver_open_mic(const retro_microphone_params_t *params); + +/** + * Driver function for closing an open microphone. + * Does nothing if the provided microphone is \c NULL. + * @param microphone Pointer to the microphone to close. + * Will be a value that was returned by \c microphone_driver_open_mic. + */ +void microphone_driver_close_mic(retro_microphone_t *microphone); + +/** + * Enables or disables the microphone. + * + * @returns \c true if the microphone's active state was set, + * \c false if there was an error. + */ +bool microphone_driver_set_mic_state(retro_microphone_t *microphone, bool state); + +/** + * Queries the active state of the microphone. + * Inactive microphones return no audio, + * and it is an error to read from them. + * + * @param microphone The microphone to query. + * @return The active state of \c microphone. + * \c true if the microphone is ready to accept input, + * \c false if not. + */ +bool microphone_driver_get_mic_state(const retro_microphone_t *microphone); + +/** + * Reads a particular number of samples from a microphone + * and stores it in a buffer. + * This should be called by the core each frame if there's an active microphone. + * Will block until the buffer is filled, + * so don't ask for more than you'll use in a single frame. + * + * @param microphone The microphone from which samples will be read. + * @param samples The buffer in which incoming samples will be stored. + * @param num_samples The available size of the provided buffer, + * in samples (\em not bytes). + * @return The number of samples that were read, or -1 if there was an error. + */ +int microphone_driver_read(retro_microphone_t *microphone, int16_t* samples, size_t num_samples); + +bool microphone_driver_get_effective_params(const retro_microphone_t *microphone, retro_microphone_params_t *params); + +/** + * A trivial backend with no functions and an identifier of "null". + * Effectively disables mic support or serves as a stand-in + * on platforms that lack mic backends. + */ +extern microphone_driver_t microphone_null; + +/** + * The ALSA-backed microphone driver. + */ +extern microphone_driver_t microphone_alsa; + +/** + * The multithreaded ALSA-backed microphone driver. + */ +extern microphone_driver_t microphone_alsathread; + +/** + * The SDL-backed microphone driver. + */ +extern microphone_driver_t microphone_sdl; + +/** + * The WASAPI-backed microphone driver. + */ +extern microphone_driver_t microphone_wasapi; + +/** + * @return Pointer to the global microphone driver state. + */ +microphone_driver_state_t *microphone_state_get_ptr(void); + +/** + * All microphone drivers available for use in this build. + * The contents of this array depend on the build configuration + * and target platform. + */ +extern microphone_driver_t *microphone_drivers[]; + +bool microphone_driver_init_internal(void *settings_data); + +bool microphone_driver_deinit(bool is_reset); + +bool microphone_driver_find_driver( + void *settings_data, + const char *prefix, + bool verbosity_enabled); + +bool microphone_driver_get_devices_list(void **ptr); + +#endif /* RETROARCH_MICROPHONE_DRIVER_H */ \ No newline at end of file diff --git a/command.h b/command.h index dd2b6ada4d..2bc6687db4 100644 --- a/command.h +++ b/command.h @@ -258,7 +258,14 @@ enum event_command CMD_EVENT_PRESENCE_UPDATE, CMD_EVENT_OVERLAY_NEXT, CMD_EVENT_OSK_TOGGLE, - +#ifdef HAVE_MICROPHONE + /* Stops all enabled microphones. */ + CMD_EVENT_MICROPHONE_STOP, + /* Starts all enabled microphones */ + CMD_EVENT_MICROPHONE_START, + /* Reinitializes microphone driver. */ + CMD_EVENT_MICROPHONE_REINIT, +#endif /* Deprecated */ CMD_EVENT_SEND_DEBUG_INFO }; diff --git a/config.def.h b/config.def.h index 54200c1ca7..86c89d4456 100644 --- a/config.def.h +++ b/config.def.h @@ -1084,10 +1084,13 @@ /* Output samplerate. */ #if defined(GEKKO) || defined(MIYOO) #define DEFAULT_OUTPUT_RATE 32000 +#define DEFAULT_INPUT_RATE 32000 #elif defined(_3DS) || defined(RETROFW) #define DEFAULT_OUTPUT_RATE 32730 +#define DEFAULT_INPUT_RATE 32730 #else #define DEFAULT_OUTPUT_RATE 48000 +#define DEFAULT_INPUT_RATE 48000 #endif /* Audio device (e.g. hw:0,0 or /dev/audio). If NULL, will use defaults. */ @@ -1098,8 +1101,10 @@ #if defined(ANDROID) || defined(EMSCRIPTEN) || defined(RETROFW) || defined(MIYOO) /* For most Android devices, 64ms is way too low. */ #define DEFAULT_OUT_LATENCY 128 +#define DEFAULT_IN_LATENCY 128 #else #define DEFAULT_OUT_LATENCY 64 +#define DEFAULT_IN_LATENCY 64 #endif /* Will sync audio. (recommended) */ @@ -1141,6 +1146,16 @@ * Avoids crackling */ #define DEFAULT_AUDIO_FASTFORWARD_SPEEDUP false +#ifdef HAVE_MICROPHONE +/* Microphone support */ +#define DEFAULT_MICROPHONE_ENABLE true +#define DEFAULT_MICROPHONE_DEVICE NULL + +#ifdef HAVE_WASAPI +#define DEFAULT_WASAPI_MICROPHONE_SH_BUFFER_LENGTH 0 +#endif +#endif + /* MISC */ /* Enables displaying the current frames per second. */ diff --git a/configuration.c b/configuration.c index 4c932f0010..82adc6157c 100644 --- a/configuration.c +++ b/configuration.c @@ -143,9 +143,18 @@ enum audio_driver_enum AUDIO_NULL }; +enum microphone_driver_enum +{ + MICROPHONE_ALSA = AUDIO_NULL + 1, + MICROPHONE_ALSATHREAD, + MICROPHONE_SDL2, + MICROPHONE_WASAPI, + MICROPHONE_NULL +}; + enum audio_resampler_driver_enum { - AUDIO_RESAMPLER_CC = AUDIO_NULL + 1, + AUDIO_RESAMPLER_CC = MICROPHONE_NULL + 1, AUDIO_RESAMPLER_SINC, AUDIO_RESAMPLER_NEAREST, AUDIO_RESAMPLER_NULL @@ -534,6 +543,23 @@ static const enum audio_driver_enum AUDIO_DEFAULT_DRIVER = AUDIO_EXT; static const enum audio_driver_enum AUDIO_DEFAULT_DRIVER = AUDIO_NULL; #endif +#if defined(HAVE_MICROPHONE) +#if defined(HAVE_WASAPI) +/* The default mic driver on Windows is WASAPI if it's available. */ +static const enum microphone_driver_enum MICROPHONE_DEFAULT_DRIVER = MICROPHONE_WASAPI; +#elif defined(HAVE_ALSA) && defined(HAVE_THREADS) +/* The default mic driver on Linux is the threaded ALSA driver, if available. */ +static const enum microphone_driver_enum MICROPHONE_DEFAULT_DRIVER = MICROPHONE_ALSATHREAD; +#elif defined(HAVE_ALSA) +static const enum microphone_driver_enum MICROPHONE_DEFAULT_DRIVER = MICROPHONE_ALSA; +#elif defined(HAVE_SDL2) +/* The default fallback driver is SDL2, if available. */ +static const enum microphone_driver_enum MICROPHONE_DEFAULT_DRIVER = MICROPHONE_SDL2; +#else +static const enum microphone_driver_enum MICROPHONE_DEFAULT_DRIVER = MICROPHONE_NULL; +#endif +#endif + #if defined(RS90) || defined(MIYOO) static const enum audio_resampler_driver_enum AUDIO_DEFAULT_RESAMPLER_DRIVER = AUDIO_RESAMPLER_NEAREST; #elif defined(PSP) || defined(EMSCRIPTEN) @@ -919,6 +945,37 @@ const char *config_get_default_audio(void) return "null"; } +#if defined(HAVE_MICROPHONE) +/** + * config_get_default_microphone: + * + * Gets default microphone driver. + * + * Returns: Default microphone driver. + **/ +const char *config_get_default_microphone(void) +{ + enum microphone_driver_enum default_driver = MICROPHONE_DEFAULT_DRIVER; + + switch (default_driver) + { + case MICROPHONE_ALSA: + return "alsa"; + case MICROPHONE_ALSATHREAD: + return "alsathread"; + case MICROPHONE_WASAPI: + return "wasapi"; + case MICROPHONE_SDL2: + return "sdl2"; + case MICROPHONE_NULL: + break; + } + + return "null"; +} +#endif + + const char *config_get_default_record(void) { enum record_driver_enum default_driver = RECORD_DEFAULT_DRIVER; @@ -1461,6 +1518,11 @@ static struct config_array_setting *populate_settings_array(settings_t *settings SETTING_ARRAY("discord_app_id", settings->arrays.discord_app_id, true, DEFAULT_DISCORD_APP_ID, true); SETTING_ARRAY("ai_service_url", settings->arrays.ai_service_url, true, DEFAULT_AI_SERVICE_URL, true); SETTING_ARRAY("crt_switch_timings", settings->arrays.crt_switch_timings, false, NULL, true); +#ifdef HAVE_MICROPHONE + SETTING_ARRAY("microphone_device", settings->arrays.microphone_device, false, NULL, true); + SETTING_ARRAY("microphone_driver", settings->arrays.microphone_driver, false, NULL, true); + SETTING_ARRAY("microphone_resampler", settings->arrays.microphone_resampler, false, NULL, true); +#endif #ifdef HAVE_LAKKA SETTING_ARRAY("cpu_main_gov", settings->arrays.cpu_main_gov, false, NULL, true); SETTING_ARRAY("cpu_menu_gov", settings->arrays.cpu_menu_gov, false, NULL, true); @@ -2094,6 +2156,15 @@ static struct config_bool_setting *populate_settings_bool( SETTING_BOOL("gcdwebserver_alert", &settings->bools.gcdwebserver_alert, true, true, false); #endif +#ifdef HAVE_MICROPHONE + SETTING_BOOL("microphone_enable", &settings->bools.microphone_enable, true, DEFAULT_MICROPHONE_ENABLE, false); + +#ifdef HAVE_WASAPI + SETTING_BOOL("microphone_wasapi_exclusive_mode", &settings->bools.microphone_wasapi_exclusive_mode, true, DEFAULT_WASAPI_EXCLUSIVE_MODE, false); + SETTING_BOOL("microphone_wasapi_float_format", &settings->bools.microphone_wasapi_float_format, true, DEFAULT_WASAPI_FLOAT_FORMAT, false); +#endif +#endif + *size = count; return tmp; @@ -2419,6 +2490,16 @@ static struct config_uint_setting *populate_settings_uint( SETTING_UINT("steam_rich_presence_format", &settings->uints.steam_rich_presence_format, true, DEFAULT_STEAM_RICH_PRESENCE_FORMAT, false); #endif +#ifdef HAVE_MICROPHONE + SETTING_UINT("microphone_latency", &settings->uints.microphone_latency, false, 0 /* TODO */, false); + SETTING_UINT("microphone_resampler_quality", &settings->uints.microphone_resampler_quality, true, DEFAULT_AUDIO_RESAMPLER_QUALITY_LEVEL, false); + SETTING_UINT("microphone_block_frames", &settings->uints.microphone_block_frames, true, 0, false); + SETTING_UINT("microphone_rate", &settings->uints.microphone_sample_rate, true, DEFAULT_INPUT_RATE, false); +#ifdef HAVE_WASAPI + SETTING_UINT("microphone_wasapi_sh_buffer_length", &settings->uints.microphone_wasapi_sh_buffer_length, true, DEFAULT_WASAPI_MICROPHONE_SH_BUFFER_LENGTH, false); +#endif +#endif + *size = count; return tmp; @@ -2525,6 +2606,9 @@ void config_set_defaults(void *data) int size_settings_size = sizeof(settings->sizes) / sizeof(settings->sizes.placeholder); const char *def_video = config_get_default_video(); const char *def_audio = config_get_default_audio(); +#ifdef HAVE_MICROPHONE + const char *def_microphone = config_get_default_microphone(); +#endif const char *def_audio_resampler = config_get_default_audio_resampler(); const char *def_input = config_get_default_input(); const char *def_joypad = config_get_default_joypad(); @@ -2629,6 +2713,16 @@ void config_set_defaults(void *data) configuration_set_string(settings, settings->arrays.audio_driver, def_audio); +#ifdef HAVE_MICROPHONE + if (def_microphone) + configuration_set_string(settings, + settings->arrays.microphone_driver, + def_microphone); + if (def_audio_resampler) /* not a typo, microphone's default sampler is the same as audio's */ + configuration_set_string(settings, + settings->arrays.microphone_resampler, + def_audio_resampler); +#endif if (def_audio_resampler) configuration_set_string(settings, settings->arrays.audio_resampler, @@ -2698,11 +2792,24 @@ void config_set_defaults(void *data) settings->uints.audio_latency = g_defaults.settings_out_latency; + if (!g_defaults.settings_in_latency) + g_defaults.settings_in_latency = DEFAULT_IN_LATENCY; + + audio_set_float(AUDIO_ACTION_VOLUME_GAIN, settings->floats.audio_volume); #ifdef HAVE_AUDIOMIXER audio_set_float(AUDIO_ACTION_MIXER_VOLUME_GAIN, settings->floats.audio_mixer_volume); #endif +#ifdef HAVE_MICROPHONE + if (DEFAULT_MICROPHONE_DEVICE) + configuration_set_string(settings, + settings->arrays.microphone_device, + DEFAULT_MICROPHONE_DEVICE); + + settings->uints.microphone_latency = g_defaults.settings_in_latency; +#endif + #ifdef HAVE_LAKKA configuration_set_bool(settings, settings->bools.ssh_enable, filestream_exists(LAKKA_SSH_PATH)); diff --git a/configuration.h b/configuration.h index 10ba1e6d00..829c914d5e 100644 --- a/configuration.h +++ b/configuration.h @@ -160,6 +160,14 @@ typedef struct settings unsigned audio_block_frames; unsigned audio_latency; +#ifdef HAVE_MICROPHONE + unsigned microphone_sample_rate; + unsigned microphone_block_frames; + unsigned microphone_latency; + unsigned microphone_wasapi_sh_buffer_length; + unsigned microphone_resampler_quality; +#endif + unsigned fps_update_interval; unsigned memory_update_interval; @@ -435,6 +443,12 @@ typedef struct settings char input_keyboard_layout[64]; +#ifdef HAVE_MICROPHONE + char microphone_driver[32]; + char microphone_resampler[32]; + char microphone_device[255]; +#endif + #ifdef ANDROID char input_android_physical_keyboard[255]; #endif @@ -600,6 +614,15 @@ typedef struct settings bool audio_fastforward_mute; bool audio_fastforward_speedup; +#ifdef HAVE_MICROPHONE + /* Microphone */ + bool microphone_enable; +#ifdef HAVE_WASAPI + bool microphone_wasapi_exclusive_mode; + bool microphone_wasapi_float_format; +#endif +#endif + /* Input */ bool input_remap_binds_enable; bool input_autodetect_enable; @@ -1041,6 +1064,17 @@ const char *config_get_default_video(void); **/ const char *config_get_default_audio(void); +#if defined(HAVE_MICROPHONE) +/** + * config_get_default_microphone: + * + * Gets default microphone driver. + * + * Returns: Default microphone driver. + **/ +const char *config_get_default_microphone(void); +#endif + /** * config_get_default_audio_resampler: * diff --git a/defaults.h b/defaults.h index ee3e851d1a..6d2330158d 100644 --- a/defaults.h +++ b/defaults.h @@ -80,6 +80,7 @@ struct defaults #endif #endif int settings_out_latency; + int settings_in_latency; #ifdef HAVE_MENU unsigned menu_materialui_menu_color_theme; #endif diff --git a/driver.h b/driver.h index 8eef1776a7..93efae9b73 100644 --- a/driver.h +++ b/driver.h @@ -41,7 +41,8 @@ enum DRIVER_BLUETOOTH, DRIVER_WIFI, DRIVER_LED, - DRIVER_MIDI + DRIVER_MIDI, + DRIVER_MICROPHONE }; enum @@ -56,7 +57,41 @@ enum DRIVER_BLUETOOTH_MASK = 1 << DRIVER_BLUETOOTH, DRIVER_WIFI_MASK = 1 << DRIVER_WIFI, DRIVER_LED_MASK = 1 << DRIVER_LED, - DRIVER_MIDI_MASK = 1 << DRIVER_MIDI + DRIVER_MIDI_MASK = 1 << DRIVER_MIDI, + DRIVER_MICROPHONE_MASK = 1 << DRIVER_MICROPHONE +}; + +/** + * These flags indicate special requirements or requests + * of a driver's setup or teardown process. + * + * They are passed to \c drivers_init and \c driver_deinit. + * Not all drivers will need them. + * + * @see drivers_init + * @see driver_deinit + */ +enum driver_lifetime_flags +{ + /** + * Indicates that the driver is being reset. + * When passed \c driver_deinit, indicates that the targeted drivers + * are about to be reinitialized. + * When passed to \c driver_init, indicates that the targeted drivers + * are in the middle of being reinitialized. + * + * This is useful for drivers that provide core-accessible resource handles, + * such as the microphone driver. + * When closed by normal means, such drivers will de-allocate the resources + * that their opened handles represent. + * If the game isn't being exited, then these resources would effectively + * be closed while the core might still be using them. + * + * This flag can be used to ensure that existing core-accessible handles + * are reinitialized with valid resources + * before the core notices that anything's wrong. + */ + DRIVER_LIFETIME_RESET = 1 << 0 }; enum driver_ctl_state @@ -112,7 +147,7 @@ void driver_set_nonblock_state(void); * @flags determines which drivers get initialized. **/ void drivers_init(settings_t *settings, int flags, - bool verbosity_enabled); + enum driver_lifetime_flags lifetime_flags, bool verbosity_enabled); /** * Driver ownership - set this to true if the platform in @@ -129,7 +164,7 @@ void drivers_init(settings_t *settings, int flags, * Typically, if a driver intends to make use of this, it should * set this to true at the end of its 'init' function. **/ -void driver_uninit(int flags); +void driver_uninit(int flags, enum driver_lifetime_flags lifetime_flags); void retro_input_poll_null(void); diff --git a/gfx/video_driver.c b/gfx/video_driver.c index 3e0188c42c..17999ba1e1 100644 --- a/gfx/video_driver.c +++ b/gfx/video_driver.c @@ -3843,12 +3843,12 @@ static void video_driver_reinit_context(settings_t *settings, int flags) video_st->hw_render_context_negotiation; memcpy(&hwr_copy, hwr, sizeof(hwr_copy)); - driver_uninit(flags); + driver_uninit(flags, DRIVER_LIFETIME_RESET); memcpy(hwr, &hwr_copy, sizeof(*hwr)); video_st->hw_render_context_negotiation = iface; - drivers_init(settings, flags, verbosity_is_enabled()); + drivers_init(settings, flags, DRIVER_LIFETIME_RESET, verbosity_is_enabled()); } void video_driver_reinit(int flags) diff --git a/griffin/griffin.c b/griffin/griffin.c index 6838ba349d..59819eb40f 100644 --- a/griffin/griffin.c +++ b/griffin/griffin.c @@ -836,6 +836,9 @@ RSOUND AUDIO ============================================================ */ #include "../audio/audio_driver.c" +#ifdef HAVE_MICROPHONE +#include "../audio/microphone_driver.c" +#endif #if defined(__PS3__) || defined (__PSL1GHT__) #include "../audio/drivers/ps3_audio.c" #elif defined(XENON) @@ -864,6 +867,9 @@ AUDIO #if defined(HAVE_SDL2) #include "../audio/drivers/sdl_audio.c" +#ifdef HAVE_MICROPHONE +#include "../audio/drivers_microphone/sdl_microphone.c" +#endif #endif #ifdef HAVE_DSOUND @@ -872,6 +878,11 @@ AUDIO #ifdef HAVE_WASAPI #include "../audio/drivers/wasapi.c" +#include "../audio/common/wasapi.c" + +#ifdef HAVE_MICROPHONE +#include "../audio/drivers_microphone/wasapi.c" +#endif #endif #ifdef HAVE_SL @@ -883,7 +894,14 @@ AUDIO #include "../audio/drivers/alsa_qsa.c" #else #include "../audio/drivers/alsa.c" +#include "../audio/common/alsa.c" #include "../audio/drivers/alsathread.c" +#include "../audio/common/alsathread.c" + +#ifdef HAVE_MICROPHONE +#include "../audio/drivers_microphone/alsa.c" +#include "../audio/drivers_microphone/alsathread.c" +#endif #endif #endif @@ -1459,6 +1477,8 @@ XML ============================================================ */ #include "../libretro-common/audio/conversion/s16_to_float.c" #include "../libretro-common/audio/conversion/float_to_s16.c" +#include "../libretro-common/audio/conversion/stereo_to_mono_float.c" +#include "../libretro-common/audio/conversion/mono_to_stereo_float.c" #ifdef HAVE_AUDIOMIXER #include "../libretro-common/audio/audio_mix.c" #endif diff --git a/intl/msg_hash_chs.c b/intl/msg_hash_chs.c index ab0ac726ee..4e21df0feb 100644 --- a/intl/msg_hash_chs.c +++ b/intl/msg_hash_chs.c @@ -831,6 +831,22 @@ int msg_hash_get_help_chs_enum(enum msg_hash_enums msg, char *s, size_t len) strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_VALUE_NO_INFORMATION_AVAILABLE), len); } break; +#ifdef HAVE_MICROPHONE + case MENU_ENUM_LABEL_MICROPHONE_RESAMPLER_DRIVER: + { + const char *lbl = settings ? settings->arrays.microphone_resampler : NULL; + + if (string_is_equal(lbl, msg_hash_to_str(MENU_ENUM_LABEL_AUDIO_RESAMPLER_DRIVER_SINC))) + strlcpy(s, + "Windowed SINC implementation.", len); + else if (string_is_equal(lbl, msg_hash_to_str(MENU_ENUM_LABEL_AUDIO_RESAMPLER_DRIVER_CC))) + strlcpy(s, + "Convoluted Cosine implementation.", len); + else if (string_is_empty(s)) + strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_VALUE_NO_INFORMATION_AVAILABLE), len); + } + break; +#endif case MENU_ENUM_LABEL_VIDEO_SHADER_PRESET: snprintf(s, len, "载入预设渲染器. \n" diff --git a/intl/msg_hash_lbl.h b/intl/msg_hash_lbl.h index a38c1d8fcd..fe4af4408f 100644 --- a/intl/msg_hash_lbl.h +++ b/intl/msg_hash_lbl.h @@ -337,6 +337,56 @@ MSG_HASH( MENU_ENUM_LABEL_AUDIO_WASAPI_SH_BUFFER_LENGTH, "audio_wasapi_sh_buffer_length" ) +#ifdef HAVE_MICROPHONE +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_ENABLE, + "microphone_enable" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_RESAMPLER_DRIVER, + "microphone_resampler_driver" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_DEVICE, + "microphone_device" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_SETTINGS, + "microphone_settings" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_DRIVER, + "microphone_driver" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_INPUT_RATE, + "microphone_input_rate" +) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_LATENCY, + "microphone_latency" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_BLOCK_FRAMES, + "microphone_block_frames" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_WASAPI_EXCLUSIVE_MODE, + "microphone_wasapi_exclusive_mode" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_WASAPI_FLOAT_FORMAT, + "microphone_wasapi_float_format" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_WASAPI_SH_BUFFER_LENGTH, + "microphone_wasapi_sh_buffer_length" + ) +MSG_HASH( + MENU_ENUM_LABEL_MICROPHONE_RESAMPLER_QUALITY, + "microphone_resampler_quality" + ) +#endif MSG_HASH( MENU_ENUM_LABEL_AUTOSAVE_INTERVAL, "autosave_interval" @@ -979,6 +1029,16 @@ MSG_HASH( MENU_ENUM_LABEL_DEFERRED_DROPDOWN_BOX_LIST_AUDIO_DEVICE, "deferred_dropdown_box_list_audio_device" ) +#ifdef HAVE_MICROPHONE +MSG_HASH( + MENU_ENUM_LABEL_DEFERRED_DROPDOWN_BOX_LIST_MICROPHONE_DEVICE, + "deferred_dropdown_box_list_microphone_device" + ) +MSG_HASH( + MENU_ENUM_LABEL_DEFERRED_MICROPHONE_SETTINGS_LIST, + "deferred_microphone_settings_list" + ) +#endif MSG_HASH( MENU_ENUM_LABEL_DEFERRED_CONFIGURATIONS_LIST, "deferred_configurations_list" diff --git a/intl/msg_hash_us.c b/intl/msg_hash_us.c index cea29e019f..517cff9046 100644 --- a/intl/msg_hash_us.c +++ b/intl/msg_hash_us.c @@ -244,6 +244,22 @@ int msg_hash_get_help_us_enum(enum msg_hash_enums msg, char *s, size_t len) strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_VALUE_NO_INFORMATION_AVAILABLE), len); } break; +#ifdef HAVE_MICROPHONE + case MENU_ENUM_LABEL_MICROPHONE_RESAMPLER_DRIVER: + { + const char *lbl = settings ? settings->arrays.microphone_resampler : NULL; + + if (string_is_equal(lbl, msg_hash_to_str(MENU_ENUM_LABEL_AUDIO_RESAMPLER_DRIVER_SINC))) + strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_HELP_AUDIO_RESAMPLER_DRIVER_SINC), len); + else if (string_is_equal(lbl, msg_hash_to_str(MENU_ENUM_LABEL_AUDIO_RESAMPLER_DRIVER_CC))) + strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_HELP_AUDIO_RESAMPLER_DRIVER_CC), len); + else if (string_is_equal(lbl, msg_hash_to_str(MENU_ENUM_LABEL_AUDIO_RESAMPLER_DRIVER_NEAREST))) + strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_HELP_AUDIO_RESAMPLER_DRIVER_NEAREST), len); + else + strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_VALUE_NO_INFORMATION_AVAILABLE), len); + } + break; +#endif case MENU_ENUM_LABEL_VIDEO_SHADER_PRESET: strlcpy(s, msg_hash_to_str(MENU_ENUM_LABEL_HELP_VIDEO_SHADER_PRESET), len); break; diff --git a/intl/msg_hash_us.h b/intl/msg_hash_us.h index 11c4e503f5..4afc43fe2a 100644 --- a/intl/msg_hash_us.h +++ b/intl/msg_hash_us.h @@ -1162,7 +1162,7 @@ MSG_HASH( ) MSG_HASH( MENU_ENUM_SUBLABEL_AUDIO_SETTINGS, - "Change audio output settings." + "Change audio input/output settings." ) MSG_HASH( MENU_ENUM_LABEL_VALUE_INPUT_SETTINGS, @@ -1632,6 +1632,20 @@ MSG_HASH( MENU_ENUM_LABEL_HELP_AUDIO_DRIVER_JACK, "Jack Audio Connection Kit driver." ) +#ifdef HAVE_MICROPHONE +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_DRIVER, + "Microphone" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_DRIVER, + "Microphone driver to use." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_RESAMPLER_DRIVER, + "Microphone Resampler" + ) +#endif MSG_HASH( MENU_ENUM_LABEL_VALUE_AUDIO_RESAMPLER_DRIVER, "Audio Resampler" @@ -2481,6 +2495,16 @@ MSG_HASH( MENU_ENUM_SUBLABEL_AUDIO_OUTPUT_SETTINGS, "Change audio output settings." ) +#ifdef HAVE_MICROPHONE +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_SETTINGS, + "Microphone" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_SETTINGS, + "Change audio input settings." + ) +#endif MSG_HASH( MENU_ENUM_LABEL_VALUE_AUDIO_RESAMPLER_SETTINGS, "Resampler" @@ -2661,6 +2685,78 @@ MSG_HASH( "Desired audio latency in milliseconds. Might not be honored if the audio driver can't provide given latency." ) +#ifdef HAVE_MICROPHONE +/* Settings > Audio > Input */ +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_ENABLE, + "Microphone" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_ENABLE, + "Enable audio input in supported cores. Has no overhead if the core isn't using a microphone." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_DEVICE, + "Device" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_DEVICE, + "Override the default input device the microphone driver uses. This is driver dependent." + ) +MSG_HASH( + MENU_ENUM_LABEL_HELP_MICROPHONE_DEVICE, + "Override the default input device the microphone driver uses. This is driver dependent." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_RESAMPLER_QUALITY, + "Resampler Quality" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_RESAMPLER_QUALITY, + "Lower this value to favor performance/lower latency over audio quality, increase for better audio quality at the expense of performance/lower latency." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_INPUT_RATE, + "Default Input Rate (Hz)" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_INPUT_RATE, + "Audio input sample rate, used if a core doesn't request a specific number." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_LATENCY, + "Audio Input Latency (ms)" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_LATENCY, + "Desired audio input latency in milliseconds. Might not be honored if the microphone driver can't provide given latency." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_WASAPI_EXCLUSIVE_MODE, + "WASAPI Exclusive Mode" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_WASAPI_EXCLUSIVE_MODE, + "Allow RetroArch to take exclusive control of the microphone device when using the WASAPI microphone driver. If disabled, RetroArch will use shared mode instead." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_WASAPI_FLOAT_FORMAT, + "WASAPI Float Format" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_WASAPI_FLOAT_FORMAT, + "Use floating-point input for the WASAPI driver, if supported by your audio device." + ) +MSG_HASH( + MENU_ENUM_LABEL_VALUE_MICROPHONE_WASAPI_SH_BUFFER_LENGTH, + "WASAPI Shared Buffer Length" + ) +MSG_HASH( + MENU_ENUM_SUBLABEL_MICROPHONE_WASAPI_SH_BUFFER_LENGTH, + "The intermediate buffer length (in frames) when using the WASAPI driver in shared mode." + ) +#endif + /* Settings > Audio > Resampler */ MSG_HASH( diff --git a/libretro-common/audio/conversion/float_to_s16.c b/libretro-common/audio/conversion/float_to_s16.c index 770981d9c1..6f65fe1588 100644 --- a/libretro-common/audio/conversion/float_to_s16.c +++ b/libretro-common/audio/conversion/float_to_s16.c @@ -97,22 +97,25 @@ void convert_float_to_s16(int16_t *out, size_t i = 0; #if defined(__SSE2__) __m128 factor = _mm_set1_ps((float)0x8000); + /* Initialize a 4D vector with 32768.0 for its elements */ for (i = 0; i + 8 <= samples; i += 8, in += 8, out += 8) - { - __m128 input_l = _mm_loadu_ps(in + 0); - __m128 input_r = _mm_loadu_ps(in + 4); - __m128 res_l = _mm_mul_ps(input_l, factor); - __m128 res_r = _mm_mul_ps(input_r, factor); - __m128i ints_l = _mm_cvtps_epi32(res_l); - __m128i ints_r = _mm_cvtps_epi32(res_r); - __m128i packed = _mm_packs_epi32(ints_l, ints_r); + { /* Skip forward 8 samples at a time... */ + __m128 input_a = _mm_loadu_ps(in + 0); /* Create a 4-float vector from the next four samples... */ + __m128 input_b = _mm_loadu_ps(in + 4); /* ...and another from the *next* next four. */ + __m128 res_a = _mm_mul_ps(input_a, factor); + __m128 res_b = _mm_mul_ps(input_b, factor); /* Multiply these samples by 32768 */ + __m128i ints_a = _mm_cvtps_epi32(res_a); + __m128i ints_b = _mm_cvtps_epi32(res_b); /* Convert the samples to 32-bit integers */ + __m128i packed = _mm_packs_epi32(ints_a, ints_b); /* Then convert them to 16-bit ints, clamping to [-32768, 32767] */ - _mm_storeu_si128((__m128i *)out, packed); + _mm_storeu_si128((__m128i *)out, packed); /* Then put the result in the output array */ } samples = samples - i; i = 0; + /* If there are any stray samples at the end, we need to convert them + * (maybe the original array didn't contain a multiple of 8 samples) */ #elif defined(__ALTIVEC__) int samples_in = samples; @@ -165,6 +168,8 @@ void convert_float_to_s16(int16_t *out, } #endif + /* This loop converts stray samples to the right format, + * but it's also a fallback in case no SIMD instructions are available. */ for (; i < samples; i++) { int32_t val = (int32_t)(in[i] * 0x8000); diff --git a/libretro-common/audio/conversion/mono_to_stereo_float.c b/libretro-common/audio/conversion/mono_to_stereo_float.c new file mode 100644 index 0000000000..838d8c2a44 --- /dev/null +++ b/libretro-common/audio/conversion/mono_to_stereo_float.c @@ -0,0 +1,44 @@ +/* Copyright (C) 2010-2023 The RetroArch team + * + * --------------------------------------------------------------------------------------- + * The following license statement only applies to this file (mono_to_stereo.c). + * --------------------------------------------------------------------------------------- + * + * Permission is hereby granted, free of charge, + * to any person obtaining a copy of this software and associated documentation files (the "Software"), + * to deal in the Software without restriction, including without limitation the rights to + * use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, + * and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, + * INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. + * IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, + * WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. + */ +#include +#include + +#include