MMIR Framework

Class: MediaManager

mmir. MediaManager

mmir.MediaManager

manager/mediaManager.js, line 3
The MediaManager gives access to audio in- and output functionality. Depending on its configuration, the MediaManager loads different implementation modules (plugins) that realize the interface-functions differently. See directory mmirf/env/media for available plugins. This "class" is a singleton - so that only one instance is in use.

Members

_logmmir.tools.Logger

A logger for the MediaManager and its plugins/modules.

This logger MAY be used by media-plugins and / or tools and helpers related to the MediaManager.

This logger SHOULD NOT be used by "code" that non-related to the MediaManager

Default Value:
  • mmir.Logger (logger instance for mmir.MediaManager)

ctxObject

Execution context for plugins TODO add doc
Default Value:
  • Object (empty context, i.e. plugins are loaded into the "root context", and no plugins loaded into the execution context)

waitReadyImplmmir.env.media.IWaitReadyIndicator

Wait indicator, e.g. for speech input:

provides 2 functions:
preparing(): if called, the implementation indicates that the "user should wait"
ready(): if called, the implementation stops indicating that the "user should wait" (i.e. that the system is ready for user input now)

If not set (or functions are not available) will do nothing

Default Value:
  • Object (no implementation set)
See:
Example
//define custom wait/ready implementation:
var impl = {
	preparing: function(str){
		console.log('Media module '+str+' is preparing...');
	},
	ready: function(str){
		console.log('Media module '+str+' is ready now!');
	}
};

//configure MediaManager to use custom implementation:
mmir.MediaManager.waitReadyImpl = impl;

//-> now plugins that call  mmir.MediaManager._preparing() and  mmir.MediaManager._ready()
//   will invoke the custom implementation's functions.

Methods

protected_addListenerObserver(eventName, observerCallback)

manager/mediaManager.js, line 1283
Add an observer for registration / removal of event-handler. The observer gets notified,when handlers are registered / removed for the event. The observer-callback function will be called with the following arguments (eventName, ACTION_TYPE, eventHandler) where
  • eventName: String the name of the event that should be observed
  • ACTION_TYPE: the type of action: "added" if the handler was registered for the event, "removed" if the the handler was removed
  • eventHandler: the handler function that was registered or removed
Name Type Description
eventName String
observerCallback function
See:

protected_emitEvent(eventName, args)

manager/mediaManager.js, line 1236
Helper for firing / triggering an event. This should only be used by media plugins (that handle the eventName).
Name Type Description
eventName String
args any optional repeatable OPTIONAL the arguments (event data) with which the event-handlers will be called.
See:
Example
// will invoke listeners for "someevent":
mmir.media._emitEvent('someevent');
// will invoke listeners for "otherevent" with the 2 event data arguments:
mmir.media._emitEvent('otherevent', withTwo, dataParameters);

protected_fireEvent(eventName, argsArray)

manager/mediaManager.js, line 1209
Helper for firing / triggering an event. This should only be used by media plugins (that handle the eventName).
Name Type Description
eventName String
argsArray Array the list of arguments with which the event-handlers will be called.
Deprecated
See:

protected_get_mmir(){mmir}

manager/mediaManager.js, line 1927
Get the core mmir-lib from the global namespace. NOTE: may be undefined, if the core is not visible in the global namespace.
Returns:
Type Description
mmir the core mmir-lib (or undefined, if the mmir-lib is not available in the global namespace)
Example
NOTE should only be used by plugin implementations.

protected_notifyObservers(eventName, actionType, eventHandler)

manager/mediaManager.js, line 1255
Helper for notifying listener-observers about changes (adding/removing listeners). This should only be used by media plugins (that handle the eventName).
Name Type Description
eventName String
actionType String the change-type that occurred for the event/event-handler: one of ["added" | "removed"].
eventHandler function the event-handler function that has changed.
See:

protected_preparing(moduleName)

manager/mediaManager.js, line 1505
This function is called by media plugin implementations (i.e. modules) to indicate that they are preparing something and that the user should wait.

The actual implementation for _preparing(String) is given by waitReadyImpl.preparing (if not set, then calling _preparing(String) will have no effect.

Name Type Description
moduleName String the module name from which the function was invoked
See:

protected_ready(moduleName)

manager/mediaManager.js, line 1529
This function is called by media plugin implementations (i.e. modules) to indicate that they are now ready and that the user can start interacting.

The actual implementation for _ready(String) is given by the waitReadyImpl implementation (if not set, then calling _ready(String) will have no effect.

Name Type Description
moduleName String the module name from which the function was invoked
See:

protected_removeListenerObserver()

manager/mediaManager.js, line 1293
Remove an observer that gets notified on registration / removal of event-handler.
See:

addListener(eventName, eventHandler)

manager/mediaManager.js, line 1127
Adds the handler-function for the event. This function calls _notifyObservers for the eventName with actionType "added". Event names (and firing events) are specific to the loaded media plugins. TODO list events that the default media-plugins support * "miclevelchanged": fired by AudioInput plugins that support querying the microphone (audio input) levels A plugin can tigger / fire events using the helper _emitEvent of the MediaManager. Media plugins may observe registration / removal of listeners via _addListenerObserver and _removeListenerObserver. Or get and iterate over listeners via getListeners.
Name Type Description
eventName String
eventHandler function

cancelRecognition()

manager/mediaManager.js, line 721
Cancel currently active speech recognition. Has no effect, if no recognition is active.

cancelSpeech()

manager/mediaManager.js, line 1009
Cancel current synthesis.

createEmptyAudio(){mmir.env.media.IAudio}

manager/mediaManager.js, line 905
Get an empty audio object. This can be used as dummy or placeholder for a "real" audio object. The audio object exports the following functions: 
play()
stop()
release()
enable()
disable()
setVolume(number)
getDuration()
isPaused()
isEnabled()
Note: enable() and disable() will set the internal enabled-state, which can be queried via isEnabled(). play() and stop() will set the internal playing-state, which can be queried via isPaused() (note however, that this empty audio does not actually play anything. setVolume() sets the internal volume-value. getDuration() will always return 0.
See:
Returns:
Type Description
mmir.env.media.IAudio the audio

getAudio()

manager/mediaManager.js, line 861
Get an audio object for the audio file specified by URL URL or by WAV data. NOTE that getWAVAsAudio may not be supported by all modules! Convenience function for getURLAsAudio and getWAVAsAudio: if first argument is a String, then getURLAsAudio will be invoked, otherwise getWAVAsAudio (if the module supports this function).

getFunc(ctx, funcName)

manager/mediaManager.js, line 1408
Returns function funcName from "sub-module" ctx.

If there is no funcName in "sub-module" ctx, then funcName from the "main-module" (i.e. from the MediaManager instance itself) will be returned.

NOTE that the returned functions will always execute within the context of the MediaManager instance (i.e. this will refer to the MediaManager instance).

Name Type Description
ctx String the execution context, i.e. "sub-module", in which to execute funcName.
If falsy, the "root-module" will used as execution context.
funcName String the function name
Throws:
if funcName does not exist in the requested Execution context.
Or if ctx is not falsy but there is no valid execution context ctx in MediaManager.
Type
ReferenceError
Example
//same as mmir.MediaManager.ctx.android.textToSpeech("...", function...):
	mmir.MediaManager.getFunc("android", "textToSpeech")("some text to read out loud",
		function onFinished(){ console.log("finished reading."); }
	);

 //same as mmir.MediaManager.textToSpeech("...", function...):
 //... IF the defaultExecId is falsy
 //    (i.e. un-changed or set to falsy value via setDefaultExec())
	mmir.MediaManager.getFunc(null, "textToSpeech")("some text to read out loud",
		function onFinished(){ console.log("finished reading."); }
	);

getListeners(){Array.<function()>}

manager/mediaManager.js, line 1181
Get list of registered listeners / handlers for an event.
Returns:
Type Description
Array.<function()> of event-handlers. Empty, if there are no event handlers for eventName

getRecognitionLanguages()

manager/mediaManager.js, line 739
get list of supported languages for ASR (may not be supported by all plugins).

getSpeechLanguages()

manager/mediaManager.js, line 1028
get list of supported languages for TTS (may not be supported by all plugins).

getURLAsAudio(url, onPlayedCallback, failureCallback, onLoadedCallback){mmir.env.media.IAudio}

manager/mediaManager.js, line 837
Get an audio object for the audio file specified by URL. The audio object exports the following functions: 
play()
stop()
release()
enable()
disable()
setVolume(number)
getDuration()
isPaused()
isEnabled()
NOTE: the audio object should only be used, after the onLoadedCallback was triggered.
Name Type Description
url String
onPlayedCallback function optional OPTIONAL
failureCallback function optional OPTIONAL
onLoadedCallback function optional OPTIONAL
See:
Returns:
Type Description
mmir.env.media.IAudio the audio

getVoices(options, successCallback, failureCallback)

manager/mediaManager.js, line 1062
get list of supported voices for TTS (may not be supported by all plugins).
Name Type Description
options String | VoiceOptions optional OPTIONAL if String, the language code (optionally with country code) for which the voices should be listed. if VoiceOptions: options.language: {String} OPTIONAL the language code options.details: {Boolean} OPTIONAL if TRUE the returned list contains VoiceDetail objects with {name: STRING, language: STRING, gender: "female" | "male" | "unknown"}
successCallback function the success callback: successCallback(Array)
failureCallback function the error callback: failureCallback(err)

hasListeners(){Boolean}

manager/mediaManager.js, line 1192
Check if at least one listener / handler is registered for the event.
Returns:
Type Description
Boolean true if at least 1 handler is registered for eventName; otherwise false.

asyncinit(successCallback, failureCallback, listenerList){Object}

manager/mediaManager.js, line 1821
Object containing the instance of the class {{#crossLink "audioInput"}}{{/crossLink}} If listenerList is provided, each listener will be registered after the instance is initialized, but before media-plugins (i.e. environment specfific implementations) are loaded. Each entry in the listenerList must have fields name (String) and listener (Function), where
name: is the name of the event
listener: is the listener implementation (the signature/arguments of the listener function depends on the specific event for which the listener will be registered)
Name Type Description
successCallback function optional OPTIONAL callback that gets triggered after the MediaManager instance has been initialized.
failureCallback function optional OPTIONAL a failure callback that gets triggered if an error occurs during initialization.
listenerList Array.<Object> optional OPTIONAL a list of listeners that should be registered, where each entry is an Object with properties: 
					{
						name: String the event name,
						listener: Function the handler function
					}
Returns:
Type Description
Object a Deferred object that gets resolved, after the mmir.MediaManager has been initialized.

protectedloadFile()

manager/mediaManager.js, line 1909
loads a file. If the file implements a function initialize(f) where the function f is called with a set of functions e, then those functions in e are added to the visibility of audioInput, and will from now on be applicable by calling mmir.MediaManager.(). *
Example
NOTE should only be used by plugin implementations for loading (dependent/sub-) plugins.

off(eventName, eventHandler)

manager/mediaManager.js, line 1172
Add an event listener.
Name Type Description
eventName String the name of the event
eventHandler function the event handler / callback function
See:

on(eventName, eventHandler)

manager/mediaManager.js, line 1159
Add an event listener.
Name Type Description
eventName String the name of the event
eventHandler function the event handler / callback function
See:

perform(ctx, funcName, args)

manager/mediaManager.js, line 1334
Executes function funcName in "sub-module" ctx with arguments args.

If there is no funcName in "sub-module" ctx, then funcName from the "main-module" (i.e. from the MediaManager instance itself) will be used.

Name Type Description
ctx String the execution context, i.e. "sub-module", in which to execute funcName.
If falsy, the "root-module" will used as execution context.
funcName String the function name
args Array the arguments for function "packaged" in an array
Throws:
if funcName does not exist in the requested Execution context.
Or if ctx is not falsy but there is no valid execution context ctx in MediaManager.
Type
ReferenceError
Example
//same as mmir.MediaManager.ctx.android.textToSpeech("...", function...):
	mmir.MediaManager.perform("android", "textToSpeech", ["some text to read out loud",
		function onFinished(){ console.log("finished reading."); }
	]);

 //same as mmir.MediaManager.textToSpeech("...", function...)
 //... IF the defaultExecId is falsy
 //    (i.e. un-changed or set to falsy value via setDefaultExec())
	mmir.MediaManager.perform(null, "textToSpeech", ["some text to read out loud",
		function onFinished(){ console.log("finished reading."); }
	]);

play()

manager/mediaManager.js, line 799
Play audio file from the specified URL or WAV data. Convenience function for playWAV and playURL: if first argument is a String, then playURL will be invoked, otherwise playWAV.

playURL()

manager/mediaManager.js, line 777
Play audio file from the specified URL.

playWAV()

manager/mediaManager.js, line 759
Play PCM audio data.

asyncrecognize(options, statusCallback, failureCallback)

manager/mediaManager.js, line 547
Start speech recognition with end-of-speech detection: the recognizer automatically tries to detect when speech has finished and triggers the status-callback accordingly with results.
Name Type Description
options PlainObject optional OPTIONAL options for Automatic Speech Recognition: 
{
			  success: OPTIONAL Function, the status-callback (see arg statusCallback)
			, error: OPTIONAL Function, the error callback (see arg failureCallback)
			, language: OPTIONAL String, the language for recognition (if omitted, the current language setting is used)
			, intermediate: OTPIONAL Boolean, set true for receiving intermediate results (NOTE not all ASR engines may support intermediate results)
			, results: OTPIONAL Number, set how many recognition alternatives should be returned at most (NOTE not all ASR engines may support this option)
			, mode: OTPIONAL "search" | "dictation", set how many recognition alternatives should be returned at most (NOTE not all ASR engines may support this option)
			, eosPause: OTPIONAL "short" | "long", length of pause after speech for end-of-speech detection (NOTE not all ASR engines may support this option)
			, disableImprovedFeedback: OTPIONAL Boolean, disable improved feedback when using intermediate results (NOTE not all ASR engines may support this option)
		}
statusCallback function optional OPTIONAL callback function that is triggered when, recognition starts, text results become available, and recognition ends. The callback signature is: 
				callback(
					text: String | "",
					confidence: Number | Void,
					status: "FINAL"|"INTERIM"|"INTERMEDIATE"|"RECORDING_BEGIN"|"RECORDING_DONE",
					alternatives: Array<{result: String, score: Number}> | Void,
					unstable: String | Void
				)
Usually, for status "FINAL" | "INTERIM" | "INTERMEDIATE" text results are returned, where 
			  "INTERIM": an interim result, that might still change
			  "INTERMEDIATE": a stable, intermediate result
			  "FINAL": a (stable) final result, before the recognition stops
If present, the unstable argument provides a preview for the currently processed / recognized text.
NOTE that when using intermediate mode, status-calls with "INTERMEDIATE" may contain "final intermediate" results, too.
NOTE: if used in combination with options.success, this argument will supersede the options
failureCallback function optional OPTIONAL callback function that is triggered when an error occurred. The callback signature is: callback(error)
NOTE: if used in combination with options.error, this argument will supersede the options

removeListener(eventName, eventHandler){Boolean}

manager/mediaManager.js, line 1145
Removes the handler-function for the event. Calls _notifyObservers for the eventName with actionType "removed", if the handler was actually removed.
Name Type Description
eventName String
eventHandler function
Returns:
Type Description
Boolean true if the handler function was actually removed, and false otherwise.

setDefaultCtx(ctxId)

manager/mediaManager.js, line 1479
Set the default execution context. If not explicitly set, or set to a falsy value, then the "root" execution context is the default context.
Name Type Description
ctxId String the new default excution context for loaded media modules (if falsy the default context will be the "root context")
Throws:
if ctxId is no valid context
Type
ReferenceError
Example
//if context "nuance" exists:
mmir.MediaManager.setDefaultCtx("nuance")

// -> now the following calls are equal to mmir.MediaManager.ctx.nuance.textToSpeech("some text")
mmir.MediaManager.perform(null, "textToSpeech", ["some text"]);
mmir.MediaManager.getFunc(null, "textToSpeech")("some text");

//reset to root context:
mmir.MediaManager.setDefaultCtx(false);

// -> now the following call is equal to mmir.MediaManager.textToSpeech("some text") again
mmir.MediaManager.perform("textToSpeech", ["some text"]);

setTextToSpeechVolume(newValue)

manager/mediaManager.js, line 1085
Set the volume for the speech synthesis (text-to-speech).
Name Type Description
newValue Number TODO specify format / range

asyncstartRecord(options, statusCallback, failureCallback)

manager/mediaManager.js, line 622
Start continuous speech recognition: The recognizer continues until stopRecord is called.
Name Type Description
options PlainObject optional OPTIONAL options for Automatic Speech Recognition: 
{
			  success: OPTIONAL Function, the status-callback (see arg statusCallback)
			, error: OPTIONAL Function, the error callback (see arg failureCallback)
			, language: OPTIONAL String, the language for recognition (if omitted, the current language setting is used)
			, intermediate: OTPIONAL Boolean, set true for receiving intermediate results (NOTE not all ASR engines may support intermediate results)
			, results: OTPIONAL Number, set how many recognition alternatives should be returned at most (NOTE not all ASR engines may support this option)
			, mode: OTPIONAL "search" | "dictation", set how many recognition alternatives should be returned at most (NOTE not all ASR engines may support this option)
			, eosPause: OTPIONAL "short" | "long", length of pause after speech for end-of-speech detection (NOTE not all ASR engines may support this option)
			, disableImprovedFeedback: OTPIONAL Boolean, disable improved feedback when using intermediate results (NOTE not all ASR engines may support this option)
		}
statusCallback function optional OPTIONAL callback function that is triggered when, recognition starts, text results become available, and recognition ends. The callback signature is: 
				callback(
					text: String | "",
					confidence: Number | Void,
					status: "FINAL"|"INTERIM"|"INTERMEDIATE"|"RECORDING_BEGIN"|"RECORDING_DONE",
					alternatives: Array<{result: String, score: Number}> | Void,
					unstable: String | Void
				)
Usually, for status "FINAL" | "INTERIM" | "INTERMEDIATE" text results are returned, where 
			  "INTERIM": an interim result, that might still change
			  "INTERMEDIATE": a stable, intermediate result
			  "FINAL": a (stable) final result, before the recognition stops
If present, the unstable argument provides a preview for the currently processed / recognized text.
NOTE that when using intermediate mode, status-calls with "INTERMEDIATE" may contain "final intermediate" results, too.
NOTE: if used in combination with options.success, this argument will supersede the options
failureCallback function optional OPTIONAL callback function that is triggered when an error occurred. The callback signature is: callback(error)
NOTE: if used in combination with options.error, this argument will supersede the options
See:

asyncstopRecord(options, statusCallback, failureCallback)

manager/mediaManager.js, line 694
Stops continuous speech recognition: After startRecord was called, invoking this function will stop the recognition process and return the result by invoking the succesCallback. Note, that the statusCallback may not return an actual text result (i.e. the last text result may have been return in the statusCallback of the startRecord() call)
Name Type Description
options PlainObject optional OPTIONAL options for stopping the Automatic Speech Recognition: 
{
			  success: OPTIONAL Function, the status-callback (see arg statusCallback)
			, error: OPTIONAL Function, the error callback (see arg failureCallback)
		}
statusCallback function optional OPTIONAL callback function that is triggered when, recognition starts, text results become available, and recognition ends. The callback signature is: 
				callback(
					text: String | "",
					confidence: Number | Void,
					status: "FINAL"|"INTERIM"|"INTERMEDIATE"|"RECORDING_BEGIN"|"RECORDING_DONE",
					alternatives: Array<{result: String, score: Number}> | Void,
					unstable: String | Void
				)
Usually, for status "FINAL" | "INTERIM" | "INTERMEDIATE" text results are returned, where 
			  "INTERIM": an interim result, that might still change
			  "INTERMEDIATE": a stable, intermediate result
			  "FINAL": a (stable) final result, before the recognition stops
If present, the unstable argument provides a preview for the currently processed / recognized text.
NOTE that when using intermediate mode (as option in startRecord()), status-calls with "INTERMEDIATE" may contain "final intermediate" results, too.
failureCallback function optional OPTIONAL callback function that is triggered when an error occurred. The callback signature is: callback(error)
See:

textToSpeech()

manager/mediaManager.js, line 991
Deprecated
  • use tts instead

    tts(options, onPlayedCallback, failureCallback, onReadyCallback)

    manager/mediaManager.js, line 966
    Synthesizes ("read out loud") text.
    Name Type Description
    options String | Array.<String> | PlainObject optional OPTIONAL if String or Array of Strings synthesizes the text of the String(s).
    For an Array: each entry is interpreted as "sentence"; after each sentence, a short pause is inserted before synthesizing the the next sentence
    for a PlainObject, the following properties should be used: 
    {
			  text: String | String[], text that should be read aloud
			, pauseDuration: OPTIONAL Number, the length of the pauses between sentences (i.e. for String Arrays) in milliseconds
			, language: OPTIONAL String, the language for synthesis (if omitted, the current language setting is used)
			, voice: OPTIONAL String, the voice (language specific) for synthesis; NOTE that the specific available voices depend on the TTS engine
			, success: OPTIONAL Function, the on-playing-completed callback (see arg onPlayedCallback)
			, error: OPTIONAL Function, the error callback (see arg failureCallback)
			, ready: OPTIONAL Function, the audio-ready callback (see arg onReadyCallback)
		}
    onPlayedCallback function optional OPTIONAL callback that is invoked when the audio of the speech synthesis finished playing: 
    onPlayedCallback()

    NOTE: if used in combination with options.success, this argument will supersede the options
    failureCallback function optional OPTIONAL callback that is invoked in case an error occurred: 
    failureCallback(error: String | Error)

    NOTE: if used in combination with options.error, this argument will supersede the options
    onReadyCallback function optional OPTIONAL callback that is invoked when audio becomes ready / is starting to play. If, after the first invocation, audio is paused due to preparing the next audio, then the callback will be invoked with false, and then with true (as first argument), when the audio becomes ready again, i.e. the callback signature is: 
    onReadyCallback(isReady: Boolean, audio: IAudio)

    NOTE: if used in combination with options.ready, this argument will supersede the options