As the web has developed, use of voice technologies can allow additional inputs and outputs from devices and browsers on those devices. In particular, being able to listen to a page, parts of a page, or use voice to enter text instead of typing could be easier because mobile phone manufcturers already have a lot of development and experience with voice technologies. For the human user, it also increases access, choice which be directly relevant to their immediate location and context when they want to access mobile web information.

"The challenge will be to effect a transition between the textual world and the multimedia, to communicate complex ideas in a manner accessible to people using audio and video. Neither medium favours the abstract (and neither do their consumers today) and each medium imposes channel limitations (you can't skim an audio or video file). It will be necessary, in my view, to evolve a form of language that combines audio, video and text, to in other words combine the subtlety and expressiveness of text with the emotion and immediacy of audio and video. And so when we are thinking about mobile applications, probably the primary challenge will be to enable people not only to receive relevant content in this multimedia language, but to compose, on the fly, with no more real difficulty than (say) speaking, multimedia messages, that can be beamed to colleagues or broadcast to the world."1

For a sample multimodal application:

1. Should it be a mobile web page with a basic form ? This would not appear to be what xhtml+voice forms have already been designed for (e.g. examples so far seem to be things like using a form to find something or order pizza etc), but was wondering whether you could create a basic mobile web page, with options when selected by voice input would return a web page template as a file, to the user either via SMS or the web. If you’re going to be online anyway, would using voice add any value in terms of speeding up the process, so you could edit / create mobile web pages on the go ? Hmmm, not sure, but having both a visual and voice form would make it more accessible.

2. If not a whole mobile web page, could it be a regular web page with voice annotated ‘elements’ on the page that could produce a similar result ? Would it be better than a mobile web form ? Don’t know yet until try.

3. Due to limits of connectivity / bandwidth / cost / time, would it be more useful just to provide a standalone mobile voice application that could produce a set of templates and is it possible ? Could any files or the code be generated by a form similar to the above to generate either a voice or text SMS? (As above, re using voiceSMS - from an accessibility point of view - is better to have both a text and voice option?)

X+V - forms designed in blocks - each question that might be asked has a surrounding block of code, can specify time and tell the browser that you've finished talking or listening. You can put the voicexml directly into the Xhtml coding or reference it as an external voicexml file (based on everything I’ve understood so far, external is better because you can reuse it, even within the same document). You set up grammar files to help with understanding i.e. a file will include typical words that you might expect to hear in response to a prompt/question

IBM leading research and specifications,

Opera

Multimodal web interfaces

Voxeo - for testing - via Skype

Refs:

1. Downes S, mLearning Tools (August 9, 2006)  Available at: http://www.downes.ca/post/38519