Gnomedex 2008
We were honored to be invited to Gnomedex 2008. This conference featured many speakers on a variety of topics related to technology, blogging and the internet in general. More information can be found at the conference web site. Note that the talk is about half an hour and the videos are relatively large.
Winter 2008
After our first university press release, two news organizations came to film segments about the Vocal Joystick. These were sent to local news stations; a version of one of them can be found under the video section on our news page.
06/2007
The following video presents Voicebot, a robotic arm controlled entirely though the Vocal Joystick. The arm itself is a Lynx 6 robotic arm, by Lynxmotion. The two clips in this video demonstrate forward and inverse kinematic control modes, and show how the Vocal Joystick is a viable option for 3-D control of a robotic limb.
VoicebotThe first clip in the video demonstrates forward kinematic control. In this method, each joint angle is controlled explicitly by a vocal parameter, and the "ck" sound changes from shoulder and elbow joint control to wrist control. In the second video clip, inverse kinematic control is used to position the gripper in cartesian space using vocal parameters, and again the "ck" sound toggles changes in gross positioning to changes in wrist orientation. Both clips demonstrate picking up and moving a small object on a table, where the 'ch' discrete sound is used to open and close the gripper.
10/2006
The following videos were shown as a part of the presentation of the Vocal Joystick system at the ASSETS 2006 conference in Portland, Oregon.
- Playing Fish Tales -- demonstration of playing a video game with VJ in 8-direction mode.
- Navigating a Google Earth map -- demonstrates the use of VJ for map navigation. The four vowels in the cardinal directions are used for panning, and two vowels along one of the diagonal axes are used for zooming in and out.
- Drawing Snoopy -- shows the continuous and fluid direction changing capability of VJ in the context of a drawing application. An 8-direction mode with adaptive filtering was used, allowing for smooth paths.
- Browsing the ASSETS Web Site (VJ) -- VJ in 8-direction mode being used to browse a standard web page.
- Browsing the ASSETS Web Site (Dragon Naturally Speaking) -- comparative demonstration of what the web browsing task is like when using an existing speech-based cursor control alternative available in the Dragon Naturally Speaking speech recognition software.
08/2006
The videos below show a VJ system with more degrees of freedom. The first video demonstrates playing video games using the VJ which allows movements in 8 directions. The second video demonstrates using five vowels plus pitch to control a simulated three-joint robotic arm. The third video demonstrate a four-way vowel classifier which enables diagonal movements via adaptive filtering. In addition, we have implemented a new velocity control scheme based on human perception of loud, normal and quiet voice. The vowel quality, loudness and discrete sounds can be adapted using an adaptation tool which is shown in the last video.
- Playing Fish Tales
- A Simulated Robotic Arm
- Browsing the World Wildlife Foundation (WWF) Website
- An Adaptation Tool
Early videos
04/2006
The following was part of a submission to the ICSLP conference, now known as Interspeech.
ICSLP Submission08/2005
06/2005
UIST SubmissionAn edited video showing several real-world VJ applications, including browsing a news site, playing a computer game, using Google Maps and a visualization tool. The system is based on a four-way vowel classifier, and diagonal movements can be enabled by filering techniques. The velocity is normalized for each vowel so that the cursor moving speed in different directions is more uniform. Voice-less consonants are again used as discrete sounds, but with a more robust rejection scheme, to reject breathing and extraneous speech.