Google Audio Indexing

[any ANONYMOUS forum user]

What is Google Audio Indexing?
Google Audio Indexing (Gaudi) is a new technology from Google that allows users to better search and watch videos from various YouTube channels. It uses speech technology to find spoken words inside videos and lets the user jump to the right portion of the video where these words are spoken.

Why is Google doing it?
Google's mission is "to organize the world's information and make it universally accessible and useful". As more video content is being created everyday, Google Audio Indexing tries to make it easier for people to find and consume spoken content from videos on the Web.

Why focus on election material?
Google Audio Indexing is a showcase for speech technology. Political videos and election materials are a special case of broadcast news content, a domain that has received a lot of academic and industry attention and is known to perform well.
By making the technology available to a wide audience, we hope to both offer a useful service and learn what our users think of this new technology.

Moreover, on this election year, what information could be more important than what describes the views, actions and platforms of the two presidential candidates? Google Audio Indexing and the Google Elections Video Search gadget are part of a broader effort around politics, with initiatives such as CNN/YouTube debates , YouChoose , and Google Checkout for political contributions .

How is Google Audio Indexing different from the Google Elections Video Search gadget?
Both are using the exact same underlying technology. The new features from Google Audio Indexing are being incorporated into the gadget.
The scope of the Google Elections Video Search gadget is restricted to the US election. The aim of Google Audio Indexing on Google Labs is broader and the US election is just a first step. We see it as an experiment platform where we can learn what features make the best user experience for people looking for spoken content on the Web.

How do I use it?
With Google Audio Indexing, you can use speech technology to find videos and to search for spoken content inside a video.

Searching for videos
To find a set of videos, simply type a query in the search box and press the "Search videos" button. The results of your search will appear in the left column. You can refine your search using channel filters, the active filter showing in bold. A channel filter corresponds to one or more YouTube channels. In the context of the US election, you can choose videos from the McCain channel, the Obama channel or from all YouTube political channels. For each result, we provide a thumbnail of the video, its title, the time since it was published, the duration and the number of times the query terms are mentioned (i.e. spoken) in the video. By clicking on a result, the video will be displayed in the right column. If you need to see more results, you can navigate between result pages using the links at the bottom of the left column (previous, next, etc.), just like on most Google Search products.

Watching a video
Once you have selected a video, it will appear on the right side of the screen, in the embedded YouTube player, with the list of mentions.
Google Audio Indexing enhances the YouTube player to display mentions of the query terms as yellow markers on the player timeline. For space reasons, we cannot always display all the mentions on the timeline. The top 10 mentions are displayed as a list under the player.
To read the transcript for a given mention, just mouse over the corresponding yellow marker. To listen to the corresponding audio, just click on the marker: the player will automatically jump to the right time in the video. You can also click on the play button located on the left side of the mention.

Searching inside a video
With Google Audio Indexing, you can also restrict your search to the content of a given video. By default, the "Search inside this video" uses the same query term as the video search.
To search for a different term, just type your query in the text form under the video player and press the "Search within this video" button. The mention results are displayed underneath. The yellow markers on the timeline are updated accordingly and you can access the information just like before, using mouse over.

How does it work?
Google Audio Indexing uses speech technology to transform spoken words into text and leverages the Google indexing technology to return the best results to the user.
The returned videos are ranked based -- among other things -- on the spoken content, the metadata, the freshness.
We periodically crawl the YouTube political channels for new content. As soon as a new video is uploaded to YouTube, it is processed by our system and made available in our index for people to search.

What video content can I search?
Google Audio Indexing searches only those videos uploaded on the YouTube political channels. If a video is on the YouTube political channels, it will be in the index.

Candidates control the video content released to the public using Google Audio Indexing technology by controlling the content they upload to YouTube channels.

Is Google using its own speech recognition technology?
Yes. The speech research group at Google has developed its own speech recognition system (called Gaudi, for Google Audio Indexing), which powers both Google Audio Indexing and the Google Elections Video Search gadget.