In order to analyze an audio file or the audio track of a video, technology must be able to understand the meaning of the interaction, whether live or recorded. This process is affected by a variety of factors: the speaker's language, dialect or accent, as well as background noise or interference. Due to the variable in speech and language, legacy approaches like phoneme matching and word spotting alone are not enough to determine what is truly being said. Autonomy Virage's SoftSound delivers sophisticated audio recognition and analysis technology that processes spoken interactions based on their conceptual content, not just the way they sound.
History of Speech Technology
Speech technology has gone through several phases of innovation, each one building upon the shortcomings of the previous generation. Many remember using speech technology over the phone to remotely program an interactive video recording system. These systems were able to recognize a limited number of keywords such as "Yes" and "No" or the number "5." If more than one word was present, these systems needed a pause of silence in between to differentiate the words. Unfortunately, conversational speech does not naturally have these pauses.
The next evolution in speech technology was phonetic indexing. This was a fast way of finding matches as it was only looking for base phonemes and specific grouping of phonemes. Unfortunately, this system had many false positives as sometimes words may appear in other words, such as "cat" in catastrophe. It was also sensitive to background noise, bandwidth of the call and, particularly, accents as the phonemes will differ considerably.
Phonetic
Keyword
Boolean
Parametric
Conceptual
Pattern Recognition
Data
Voice
Video
E-mail
Chat
Databases
Structured
Unstructured
Autonomy's Approach to Audio Search
A language model was developed to address the issues presented by phonetic indexing. This technique used a dictionary and a pre-defined language model and gave a highly accurate recognition rate that could also find phrases. However, the language model could not distinguish between homophones (e.g. 'eye' and 'I') and heteronyms (e.g. 'The bass player ate bass') and was confined to the pre-set dictionary.
To make the language model more flexible, a self-learning language model was developed that could learn new words. This improved model was highly accurate and could be set up without the need for the dictionary, but required massive computational requirements.
Today, the latest generation of speech technology delivers conceptual search. This approach utilizes advanced mathematics and complex algorithms to derive meaning from speech. Conceptual search addresses the shortcomings of previous speech technology models and provides the most accurate way of recognizing and finding speech because it understands what is being said. It can distinguish between homophones, heteronyms, as well as find and group things by concept. It can also find related information based on meaning and has lower computational need than some of the earlier generations of speech recognition technology.
Phoneme Processing
Phonemes are the smallest discreet sound-parts of language and form the basic components of any word. Phoneme matching attempts to break down words into their constituent phonemes and then match searched terms to combinations of phonemes as they occur in the audio stream. While this approach does not require a dictionary, it is limited in its accuracy and inability to make conceptual matched.
Phoneme processing is commonly used approach to audio recognition, but is frequently inaccurate and often returns high levels of false positives. Because words are treated simply as combinations of sounds with no awareness of their meaning in context, the system cannot differentiate between the required data, homophones, and phrases that share the same phoneme but bear no conceptual relation to the search terms. For example, the sentence "The computer can recognize speech" contains many of the same basic phoneme components as "The oil spill will wreck a nice beach," while the meaning is entirely different. Phoneme processing cannot account for multiple expression of the same concept, so any information that is related to the search term but does not contain the same phoneme will not be returned.
Word Spotting
As with phoneme matching, word spotting techniques search for words out of context, so they are unable to differentiate between homophones and homonyms. Because the system relies on exact sound matches, it is also unable to account for changes in pronunciation that affect sound, but not the actual concepts behind spoken words, such as plurals. As with other purely phonetic approaches, word spotting cannot make conceptual associations and will frequently miss related information that is not included in the search terms.
Autonomy offers all methods and has developed these systems originally for use in the UK and US Intelligence community.
Natural Language Processing
Natural Language Processing (NLP) is a form of human-to-computer interaction where the elements of human language, be it spoken or written, are formalized so that a computer can perform value-adding tasks based on that interaction. Autonomy's approach differs from standard NLP use in that it is still able to harness the power of IDOL's conceptual analysis. Autonomy's NLP technology functions independently of linguistic restraints, giving Autonomy's software universal application possibilities anywhere in the world.
"Autonomy excels in natural language processing techniques for understanding queries in multiple languages."
Traditionally, multimedia content has been considered an unwieldy resource, requiring considerable man-hours to extract tangible benefits and returns. As a consequence, most rich media assets remain dormant in the different repositories or lost on local user machines, thus creating duplication and inefficient storage management. The majority of organizations have failed to utilize the valuable intelligence contained within resources such as recorded meetings and training videos, and frequently fail to reuse video content such as broadcast content or marketing promotional videos. Using advanced image and audio analysis engines that watch, listen to and read a video signal in real-time, Autonomy Virage delves into the video file itself to extract the meaning of the information it contains.
Autonomy Virage delves deep within the video file itself rather than relying on human-defined metatags which are subjective and limited in scope. It then provides a highly detailed, time-encoded, comprehensive range of data which users, or IDOL itself, can search through to find relevant content with pinpoint accuracy. The full range of IDOL's functions can be performed on video content and video can be automatically cross-referenced with any other form of information.
Deep Video Indexing (DVI) uniquely tackles the challenge of asset identification and management through its use of a cascade application method that layers technology analytics to create a unique fingerprint for all video assets. This fingerprint is a flexible representation of the asset whose characteristics can be extracted or used to determine similarities between assets. Due to this unique, flexible mode of asset representation within the IDOL conceptual index, many of the challenges that traditionally plague the video medium are overcome. Issues such as spherical distortion, artifacts from low resolution transcoding, screen inclusion (i.e. assets within assets), audio distortions and audio mismatching (e.g. translations, misalignment) do not affect the integrity of the representation.
Openly configurable, the DVI functionality uses numerous proprietary approaches in a configurable lattice which can be weighted within the DVI fingerprint as separate entities or as a whole unit. These technological approaches include:
Texture trajectory analysis
Advanced Optical Character Recognition (OCR)
Spectrum trajectory analysis
Advanced scene analysis
Autonomy Virage can extract a comprehensive range of data from multimedia resources, including full transcripts of audio streams, on-screen character recognition, keyframes, facial recognition and speaker information, all of which is linked to the original video file, allowing users to locate content with pinpoint accuracy. Utilizing the power of IDOL, Autonomy Virage provides users with an unrivalled range of video analysis tools such as scene detection, 'find similar' functions and conceptual analysis such as automatic hyperlinking of related content, categorization and clustering.
In addition, video content can be automatically cross-referenced with any other form of information such as PowerPoint presentations, Word documents or web pages. In this way, Autonomy Virage ensures that rich media is immediately fully searchable and accessible by any user.
Summary: ...convert video and audio natural language to text and time synchronize with a streaming preview of the content. Video assets can be quickly and easily found with pinpoint accuracy to the exact location within a video where a word or phrase is spoken. Virage MediaBin also provides a new and intuitive user...
Summary: ...resources to stop emerging crime trends as they first appear and hence help reduce policing costs in the long-term. 4. Voice Recognition Autonomy’s format independent technology is also being used in crime reporting centres. Every call is recorded and Autonomy’s patented speech recognition technology...
Summary: ...Orange Telecom Turns Insight to Action with Speech Analytics - case study. • Equipment Releases/Issues – Speech analytics enables the business to retrieve agent/customer conversations with mentions of specific equipment (handsets) or issue types (such as defective batteries,accessories, etc.) • Coaching/...
Summary: ...only find a specific feature programme, but also a specific place within the programme that mentions the topic they are searching for. As IDOL’s search functionality is based on the underlying meaning of words and concepts spoken on the radio, search results returned to online users are always of maximum...
Summary: ...and document management systems. The firm plans to consolidate its future content management and governance initiatives around the IDOL platform thus helping to extract better returns on investment. “A sound information management strategy is fundamental to the modern law firm. We clearly see HP Autonomy’s...
Summary: ...media type or system. Phase I of the project—an electronic records and document management system (EDMS) with evidence linking - was implemented. “The software integrates with our environment of Lotus Notes and Word, so users don’t need to know anything about WorkSite or RecordsManager. They just...
Summary: ...thus moving beyond traditional approaches which depend heavily on grammatical structure or specific vocabulary. • IDOL's unique AQG refines search results into clusters thus giving KLA employees a bird's eye view of data in the organization and allowing them to spot hot trends and topics • Automatic...
Summary: ...Leon County Schools - case study. This was limited by the use of static Microsoft Word and Adobe PDF files that had to be printed out, faxed or mailed to obtain proper approvals. Leon County soon found out that this system was not adequate enough to effectively manage their 200+ official forms, and it...
Summary: ...implementation. • Search utilisation has risen 10 folds – Internal users and External customers started using search as a primary tool to source information. • Increased traffic, click-through and conversion rates. Background Optus is an Australian leader in integrated telecommunications, delivering...
Summary: ...suggests content that is contextually linked thus ensuring clients have full visibility over related content; information clustering provides a bird's eye view of data grouping similar items together to spot new or hot trends and topics; and categorisation of content automatically generates taxonomies...
Summary: ...Cheshire Constabulary: Powered by IDOL - Case Study. POWER PROMOTE PROTECT Cheshire Constabulary: Powered by IDOL Organization Background Cheshire Constabulary is responsible for policing a population of approximately one million people throughout Halton, Warrington, Cheshire East and West Cheshire & Chester....
Summary: ...website. In another use of Aurasma, when a conference attendee aimed their mobile device at an ad in the conference guide book, an introductory video launched showing university president Richard H. Hart, MD, Dr PH speaking. Users also had the option to click-through to a website that was specifically...
This is a small selection of the Autonomy case studies available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...able to understand the meaning of live or recorded communication. Sophisticated audio recognition and analysis technology process spoken interactions based on their conceptual content, not just by the way they sound. This approach provides the most accurate way of recognizing and finding speech because...
Summary: ...processed into meaningful visual paradigms for rapid categorization and analysis. IDOL’s speaker independent, multilingual speech analytics technology allows investigators to rapidly locate conversations containing specific search phrases and speakers. as well as edit, search, and hyperlink to audio...
Summary: ...based on their conceptual content, not just the way they sound. Autonomy Marketing Analytics solutions derives a hypothesis of the concepts present in speech and analyzes elements such as topics discussed, speaker identity and gender, emotions contained in the conversation, and excessive silence or cross-talk....
Summary: ...to produce a sanctioned communication. Product Brief Highlights • Unified platform for EDD and review of audio in-context with email and documents • Full suite of award-winning eDiscovery services • Multiple speech analysis techniques for highest level of accuracy • Flexible, keyword, phoneme,...
Summary: ...can convert audio to text using an extensive vocabulary and out-of-the-box speaker independent technology. The solution detects changes among speakers and identifies copy by speaker. Once captured, key words and phrases can be detected and flagged for further attention. 3 Automatic vehicle plate recognition...
Summary: ...search techniques to mine your data, no matter how complex or unique. You can search on any combination of: ??File type and defi ned data fields, such as sender, recipient, date sent or date archived ??Free-form text/keyword or word phrases inside message content or attachments ??Data repositories that...
Summary: ...a comprehensive, end-to-end solution that encompasses every aspect of vehicle recognition and tracking, together with advanced recording, retrieval, and analytical capabilities. Intelligent core technologies Autonomy ANPR/ALPR combines multiple state-of-the-art Optical Character Recognition (OCR) techniques...
Summary: ...cerecords management through automatic classification and indexing of documents “This system has had so many benefits for us. It has enabled us to spot problems more quickly and easily and it has reduced our workload. It has really paid dividends” —Rick Faulkner, Health and Safety and Environmental...
Summary: ...Faster and much less expensive than manual transcription leveraging an array of speech analytics and automation to save time and money • Battle tested and sophisticated speech technology overcomes issues around audio quality, accents / dialects, people talking over each other, etc.
...
Summary: ...legal teams can extract only the information most relevant to a case while overcoming variables of audio such as quality, accents and dialects, cross-talk and background noise. The native audio review automatically listens to and mines audio data using a combination of search techniques, including conceptual...
This is a small selection of the Autonomy Product Briefs available, please visit our publications site at http://publications.autonomy.com/ for more information.
Summary: ...merely use keywords and cannot contend with variant expressions of an idea, IDOL understands the language behind the spoken words, even distinguishing homophones and phrases. For instance, IDOL would not be ensnared by the following gaffe that typifies technologies dependent solely on phoneme processing:...
Summary: ...Building a GARP®-compliant solution. This process is afected by a variety of factors, such as the speaker’s language, dialect or accent, as well as background noise or interference. Due to the variables in speech and language, legacy approaches like phoneme matching and word-spotting alone are not...
Summary: ...thereby distinguishing homophones Use of both phonetic and conceptual approaches to offer unique high-level functions Audio, video, email and chat to be indexed and searched alongside other data formats Unrivalled accuracy by understanding words in context Speaker and audio segmentation, speaker recognition...
Summary: ...return files that contain the terms queried by the user, relying on vocabularies in the system. It is simple and intuitive. Phonetics goes deeper by matching search terms to words broken down into their phonemes, or smallest compositions of speech, and matches them to words, even though they may not be...
Summary: ...govern, and make available for review, especially when it exists in multiple repositories, formats, and languages. Your speech recognition engine will need to account for the variability in speech such as language, dialect, accent, or tone by using a combination of acoustic models, a language model, pronunciation...
Summary: ...dependent on synthetic tags. Accuracy Autonomy’s pioneering MBC platform provides advanced and accurate enterprise retrieval by encompassing and transcending existing search mechanisms. By searching using the concepts defined by the given words, and not just keyword techniques, it provides superior...
Summary: ...user input, this is the fi rst system seamlessly to connect the spoken word with a fi nal formatted document, and is able to deliver on the promise of increased effi ciency dramatically improving document turnaround times. Capturing and recognising speech is part of the challenge but the end result needs...
Summary: ...This result is then extended, by more sophisticated mathematical arguments, to instances when units are related. Natural languages contain a high degree of redundancy and nonessential content. For example, a conversation in a noisy room can be understood even when some of the words cannot be heard, and...
Summary: ...on words or concepts that are spoken during the call. Activity events are based on desktop activity such as screens and fields used while on the call. Screen Masking utilizes activity events from the agent’s desktop, including the applications, screens, and fields used by the agents. The activity event...
Summary: ...Information Theory to apply a technique known as Adaptive Probabilistic Concept Modelling* to the problem of understanding unstructured information. This technique builds a conceptual understanding of the information it processes by analyzing the patterns of arbitrary signals or symbols in that information....
Summary: ...their context. “A black and white flash jumped into the sea and ap a fish in its beak” Is it a penguin? This approach is similar to understanding a conversation in a noisy room, where you can grasp the context of the discussion even when some of the words cannot be heard—or grasping the essence...
Summary: ...to determine the characteristics that give the text meaning. As this technology is based on probabilistic modeling, it does not use any form of language dependent parsing or dictionaries. Words are treated as abstract symbols of meaning and the engine derives its understanding through the context of their...
This is a small selection of the Autonomy White Papers available, please visit our publications site at http://publications.autonomy.com/ for more information.
Related Case Studies
top of page
RSS Feeds
Find us on Facebook
Follow us on Twitter
See us on YouTube
Find us on LinkedIn
