- Duration: 2 days
- Skill-level: Foundation-level Taming Text skills for Intermediate skilled team members. This is not a basic class.
- Targeted Audience: This course is geared for those who want to explore how to automatically organize text using approaches such as full-text search, proper name recognition, clustering, tagging, information extraction, and summarization
- Hands-on Learning: This course is approximately 50% hands-on lab to 50% lecture ratio, combining engaging lecture, demos, group activities and discussions with machine-based student labs and exercises. Student machines are required.
- Delivery Format: This course is available for onsite private classroom presentation.
- Customizable: This course may be tailored to target your specific training skills objectives, tools of choice and learning goals.
There is so much text in our lives, we are practically drowning in it. Fortunately, there are innovative tools and techniques for managing unstructured information that can throw the smart developer a much-needed lifeline. You’ll find them in this course.
Taming Text is a practical, example-driven guide to working with text in real applications. This course introduces you to useful techniques like full-text search, proper name recognition, clustering, tagging, information extraction, and summarization. You’ll explore real use cases as you systematically absorb the foundations upon which they are built.
Written in a clear and concise style, this course avoids jargon, explaining the subject in terms you can understand without a background in statistics or natural language processing. Examples are in Java, but the concepts can be applied in any language.
Working in a hands-on learning environment, led by our Taming Text instructor, students will learn about and explore:
- Taming Text is a hands-on, example-driven guide to working with unstructured text in the context of real-world applications.
- explores how to automatically organize text using approaches such as full-text search, proper name recognition, clustering, tagging, information extraction, and summarization.
- guides you through examples illustrating each of these topics, as well as the foundations upon which they are built.
Topics Covered: This is a high-level list of topics covered in this course. Please see the detailed Agenda below
- When to use text-taming techniques
- Important open-source libraries like Solr and Mahout
- How to build text-processing applications
Audience & Pre-Requisites
Pre-Requisites: Students should have
- Basic to Intermediate IT Skills, Microsoft azure and Machine Learning knowledge
- Good foundational mathematics or logic skills
Course Agenda / Topics
- Getting started taming textfree
- Why taming text is important
- Preview: A fact-based question answering system
- Understanding text is hard
- Text, tamed
- Text and the intelligent app: search and beyond
- Foundations of taming text
- Foundations of language
- Common tools for text processing
- Preprocessing and extracting content from common file formats
- Search and faceting example: Amazon.com
- Introduction to search concepts
- Introducing the Apache Solr search server
- Indexing content with Apache Solr
- Searching content with Apache Solr
- Understanding search performance factors
- Improving search performance
- Search alternatives
- Fuzzy string matching
- Approaches to fuzzy string matching
- Finding fuzzy string matches
- Building fuzzy string matching applications
- Identifying people, places, and things
- Approaches to named-entity recognition
- Basic entity identification with OpenNLP
- In-depth entity identification with OpenNLP
- Performance of OpenNLP
- Customizing OpenNLP entity identification for a new domain
- Clustering text
- Google News document clustering
- Clustering foundations
- Setting up a simple clustering application
- Clustering search results using Carrot2
- Clustering document collections with Apache Mahout
- Topic modeling using Apache Mahout
- Examining clustering performance
- Classification, categorization, and tagging
- Introduction to classification and categorization
- The classification process
- Building document categorizers using Apache Lucene
- Training a naive Bayes classifier using Apache Mahout
- Categorizing documents with OpenNLP
- Building a tag recommender using Apache Solr
- Building an example question answering systemfree
- Basics of a question answering system
- Installing and running the QA code
- A sample question answering architecture
- Understanding questions and producing answers
- Steps to improve the system
- Untamed text: exploring the next frontier
- Semantics, discourse, and pragmatics: exploring higher levels of NLP
- Document and collection summarization
- Relationship extraction
- Identifying important content and people
- Detecting emotions via sentiment analysis
- Cross-language information retrieval