UPDATE May 2018: Read a current account of our progress here.
Today we launch early access to our AI-driven product Wootric CXInsight™. This capability is the culmination of many months of focus on applying artificial intelligence -- machine learning and natural language processing (NLP) in our case -- to customer experience feedback. Our algorithms are now able to automatically categorize, aka tag, and assign sentiment to qualitative feedback for several product categories.
For the AI geeks out there, we want to share more about how Wootric is approaching the machine learning challenge in the customer feedback management space.
Our training data for customer feedback analysis
Wootric has collected millions of real-world customer feedback comments for products across all possible industries -- ranging from state of the art SaaS software and services to enterprise products that have been around for a while. A lot of that data has been carefully categorized by our customers.
This trove of categorized feedback serves as training data for continually improving the proprietary algorithms that Wootric has developed to automatically analyze qualitative feedback.
State of the art text classification requires high quality manually labeled data. Anybody claiming otherwise is, in our opinion, just blowing smoke.
Technology approach behind Wootric's AI
Here are some specifics about the process we have gone through (and continue to pursue) to build AI that can deliver meaningful insights for our customers.
- Our customers have categorized thousands and thousands of feedback themselves which are of very high quality and consistency.
- For product categories where we don’t have customer categorized data, we have spent hundreds of engineering hours to manually categorize.
- We have experimented with several “old school” algorithms like bag of words, LDA and Word2Vec along with state of the art algorithms based on LSTM and CNN.
- We have tuned these algorithms for short human generated product and service feedback. Most examples you find on the web are related to news articles, wikipedia entries etc and these are not directly applicable to problem we are solving for our customers.
- Our list of what does not work is longer than what works. ;-) We will write more about this on subsequent blogs.
- We have incorporated and explored approaches from publications that have come out of leading research institutions.
- Stanford NLP
- Google’s word2vec and several other research papers on LSTM
- Recommendations from papers at jair.org
- Facebook NLP research papers
- Berkeley NLP Group
- We have built an online learning system to continuously improve our quality of data and tune our models through the help of human-in-the-loop mechanism. We have made it super easy for our customers to mark misclassified feedback which is then used by our algorithms to re-learn. Obviously there are checks and balances in place to make sure overall accuracy is moving in right direction.
- Our engineering team has a strong background in mathematics -- statistics and linear algebra in particular -- along with big data processing. We were a bit rusty with mathematics when we started to be honest. ;-)
This is just the beginning though. We plan to share more details about our approach as it evolves. Please tweet us @wootric for more details.
Learn more about Wootric CXInsight™.