Is There A Way To Reverse Stem In Python Nltk?

I have a list of stems in NLTK/python and want to get the possible words that create that stem. Is there a way to take a stem and get a list of words that will stem to it in python

Solution 1:

To the best of my knowledge the answer is No, and depending on the stemmer it might be difficult to come up with an exhaustive search for reverting the effect of the stemming rules and the results would be mostly invalid words by any standard. E.g for Porter stemmer:

from nltk.stem.porter import *
stemmer = PorterStemmer()
stemmer.stem('grabfuled')
# results in "grab" 

So, a reverse function would generate "grabfuled" as one of the valid words as "-ed" and "-ful" suffixes are removed consecutively in the stemming process. However, given a valid lexicon, you can do the following which is independent of the stemming method:

from nltk.stem.porter import *
from collections import defaultdict

vocab = set(['grab', 'grabbing', 'grabbed', 'run', 'running', 'eat'])

# Here porter stemmer, but can be any other stemmer too
stemmer = PorterStemmer()

d = defaultdict(set)
for v in vocab:
    d[stemmer.stem(v)].add(v)  

print(d)
# defaultdict(<class 'set'>, {'grab': {'grab', 'grabbing', 'grabbed'}, 'eat': {'eat'}, 'run': {'run', 'running'}})

Now we have a dictionary that maps stems to the valid words that can generate them. For any stem we can do the following:

print(d['grab'])
# {'grab', 'grabbed', 'grabbing'}

For building the vocabulary you can tokenize a corpus or use nltk's built-in dictionary of English words.