pythonpackageruntime-errorhttp-errorstopiteration

Python - Scrubadub.clean not working - Cannot properly scrub text PII + HTTP Error 503


I'm sorry and this is probably a basic question but as I am learning scrubadub and trying to get it work myself on Jupyter notebook. It kept showing - HTTP Error 503: Service Unavailable This is what I input, which is exactly the same as the scrubadub documentation.

text = u"John is a cat"
scrubadub.clean(text, replace_with='placeholder') 
u"{{NAME}} is a cat"

And this is the error message I got:

HTTPError                                 Traceback (most recent call last)
<ipython-input-92-5b0754baae94> in <module>()
      1 text = u"John is a cat"
----> 2 scrubadub.clean(text, replace_with='placeholder')
      3 u"{{NAME}} is a cat"

/anaconda3/lib/python3.7/site-packages/scrubadub/__init__.py in clean(text, cls, **kwargs)
     14     cls = cls or Scrubber
     15     scrubber = cls()
---> 16     return scrubber.clean(text, **kwargs)

/anaconda3/lib/python3.7/site-packages/scrubadub/scrubbers.py in clean(self, text, **kwargs)
     55         clean_chunks = []
     56         filth = Filth()
---> 57         for next_filth in self.iter_filth(text):
     58             clean_chunks.append(text[filth.end:next_filth.beg])
     59             clean_chunks.append(next_filth.replace_with(**kwargs))

screenshot for error#1 screenshot for error#2

I also tried below code here and I get error messages as well, I'm guessing if I missing any arguments in the code...

import scrubadub
class MyFilth(scrubadub.filth.base.Filth):
    type = 'mine'
class MyDetector(scrubadub.detectors.base.Detector):
    filth_cls = MyFilth
    def iter_filth(self, text):
       # do something here
       pass

scrubber = scrubadub.Scrubber()
scrubber.add_detector(MyDetector)

text = u"My stuff can be found there"
scrubadub.clean(text)
u"{{MINE}} can be found there."

StopIteration                             Traceback (most recent call last)
/anaconda3/lib/python3.7/site-packages/scrubadub/detectors/base.py in iter_filth(self, text)
     21         if self.filth_cls.regex is None:
---> 22             raise StopIteration
     23         for match in self.filth_cls.regex.finditer(text):

StopIteration: 

The above exception was the direct cause of the following exception:

RuntimeError                              Traceback (most recent call last)
<ipython-input-94-2cc23d003da7> in <module>()
     11 
     12 text = u"My stuff can be found there"
---> 13 scrubadub.clean(text)
     14 u"{{MINE}} can be found there."

/anaconda3/lib/python3.7/site-packages/scrubadub/__init__.py in clean(text, cls, **kwargs)
     14     cls = cls or Scrubber
     15     scrubber = cls()
---> 16     return scrubber.clean(text, **kwargs)

Solution

  • There is an open issue for the same on github as scrubadub seems to not working very good with python 3.7 which you are currently using.

    I was able to reproduce it with 3.7 without a notebook as well. So it is a notebook issue for sure.

    As an interim solution, changing your env to 3.6 (or worst case 2.7 highly not recommended) works.

    https://github.com/datascopeanalytics/scrubadub/issues/40Stop Iteration issue