pythoniteratorplonedata-migrationtransmogrifier

It is possible to run a transmogrifier section after all other have completely run?


I'm importing content to Plone using a transmogrifier pipeline and, in order to fix various aspects like images, links and related content, I need to run my section just after all content has been created and indexed.

I need this because I want to use the catalog tool in order to search for content by path and use its UUID to referrer to it.

Is that possible using transmogrifier or it's better to do it using any other of the technologies available, like a simple upgrade step?

I was thinking on using a pattern similar to the source section:

from collective.transmogrifier.interfaces import ISection
from collective.transmogrifier.interfaces import ISectionBlueprint

class DoSomethingAtTheVeryEndSection(object):

    classProvides(ISectionBlueprint)
    implements(ISection)

    def __init__(self, transmogrifier, name, options, previous):
        self.previous = previous

    def __iter__(self):
        for item in self.previous:
            yield item

        for item in self.previous:
            do_something()

Is this a good idea?


Solution

  • Yes, it is a good idea to make a PostProcess section, the only problem is that the self.previous generator can't be called 2 times this way.

    A workaround is to use itertools.tee to duplicate the generator, this way you can walk two times into the generator:

    from collective.transmogrifier.interfaces import ISection
    from collective.transmogrifier.interfaces import ISectionBlueprint
    
    import itertools
    
    
    class DoSomethingAtTheVeryEndSection(object):
    
        classProvides(ISectionBlueprint)
        implements(ISection)
    
        def __init__(self, transmogrifier, name, options, previous):
            self.previous = previous
    
        def __iter__(self):
            self.previous, self.postprocess = itertools.tee(self.previous)
            for item in self.previous:
                yield item
    
            for item in self.postprocess:
                do_something()