qtpyqtpyside

QMovie is stuttering / lagging when using an animated .webp file on both PyQt6 and PySide6

The animated .webp file on the left is being played on Google Chrome, while the one on the right with a QMovie.

I've tested this on PyQt6 6.7.1, 6.8.1 and 6.9.1, and also on PySide6 6.8.2. There doesn't seem to be any difference between these versions as it stutters the same on all.

I've isolated the issue like this:

from PyQt6.QtWidgets import QApplication, QLabel
from PyQt6.QtGui import QMovie
import sys
import resources

app = QApplication(sys.argv)
label = QLabel()
label.setStyleSheet("background: white;")
movie = QMovie(':/Animations/a_mode.webp')
movie.setCacheMode(QMovie.CacheMode.CacheAll)
print(movie.cacheMode())
print(movie.format())
label.setMovie(movie)
label.setFixedSize(50, 50)
label.show()
movie.start()
sys.exit(app.exec())

I've tried:

Loading the .webp from a .qrc file, and also an absolute path.

Enabling the CacheAll mode.

Testing on both PyQt6 and PySide6.

Neither of those things made an improvement

Link to the .webp animation: https://ibb.co/jv1G3dWh

Solution

  • The issue can be confirmed across multiple platforms and Qt versions. Generally, Qt is "slower" in displaying the animation compared to other programs, specifically web browsers.

    Some background

    Besides GIF, there are other competing image formats that support animations and available in Qt: MNG and WebP (APNG doesn't seem officially supported, even though there is a third-party plugin).
    Both of those formats, to some extent, may consider a "legacy compatibility" with the original GIF format and how it was/is commonly implemented, including playback speed.

    Although modern browsers will statistically display animated contents more or less consistently, there can be some exceptions in the implementation.

    Historically, some browsers used to override the frame delay if it was too short, and in their evolution those overrides may have been considered for consistency. See this related post on webmasters stackexchange: Why is this GIF's animation speed different in Firefox vs. IE? (also read the reference links in its answer).

    It's also important to consider that animated images are implicitly without audio (therefore there is less need for timing accuracy) and every frame could have a different duration, making it more difficult to balance performance and accuracy.

    The Qt implementation and related bug

    QMovie has a relatively simple implementation based on some arbitrary assumptions, and treats all animated image formats in the same way.
    Specifically, it adjusts the "next frame delay" only based on the time needed to load the current frame, considering what the underlying QImageLoader tells it: if the next frame should be presented after 30ms and loading the image will take 25ms, it will then set (but not actually schedule) the next frame after 25ms.

    Once the new frame has been loaded, QMovie emits its updated() signal, which eventually calls all the connected slots (including the internal one used by QLabel), and finally starts the timer with the interval considered above.

    The problem with this is that those slots are normally connected using a direct connection, meaning that every function connected to the signal will be blocking until it has returned. The result is that the update doesn't consider the possible overhead caused by those functions the event loop, therefore a QMovie will always be slower than the sum of the declared duration of each frame, because it will begin to schedule the next frame only after the event loop allows it.

    The actual implementation can be seen in the source code (look for the void QMoviePrivate::_q_loadNextFrame(bool starting) line).

    There is an actual bug report about this (QTBUG-133747), it was reported for 6.8.2, but the source of the problem obviously comes from much earlier versions, and there has been no activity on that report since its submission.
    In my opinion, I sincerely doubt this will ever be fixed, unless extreme inconsistencies will be found and reported: image animations don't commonly require such a precise accuracy (since there's usually no need for syncing) and the Qt developer team certainly has more important priorities. Yet, you could still consider to sign up the Qt bug report system and vote that bug.

    A possible solution

    As long as we want to use and respect the QMovie API, we can get some level of control over the timing accuracy of frames.

    A possibility is to create a QMovie subclass and eventually use its features in order to try to improve the presentation speed.

    The following example uses a method that periodically checks the elapsed time between frames or the whole playback time, and eventually makes some adjustments in order to make the animation more consistent with the expected frame delays.

    It has two ways of doing it:

    The former method checks the actual duration required for the previous cycle and eventually adjusts the QMovie speed property in order to compensate for the next cycle. It's less accurate, but also provides slightly better performance since it only adjusts the speed only once for every cycle; unfortunately this also means that it's not directly possible to set a custom speed using the common API.

    The latter method is usually more accurate, as it checks the elapsed time from the previous frame and eventually anticipates the time when the next frame should appear. The performance cost of that provides a statistically more accurate timing of frame presentation, even though there is the tendency to be always a bit slower for high frame rates and a bit too fast for low frame rates.

    Here is the implementation:

    class PreciseMovie(QMovie):
    def __init__(self, *args, **kwargs):
        self._frameCheck = kwargs.pop('frameCheck', False)
        super().__init__(*args, **kwargs)
        self.setCacheMode(self.CacheMode.CacheAll)

        self._computeDurations()

        self._eTimer = QElapsedTimer()
        self._frameTimer = QTimer(self)
        self._frameTimer.setSingleShot(True)

        self._frameTimer.timeout.connect(self.jumpToNextFrame)
        self.stateChanged.connect(self._stateChanged)
        self.frameChanged.connect(
            self._fixDurationFrame 
            if self._frameCheck 
            else self._fixDurationCycle
        )

    def _computeDurations(self):
        self._frameDurations = []
        if not self.isValid():
            self._duration = 0
            return
        ir = QImageReader(self.device())
        while not ir.read().isNull():
            self._frameDurations.append(ir.nextImageDelay())
        self._duration = sum(self._frameDurations)
        self.device().seek(0)

    def _fixDurationCycle(self, i):
        if (
            i == 0 
            and self._duration 
            and self.state() == self.MovieState.Running
        ):
            if self._eTimer.isValid():
                elapsed = self._eTimer.restart()
                if elapsed and self._duration != elapsed:
                    self.setSpeed(round(elapsed * 100 / self._duration))
            else:
                self._eTimer.start()
                self.setSpeed(100)

    def _fixDurationFrame(self, i):
        if self.state() != self.MovieState.Running or not self._duration:
            return
        if not self._eTimer.isValid():
            self._eTimer.start()
        else:
            self._frameTimer.stop()
            frame = self.currentFrameNumber()
            speed = 100 / self.speed()
            duration = self._frameDurations[frame - 1]
            elapsed = self._eTimer.restart()
            offset = elapsed - duration * speed
            if offset > 0:
                nextDuration = self._frameDurations[frame] * speed - offset
                if nextDuration > 0:
                    self._frameTimer.start(round(nextDuration))
                else:
                    self.jumpToNextFrame()

    def _stateChanged(self, state):
        if state != self.MovieState.Running:
            self._eTimer.invalidate()

    @pyqtProperty(bool)
    def frameCheck(self):
        return self._frameCheck

    @frameCheck.setter
    def frameCheck(self, frameCheck):
        if self._frameCheck != frameCheck:
            if self._frameCheck:
                dis = self._fixDurationFrame
                con = self._fixDurationCycle
            else:
                dis = self._fixDurationCycle
                con = self._fixDurationFrame

            self.frameChanged.disconnect(dis)
            self.frameChanged.connect(con)

    def checkEachFrame(self):
        return self.frameCheck

    def setEachFrameChecked(self, frameCheck):
        self.frameCheck = frameCheck

    # the following are "pseudo-overrides"

    def setFileName(self, fileName):
        super().setFileName(fileName)
        self._computeDurations()

    def setDevice(self, device):
        super().setDevice(device)
        self._computeDurations()

    Note that I specifically added a custom property (frameCheck) and related getter/setter functions with different names, in order to keep consistency with the Qt property system and allow common callables for getting/setting the property. It is possible to create custom properties and keep the callable syntax, but that's out of the scope of this post.

    Considerations and test comparisons

    Since image animation potentially involves frames having very different durations, the playback accuracy may be unexpected.

    Also, the above attempt presents an "inverse drawback". While the default QMovie will always be as fast or slower than expected, my approach will almost always be as fast or, possibly, faster than expected.

    Furthermore, the _fixDurationCycle approach makes it impossible to set a custom speed. While that's an uncommon requirement, it's necessary to consider it.
    I will consider some possible improvements in order to work around that, so that an "accurate" speed property would always be consistent, even when actually changing the internal speed.

    Finally, it's quite possible that using an animated image with very different frame delays will give unexpected results: they will probably be acceptable, but still not consistent with other programs/browsers.

    In any case, I prepared a test code to show the differences between the default QMovie implementation and the custom class (with different check modes):

    from PyQt5.QtCore import *
from PyQt5.QtGui import *
from PyQt5.QtWidgets import *

PATH = 'a-mode.webp'


class PreciseMovie(QMovie):
    ... # as above

def makeTest(cls, **kwargs):
    def frameChanged(f):
        if f != 0:
            return
        nonlocal count
        count += 1
        countLabel.setNum(count)
        if not et.isValid():
            et.start()
            return
        elapsed = et.restart()
        times.append(elapsed)
        timeLabel.setText(f'{sum(times) // len(times)}ms')
        delays.append(elapsed - duration)
        avgDelayLabel.setText(f'{sum(delays) // len(delays)}ms')
        totDelayLabel.setText(f'{sum(delays)}ms')

    def stateChanged(s):
        if s != QMovie.MovieState.Running:
            et.invalidate()

    count = 0
    times = []
    delays = []
    et = QElapsedTimer()

    movie = cls(PATH, **kwargs)
    movies.append(movie)
    movie.setCacheMode(movie.CacheAll)

    movieLabel = QLabel()
    movieLabel.sizeHint = lambda: defaultSize
    movieLabel.setAlignment(Qt.AlignCenter)
    movieLabel.setSizePolicy(
        QSizePolicy.PolicyExpanding, QSizePolicy.Policy.Expanding)
    movieLabel.setMovie(movie)

    clsText = cls.__name__
    if cls == PreciseMovie:
        if movie.frameCheck:
            clsText += ' by frame'
        else:
            clsText += ' by loop'
    clsLabel = QLabel(clsText)
    clsLabel.setAlignment(Qt.Alignment.AlignCenter)
    countLabel = QLabel('0')
    timeLabel = QLabel('0ms')
    avgDelayLabel = QLabel('0ms')
    totDelayLabel = QLabel('0ms')

    lay = QGridLayout()
    lay.addWidget(clsLabel, 0, 0, 1, 2)
    lay.addWidget(movieLabel, 1, 0, 1, 2)
    lay.addWidget(QLabel('Loop count:'), 2, 0)
    lay.addWidget(countLabel, 2, 1)
    lay.addWidget(QLabel('Avg time:'), 3, 0)
    lay.addWidget(timeLabel, 3, 1)
    lay.addWidget(QLabel('Avg delay:'), 4, 0)
    lay.addWidget(avgDelayLabel, 4, 1)
    lay.addWidget(QLabel('Total delay:'), 5, 0)
    lay.addWidget(totDelayLabel, 5, 1)

    movie.frameChanged.connect(frameChanged)
    movie.stateChanged.connect(stateChanged)

    return lay

def toggle():
    func = 'stop' if movies[0].state() else 'start'
    if movies[0].state():
        func = 'stop'
        text = 'Play'
    else:
        func = 'start'
        text = 'Stop'

    for m in movies:
        getattr(m, func)()
    button.setText(text)


app = QApplication([])

duration = 0
ir = QImageReader(PATH)
defaultSize = ir.size()
while not ir.read().isNull():
    duration += ir.nextImageDelay()

titleLabel = QLabel(f'"{PATH}" - {ir.imageCount()} frames, {duration}ms')
titleLabel.setAlignment(Qt.Alignment.AlignCenter)

movies = []
topLay = QHBoxLayout()
topLay.addLayout(makeTest(QMovie))
topLay.addWidget(QFrame(frameShape=QFrame.Shape.VLine))
topLay.addLayout(makeTest(PreciseMovie, frameCheck=True))
topLay.addWidget(QFrame(frameShape=QFrame.Shape.VLine))
topLay.addLayout(makeTest(PreciseMovie))

win = QWidget()
mainLay = QVBoxLayout(win)
mainLay.addWidget(titleLabel)
mainLay.addLayout(topLay)

button = QPushButton('Play')
mainLay.addWidget(button)
button.clicked.connect(toggle)

win.show()
app.exec()

    We already know that the default QMovie is slower, but it's interesting to consider the differences between the behavior of _fixDurationCycle() and _fixDurationFrame().
    The _fixDurationCycle() approach seems more accurate for consistent frame delays, and when running for a lot of time. The _fixDurationFrame() way, instead, seems more accurate for shorter runs and/or very inconsistent frame delays.

    Right now I cannot suggest one way or the other. The only consideration you may want to follow is the performance, especially when dealing with lots of UI elements that may require frequent updates.