composer-phpmediawikiguzzlemwclient

Guzzle update breaks Mediawiki 1.39.3 LTS - how can dependency hell be avoided for the future?

Just got a:

Fatal error: Declaration of MWCallbackStream::write($string) must be compatible with Psr\Http\Message\StreamInterface::write(string $string): int in /var/www/html/includes/http/MWCallbackStream.php on line 49

which manifests itself downstream as a

Did not get a valid JSON response from the server. Check that you used the correct hostname. If you did, the server might be wrongly configured or experiencing temporary problems.

in the mwclient library I am using when trying to copy a page from one Mediawiki to another.

This happened when implementing https://github.com/WolfgangFahl/py-yprinciple-gen/issues/31

with the following code

  def push(self):
        """
        push according to my command line args
        """
        if not self.args.source:
            raise "missing source wiki"
        if self.args.topics:
            topic_names=self.args.topics
        else:
            topic_names=self.context.topics.keys()
        login=self.args.login
        force=self.args.force
        ignore=True
        wikiPush=WikiPush(fromWikiId=self.args.source,toWikiId=self.smwAccess.wikiId,login=login,verbose=not self.args.quiet,debug=self.args.debug)
        if not self.args.quiet:
            print(f"pushing concept {self.args.context} from {self.args.source} to {self.wikiId} ...")
        all_failed=[]
        for topic_name in topic_names:
            failed=wikiPush.push(pageTitles=[f"Concept:{topic_name}"],force=force,ignore=ignore,withImages=True)
            all_failed.extend(failed)
            pass
        if len(all_failed)>0:
            print(f"Warning {len(all_failed)} uploads failed")

but can also be reproduced with a simple command line such as:

wikipush -l -s master -t crm -p "Concept:Action"
copying 1pages from master to crm
1/1 ( 100%): copying Concept:Action ...❌:Did not get a valid JSON response from the server. Check that you used the correct hostname. If you did, the ...

while

wikipush -l -s master -t wiki -p "Concept:Action" -f
copying 1pages from master to wiki
1/1 ( 100%): copying Concept:Action ...✅

happily works if not targeting a MediaWiki 1.39 LTS.

https://wiki.bitplan.com/index.php/Concept:Action is not special compared to other pages that can be copied/pushed with no problem.

The problem appears with certain pages and I couldn't idenfify a pattern what content might cause the problem therefore debugging was quite hard and took a while.

The JSON response is simply empty due to the fatal crash which could only be found out by setting Mediawiki to debug mode with:

error_reporting( -1 );
ini_set( 'display_errors', 1 );
$wgShowExceptionDetails=true;

my questions are:

  1. How can an update of guzzle break my MediaWiki seemingly easily?
  2. What's the reason for this behavior?
  3. How can this kind of problem be avoided in the future?

I didn't find the answers in https://phabricator.wikimedia.org/T335073 - there is only a work around but the underlying fundamental problem of why guzzle updates break a Mediawiki LTS version is not discussed.

grep guzzle composer.json 
        "guzzlehttp/guzzle": "7.4.5",

More often than not CIs keep breaking these days for reasons that are fully unclear. Both this question and AttributeError: module 'sqlalchemy' has no attribute '__all__' are of this type although the enviroments are totally different. The linking element is IMHO how the dependencies have been specified. The devil might be in the details e.g. asking for 2.4 with a ^i n composer but getting an incompatible 2.5. Why is 2.5 incompatible? Why is an LTS version assuming that later versions will be compatible and doesn't pin things in the first play. Is pinning actually a good solution here? These are not support question but very general ones which I'd like to get answers for specific to this case. How do we avoid these problems in the future?

Solution

  • I'll only answer on the dependency hell question. This is "Semantic Versionings' (Semver) fault" as it allows to ignore the dependencies of the dependency that is versioned.

    If there are overlapping dependencies, e.g. typically in PHP that is the PHP runtime version, and code is not provided in a backwards compatible fashion, systems of multiple dependencies can easily break regardless the semantic versioning approach would have lurked a person into a different understanding.

    As Nico Haase has pointed out:

    It's not an update by Mediawiki or Guzzle that caused this problem, but the one listed at github.com/php-fig/http-message/releases/tag/1.1

    This is in an interface package that is adding a new feature which is backwards incompatible. Normally this would require to raise a major version, however as this is only by the runtime (so a dependency it depends on it), it is technically correct to not raise the major version number.

    However, and this is practically the typical PHP situation, your own application then - when it consumes such dependencies - must pin the dependency versions (e.g. lock). You can not benefit of the easy updating path semver originally announced as its main benefit. YMMV, e.g. every project is different and there may be less hard constraints with development only dependencies as they fail fast and may not break the whole project. But then each development build should capture the versions of all dependencies in use (e.g. a copy of vendor/compsoer/installed.json and the build environment, e.g. php version, configuration etc.).

    A quick list to outline the rationale:

    In general to not adhere to Semver down to the PHP language level (IMHO this could work pretty well, even there are some shifts during a level) can turn out to be very unfortunate, but there is no clear consensus about this in the PSR community, nor in the PHP community overall.

    And if you allow me a personal comment, I'm not even sure we've understood the problem yet.

    The grandfathering contract is/was to have backwards compatibility for the runtime, which is also fail-safe.

    But with more youngsters (gladly!), you get more tigers that eventually want to be the tiger team, therefore: require & pin your dependencies. This is especially becoming more and more true as the yearly release cycle of the PHP runtime starts to show more global effects. Also showing the flaw in its birth: We have yearly releases, but there is no commitment how many years at least a major version will have (I'd assume a demand of a minimum of 8 +/- 2 releases for PHP 8 and then a better distribution which versions are maintained for how long to reduce the number of releases per major in the long run).

    So from a fail-safe approach with a fail-safe language, we're now going into a fail-first language that renders its fail-safe production use void without offering any comfort to deal with such inalienable PHP runtime changes.

    As Semver in the PHP world is most often all and about Composer, for a PHP Composer project with dependencies from top of my head I'd suggest the following: