pythonlibtorrentmagnet-uri

python libtorrent How long should i wait for meta data while using magnet URI?

i am trying to download meta data from DHT by providing them magnet URI , but some times it takes more than 5 minutes for single URI .

i am using code like this

while (not handle.has_metadata()):
    try:
        sleep(1)
    except KeyboardInterrupt:
        print("Aborting...")
        ses.pause()
        print("Cleanup dir " + tempdir)
        shutil.rmtree(tempdir)
        sys.exit(0)
ses.pause()
print("Done")

so how long should i have to wait for meta data ? or i can keep that handle active while creating new handle for new magnet uri to get meta data ?

is there timeout settings or something like that ?

UPDATE :

What i mean is , is there a magic number , lets say X minutes . If it cant fetch meta data in X minutes then it cant fetch metadata in lets say 24 hours.

or is it possible , that it can fetch in 24 hours but not first x minutes ?

how does this exactly work ?

Solution

  • as Borealid points out, there's no way of being certain that someone with metadata does not exist in the world (but is offline at the moment).

    If you're interested in knowing whether someone has metadata right now (more or less) my basic suggestion would be that you wait at least one DHT announce interval.

    There are a few steps that need to be successful:

    1. DHT bootstrap (finding DHT nodes)
    2. DHT announce (finding BitTorrent peers)
    3. connecting to peers
    4. having a peer that has the metadata (and supports the metadata extension). The vast majority of peers supports this extension.

    Technically, each step could have a separate timeout. Assuming that you are bootstrapped and have a working connection to the DHT, the next concern is to actually make sure the announce to the DHT works and completes.

    Torrents are supposed to announce to the DHT every 15 minutes. It's configurable in libtorrent though, called dht_announce_interval. If there's some problem with the DHT causing it to miss an update (say, the DHT isn't fully bootstrapped when you add the magnet link) you may have to wait for another 15 minutes for it to announce again.

    Another thing to keep in mind is that in libtorrent, the DHT announces are attempted to be distributed more or less evenly over time. This means the first attempt may not be immediately when the magnet link is added.

    To be more certain of whether there are any peers, you can also force_update the DHT (call force_dht_announce() on the torrent_handle object). If you do this a few minutes in (in case you haven't found any peers) it may also counter any issue making the first announce fail.

    Once you have a connected bittorrent peer, you can be fairly certain the DHT announce succeeded (unless you got it from local peer discovery I suppose). When looking at the peer list (torrent_handle::get_peer_info()) each peer_info entry has a source flags field, which can tell you whether it came from the DHT or not.

    Once you have a peer, you may want to wait one PEX interval (60 seconds) to make sure you have a chance to learn about more peers before giving up, in case the peer itself doesn't have the metadata or isn't sending it.

    There's no simple way of telling whether a peer supports the metadata or pex extensions specifically, but if it doesn't support the extension protocol, it won't support either. You can tell that via the peer_info::flags & peer_info::supports_extensions.

    As long as peer_info::connecting or peer_info::handshake bits are set in the flag field, the peer may just be a random IP that used to be part of the swarm. It's not safe to assume a peer is alive or exists until those bits are cleared.