artificial-intelligencegpt-2

How can text completion using the GPT-2 language model generate a full URL?


I found this auto text completion on Mr Fabrice Bellard's website. Then I ask like in the picture: What I asked the machine

So my question is: Is the respond text is generated randomly or somehow controlled by the text I typed (and if it's controlled by the text I typed, why it isn't related to the question I typed?)? And how's possible for it to generate a link to a web page (which cannot be accessed by me)?

I'm new to AI and neutral networks (and that sort of thing), so forgive me if this is a stupid question (because I'm so curious about it).


Solution

  • Gtp-2 was trained on massive amounts of text all around the internet and is able to generate text by predicting the next word in a sequence of tokens. In theory, the content generated should be driven by the input you provide. Beware that the URLs generated are not real, the model is inventing (hallucinating) them.

    You might also to check GPT-3 as it does a much better job at generating text following the context of the input.