LESSWRONG
LW

Comment Permalink
Vladimir_Nesov33

Tagging is conditioning (see also). If instead of having SSL model learn text, you have it learn (summary, text), then it learns to predict text from summary. A summary can be any measurement of text such that some values of that measurement can be a useful query. For example, if text is a chess game, a summary could be a statement of who wins. Then, starting a prompt with a claim of winning will tend to lead to a winning game. Similarly, if summary says if text is a valid argument, you gain the ability to query for valid arguments. Finally, summary can well be written by a language model, using a prompt that includes text and instructions to write an appropriate summary. If summary is a free-form text description, queries can become arbitrary texts as well, including those not appearing in the training dataset.

Reply
See in context
by Viliam
1 min read
A
17

26

(Epistemic status: I do not understand GPT deeply, so this is just a random idea.)

If I understand it correctly, GPT learns from existing texts. Lots of texts.

Would it be possible to make GPT smarter by simply giving it smarter text? Of course, writing tons of smarter text would be a lot of work, but what about annotating the existing text, like "take this more seriously" and "take this less seriously"? (From technical perspective, maybe the GPT should read the text marked as serious five times?) Assuming that the annonation is roughly correct, would this improve the results?

*

If yes, the problem is how to select smarter texts, especially if we want lots of them? But I think some good guesses can be made:

  • High-school textbooks. They should be educational and relatively uncontroversial (settled science). There probably are some reviews of textbooks, so only annotate the ones with good reviews.
  • Some parts of Reddit are probably much better than average, in the sense that smart content gets upvoted. So include the upvoted comments.
GPTAI
Frontpage

26

GPT learning from smarter texts?
New Answer
New Comment

1 Answers sorted by
top scoring

Razied

102

I think the more serious and interesting problem is how to prompt GPT to produce more serious text. If we just give it all existing text to learn from, it will have to model both the stupid and the smart, which is fine, since a smart person should be able to impersonate a stupid one without losing their capabilities. The problem we seem to have is that we can't efficiently tell GPT when to pretend to be stupid and when not to do so, right now it seems to be picking up on cues we give it from the prompt. If our prompt looks like something someone on reddit wrote, the response is also stupid, if the prompt looks like a paragraph from a scientific paper, the response is likely to be GPT's best attempt at reasoning. 

So in my view one of the best ways to improve GPT is to tag every piece of text it learns from with its provenance and date of publication. These would be  "tags from god" so to speak, so even if a piece of text it trains on is a story that starts with "it was the year 2034...", the tag for that piece of text would specify "publication date: 1985", so GPT would not be fooled into confusing stories written about the future, with the actual future. It would also not confuse scientific articles with just random blog posts, or again stories. So when we prompt it at inference-time and give it "Introduction to Fusion Plant Design, Cambridge University press, Chapter 1: ", we can tag our prompt with both "scientific article" and "publication date: 2040" in order to ensure that GPT is trying to produce a legitimate article and actually trying to model the future progress of science. Giving it this prompt right now would likely produce rubbish, because it might infer that the title to that textbook appears in a random comment like this one, and so GPT wouldn't actually try to produce a textbook.

[-]Radford Neal70

Tagging all text with the date it was produced is very important for other reasons as well.  (And as far as I know, it isn't done at present.)

With such date tags, one could ask GPT to produce text for a specific date.  For example, one could ask it to continue the prompt "I think people of different races should" with dates of 1880, 1920, 1960, and 2000 to see how racial attitudes have changed over the years, which is very interesting historical information.  It also would allow one to force a 2023 date for normal responses, to avoid unwante... (read more)

Reply
4gwern
It has only been done by "Time-Aware Language Models as Temporal Knowledge Bases", Dhingra et al 2021 (and done mostly in passing by earlier experiments in providing metadata prefixes like CTRL). Temporal reasoning scales, unsurprisingly: even without explicit metadata, which would be extremely hard to get reliably for most cases (eg Common Crawl - dating random web pages at scale? gl), there tend to be lots of implicit clues in text, such as the URL structure of news articles (CTRL gives the example of https://www.cnn.com/style/09/20/2018/george-clooney-interview), and this probably serves as scaffolding for helping understand the internal evidence of undated text. You can already prompt a model like GPT-3 with dates already, so you wouldn't be creating any qualitatively new capabilities. So including more metadata (of every kind, not just dating) is a good idea, but not necessary and may be a bad use of expert human labor: probably it'd be so cheap that it's worth hand-engineering in for clean sources like Wikipedia or Twitter or Reddit or academic datasets where you can be sure of the date easily, but then less so for the bulk of the dataset coming from Common Crawl etc.
2Razied
Oh! Pretty cool, I hadn't thought of that effect. Another consequence of tagging all text with the date that seems particularly interesting to me is that it allows us to query GPT on its beliefs about the future in a more direct way. Say you want to know if GPT believes that a war would break out in France by 2040, we could ask GPT to give us the likelihood for the text "France enters war!" tagged with "journal: New York Times; Date: 2040". We could see how the likelihood for that phrase changes with different tagged dates in order to see what GPT believes. We can repeat this with any sort of headlines we want. We only need that GPT believes that the New York Times is a relatively accurate source of facts about the real world for this to work. A further trick is that we can ask GPT about how its own outputs will affect the world. Suppose we ask GPT to produce that "Fusion Plant Design" textbook I mentioned in the first comment. We can then take the textbook it outputs, change its date tag to 2023, introduce it into GPT's training set and take a small gradient step with it, this essentially makes GPT "aware" that the textbook now exists in the world, as if it was released publicly. We then ask this updated model about the likelihood of future war in France through the same way as above. In effect this allows us to answer the question "Does GPT think that releasing this textbook will increase or decrease the likelihood of war in France by 2040?". It would be hopeless to directly ask it that question, because no human could possibly know the answer to that, so GPT won't give it directly, but we can still tease it out if we use date-tagged data like this.
[-]Yejun Y.42

I think that prompting is definitely important. I've found that GPT as it is now can mimic any given author's style with great accuracy as long as it's given that author's text inside of the prompt. For example, "write a short story in the style of Nabokov" gives you a bland short story, while prompting with his verbatim text produces a pretty faithful continuation. 

Reply
[-]Vladimir_Nesov33

Tagging is conditioning (see also). If instead of having SSL model learn text, you have it learn (summary, text), then it learns to predict text from summary. A summary can be any measurement of text such that some values of that measurement can be a useful query. For example, if text is a chess game, a summary could be a statement of who wins. Then, starting a prompt with a claim of winning will tend to lead to a winning game. Similarly, if summary says if text is a valid argument, you gain the ability to query for valid arguments. Finally, summary can we... (read more)

Reply
1 comment, sorted by
top scoring
Click to highlight new comments since:
[-]Owain_Evans20

See the Galatica model (https://arxiv.org/abs/2211.09085) from Meta. It's trained on a curated dataset of scientific papers, reference materials and scientific knowledge bases (with only a very small % of random internet text). IIRC the benefits of this seem limited (better to train on a bigger dataset and use other techniques to make the model access the sciencey parts of the training set).

Reply
Moderation Log
More from Viliam
View more
Curated and popular this week
A
11