<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p><br>
</p>
<div class="moz-cite-prefix">Am 18.10.2023 um 20:23 schrieb Quincas
Moreira via Synth-diy:<br>
</div>
<blockquote type="cite"
cite="mid:CA+u0Jtn0AwXt+MFcVd59qYayPMwUqsEg1k4o+90vd8Gvqw3+kw@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="auto">Interesting, but my experience is not that it
continues my sentences, rather that it replies to my queries
with very useful information and ideas. I know nothing is
original and it’s derived from farming existing information, but
the result is far more interesting, engaging, useful than simple
predictive text. And it has already learned language it was not
trained on, etc. I’m not scared of it, but I’m intrigued and
interested, it’s neural network and seems to be evolving beyond
what even its programmers expected.</div>
</blockquote>
<p>If I understand this right, to turn this "continuation
enginge"(aka GPT) into a chat bot (aka ChatGPT), it's actually
trained on pairs of Question / Answers, so it continues what the
most likely answer in response to your question is. <br>
</p>
<p>This training data is (was) largely prepared by humans (so far).
I think some groups has harnessed training data prepaired by an AI
to feed into their own project greatly reducing the cost for that.
<br>
</p>
<p><br>
</p>
<p>Best,</p>
<p> René<br>
</p>
<p><br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CA+u0Jtn0AwXt+MFcVd59qYayPMwUqsEg1k4o+90vd8Gvqw3+kw@mail.gmail.com">
<div><br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed 18 Oct 2023 at 11:43
cheater cheater via Synth-diy <<a
href="mailto:synth-diy@synth-diy.org"
moz-do-not-send="true" class="moz-txt-link-freetext">synth-diy@synth-diy.org</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;padding-left:1ex;border-left-color:rgb(204,204,204)">>
I have to say I am very excited to see where this GPT thing
goes but also a little frightened by it.<br>
<br>
most people who say they are either excited or frightened by
GPT say<br>
that because they are mystified by the software, in turn
because they<br>
don't know what it does. So let me give you a short
description.<br>
<br>
My background: I worked as a software engineer in some of
the most<br>
famous AI startups on the recent market, which created less
public<br>
competitors to GPT and ChatGPT.<br>
<br>
The short of it is: remember on your smart phone, when
you're typing<br>
out a message, and it shows you the next word you might type
above the<br>
keyboard? And you can tap it? Sometimes you can keep tapping
and a<br>
sentence will come out? That's the core idea behind GPT.<br>
<br>
Basically, what GPT does - "Generative Predictive Text" is
the<br>
original moniker which later got rebranded to sound more
mystifying -<br>
is that given a start of a sentence, it finishes that
sentence in the<br>
most expected way.<br>
<br>
So let's say you start with:<br>
<br>
Trees are<br>
<br>
GPT has read every text on the planet. It has a frequency
table of<br>
every word that comes after "Trees are". Example
continuations are:<br>
<br>
Trees are green ... (rank 72)<br>
Trees are large ... (rank 1)<br>
Trees are wooden ... (rank 15)<br>
<br>
It finds out the most popular word after "Trees are" and
tacks it on.<br>
<br>
Then, it repeats it with the next one. For example, let's
say the most<br>
popular word was "wooden". Then the new prompt for it is:<br>
<br>
Trees are large<br>
<br>
continuations for this might be:<br>
<br>
Trees are large plants ... (rank 7)<br>
Trees are large, green ... (rank 52999)<br>
Trees are large and ... (rank 122)<br>
<br>
and so on.<br>
<br>
Now OpenAI's GPT actually takes more context than two words.
It'll<br>
look at the whole paragraph you put in, and figure out the
next most<br>
probable word to tack on to the end. But it only ever does
that: it<br>
goes one, word, by, one, word.<br>
<br>
GPT isn't smart. It doesn't know what trees are. When you
ask it what<br>
trees are it doesn't think to itself "hmm, what is my
definition of a<br>
tree, an object I know of?". For GPT, trees don't exist. It
has no<br>
object permanence - like a toddler. If we started a
campaign, where on<br>
every forum, mailing list, news website, and encyclopedia we
say that<br>
trees are made out of metal, GPT 5 will soon enough start
telling<br>
people that:<br>
<br>
Trees are made out of _____ (inserted most popular word:
"metal").<br>
<br>
It's like the kid taken to the blackboard that doesn't know
how to<br>
answer the teacher's question: "Johnny, what is the capitol
of<br>
Colombia?" "It's... uh... er... uh..." (2 minutes pass) "OK,
Johnny,<br>
B...." "Berlin?" "Bo...." "Bo...dapest?" "Bog..." "Bog
roll!"<br>
<br>
There's no reason to be scared of a precocious phone
keyboard.<br>
<br>
And it isn't going anywhere, because interesting output
requires<br>
operating on concepts - not just doing guess-the-next-word.<br>
<br>
GPT has one great application: it's great for if you want to
be lied<br>
to. It's great on assignments like "tell me a sci fi story"
or "tell<br>
me about faeries". But otherwise it has the IQ of an
absolute idiot.<br>
<br>
If you want to understand how GPT "thinks", play an online
game called<br>
Semantle. (google it, I don't want to put in links and end
up in spam<br>
folders). Once you've won a few games, you know a little bit
about how<br>
predictive text sees the world of words.<br>
<br>
On Thu, Oct 12, 2023 at 10:27 PM Kevin Walsh via Synth-diy<br>
<<a href="mailto:synth-diy@synth-diy.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">synth-diy@synth-diy.org</a>>
wrote:<br>
><br>
> A quiet week so...<br>
><br>
> OpenAI.com GPT4.0 has just released image analysis.<br>
><br>
> I tried it for a VCO circuit (Schmitt/Inverter) and it
gave a decent explanation of the circuit.<br>
><br>
> I got it to write Arduino code for a MIDI controlled
baby8 sequencer with nothing but prompts.<br>
><br>
> I have to say I am very excited to see where this GPT
thing goes but also a little frightened by it.<br>
><br>
> Thoughts?<br>
> _______________________________________________<br>
> Synth-diy mailing list<br>
> <a href="mailto:Synth-diy@synth-diy.org"
target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">Synth-diy@synth-diy.org</a><br>
> <a
href="http://synth-diy.org/mailman/listinfo/synth-diy"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://synth-diy.org/mailman/listinfo/synth-diy</a><br>
> Selling or trading? Use <a
href="mailto:marketplace@synth-diy.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">marketplace@synth-diy.org</a><br>
<br>
_______________________________________________<br>
Synth-diy mailing list<br>
<a href="mailto:Synth-diy@synth-diy.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">Synth-diy@synth-diy.org</a><br>
<a href="http://synth-diy.org/mailman/listinfo/synth-diy"
rel="noreferrer" target="_blank" moz-do-not-send="true"
class="moz-txt-link-freetext">http://synth-diy.org/mailman/listinfo/synth-diy</a><br>
Selling or trading? Use <a
href="mailto:marketplace@synth-diy.org" target="_blank"
moz-do-not-send="true" class="moz-txt-link-freetext">marketplace@synth-diy.org</a><br>
</blockquote>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Synth-diy mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Synth-diy@synth-diy.org">Synth-diy@synth-diy.org</a>
<a class="moz-txt-link-freetext" href="http://synth-diy.org/mailman/listinfo/synth-diy">http://synth-diy.org/mailman/listinfo/synth-diy</a>
Selling or trading? Use <a class="moz-txt-link-abbreviated" href="mailto:marketplace@synth-diy.org">marketplace@synth-diy.org</a>
</pre>
</blockquote>
</body>
</html>