An update concerning the use of ChatGPT as an alternative to web search engines

I gained some more experience in using ChatGPT and I would like to share it with the community. ChatGPT is very useful especially because it has a notion of the entire conversation, therefore it can help pinpointing information necessary to solve a given problem with followup questions way better than a session composed of reading various material obtained using a web search engine.

However, I found in my small sample that it is somehow wrong between 0% and 50% of the times, according to how common is the problem you are discussing. For example, if I would ask questions that concern all Linux distributions, it is probably right more often than not. However, if I were to ask question specific to IPFire, the opposite is true, he is wrong more often than not.

The positive news is that it is wrong frequently in small details. The bad news is that it is wrong in small details, if you catch my drift (given that you are not the current incarnation of ChatGPT, you should).

I had a long conversation when it gave me this piece of code in bash:

#!/bin/bash

set -e

my_var="applebananaorangebanana"
short="${my_var%banana}"
long="${my_var%%banana}"

echo "Short suffix result: $short"
echo "Long suffix result: $long"

which according to ChatGPT should give you the following:

Short suffix result: applebananaorange
Long suffix result: apple

instead the script gives you:

Short suffix result: applebananaorange
Long suffix result: applebananaorange

because the script is omitting the glob * character:

short="${my_var%banana*}"
long="${my_var%%banana*}"

When I pointed out the error, he quickly acknowledged its mistake and updated the script correctly. However, when I deleted the conversation, it forgot the correction. In a subsequent conversation I sent a feedback to the developers of the project, therefore very likely it will give a correct answer sometime in the future.

My conclusion is the following: we are at the dawn of this new technology, as we went from 0 to 1. From 1 to 10 or 100 it will be much, much easier and faster. I won’t comment on the implications for society at large except saying that it is clear that the impact will be huge.

What I will say is this: in few years you will be able to describe in human terms a code you want to have in a given language and OS and it will CORRECTLY give you one all the time (as opposed to some of the time which is the present situation). The opposite is also true, you will be able to past some code and it will explain it to you in human terms. very likely, you will be able to post an entire code base and ask for different explanations at different level of resolution. The barrier for learning to code or simply understanding someone else code will be reduced dramatically.

I am not sure it will be able to replace entirely software engineers (and writers, customer care, human resource … etc.). Probably not any time soon, at least in the case of software engineers.

But it will be able to train unskilled individuals to get to a much higher level than they would get today due to their given IQ, personality, motivation and natural inclination. Opensource software projects will benefit from a more competent community. This is one impact ChatGPT will probably have on IPFire, as an example of Open Source project. On the other end, the project leaders will have a much harder time keeping bugs (intentional or not) out of the code base as the community of developers will inevitably increase, while decreasing the average value of the collective skill-set.

5 Likes

That is some of the more positive points and uses of ChatGPT.
I trained it on a few short stories I have written inspired by an MMO. It did some relevant rewrites, I was able to change “person” mode and some other things. All in all, my result turned up better than before, but only after I did some additional editing, adding my personal tone, that was mostly removed.

You may have noticed the ChatGPT has its own particular way of putting words together, thus a text can be identified as written by ChatGPT, if you do not edit it afterwards.

When this sprung I knew it would be the number one source for essay and text work cheating… ever. And so it has become.

Having said that you bring up some very relevant aspects for developers and scientific and statistical usage that will probably become easier. Also learning material. Having read 3-4 networking books, including Wikipedia, on OSI and similar basic concept, ChatGPT gave me the best and simplest definition of OSI I have ever read.

I think ChatGPT requires examples and elaborations of what you ask it, to present a correct answer, in particular when related to programming.

Then we have the security related issues. ChatGPT will be, and is already, targeted by obscure forces and will eventually have to implement higher security if it does not want to get hacked and cracked to death…

4 Likes

Good point! Its the age-old adage “garbage in - garbage out”. Bigger problems will come when it decides to deliberately give garbage/erroneous/false answers

1 Like

… and it is already helping to create code that can be use in bad ways…

:face_with_raised_eyebrow:

But it will be able to train unskilled individuals to get to a much higher level … due to their given IQ, personality, motivation and natural inclination. Opensource software projects will benefit from a more competent community.

Tottaly agree; Waited 40 years to see a throw dice program in BASIC :smiley:
Even gave me the code to get more lifes in manic minner game. Computers are a puzzle and as long as they puzzle, puzzlers will puzzle :wink:

cfusco, I was actually thinking about it like 2 days ago of why there isn’t yet a ChatGPT version (community free) made or run by the Linux community. The current one is kind of commercial and so it’ll be loaded (as far as responses and predictions) with what ever the tight OpenAI and ChatGPT devs decides to advertise and load it with… If there were a free open source ChatGPT (under any other name) It’d be even more powerful than the current ones. Open community tends to allow everyone around the world to chip in, rather than what ever those two companies are tailoring :sewing_needle: it commercially. Just my two cents. Reason Linux (all it’s variants) is so awesome is due to the open community.