Welcome Guest ( Log In | Register )

Outline · [ Standard ] · Linear+

 ChatGPT 50% wrong answers to programming questions, ChatGPT is not the ultimate solution

views
     
TSTaikor.Taikun
post Oct 1 2023, 01:49 PM, updated 12 months ago

Regular
******
Senior Member
1,019 posts

Joined: Sep 2018
QUOTE
ChatGPT gives wrong answers to programming questions more than 50% of the time
By Ross Kelly, published August 08, 2023

Some developers may be placing too much faith in generative AI, experts warn.

ChatGPT has been found to produce incorrect answers for over half of the software engineering questions posed, according to fresh research.

The Purdue University study saw researchers analyze ChatGPT answers to 517 Stack Overflow questions, with the aim of assessing the “correctness, consistency, comprehensiveness, and conciseness” of answers presented by the generative AI tool.

Researchers reported 52% of answers to programming-related queries were inaccurate, while more than three-quarters (77%) were deemed “verbose”.

A key talking point from the study centered around user interpretation of answers presented by ChatGPT, as well as the perceived legitimacy of answers produced by the chatbot.

https://www.itpro.com/technology/artificial...-50-of-the-time
PATAR
post Oct 16 2023, 02:41 PM

Getting Started
**
Junior Member
197 posts

Joined: Jan 2018
From: Sweden/Kedah
In my own experience Chat-GPT is great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though
Natsukashii
post Oct 16 2023, 02:44 PM

^^
******
Senior Member
1,084 posts

Joined: Jul 2022
QUOTE(PATAR @ Oct 16 2023, 02:41 PM)
In my own experience Chat-GPT is  great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though
*
Have you tried Bing?

If yes, so despite flaws, ChatGPT is still better than the others?


TSTaikor.Taikun
post Oct 16 2023, 03:07 PM

Regular
******
Senior Member
1,019 posts

Joined: Sep 2018
QUOTE(PATAR @ Oct 16 2023, 02:41 PM)
In my own experience Chat-GPT is  great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though
*
Still a useful tool for many ppl

QUOTE(Natsukashii @ Oct 16 2023, 02:44 PM)
Have you tried Bing?

If yes, so despite flaws, ChatGPT is still better than the others?
*
I think ChatGPT has already established itself in the mainstream
PATAR
post Oct 16 2023, 04:54 PM

Getting Started
**
Junior Member
197 posts

Joined: Jan 2018
From: Sweden/Kedah
QUOTE(Natsukashii @ Oct 16 2023, 02:44 PM)
Have you tried Bing?

If yes, so despite flaws, ChatGPT is still better than the others?
*
I prefer chat-gpt.
When and if i can use GPT-4 with Bing on my desktop I think it will be better. I already get that on my mobile.
But so far I only get GPT-3 on Bing on desktop.
MatQuasar
post Oct 27 2023, 07:58 PM

Casual
***
Validating
329 posts

Joined: Jun 2023
Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks

https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/

QUOTE
Goodbye programmers?
Perhaps soon, but not yet. ....

So, for the time being, I think we’re going to see a rapid rise in human-AI cooperation rather than outright replacement.

flashang
post Oct 27 2023, 11:00 PM

Casual
***
Junior Member
355 posts

Joined: Aug 2021


QUOTE(MatQuasar @ Oct 27 2023, 07:58 PM)
Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks

https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/
*
You may use machine to produce software like a factory.

But you may need engineer to design,
and technician to find out if there is some bugs or backdoor.

smile.gif


TSTaikor.Taikun
post Oct 28 2023, 08:35 AM

Regular
******
Senior Member
1,019 posts

Joined: Sep 2018
QUOTE(MatQuasar @ Oct 27 2023, 07:58 PM)
Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks

https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/
*
QUOTE(flashang @ Oct 27 2023, 11:00 PM)
You may use machine to produce software like a factory.

But you may need engineer to design,
and technician to find out if there is some bugs or backdoor.

smile.gif
*
I don’t think it’s a full-fledge solution. Just a proof of concept. Chatgpt can be wrong 50% of the time. And yes, we still need engineer to fine tune the codes
zenix
post Dec 30 2023, 11:35 AM

Pirate Captain
*******
Senior Member
6,249 posts

Joined: Jul 2006
no matter what non-IT bosses think of AI, it is not matured enough to replace anyone, but it is a good tool to guide people, push you in the right direction, or just like talking to someone about your problems until u find the answer yourself.

chat buddy for introverts tongue.gif
SUSifourtos
post Dec 30 2023, 11:39 AM

Look at all my stars!!
*******
Senior Member
2,256 posts

Joined: Feb 2012



this olis not how u use gpt...

Create custom ai assistance for boss, charge 150-350 per month.
SUSSihambodoh
post Dec 30 2023, 11:39 AM

Casual
***
Junior Member
436 posts

Joined: Dec 2021

My experience with it is bad. Too much hallucination.


gooroojee
post Dec 30 2023, 11:39 AM

LowYee
*******
Senior Member
2,193 posts

Joined: Feb 2012
I think ChatGPT was initially genius when it's user base and training data came from the top 0.001% of the world's population.

And then they let it loose in the world, where half the world's population is below average IQ... hehehe
lolzcalvin
post Jan 3 2024, 10:56 PM

shibe in predicament
******
Senior Member
1,596 posts

Joined: Mar 2014
From: 75°26'11.6"S, 136°16'16.0"E


just saying that this is true, but I'd make that 50% correct answers to be even lower
used chatgpt to help kicking start a java spring project (was completely inexperienced in java and spring)
many answers given by chatgpt either didn't make sense or outright wrong
ok for boilerplate, but bad for many specifics
frequency
post Jan 4 2024, 09:10 AM

我要挑战十个!
*******
Senior Member
2,567 posts

Joined: Jan 2003


sometimes chatgpt just like the boy answering questions in final exam

die die must answer something and make you believe it is the correct answer

you need verify the answer
Kayziealnatz
post Oct 13 2024, 03:57 AM

Getting Started
**
Junior Member
178 posts

Joined: Oct 2024
QUOTE(MatQuasar @ Oct 27 2023, 08:58 PM)
Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks

[url=https:// vulcanpost.com/843379/ team-of-ai-bots-develops-software-in-7-minutes-instead-of-4-weeks/]https:// vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/[/url]
*
If AI can do almost all we can do, then don't give them the ability to do maintenence on themselves, then only we won't be replaced, ya?
15cm
post Oct 17 2024, 11:17 PM

Casual
***
Junior Member
423 posts

Joined: Apr 2022
chatgpt is ok for popular programming language

the more obscure the thing you are doing, the higher chances you get bullshit answer

the irony is for knowledge that are more secure is where you need chatgpt the most

so its not all that useful. i use chatgpt to do task i already know how to do but too lazy to do myself because i can verify the output

i was doing kubernetes, terraform, github workflow all those stuff

the bloody chatgpt gave me wrong answer when i ask me to proof read the .kube deployment file. it die die tell me i must create volume mount but i actually dont because i already COPY the file over using docker

wasted 3 hours of my life wondering whats wrong
cubiclecarbonate
post Oct 28 2024, 03:38 PM

On my way
****
Junior Member
558 posts

Joined: Jul 2011


Hmmm. I think it depends on how the user entered their inputs. Or how strong their programming fundamental and knowledge. User really needs the hands on experience prior to the expected response.

In my case, my productivity goes beyond 100% improvement. Like 2-3 sprints can be shortened to halfday or a day max. Excluding testing.
danieln
post Nov 5 2024, 08:11 PM

Regular
******
Senior Member
1,307 posts

Joined: Feb 2008
I just use copilot on windows, tested using ollama as well.. LOL

but I don't ask AI to write the whole function, but only help on certain parts. so far it does help on getting code written faster, but you have to make sure to know what code you paste in, else it will be a nightmare
shyan90's
post Nov 11 2024, 12:38 PM

To Be Old and Wise You Must First be Young and Stupid
******
Senior Member
1,204 posts

Joined: Jun 2015
From: Malaysia


They not provide sufficient details isit? So far i have no problem to use Chatgpt to generate some script...but need put very detail to get the correct 1

babisotong
post Dec 16 2024, 11:25 AM

On my way
****
Junior Member
654 posts

Joined: Feb 2013


https://chatgpt.com/ is good for detailed explanation and example
and
https://poe.com/ChatGPT short and simple

both answer/code need to be tested and fix to work on your solution.

2 Pages  1 2 >Top
 

Change to:
| Lo-Fi Version
0.0183sec    0.74    5 queries    GZIP Disabled
Time is now: 23rd December 2025 - 04:39 AM