Outline ·
[ Standard ] ·
Linear+
ChatGPT 50% wrong answers to programming questions, ChatGPT is not the ultimate solution
|
TSTaikor.Taikun
|
Oct 1 2023, 01:49 PM, updated 12 months ago
|
|
QUOTE ChatGPT gives wrong answers to programming questions more than 50% of the timeBy Ross Kelly, published August 08, 2023Some developers may be placing too much faith in generative AI, experts warn. ChatGPT has been found to produce incorrect answers for over half of the software engineering questions posed, according to fresh research.The Purdue University study saw researchers analyze ChatGPT answers to 517 Stack Overflow questions, with the aim of assessing the “correctness, consistency, comprehensiveness, and conciseness” of answers presented by the generative AI tool. Researchers reported 52% of answers to programming-related queries were inaccurate, while more than three-quarters (77%) were deemed “verbose”. A key talking point from the study centered around user interpretation of answers presented by ChatGPT, as well as the perceived legitimacy of answers produced by the chatbot. https://www.itpro.com/technology/artificial...-50-of-the-time
|
|
|
|
|
|
PATAR
|
Oct 16 2023, 02:41 PM
|
Getting Started

|
In my own experience Chat-GPT is great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though
|
|
|
|
|
|
Natsukashii
|
Oct 16 2023, 02:44 PM
|
|
QUOTE(PATAR @ Oct 16 2023, 02:41 PM) In my own experience Chat-GPT is great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though Have you tried Bing? If yes, so despite flaws, ChatGPT is still better than the others?
|
|
|
|
|
|
TSTaikor.Taikun
|
Oct 16 2023, 03:07 PM
|
|
QUOTE(PATAR @ Oct 16 2023, 02:41 PM) In my own experience Chat-GPT is great at pushing me in the right direction to solve a coding problem. But bad at making a totally compliant solution. Google Bard is worse though Still a useful tool for many ppl QUOTE(Natsukashii @ Oct 16 2023, 02:44 PM) Have you tried Bing? If yes, so despite flaws, ChatGPT is still better than the others? I think ChatGPT has already established itself in the mainstream
|
|
|
|
|
|
PATAR
|
Oct 16 2023, 04:54 PM
|
Getting Started

|
QUOTE(Natsukashii @ Oct 16 2023, 02:44 PM) Have you tried Bing? If yes, so despite flaws, ChatGPT is still better than the others? I prefer chat-gpt. When and if i can use GPT-4 with Bing on my desktop I think it will be better. I already get that on my mobile. But so far I only get GPT-3 on Bing on desktop.
|
|
|
|
|
|
MatQuasar
|
Oct 27 2023, 07:58 PM
|
|
Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/QUOTE Goodbye programmers? Perhaps soon, but not yet. ....
So, for the time being, I think we’re going to see a rapid rise in human-AI cooperation rather than outright replacement.
|
|
|
|
|
|
flashang
|
Oct 27 2023, 11:00 PM
|
|
QUOTE(MatQuasar @ Oct 27 2023, 07:58 PM) Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/You may use machine to produce software like a factory. But you may need engineer to design, and technician to find out if there is some bugs or backdoor.
|
|
|
|
|
|
TSTaikor.Taikun
|
Oct 28 2023, 08:35 AM
|
|
QUOTE(MatQuasar @ Oct 27 2023, 07:58 PM) Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks https://vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/QUOTE(flashang @ Oct 27 2023, 11:00 PM) You may use machine to produce software like a factory. But you may need engineer to design, and technician to find out if there is some bugs or backdoor.  I don’t think it’s a full-fledge solution. Just a proof of concept. Chatgpt can be wrong 50% of the time. And yes, we still need engineer to fine tune the codes
|
|
|
|
|
|
zenix
|
Dec 30 2023, 11:35 AM
|
|
no matter what non-IT bosses think of AI, it is not matured enough to replace anyone, but it is a good tool to guide people, push you in the right direction, or just like talking to someone about your problems until u find the answer yourself. chat buddy for introverts
|
|
|
|
|
|
SUSifourtos
|
Dec 30 2023, 11:39 AM
|
|
this olis not how u use gpt...
Create custom ai assistance for boss, charge 150-350 per month.
|
|
|
|
|
|
SUSSihambodoh
|
Dec 30 2023, 11:39 AM
|
|
My experience with it is bad. Too much hallucination.
|
|
|
|
|
|
gooroojee
|
Dec 30 2023, 11:39 AM
|
|
I think ChatGPT was initially genius when it's user base and training data came from the top 0.001% of the world's population.
And then they let it loose in the world, where half the world's population is below average IQ... hehehe
|
|
|
|
|
|
lolzcalvin
|
Jan 3 2024, 10:56 PM
|
|
just saying that this is true, but I'd make that 50% correct answers to be even lower used chatgpt to help kicking start a java spring project (was completely inexperienced in java and spring) many answers given by chatgpt either didn't make sense or outright wrong ok for boilerplate, but bad for many specifics
|
|
|
|
|
|
frequency
|
Jan 4 2024, 09:10 AM
|
|
sometimes chatgpt just like the boy answering questions in final exam
die die must answer something and make you believe it is the correct answer
you need verify the answer
|
|
|
|
|
|
Kayziealnatz
|
Oct 13 2024, 03:57 AM
|
Getting Started

|
QUOTE(MatQuasar @ Oct 27 2023, 08:58 PM) Goodbye programmers: Team of AI bots develops software in 7 minutes instead of 4 weeks [url=https:// vulcanpost.com/843379/ team-of-ai-bots-develops-software-in-7-minutes-instead-of-4-weeks/]https:// vulcanpost.com/843379/team-of-ai-bo...ead-of-4-weeks/[/url] If AI can do almost all we can do, then don't give them the ability to do maintenence on themselves, then only we won't be replaced, ya?
|
|
|
|
|
|
15cm
|
Oct 17 2024, 11:17 PM
|
|
chatgpt is ok for popular programming language
the more obscure the thing you are doing, the higher chances you get bullshit answer
the irony is for knowledge that are more secure is where you need chatgpt the most
so its not all that useful. i use chatgpt to do task i already know how to do but too lazy to do myself because i can verify the output
i was doing kubernetes, terraform, github workflow all those stuff
the bloody chatgpt gave me wrong answer when i ask me to proof read the .kube deployment file. it die die tell me i must create volume mount but i actually dont because i already COPY the file over using docker
wasted 3 hours of my life wondering whats wrong
|
|
|
|
|
|
cubiclecarbonate
|
Oct 28 2024, 03:38 PM
|
|
Hmmm. I think it depends on how the user entered their inputs. Or how strong their programming fundamental and knowledge. User really needs the hands on experience prior to the expected response.
In my case, my productivity goes beyond 100% improvement. Like 2-3 sprints can be shortened to halfday or a day max. Excluding testing.
|
|
|
|
|
|
danieln
|
Nov 5 2024, 08:11 PM
|
|
I just use copilot on windows, tested using ollama as well.. LOL
but I don't ask AI to write the whole function, but only help on certain parts. so far it does help on getting code written faster, but you have to make sure to know what code you paste in, else it will be a nightmare
|
|
|
|
|
|
shyan90's
|
Nov 11 2024, 12:38 PM
|
|
They not provide sufficient details isit? So far i have no problem to use Chatgpt to generate some script...but need put very detail to get the correct 1
|
|
|
|
|
|
babisotong
|
Dec 16 2024, 11:25 AM
|
|
https://chatgpt.com/ is good for detailed explanation and example and https://poe.com/ChatGPT short and simple both answer/code need to be tested and fix to work on your solution.
|
|
|
|
|