Renneder@sh.itjust.worksM to BecomeMe@sh.itjust.works · 1 year agoChatGPT gets code questions wrong 52% of the timewww.theregister.comexternal-linkmessage-square9fedilinkarrow-up142arrow-down13cross-posted to: [email protected]
arrow-up139arrow-down1external-linkChatGPT gets code questions wrong 52% of the timewww.theregister.comRenneder@sh.itjust.worksM to BecomeMe@sh.itjust.works · 1 year agomessage-square9fedilinkcross-posted to: [email protected]
minus-squareQfuiyh@lemm.eelinkfedilinkarrow-up2arrow-down1·1 year agoTitle feels misleading, it gets stack overflow questions wrong 52% of the time However it got 77% of easy Leetcode questions correct. Also I believe that’s first try, which is not generally how chatgpt should be used. Also also, you should probably be using a coding specific model if you want good coding results
minus-squarenanoUFO@sh.itjust.workslinkfedilinkarrow-up5·1 year agoEvery leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.
minus-squareHackerJoe@sh.itjust.workslinkfedilinkarrow-up2·1 year agoBut it confidently explains the wrong answers. I just hope politicians don’t find out how to use it. It’ll be our doom.
Title feels misleading, it gets stack overflow questions wrong 52% of the time
However it got 77% of easy Leetcode questions correct. Also I believe that’s first try, which is not generally how chatgpt should be used.
Also also, you should probably be using a coding specific model if you want good coding results
Every leetcode question has been answered a billion times and you train it on those billions of answers, it should get those right.
But it confidently explains the wrong answers.
I just hope politicians don’t find out how to use it. It’ll be our doom.