> maybe the tests aren't the best designs given there is no way I could review that many tests in 3 hours,
If you haven't reviewed and signed off then you have to assume that the stuff is garbage.
This is the crux of using AI to create anything and it has been a core rule of development for many years that you don't use wizards unless you understand what they are doing.
I used a static analysis code coverage tool to guarantee it was checking the logic, but I did not verify the logic checking myself. The biggest risk is that I have no way of knowing that I codified actual bugs with tests, but if that's true those bugs were already there anyways.
I'd say for what I'm trying to do - which is upgrade a very old version of PHP to something that is supported, this is completely acceptable. These are basically acting as smoke tests.
You need to be a bit careful here. A test that runs your function and then asserts something useless like 'typeof response == object' will also meet those code coverage numbers.
In reality, modern LLMs write tests that are more meaningful than that, but it's still worth testing the assumption and thinking up your own edge cases.
Thanks. Do you mean the Totally Under Control documentary from 2020?
One question, have you ever considered the opposite of what you're saying to be true, or looked for the evidence for that? Saying because I've heard of and looked at both opinions you've expressed in your comment and heard and seen evidence for them to be true. I also did the opposite. And looking at the opposite seemed more true objectively and with the emotions and popular biases like authority bias and other harmful ones removed.
"If you can't see anything wrong with the side you agree with, and you can't see anything right with the side you disagree with, you've been manipulated." Very wise quote.
Authority bias is the cognitive bias where we give disproportionate weight to the opinion of someone perceived as an authority figure, even outside their domain of expertise.
What a lot of people outside the scientific field fail to realise that when a claim is made by an expert in their field, it is peer reviewed and challenged or accepted.
The whole point of science is that you can challenge wrong ideas and change your perspective.
Like I said though. If you are just going to the internet to find something that aligns with what you believe, that isn't peer review.
> Authority bias is where an expert in field X claims to have knowledge in field Y when they don't.
Did you not share this above? This contradicts to me your definition here, which is just copying what I shared and stating it as thought its now your opinion. Comes off like gaslighting by the way.
> write down some really good documentation and explanations that are then read by the other stakeholders, so that these, at the end, also have a very deep knowledge about the topic.
I wish. The reality at the moment is someone gets an LLM to do it and the other person uses an LLM to read it.
Two major issues occur unless they have experience.
1. The developer will often have what is called "Acquired knowledge". That is information that is relevant but isn't in any of the files and the developer assumes other developers know what they know.
2. Often is the case that there is more information required that doesn't sit inside the code and is not evident to get the program to work. Quickest way to find that is to get a newbie on a clean machine to follow only the instructions.
> Technical writing is part of the job of software engineering.
Where I work we have professional technical writers and the quality vs your typical SW engineer is night and day. Maybe you got lucky with the rare SW engineer that can technical write.
Nice read after the earlier post saying fire all your tech writers. Good post.
One thing to add is that the LLM doesn't know what it can't see. It just amplifies what is there. Assumed knowledge is quite common with developers and their own code. Or the more common "it works on my machine" because something is set outside of the code environment.
Sadly other fields are experiencing the same issue of someone outside their field saying AI can straight up replace them.
If you haven't reviewed and signed off then you have to assume that the stuff is garbage.
This is the crux of using AI to create anything and it has been a core rule of development for many years that you don't use wizards unless you understand what they are doing.
reply