On the other side of the equation I've been spending much more time on code-review on an open source project I maintain, because developers are much more productive and I still code-review at the same speed.
The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:
- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.
- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.
- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.
I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.
But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.
With respect to the first issue you raise, I would perhaps start including prompts in comments. This is a little sneaky sure. And maybe explicitly putting them in a markdown would be better. But there's the risk that markdown won't be loaded. Perhaps it might be possible to inject the file into context via a comment, I've never tried that though and I doubt every assistant will act in a consistent way. The comment method is probably the best bet IMO.
Forgive me because this is a bit of a tangential rant on the second issue, but Gemini Pro 3 was absolutely heinous about this so I cancelled my sub. I'm completely puzzled what it's supposed to be good for.
To your third issue, you should maybe consider building a dataset from those interactions... you might be able to train a LoRA on them and use it as a first pass before you lift a finger to scroll through a PR.
I think a really big issue is that there is a lack of consistency in the use of AI for SWE. There are a lot of models and poorly designed agents/assistants with really unforgivable performance and people just blindly using them without caring about the outputs amounts to something that is kind of Denial-of-Service-y and I keep seeing this issue be raised over and over again.
At the risk of sounding elitist, the world might be a better place for project maintainers when the free money stops rolling into the frontier labs to offer anyone and everyone free use of the models...never give a baby powertools and so on.
That basically matches, in broad outline, what I see from AI use in an enterprise environment; absent a radical change, I think the near term impact of AI on software development is going to be too increase velocity while shifting workload to less (but not zero) code writing and more code reviewing and knowing when you need to prompt-and-review vs. hand-code.
imho that is just silly ... I can see various ways censorship and freedom and common good at the same time. Actually, I can imagine different set ups where this could work...
But then, you have to define these things. E.g.: freedom of person "A" to kill person "B" infringes on person "B" freedom of come and go and not be killed (by "A" or anyone else) ... so what is freedom. "Common good" is even more complicated ... who should defined it ? And how ?
On the other topic, I for one think that censorship of AI generated content and fake news, as well as AI generated ordering of results should be censored. But it's not that easy, and implementing that is an even bigger can of worms.
the issue is how do you prove the content was written by AI?
> But then, you have to define these things. E.g.: freedom of person "A" to kill person "B" infringes on person "B" freedom of come and go and not be killed (by "A" or anyone else) ... so what is freedom. "Common good" is even more complicated ... who should defined it ? And how ?
even worse - how do you make sure the definition of such terms stays up to date with changing times?
Of course one can generalize using the colloquial "Here in Europe". And generalization is useful -- one cannot go into all the complexity and details all the time, at some point one has to summarize/generalize an argument.
Yes, Europe is not a monolithic bloc, but there is a large fraction that is less sex focused, it's a fair generalization and comment to express that.
Go is as good a language as any for an ML framework. Better if you buy into it being better for its simplicity. It's clearly a worse if you factor in the ecosystem that already exists in Python, and one is leveraging that.
The C FFI ("foreign function interface") doesn't play a role at the ML Framework level. One does not need to call a C function at a fine-grained level. One step of inference can be in the milliseconds time scale for LLMs, the C FFI costs are almost 5 orders of magnitude smaller (10s or 100s of nanoseconds?). Look, Python C bindings are also costly, and things just work.
Plus Go doesn't need a bridge to Python necessarily. It only needs a bridge to whatever Python is also using, if it wants to be performant.
Having said that, one cannot expect to implement the lower level operations in Go efficiently (well, competitively with what is best out there), or JIT-compile a computation graph to Go (doable but not as performant anyway). But that doesn't matter, fast execution of computation graphs can be implemented in C++/Rust/Cuda/etc. The ML Framework level of operations (building models, layers, etc.) can all be done in Go (or Rust, Elixir, Julia, Occam, Lisp, Basic, etc.)
But even that, like most things in life, folks think every problem needs the fastest of the fastest ... one can do matrix multiplication fast enough in Go for the majority of the small ML models out there -- for many tasks the time spent in ML inference is just a small fraction of a cost of the total system. It really doesn't matter if it is an order of magnitude slower.
Sorry but this is just plain wrong. The real issue is Cuda not Python. Go's memory model makes it to where it has to trampoline to C, this is as much as 300ns in Go which is about 10x what Python is. These things add up significantly when doing ML training.
1. The O(N^2*d) computation cost of the attention layers. For large graphs (millions of nodes) it's quickly too costly. And in some of the social network problems, the more data you feed the better is the inference on average (on a ~log scale).
2. As the paper suggests, in some cases the graph structure has important information. Flattening out everything, or fully connecting the nodes, a more accurate description of what goes on in an attention layer in this scenario, the structure is lost. The structure information can be introduced as a positional encoding -- see paper (edited/fixed): https://arxiv.org/pdf/2207.02505.pdf
So remember to do that if you attempt the attention solution.
3. Then there is overfitting, already a big issue in GNNs. Fully connecting every node with attention has less of an "inductive bias" if you will, created by the graph structure. Not sure how much it matters ...
Noob question: when folks talk about fine-tuning LLM, do they usually fine-tune the encoder (of the prompt), the decoder (that generates the text) or both ?
Both can be done. Fine-Tuning the prompt is cheaper and can be done on consumer hardware. Fine-tuning the LLM model weights is expensive and needs cloud support.
Thanks for the reply, but when you mean "fine-tuning" for the prompt, do you mean fine-tuning of the LLM encoder of the prompt right ? (The thing that transforms the prompt into a sequence of embeddings?) But that is not cheap/easy to train ...
I know some systems also allow an extra fixed embedding parameter to the prompt, that can also be fine-tuned. But that is yet another thing that can be fine-tuned very cheaply.
One way is to use categorical set splits [1] (proposed for categorical set inputs, but works for categorical features as well), used in TF-DF [1]. Greedy, and expensive to train (cheap inference though), but it gives great results.
The real issue is that I can't trust the AI generated code, or trust the AI to code-review for me. Some repeated issues I see:
- In my experience the AI doesn't integrate well with the code that there is already there: it often rewrites functionality and tend not to adhere to the project's conventions, but rather use what it is trained on.
- The AI often lacks depth into more complex issues. And because it doesn't see the broader implication of changes, it often doesn't write the tests that would cover them. Developers that wrote the PRs accept the AI tests without much investigation into the code-base. Since the changes passes the (also insufficient) tests, they send the PR to code-review.
- With AI I think (?) I'm more often the one careful deep diving into the project and re-designing the generated code in the code-review. In a way it's an indirect re-prompting.
I'm very happy with the increased PRs: they push the project forward, with great ideas of what to implement, and I'm very happy about AI increased productivity. Also, with AI developers are bolder in their contributions.
But this doesn't scale -- or I'll spend all my time code-reviewing :) I hope the AIs get better quickly.
reply