Code Comments in the Age of AI-Assisted Development

Traditional wisdom about commenting code is being challenged by the use of Large Language Models (LLMs).

Key Observations from Practice

1. Comment Volatility with LLMs

   • LLMs often strip out existing comments when editing code.
   • Instructing LLMs to preserve comments can lead to inconsistencies and errors.

2. On-Demand Commenting

   • LLMs excel at adding relevant comments to uncommented code.
   • This ability makes it less critical to maintain comments in the codebase itself.

3. Shift to External Documentation

   • Complex explanations are better suited for separate markdown files (e.g., README.md, docs/feature.md).
   • These external docs can be easily fed to LLMs alongside code for context.

4. LLM-Oriented Documentation

   • Creating documentation with LLMs in mind often results in clearer explanations for humans too.
   • Focus on explaining what LLMs might misunderstand, omitting what they typically grasp well.

Implications for Development Practices

1. Minimize In-Code Comments

• Keep code as clean and comment-free as possible.
• Rely on clear, self-documenting code structures and naming conventions.

2. Leverage LLMs for Commentary

• Use AI tools to generate comments when needed, rather than maintaining them constantly.
• This approach ensures comments are up-to-date with the current code state.

3. Prioritize External Documentation

• Shift detailed explanations, architectural decisions, and business logic to markdown files.
• Organize these docs in a way that's easy to reference alongside code.

4. Develop LLM-Friendly Documentation Practices

• When writing docs, focus on aspects that typically confuse AI:
  • Business logic rationales
  • Non-obvious performance considerations
  • Regulatory or compatibility constraints
• This approach often clarifies documentation for human readers as well.

5. Streamline Code Review Process

• Instead of relying on in-code comments, use pull request descriptions for explaining changes.
• Reference relevant sections of external documentation in PRs.

6. Adopt a Dynamic Documentation Approach

• Treat documentation as a fluid resource that can be generated or augmented by AI as needed.
• Regularly update external docs to reflect current code state and architectural decisions. You can leverage LLMs here, too.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com