Skip to Main Content Carnegie Mellon University Libraries

Best Practices for Large Language Models: Simplified format

This curated resource aims to help you integrate AI tools into your daily academic and professional workflows.

LLM use best practices

LLM Best Practices - Carnegie Mellon University Libraries

LLM Best Practices

Carnegie Mellon University Libraries

Integrating AI tools into academic workflows

Carnegie Mellon University Libraries embraces the rapid advancement of Artificial Intelligence and Large Language Models. Recognizing that these powerful tools often lack practical implementation guidance, we've compiled actionable tips with concrete examples from respected Computer Science and Software Development experts including Simon Willison, Gavin Leech, Kaj Sotala, and Jeff Kaufman. This curated resource aims to help you integrate AI tools into your daily academic and professional workflows.

Collaborative Tool, Not Replacement

Frame LLMs as intelligent collaborators rather than complete replacements for your skills. Simon Willison describes this mindset as thinking of LLMs as "an over-confident pair programming assistant who's lightning fast at looking things up, can churn out relevant examples at a moment's notice and can execute on tedious tasks without complaint".

Example: When developing a new feature, use an LLM to generate the initial code structure, then refine and improve it with your expertise rather than accepting the first output.

Source: Simon Willison

Verify Critical Information

Always fact-check information from LLMs, especially for important decisions. As Gavin Leech notes, he is "already too imprecise for [his] own liking; building in LLMs would make [him] even worse".

Example: When creating documentation with scientific claims or statistics, have the LLM generate a draft but verify key facts against reliable sources before publishing.

Source: Gavin Leech

Inspiration and Ideation

LLMs excel at generating multiple approaches to a problem. As Kaj Sotala shares: "Essay brainstorming. I'll tell Claude 'here's an essay that I started writing' and copy-paste what I've written so far to it. It will comment with ideas, possible other directions, and connections to related things".

Example: When stuck on a creative project, ask the LLM to propose several different angles or perspectives to help overcome writer's block.

Source: Kaj Sotala

Use Specific Instructions

Be explicit about what you want the LLM to do. Simon Willison explains: "For production code my LLM usage is much more authoritarian: I treat it like a digital intern, hired to type code for me based on my detailed instructions".

Example: Instead of "Help me with this function," try "Write a Python function that accepts a URL and returns the domain name using regex, with appropriate error handling for malformed URLs."

Source: Simon Willison

Leverage Context Effectively

Provide relevant context to improve LLM responses. Willison emphasizes: "Most of the craft of getting good results out of an LLM comes down to managing its context—the text that is part of your current conversation".

Example: When asking for help with a bug, include the full error message, relevant code snippets, and details about your development environment.

Source: Simon Willison

Iterate through Conversation

Treat LLM interactions as conversations rather than one-off queries. Jeff Kaufman notes how he iterates on plotting: "I often write code that assembles some data and then want to graph it. Pasting the code into Claude along with a request for the kind of plot I want typically produces something close to what I had in mind, and a good starting place for tweaks".

Example: After receiving initial code, refine it through follow-up prompts like "Make this more efficient" or "Add comments to explain the logic."

Source: Jeff Kaufman

Terminology and Concept Lookups

LLMs excel at explaining terminology or helping recall specific concepts. Kaufman shares how he uses LLMs for "questions like 'what do you call it when ...' with a verbose description and then Google the technical terms it thinks I might be gesturing at".

Example: When you remember a concept but forget its name, describe it to the LLM and use its response as a starting point for further research.

Source: Jeff Kaufman

Code Understanding

LLMs can help explain unfamiliar codebases or complex algorithms. Willison demonstrates using LLMs to analyze a GitHub repository: "I use my own files-to-prompt tool... to gather the contents of all of the files in the repo into a single stream. Then I pipe that into my LLM tool and tell it to prompt Gemini 2.0 Pro with a system prompt of 'architectural overview as markdown'".

Example: When joining a new project, feed the codebase to an LLM and ask for an overview of the architecture and key components.

Source: Simon Willison

Simplify Complex Text

LLMs can translate dense technical or academic writing into more accessible language. Sotala describes: "I found it helpful to copy-paste large parts of a philosophical text into Claude and asked 'could you explain this with simpler language and concrete examples'".

Example: When studying complex academic papers, have the LLM rephrase key paragraphs and provide concrete examples to aid understanding.

Source: Kaj Sotala

Test and Validate Generated Code

Always test LLM-generated code rather than assuming it works. Willison insists: "The one thing you absolutely cannot outsource to the machine is testing that the code actually works".

Example: After receiving a database query from an LLM, run it against a test database to verify it produces the expected results before using it in production.

Source: Simon Willison

Recognize Subject Limitations

LLMs perform better in some domains than others. As Gavin notes, "I encourage you to start by prompting it with a field you know intimately - you will be disappointed".

Example: For creative writing assistance or general brainstorming, rely more heavily on LLMs; for specialized legal or medical advice, use them only as a starting point and consult experts.

Source: Gavin Leech

Adopt Appropriate Skepticism

Adjust your level of skepticism based on the stakes and task at hand. Jeff Kaufman explains: "There are still a lot of things it gets wrong, but it turns out there are many areas where I can pretty easily check the output and where something mostly-right is a good starting point".

Example: For high-stakes tasks like financial analysis, thoroughly verify all LLM outputs; for low-stakes tasks like generating blog post titles, you can be more accepting of initial suggestions.

Source: Jeff Kaufman

Balance Automation with Skill Development

Consider whether using an LLM may prevent you from developing important skills. Gavin mentions "worrying about being deskilled by using them" as a reason for limiting LLM use.

Example: If you're learning a new programming language, solve some problems yourself before turning to an LLM, but use the LLM to check your work or suggest improvements once you've made an honest attempt.

Source: Gavin Leech

Explore Multiple Perspectives

LLMs can help you consider different viewpoints on complex topics. Gavin mentions using LLMs for "strong misreading, in which one develops an entirely new view when reading someone else. Seems like LLMs could help me do this, by producing a weak misreading of a confusing writer which I then misread properly".

Example: When analyzing a controversial policy decision, ask the LLM to argue from different stakeholder perspectives to gain a more comprehensive understanding of the issue.

Source: Gavin Leech

Domain-specific Formatting Tasks

LLMs excel at transforming content between different formats and standards. Jeff Kaufman shares how he used Claude to solve a formatting challenge: "I once asked Claude, 'I have a Google Doc file with some lines that read 'USER:' and 'ASSISTANT:'. Is there a way of programmatically making all of those lines into Heading-3?'"

Example: When converting documentation between different standards (e.g., APA to Chicago style), have the LLM reformat your content while preserving the substance.

Source: Jeff Kaufman

Present Ideas as Someone Else's

To get more objective feedback, frame your own ideas as coming from another source. Gavin notes that "to get around sycophancy I present my ideas as someone else's. ('Someone just claimed hypothesis X. What's the evidence for and against this?')"

Example: Instead of "What do you think of my theory that X causes Y?", try "A researcher recently proposed that X causes Y. What evidence supports or contradicts this hypothesis?"

Source: Gavin Leech

Structure Complex Learning Projects

LLMs can help break down overwhelming topics into manageable learning paths. Sotala describes using LLMs as a "Self-help coach. A lot of self-help books have various exercises or a complicated algorithm to follow... I'll grab a PDF of the book... upload it to Claude, and ask to be coached according to the philosophy in the book".

Example: When learning a complex new programming framework, have the LLM create a step-by-step learning plan that builds skills progressively, with relevant exercises at each stage.

Source: Kaj Sotala