LLMs.TXT: What It Is, Why It Exists, & How To Create One

If you’re anything like most of the B2B marketers i’m hearing from these days, new AI developments are giving you motion sickness.

One day your content is yours, the next it’s fueling some chatbot that will happily spit out a response for some college kids next essay.

I can’t stop progress, nor do I want to.

But there is a new tool in our arsenal to talk to these spicy autocomplete machines [1] and even give them some direction that might be useful for your brand visibility in the future.

That’s where llms.txt comes in.

It’s a simple way to tell AI models what they can and can’t do with your site.

If the words robots.txt make sense to you, that’s fantastic. You’re already halfway there (anybody else read that in Jon’s voice?).

If text files mean nothing to you, don’t worry. That’s quite alright.

This guide breaks it all down in plain English so you can keep your content in check without getting lost in the AI chaos and maybe improve the robots ability to know what you do, how you do it, and who you do it for.

Rock on.

What is llms.txt?

llms.txt is a new type of file designed to provide guidelines for how large language models (LLMs) interact with website content.

From Jeremy Howard @ llmstxt.org:

“A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.”

Similar to robots.txt—which tells web crawlers how to navigate a site—llms.txt specifies whether AI models can use, summarize, generate content from, or train on data from a website.

This file is part of a growing movement toward transparency and control over how AI models engage with publicly available online content.

Why Was llms.txt Created?

AI-powered search and content generation blew up overnight, and suddenly, website owners realized their content was being used in ways they never agreed to.

Unlike search engines that just index pages for rankings, AI models don’t just read your content.

The goal of those sneaky LLMs is to learn from it, summarize it, and sometimes straight-up repurpose it in responses. That didn’t sit right with a lot of folks.

Why It Matters To Ya

• Privacy and Content Ownership – Not everyone wants their hard work feeding the AI machine for free.
• Commercial Considerations – If a company invests in top-tier content, why should AI companies get to use it without permission?
• Customization and Control – Some businesses are fine with AI-powered search but draw the line at training or summarization.
• Standardization – The internet runs on agreed-upon rules, and this is a step toward making AI play by them.

With llms.txt, businesses, content creators, and website owners get to decide how AI interacts with their content instead of just hoping for the best.

How is llms.txt Used?

This little text file might not look like much, but it gives you real control over how AI interacts with your content.

Think of it as house rules for AI models. You decide what they can and can’t do.

Here’s what it helps with:

• Blocking AI Training – If you don’t want your content fueling the next chatbot, you can shut that down.
• Controlling AI Summarization – Maybe you’re fine with AI-powered search but don’t want your content getting chopped up into lazy summaries. Your site, your rules.
• Allowing AI Indexing Without Training – You can make sure AI-driven search results include your content without handing over your data for training.
• Making Your AI Policy Clear – No confusion, no gray areas. AI providers know exactly what they’re allowed to do with your site.

It’s simple, effective, and puts you back in control.

How to Create an llms.txt File With A Generator Tool

Lucky for you, I’ve created a simple to use llms.txt generator tool.

You can find that right here.

My advice is to keep this guide up and in front of you while using the tool just in case for context.

How to Create an llms.txt File Manually

To implement llms.txt on your website, follow these steps:

Step 1: Open a Plain Text Editor

To create llms.txt, use a plain text editor such as:

Step 2: Copy and Modify the Template

Below is a generic llms.txt template. Replace [yourwebsite.com] with your actual domain name:

# llms.txt for [yourwebsite.com]

# Website Overview
# [Brief description of your website, services, and industry]

User-agent: *
AI-Training: Disallow
AI-Generation: Allow
AI-Summarization: Allow
AI-Crawling: Allow

# Specific Directives for AI Companies
User-agent: OpenAI
AI-Training: Disallow
AI-Generation: Allow
AI-Summarization: Allow

User-agent: Google-DeepMind
AI-Training: Disallow
AI-Generation: Allow
AI-Summarization: Allow

User-agent: Anthropic
AI-Training: Disallow
AI-Generation: Allow
AI-Summarization: Allow

# Reference Content for AI Indexing
Reference-Content:
- [https://yourwebsite.com/resource1]
- [https://yourwebsite.com/resource2]
- [https://yourwebsite.com/blog-post-example]

# Sitemap for AI Indexing (Not Training)
Sitemap: https://yourwebsite.com/sitemap_index.xml

# Legal & Privacy
- [https://yourwebsite.com/terms-of-use/]
- [https://yourwebsite.com/privacy-policy/]

This file:

• Prevents AI from using your content for training
• Allows AI-powered search and summarization
• Provides references for indexing
• Specifies terms and privacy policies

Step 3: Save the File

When saving the file:

• File name: llms.txt
• Format: Plain text (.txt)
• Ensure there’s no extra extension like .txt.txt (Check by right-clicking → Properties on Windows or Get Info on Mac)

How to Upload llms.txt to Your Website

Once you’ve created llms.txt, you need to upload it to your website’s root directory. This process varies depending on your hosting provider.

General Steps for Any Web Hosting Provider

Option 1: Upload via FTP/SFTP

1. Log into Your Hosting Provider’s Control Panel
   • Find your SFTP/FTP credentials (usually under “SFTP Accounts” or “FTP Access”).
   • If you don’t have an FTP account, create one.
2. Connect Using an FTP Client
   • Download and install FileZilla (or Cyberduck for Mac).
   • Enter your hosting credentials:
     • Host: (e.g., ftp.yourwebsite.com or yourwebsite.sftp.provider.com)
     • Username: (Your FTP/SFTP username)
     • Password: (Your FTP/SFTP password)
     • Port: Use 22 for SFTP, or 21 for FTP.
   • Click Connect.
3. Navigate to Your Root Directory
   • Look for folders like wp-admin, wp-content, and wp-includes (if using WordPress).
   • The root directory is where robots.txt or wp-config.php is located.
   • Drag and drop llms.txt into this directory.

Option 2: Upload via Web Hosting File Manager

1. Log into Your Hosting Provider’s Dashboard
   • Examples: Bluehost, SiteGround, HostGator, WP Engine, GoDaddy.
2. Find the File Manager
   • Navigate to “File Manager” (usually under “Website” or “Advanced Settings”).
3. Locate Your Root Directory
   • If using WordPress, look for public_html or /root/.
4. Upload llms.txt
   • Click Upload and select llms.txt from your computer.
   • Confirm the upload.

Option 3: Use a WordPress Plugin

If using WordPress, you can install a plugin like File Manager or WP File Manager to upload llms.txt directly within the WordPress dashboard.

Make Sure You Uploaded It Correctly

After uploading llms.txt, confirm that it’s accessible by visiting:

https://yourwebsite.com/llms.txt

Do you see a live plain text file with the template you used above?

If yes > congrats, you did it!

If no > start over and use ChatGPT to guide you until it works (:

Cheers.

Want A Second Opinion About Your GenAI Search Strategy?

RZ helps our clients with considerations like LLMs.txt all the time. And so much more.

If you feel lost or even just want a second opinion on your SEO strategy, GEO strategy, or anything else, just reach out for a call.

We’re always happy to chop it up and help you solve your thorniest problems.