Should you convert your website into Markdown to help Large Language Models (LLMs) understand your content better? Is "llms.txt" worth the effort for SEO? In this episode of Search Off the Record, Martin Splitt and John Mueller from the Google Search Relations team dive deep into the history of Markdown, its rise in the AI era, and whether it holds any real weight for search engine discovery. In this episode, you'll learn: The Origins of Markdown: From John Gruber and Aaron Swartz to its status as the "language of GitHub." Markdown vs. HTML: Why the "cleanliness" of Markdown is tempting for developers but potentially risky for site structure. LLMs & Markdown: Do AI crawlers actually prefer Markdown, or are they already experts at parsing HTML? The "Parallel Version" Trap: Why creating a separate text/Markdown version of your site for AI can lead to the same maintenance nightmares as dynamic rendering. Use Cases that Make Sense: When Markdown is actually superior (like developer documentation) and when it's totally unnecessary (like your shoe catalog). Key Takeaways for SEOs & Developers: Crawlers are built for the "messy" web: Google and other engines have decades of experience parsing HTML. Don't sacrifice discovery: Headers, footers, and sidebars in HTML provide critical context for site structure that a raw Markdown file might lack. Maintenance is king: Avoid the complexity of maintaining two versions of the same content.

Chapters 0:00 - Introduction: Should we all be using Markdown? 3:45 - The history and purpose of Markdown. 7:15 - Why developers love it: Separation of style and content. 11:20 - Do crawlers need Markdown to understand your site? 14:50 - The danger of "parallel versions" and dynamic rendering lessons. 17:30 - Discussing the "llms.txt" proposal and AI agents. 21:00 - Where Markdown actually makes sense (Developer Docs). 24:00 - Final verdict: Stick to HTML for the web.

Resources Mentioned: Google Search Central: https://developers.google.com/search

Are you using Markdown for your site's frontend or just as a backend source? Let us know in the comments! Episode transcript → https://goo.gle/sotr111-transcript

Listen to more Search Off the Record → https://goo.gle/sotr-yt

Subscribe to Google Search Channel → https://goo.gle/SearchCentral

Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

#SOTRpodcast #SEO #GoogleSearch

Speakers: Martin Splitt, John Mueller

In this episode of Search Off the Record, Martin Splitt and John Mueller from Google's Search Relations team dive deep into the world of AI-assisted development. They explore the reality of "Vibe Coding", the process of building apps and websites using natural language instead of manual syntax. Whether you're a developer looking to offload tedious setup tasks or an SEO expert trying to understand how AI-generated sites impact search, this conversation is for you.

In this episode, you'll learn: * What is Vibe Coding? Understanding the shift from writing syntax to "talking" to your IDE. * The Developer's Trap: Why you still need technical knowledge (like linters, deployment scripts, and GitHub Actions) to prevent AI from breaking your project. * SEO & AI Architecture: Why you can't just "add SEO" at the end—and how to guide AI to build with canonicals and sitemaps from day one. * Tooling Breakdown: Martin and John share their experiences with AI Studio, Gemini CLI, Firebase, and GitHub. * Testing with AI Agents: How to use AI to remote control browsers (like Chromium) for automated testing.

Chapters 00:00 – Intro: What exactly is "Vibe Coding"? 01:32 – Martin's experiment with AI Studio and client-side JS. 03:30 – The "English as a Programming Language" allure. 06:00 – Why the AI makes assumptions (and why that's dangerous). 08:51 – "Sprinkling SEO" vs. Building for SEO from the start. 12:40 – Can AI test itself? Using browser agents for QA. 20:27 – The technical debt of AI: Refactoring and maintainability. 25:42 – Moving to the terminal: Gemini CLI & Cloud Code. 31:34 – Using AI to skip the setup work.

Resources Mentioned: * Google AI Studio * Firebase Hosting * Gemini CLI / Cloud Code * GitHub Actions for CI/CD

What's your experience with Vibe Coding? Let us know in the comments!

Episode transcript → https://goo.gle/sotr110-transcript

Listen to more Search Off the Record → https://goo.gle/sotr-yt

Subscribe to Google Search Channel → https://goo.gle/SearchCentral

Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

#SOTRpodcast #SEO #GoogleSearch

Speakers: Martin Splitt, John Mueller

In this episode of Search Off the Record, Martin speaks with Nikola Todorovic (director of Software Engineering at Google Search) about how AI is changing Google Search. They discuss the evolution from traditional search to AI Overviews and AI Mode, how Google tests and launches search changes, and why query behaviour is becoming more conversational and complex.

Nikola also explains the role of machine learning in Search, how features are evaluated before launch, and what site owners and SEOs should focus on as AI becomes a bigger part of the search experience. If you work in SEO or web development, this episode offers a clear look at how Google approaches AI in Search and what it means for the future of search visibility.

Episode transcript → https://goo.gle/sotr109-transcript

Listen to more Search Off the Record → https://goo.gle/sotr-yt

Subscribe to Google Search Channel → https://goo.gle/SearchCentral

Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

#SOTRpodcast #SEO #GoogleSearch

Speakers: Martin Splitt, Nikola Todorovic

In this episode of Search Off the Record, Martin and Gary turn a simple robots.txt question into a data‑driven deep dive using HTTP Archive, WebPageTest, custom JavaScript metrics, and BigQuery. They explore how millions of real robots.txt files are actually written in 2025–2026, which directives and user‑agents are most common, and what that means for modern crawling and AI bots.

Perfect for beginner to mid‑level developers and SEOs, you'll learn how large‑scale web measurement works (HTTP Archive, Chrome UX Report, Web Almanac), and how to turn raw crawl data into actionable SEO insights. Subscribe for more candid conversations about crawling, indexing, and the data behind how Google Search and the web really work.

Resources:

Web Almanac → https://almanac.httparchive.org/en/2025/ Robotstxt custom metric for the HTTP Archive → https://github.com/HTTPArchive/custom-metrics/pull/191 robots.txt parser change → https://github.com/google/robotstxt/commit/4af32e54b715442bb04cd0470e99192f0ffb9792#commitcomment-178586774

Episode transcript → https://goo.gle/sotr108-transcript

Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral

Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

#SOTRpodcast #SEO #GoogleSearch

Speakers: Martin Splitt, Gary Illyes

In this episode of Search Off the Record, Gary and Martin dig into what "page size" and "page weight" actually mean for developers, users, and search engines.

They discuss exploding web page sizes: median mobile homepages hit 2.3 MB in 2025 Web Almanac (up 3x from 2015), key insights for developers on page weight definitions, Googlebot's crawl limits, HTML bloat from structured data/images, and why size still hurts UX on slow connections despite faster networks.

If you build or maintain websites, this conversation will help you rethink how much data your pages ship, where bloat really comes from, and why page weight still matters even as connections get faster.

Resources: ​Web Almanac → https://almanac.httparchive.org/en/2025/ HTML living standard → https://html.spec.whatwg.org/multipage/ How page speed helps with conversions → https://www.thinkwithgoogle.com/marketing-strategies/app-and-mobile/mobile-page-speed-data/

Episode transcript → https://goo.gle/sotr106-transcript

Listen to more Search Off the Record → https://goo.gle/sotr-yt Subscribe to Google Search Channel → https://goo.gle/SearchCentral

Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.

#SOTRpodcast #SEO #GoogleSearch

Speakers: Martin Splitt, Gary Illyes