Renaming a page quietly 404s every link to it

v0.1.5 is the SEO release. Sitemap, robots.txt, per-page noindex, and 301s when a slug changes.

I built them in that last order first, because only one item on the list was actively destroying something. The rest were merely absent.

The one that breaks a client site

A client renames a page. Maybe "Services" becomes "What we do". They type it in the slug field, save, and the page moves.

Every inbound link to the old URL now 404s. Every share, every search result, every link somebody else put on their site. Silently and permanently, and the client has no idea it happened because from inside the panel the page looks fine.

Changing a slug now writes a 301 from the old one, and the public route consults redirects before it gives up.

The design decision that turned out to matter: the redirect stores the page, not a target slug.

Rename twice and you get two rows, both resolving to wherever the page lives now. There is no chain to follow and nothing to clean up. If it stored a target slug you would get one → two → three and either a redirect chain or a pruning job. Storing the destination as an identity rather than a location makes the problem not exist.

Two rules around it. A new page claiming a freed slug drops the redirect from it, because whoever claims a slug owns it. And an unpublished target 404s rather than redirecting, because sending somebody to a 404 is worse than the 404 itself.

I wrote the sitemap instead of installing one

There is a good package for this. spatie/laravel-sitemap is well maintained and I have used it before.

Its job is crawling a site to discover URLs. Atelier already knows every URL it has, because they are rows in a table. What was left after removing the discovery problem was about forty lines of XML.

Taking a dependency to do the part you have already done is how a small package becomes a large one. The rule I have been applying to this project is that leaf libraries solving one problem are welcome, and anything that brings a subsystem to solve a problem I do not have is not.

The sitemap lists every published, indexable page in every locale, with xhtml:link alternates and lastmod from the publish time, which the table already stored and nothing read.

Sitemap URLs from outside the CMS

A real client site is rarely only CMS pages. There is usually a blog or a services catalogue with its own model, its own panel tab and its own routes, and Atelier cannot discover any of it.

So a host app hands them over where it registers blocks:

AtelierPlugin::make()
    ->blocks(DefaultBlocks::all())
    ->sitemap([
        fn () => Post::published()->get()->map(fn (Post $post) => [
            'loc' => route('blog.show', $post),
            'lastmod' => $post->updated_at,
        ]),
    ]);

Sources run when the sitemap is requested rather than at boot, so they are free to query. Entries deduplicate on the URL. And a source that throws takes the sitemap down with it, deliberately: a sitemap quietly missing half a site looks fine and stops your blog being indexed, while one that fails is a bug somebody notices.

The package does not filter your entries either. It will not check whether a post is published, because only your model knows that, and pretending otherwise would be guessing on your behalf.

Indexing is decided per locale, not per page

This is the detail I would have got wrong without stopping to think.

A page can be ready in English and half translated in Arabic. So "hide from search engines" is a per locale toggle, and when the Arabic URL is hidden, the hreflang alternate pointing at it drops too.

An alternate pointing at a noindexed URL is the same mistake as listing that URL. You are telling a crawler "here is the Arabic version" while telling it not to index the Arabic version.

One switch, both effects: the tag and the sitemap entry. And the robots meta is only emitted when it says something, because index, follow is what every crawler already assumes and a tag repeating it is noise.

The robots.txt caveat I could not solve

Atelier serves /robots.txt pointing at the sitemap and keeping crawlers out of the panel and the preview route.

Laravel ships a real public/robots.txt, and a file on disk is served by the web server before Laravel runs at all.

So the route does nothing until you delete that file. I confirmed it on a real install rather than assuming: moved the file aside, ours appeared immediately, moved it back, gone again.

A package cannot tell which one you meant. So it is documented as a caveat rather than papered over: delete the file, or copy the Sitemap: line into yours.

Where it stands

v0.1.5. Meta, canonical, hreflang and Open Graph were already there. Now a sitemap covering both locales, a robots.txt, per-page indexing control, and 301s so a rename does not cost a client every link they have.

Still no JSON-LD, which is next. One correction to my own planning docs while I was here: FAQ structured data stopped producing rich results for ordinary sites in August 2023, so it is the cheapest item on that list rather than the highest value one. I had it written down the other way round and would have built it first for the wrong reason.