SEO: Dynamic Metadata, Sitemap & robots.txt

Semantics for SEO

Search engines love clear structure: one h1, logical headings, meaningful alt text, descriptive links, and fast LCP.

Server-rendered HTML from the App Router helps crawlers see content without waiting for client JS.

Real-life example: Good semantics is a well-labeled supermarket — crawlers find milk in Dairy, not by wandering every aisle blindly.

• Use real <h1>–<h3>, not only styled divs
• Descriptive titles and meta descriptions (unique per page)
• Internal links with clear anchor text
• Mobile-friendly layout and Core Web Vitals

sitemap.ts

Export a default function from app/sitemap.ts. Next.js serves /sitemap.xml automatically. Include static routes and dynamic course/blog URLs from your CMS or DB.

Real-life example: A sitemap is the mall directory map — "you are here" plus every shop address for visitors (and Google).

import type { MetadataRoute } from "next";

export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const base = "https://rishtaara.com";
  const courses = await getAllCourseSlugs(); // your data helper

  return [
    { url: base, lastModified: new Date(), changeFrequency: "weekly", priority: 1 },
    { url: `${base}/learn`, changeFrequency: "weekly", priority: 0.9 },
    ...courses.map((slug) => ({
      url: `${base}/learn/web-development/${slug}`,
      lastModified: new Date(),
      changeFrequency: "monthly" as const,
      priority: 0.8,
    })),
  ];
}

robots.ts & dynamic metadata for SEO

app/robots.ts controls what bots may crawl. Point sitemap to your absolute URL. Disallow private areas like /dashboard and /api.

Combine robots + sitemap + generateMetadata for a solid SEO baseline.

Real-life example: robots.txt is a polite sign on a private driveway — "deliveries welcome at the front; please skip the backyard."

import type { MetadataRoute } from "next";

export default function robots(): MetadataRoute.Robots {
  return {
    rules: {
      userAgent: "*",
      allow: "/",
      disallow: ["/dashboard/", "/api/"],
    },
    sitemap: "https://rishtaara.com/sitemap.xml",
  };
}