# robotstxt.org/ # ログインが要る画面・API・管理者向けの画面はクロールさせない(Config\Seo::ROBOTS_DISALLOW_PATHS と同じ。#892)。 # /auth は noindex を読ませるため塞がない。公開ページ・解説記事(/guide)・sitemap は塞がない。 # クローラーは自分の名前のグループがあると * のグループを読まないため、名前付きのグループにも同じ Disallow を書く。 Sitemap: https://watchbell.com/sitemap.xml User-agent: * Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: Slurp Crawl-delay: 10 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: bingbot Crawl-delay: 1200 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: msnbot Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: Yeti Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: Baiduspider Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: Baiduspider+ Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: BaiduImagespider Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: BaiduMobaider Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget User-agent: MLBot Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget user-agent: AhrefsBot Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget user-agent: Ezooms Crawl-delay: 10000 Disallow: /admin Disallow: /setting Disallow: /mock Disallow: /webservice Disallow: /widget