#sitemap Sitemap: https://schedule.freedish.in/sitemap.xml Sitemap: https://schedule.freedish.in/sitemap-pages.xml Sitemap: https://schedule.freedish.in/feeds/posts/default?orderby=UPDATED Sitemap: https://www.freedish.in/sitemap.xml Sitemap: https://www.freedish.in/sitemap-pages.xml Sitemap: https://www.freedish.in/feeds/posts/default?orderby=UPDATED Sitemap: https://hindi.freedish.in/sitemap.xml Sitemap: https://hindi.freedish.in/sitemap-pages.xml Sitemap: https://hindi.freedish.in/feeds/posts/default?orderby=UPDATED # Global crawl settings User-agent: * Allow: / # Disallow sensitive paths Disallow: /search?q= # Prevent search results indexing # Block known aggressive crawlers User-agent: AhrefsBot Disallow: / User-agent: MJ12Bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: Majestic12 Disallow: / User-agent: DotBot Disallow: / User-agent: Mail.Ru Disallow: / User-agent: Scrapy Disallow: / User-agent: 80Legs Disallow: / User-agent: Sistrix Disallow: / User-agent: Exabot Disallow: / User-agent: SpamRefresher Disallow: / User-agent: BacklinkCrawler Disallow: / User-agent: LinkChecker Disallow: / User-agent: Nmap Disallow: / User-agent: ZmEu Disallow: / User-agent: SQLMap Disallow: / User-agent: BlackWidow Disallow: / User-agent: Custo Disallow: / User-agent: DISCo Disallow: / User-agent: Download Demon Disallow: / User-agent: EirGrabber Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: EyeNetIE Disallow: / User-agent: FlashGet Disallow: / User-agent: GetRight Disallow: / User-agent: GetWeb! Disallow: / User-agent: Go!Zilla Disallow: / User-agent: Go-Ahead-Got-It Disallow: / User-agent: GrabNet Disallow: / User-agent: Grafula Disallow: / User-agent: HMView Disallow: / User-agent: HTTrack Disallow: / User-agent: Image Stripper Disallow: / User-agent: Image Sucker Disallow: / User-agent: Indy Library Disallow: / User-agent: InterGET Disallow: / User-agent: Internet Ninja Disallow: / User-agent: JetCar Disallow: / User-agent: JOC Web Spider Disallow: / User-agent: larbin Disallow: / User-agent: LeechFTP Disallow: / User-agent: Mass Downloader Disallow: / User-agent: MIDown tool Disallow: / User-agent: Mister PiX Disallow: / User-agent: Navroad Disallow: / User-agent: NearSite Disallow: / User-agent: NetAnts Disallow: / User-agent: NetMechanic Disallow: / User-agent: NetSpider Disallow: / User-agent: Net Vampire Disallow: / User-agent: NetZIP Disallow: / User-agent: Octopus Disallow: / User-agent: Offline Explorer Disallow: / User-agent: Offline Navigator Disallow: / User-agent: PageGrabber Disallow: / User-agent: Papa Foto Disallow: / User-agent: pavuk Disallow: / User-agent: PCBrowser Disallow: / User-agent: RealDownload Disallow: / User-agent: ReGet Disallow: / User-agent: SiteSnagger Disallow: / User-agent: SmartDownload Disallow: / User-agent: SuperBot Disallow: / User-agent: SuperHTTP Disallow: / User-agent: Surfbot Disallow: / User-agent: tAkeOut Disallow: / User-agent: Teleport Pro Disallow: / User-agent: VoidEYE Disallow: / User-agent: Web Image Collector Disallow: / User-agent: Web Sucker Disallow: / User-agent: WebAuto Disallow: / User-agent: WebCopier Disallow: / User-agent: WebFetch Disallow: / User-agent: WebGo IS Disallow: / User-agent: WebLeacher Disallow: / User-agent: WebReaper Disallow: / User-agent: WebSauger Disallow: / User-agent: Website eXtractor Disallow: / User-agent: Website Quester Disallow: / User-agent: WebStripper Disallow: / User-agent: WebWhacker Disallow: / User-agent: WebZIP Disallow: / User-agent: Wget Disallow: / User-agent: Widow Disallow: / User-agent: WWW-Mechanize Disallow: / User-agent: wsb25 Disallow: / User-agent: Xaldon WebSpider Disallow: / User-agent: Zeus Disallow: / User-agent: ScreamingFrogSEOSpider Disallow: / User-agent: DynoMapper Disallow: / User-agent: Sitechecker Disallow: / User-agent: Octoparse Disallow: / User-agent: ParseHub Disallow: / User-agent: Deepcrawl Disallow: / User-agent: Oncrawl Disallow: / User-agent: Import.io Disallow: / User-agent: WebHarvy Disallow: / User-agent: Zyte Disallow: /