Welcome to the Webmaster Forum.
Page 1 of 4 123 ... LastLast
Results 1 to 15 of 49
  1. #1
    Senior Member
    Join Date
    Aug 2015
    Posts
    212

    What is robots.txt ?

    Hello friends,

    Please tell me, What is robots.txt ?

  2. #2
    Senior Member
    Join Date
    Jan 2015
    Posts
    567

    Re: What is robots.txt ?

    robots.txt is text file that search information allow or not.

    Fastmailsender

  3. #3
    Member
    Join Date
    Apr 2015
    Location
    612 N. Federal Highway, Boynton Beach, Florida 33435
    Posts
    46

    Re: What is robots.txt ?

    The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

  4. #4
    Member
    Join Date
    Sep 2015
    Location
    India
    Posts
    69

    Re: What is robots.txt ?

    robots.txt is text file that is used to block some of pages which you did not want to crawled by the google bot.

  5. #5
    Junior Member
    Join Date
    Oct 2015
    Posts
    12

    Re: What is robots.txt ?

    The Robots Exclusion Protocol. It is a text file you put on your site to tell search robots which pages you would like them not to visit.

  6. #6
    Senior Member
    Join Date
    Aug 2014
    Posts
    889

    Re: What is robots.txt ?

    Robot.txt:
    It is an HTML tag placed on the source of a web page which redirects search engine spiders which files to crawl on or not.
    Agriyas Micro Jobs Script works as a Fiverr Clone Script helping you to launch your niche , lets you Gigbucks Clone redefine the Fiverr Clone Market!

  7. #7
    Member
    Join Date
    Jul 2015
    Posts
    44

    Re: What is robots.txt ?

    Robots.txt is text file that is used to allow and disallow.

  8. #8
    Member
    Join Date
    Jul 2015
    Posts
    47

    Re: What is robots.txt ?

    This file helps to give the information that how pages are indexed and crawl.

  9. #9
    Senior Member
    Join Date
    May 2014
    Posts
    610

    Re: What is robots.txt ?

    Robots.txt is common name of a text file that is uploaded to a Web site root directory and linked in the html code of the Web site.
    That is used to allow and disallow.

  10. #10
    Senior Member
    Join Date
    Sep 2015
    Location
    jalandhar
    Posts
    708

    Re: What is robots.txt ?

    Robot text file is used for dynamic websites and even php websites you can use them
    why use them because if you dont want to give any access to google

  11. #11
    Member
    Join Date
    Oct 2015
    Location
    Bangalore
    Posts
    50

    Re: What is robots.txt ?

    Robots.txt is common name of a text file that is uploaded to a Web site's root directory and linked in the html code of the Web site.

  12. #12
    Senior Member
    Join Date
    Nov 2015
    Posts
    139

    Re: What is robots.txt ?

    The robots exclusion protocol (REP), or robots.txt is a text file webmasters create to instruct robots (typically search engine robots) how to crawl and index pages on their website.

  13. #13
    Senior Member
    Join Date
    Aug 2015
    Location
    india
    Posts
    185

    Re: What is robots.txt ?

    This file informs search engine spiders about the data to be crawled.

  14. #14
    Member
    Join Date
    Nov 2015
    Posts
    60

    Re: What is robots.txt ?

    Web site owners use the /robots.txt file to give instructions about their site to web robots; this is called The Robots Exclusion Protocol.

  15. #15
    Senior Member
    Join Date
    Jun 2014
    Location
    CA
    Posts
    2,226

    Re: What is robots.txt ?

    Robots.txt file is used to control your site which part of your site is indexed by Google and which is not.

 

 
Page 1 of 4 123 ... LastLast

Similar Threads

  1. Which one is better Robots.txt or Meta robots Tag
    By adriennerich in forum Search Engine Optimization (SEO)
    Replies: 10
    Last Post: 08-05-2019, 09:03 AM
  2. What is robots.txt?
    By eagleeye dental in forum Search Engine Optimization (SEO)
    Replies: 20
    Last Post: 11-09-2015, 04:28 AM
  3. Robots.txt vs Meta Robots Tag
    By Roseanne Hortense in forum Search Engine Optimization (SEO)
    Replies: 0
    Last Post: 05-31-2014, 01:18 PM
  4. About Robots.TXT
    By easyarticles in forum Search Engine Optimization (SEO)
    Replies: 9
    Last Post: 08-30-2013, 06:19 AM
  5. using robots.txt vs meta robots tag
    By easyarticles in forum Search Engine Optimization (SEO)
    Replies: 0
    Last Post: 08-02-2013, 06:49 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Back to top

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203