Help me for Robots.txt file

Discussion in 'Search Engine Optimization (SEO)' started by jamess, Mar 7, 2013.

  1. jamess

    jamess Member

    Hello,

    I want some changes in robots.txt file of my site. but I am confused in following commands
    Disallow: /wp-content/
    Disallow: /wp-content/themes
    Disallow: /?comments=*
    Disallow: /trackback/
    Disallow: /cgi-bin
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /comments
    Disallow: /category/*/*
    Disallow: */trackback
    Disallow: */feed
    Disallow: */comments
    Disallow: /*?*
    Disallow: /*?
    Allow: /wp-content/uploads

    Please suggest me the benefits of aforementioned commands.. plz plz help me I need desperately to know about these commands .
     
  2. A8ch

    A8ch Gold Member

  3. Just2EZ

    Just2EZ Moderator

    Those should keep crawlers from indexing those WordPress directories.
    "Comments" probably should not be on that list!
    It really depends on if you want them crawled.
     

Share This Page