Configurer son serveur avec htaccess, optimisation, performance et sécurité
Qu’est ce que le fichier .htaccess ?
C’est un fichier de configuration optionnel qui définit certaines directives et paramètres pour le dossier et sous-dossier d’un serveur Apache. Le serveur Apache interprète des langages de programmation tel que le PHP. La plupart des serveurs web mutualisés sont sous apache. Il est donc important d’avoir un fichier correctement configuré.
Redirection HTTPS
Je commence par ceci, car cela ne sert à rien de configurer ce qui suit si nous changeons de page. Ce script permet de rediriger les requêtes HTTP en HTTPS. Bien sûr, si votre site n’est pas en HTTPS , pas la peine de mettre cette portion de script.
## Redirection HTTPS
# NE https://httpd.apache.org/docs/current/rewrite/flags.html#flag_ne
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP:X-Forwarded-Proto} !https
RewriteCond %{HTTPS} !=on
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [NE,L,R=301]
</IfModule>
## END Redirection HTTPS HTTPS Redirection HTTPS
Compression
En terme de performance et d’optimisation des sites web, il est aujourd’hui incontournable de mettre en place la compression des ressources afin que les ressources soit compresser lors de leur téléchargement.
Voici les directives à mettre en place.
## Compression Gzip
<IfModule mod_deflate.c>
# Active le filtre compression
SetOutputFilter DEFLATE
# Applique la compression sur les fichiers de type mime défini
AddOutputFilterByType DEFLATE text/html text/css text/plain text/xml application/x-javascript application/x-httpd-php image/svg+xml application/xhtml+xml application/xml application/rss+xml application/atom_xml
# Gestion pour les navigateurs incompatibles
BrowserMatch ^Mozilla/4 gzip-only-text/html
BrowserMatch ^Mozilla/4\.0[678] no-gzip
BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
# Ne pas mettre en cache si ces fichiers le sont deja
SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip
# Les proxies doivent donner le bon contenu
<IfModule mod_headers.c>
Header append Vary User-Agent env=!dont-vary
</IfModule>
</IfModule>
## END Compression Gzip Compression Gzip
Le Cache
Le cache web est une forme de mémoire tampon qui stocke tout ou une partie d’une page pour une durée déterminée. C’est donc cette partie que nous allons configurer.
Expire headers
Détermine le temps de validité du cache par type de fichier.
## Expire headers
<IfModule mod_expires.c>
ExpiresActive On
ExpiresDefault "access plus 1 month"
# Media files
ExpiresByType audio/ogg "access plus 1 month"
ExpiresByType image/bmp "access plus 1 month"
ExpiresByType image/gif "access plus 1 month"
ExpiresByType image/jpeg "access plus 1 month"
ExpiresByType image/png "access plus 1 month"
ExpiresByType video/mp4 "access plus 1 month"
ExpiresByType video/ogg "access plus 1 month"
ExpiresByType video/webm "access plus 1 month"
# Icon files
AddType image/x-icon .ico
ExpiresByType image/ico "access plus 1 month"
ExpiresByType image/icon "access plus 1 month"
ExpiresByType image/x-icon "access plus 1 month"
# Text files
ExpiresByType text/css "access plus 1 month"
ExpiresByType text/javascript "access plus 1 month"
ExpiresByType text/html "access plus 7200 seconds"
# Application files
ExpiresByType application/xhtml+xml "access plus 7200 seconds"
ExpiresByType application/javascript "access plus 1 month"
ExpiresByType application/x-javascript "access plus 1 month"
ExpiresByType application/x-shockwave-flash "access plus 1 month"
# Web fonts
AddType application/font-woff2 .woff2
ExpiresByType application/font-woff "access plus 1 month"
ExpiresByType application/font-woff2 "access plus 1 month"
ExpiresByType application/vnd.ms-fontobject "access plus 1 month"
ExpiresByType application/x-font-ttf "access plus 1 month"
ExpiresByType font/opentype "access plus 1 month"
ExpiresByType image/svg+xml "access plus 1 month"
ExpiresByType application/x-font-opentype "access plus 1 month"
ExpiresByType application/x-font-woff "access plus 1 month"
</IfModule>
## END Expire headers Expire Headers
Cache-control
Le Cache-Control est complémentaire du Expire Headers.
## Cache-Control Headers
<IfModule mod_headers.c>
# KILL THEM ETAGS
Header unset ETag
FileETag none
<FilesMatch "\.(ico|jpe?g|png|gif|swf|css|gz)$">
Header set Cache-Control "max-age=2592000, public"
</FilesMatch>
<FilesMatch "\.(js)$">
Header set Cache-Control "max-age=2592000, private"
</FilesMatch>
<filesMatch "\.(html|htm)$">
Header set Cache-Control "max-age=7200, public"
</filesMatch>
# Disable caching for scripts and other dynamic files
<FilesMatch "\.(pl|php|cgi|spl|scgi|fcgi)$">
Header unset Cache-Control
</FilesMatch>
</IfModule>
## END Cache-Control Headers Cache-Control
La Sécurité
La sécurité et une partie importante, voici quelques lignes a placer dans le fichier htaccess.
Le minimum
# Protection de la lecture des repertoires
<IfModule mod_autoindex.c>
Options -Indexes
</IfModule>
# Encodage des caractères
AddDefaultCharset utf-8
# Masquer les informations du serveur
ServerSignature Off
# Protege le fichier htaccess
<files .htaccess>
order allow,deny
deny from all
</files> Le minimum
Headers
Ces différentes instructions permettent d’activer certain paramètres de sécurité dans les entêtes.
## Headers - Security
<IfModule mod_headers.c>
# Secure cookie with HttpOnly and Secure flag in Apache
Header set Set-Cookie HttpOnly;Secure
# Content-Security-Policy
# Commenté car il doit être fait au cas par cas - https://content-security-policy.com/
# Header set Content-Security-Policy "default-src https: 'unsafe-inline' 'unsafe-eval'"
# HSTS - Oblige le navigateur a passer en HTTPS
Header always set Strict-Transport-Security "max-age=16070400; includeSubDomains"
# Referrer Policy
Header always set Referrer-Policy: "strict-origin-when-cross-origin"
# Access-Control-Allow-Origin
Header set Access-Control-Allow-Origin "%{HTTP_ORIGIN}e" env=HTTP_ORIGIN
# Clickjacking
Header always set X-FRAME-OPTIONS "SAMEORIGIN"
# XSS protection
Header always set X-XSS-Protection "1; mode=block"
# Nosniff (dareboost)
Header always set X-Content-Type-Options "nosniff"
# Masque la version de php
Header unset X-Powered-By
</IfModule>
## END Headers - Security Headers
Blocage Bot / Spam
Bot
Cette liste m’a été fourni par l’hébergeur siteground, afin de bloquer les robots qui harcelaient mon hébergement et me prenaient de la ressource.
# Bot Block #
<IfModule mod_rewrite.c>
RewriteEngine On
# Block Essential Bots #
#RewriteCond %{HTTP_USER_AGENT} ^(.*)msnbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MJ12bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BLEXBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SolomonoBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Yandex [NC,OR]
#RewriteCond %{HTTP_USER_AGENT} ^(.*)bingbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Baiduspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Yeti [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mail.Ru [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Ezooms [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)AhrefsBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Sosospider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sogou [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)exabot [NC,OR]
# Block Non-Essential Bots #
RewriteCond %{HTTP_USER_AGENT} ^(.*)2icommerce [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)archive.org [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Abont [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)abot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Accoona [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ActiveTouristBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)adressendeutschland [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)aipbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Alligator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)AllSubmitter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)almaden [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)anarchie [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Acunetix [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)AESOP_com_SpiderMan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Alexibot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Anonymous [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Anonymouse.org [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Apexoo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Aqua_Products [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)asterias [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ASSORT [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ATHENS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)AtHome [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Atomz [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)attach [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)autoemailspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)autohttp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BackDoorBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BackWeb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)binlar [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Badass [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BecomeBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)berts [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bitacle [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Biz360 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)bladder\ fusion [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Blog\ Checker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BlogPeople [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Blogshares\ Spiders [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bloodhound [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BatchFTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bigfoot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Black.Hole [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BlackWidow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BlowFish [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bolt [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Board\ Bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bookmark\ search\ tool [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)b2w [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)bew [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BotALot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BotRightHere [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bot\ mailto:craftbot@yahoo.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Buddy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bropwers [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Browsezilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BuiltBotTough [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Bullseye [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)BunnySlippers [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)casper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Cegbfeieh [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)CheeseBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)cmsworld [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)checkpriv [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)choppy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)clshttp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)CherryPicker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ChinaClaw [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)charlotte [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Convera [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Copernic [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)CFNetwork [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Copier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)CopyRightCheck [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)cosmos [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Crescent [NC,OR]
#RewriteCond %{HTTP_USER_AGENT} ^(.*)curl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Custo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Cyberz [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)c-spider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Default\ Browser [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Da$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DataCha0s [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Daum [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Deweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Digger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DISCo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Digimarc [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)digout4uagent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)diavol [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)discobot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DnloadMage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DIIbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)dotbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DittoSpyder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Download [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Downloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Download\ Demon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Download\ Devil [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Download\ Wonder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)dragonfly [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)dirbuster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DreamPassport [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DSurf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DTS\ Agent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)dumbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)DynaWeb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Drip [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)eCatch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ecollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)edgeio [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)efp@gmx.net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ecxi [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)e-collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EasyDL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EBrowse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ebingbong [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EirGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EmailCollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Email\ Extractor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EmailSiphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EmeraldShield [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Enterprise_Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EmailWolf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EroCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ESurf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Eval [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Everest-Vulcan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)extract [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Extractor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ExtractorPro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Express\ WebPictures [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)EyeNetIE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FairAd [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)fastlwspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FileHound [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)findlinks [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)fimap [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FlashGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Flaming\ AttackBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FHscan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)feedfinder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)fetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FEZhead [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)flicky [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FlickBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Foobot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Forex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)flunky [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Franklin\ Locator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FreshDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FrontPage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)FSurf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Gaisbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Gamespy_Arcade [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)genieBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GetBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Getleft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GetRight [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GetSmart [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GetWeb! [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Go!Zilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Google\ Wireless\ Transcoder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)g00g1e [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Go-Ahead-Got-It [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GOFORITBOT [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)gotit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Grabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GrabNet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Grafula [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)grub [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)GT::WWW [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)heritrix [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Harvest [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Hatena\ Antenna [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)hloader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HMView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)holmes [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HooWWWer [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HouxouCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HTTPGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HTTP::Lite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)httplib [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HTTPRetriever [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)HTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)humanlinks [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IBM_Planetwide [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)iCCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Image\ Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Image\ Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)imagefetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)imds_monitor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IncyWincy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ichiro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)iGetter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ia_archiver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IDBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)id-search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)id-search.org [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IlseBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Image\ Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Image\ Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Indy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InetURL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InfoNaviRobot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InstallShield\ DigitalWizard [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Industry\ Program [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InfoTekies [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Intelliseek [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InterGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Internet\ Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Iria [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IRLbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Iron33 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ISSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)IUPUI\ Research\ Bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ISC\ Systems\ iRc [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Java [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JBH\ Agent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Jakarta [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JennyBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JetCar [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)jeteye [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)jeteyebot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JoBo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JOC\ Web\ Spider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JOC [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JOC\ Web\ Spider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)JustView [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Jyxobot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)jbrofuzz [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Kenjin.Spider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Kapere [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Kenjin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)KRetrieve [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Keyword\ Density [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)KRetrieve [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ksoap [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)KWebGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LapozzBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)larbin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)leech [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LeechFTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LeechGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libwww [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libwww-perl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libwhisker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Link [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libghttp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LinksManager.com_bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LexiBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)lftp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)loader [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libWeb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)likse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)libwww-FM [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LinkextractorPro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LinkScan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)linktiger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LNSpiderguy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LocalcomBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)looksmart [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LinkWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)lmcrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Linkie [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)lwp-trivial [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)LWP::Simple [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mac\ Finder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mail\ Sweeper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)mark.blonin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MaSagool [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Magnet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mag-Net [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MarkWatch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mass [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mata\ Hari [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MCspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MetaProducts\ Download\ Express [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Microsoft\ Data\ Access [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Microsoft\ URL\ Control [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Maxthon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MFC_Tear_Sample [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Memo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Microsoft.URL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Microsoft\ URL\ Control [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MIDown [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MIIxpc [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mirror [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Missauga [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Missouri\ College\ Browse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)miner [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Missigua\ Locator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Mister [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Monster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)mothra [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)mkdb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)moget [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Moreoverbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)netscan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MovableType [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MSIECrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MSProxy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MVAClient [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MyFamilyBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MyGetRight [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)kmccrew [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)MSFrontPage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)nameprotect [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NASA\ Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Naver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Navroad [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NearSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetAnts [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)netattache [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetCarta [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)nessus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Netcraft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetMechanic [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetResearchServer [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NetZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Net\ Vampire [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NEWT\ ActiveX [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Nextopia [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NextGenSearchBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Net\ Vampire [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^NG$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NICErsPRO [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NimbleCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)noxtrumbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)NPbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Nutch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)nikto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Octopus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Openvas [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Openbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Offline [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Openfind [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)OpenTextSiteCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Oracle\ Ultra\ Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)OutfoxBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)P3P [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PackRat [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PageGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PagmIEDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)panscient [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Papa\ Foto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)python [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)pavuk [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PECL::HTTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PeoplePal [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)pcBrowser [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PHPCrawl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)perl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PerMan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PersonaPilot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PleaseCrawl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PHP\ version [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PlantyNet_WebRobot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)playstarmusic [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Plucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Pockey [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Port\ Huron [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ProPowerBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)prospector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ProWebWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Prozilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PushSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PussyCat [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)PuxaRapido [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)puf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Program\ Shareware [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Progressive\ Download [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)psbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)psycheclone [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)planetwork [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)postrank [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)purebot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)pycurl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Python-urllib [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Pump [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)QueryN [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Radiation [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RealDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RedCarpet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RedKernel [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Reaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Recorder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ReGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)relevantnoise [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Rippers [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SBIder [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)scooter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ScoutAbout [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)script [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)searchpreview [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)searchterms [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Seekbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Serious [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Shai [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)shelob [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Shim-Crawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SickleBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RepoMonkey [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RMA [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Rover [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Rsync [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Rufus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)RTG30 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SAPO [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Siphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)InternetSeer.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sitecheck [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SiteSnagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Slurpy\ Verifier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SlySearch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SmartDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sna [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Snake [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)snagger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Snapbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Snoopy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Steeler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SpaceBison [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SpankBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)spanner [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SpeedDownload [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Spegla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Sphere [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Sphider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Webscanner [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Sqworm [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Stamina [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SpiderBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sproose [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sootle [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)seekerspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Stripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)studybot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)siclab [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)skygrid [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SuperBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SuperHTTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Surfbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)SurfWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)suzuran [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Szukacz [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)sqlmap [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)tAkeOut [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)TALWinHttpClient [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)tarspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Teleport [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Toata\ dragostea [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)toCrawl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)urllib [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)User-Agent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Telesoft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Templeton [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)TestBED [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)turnit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)TurnitinBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)The\ Intraformant [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)TheNomad [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)TightTwatBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Titan [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)True_Robot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)turingos [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Twisted\ PageGetter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URLy.Warning [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URL\ Control [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URLy\ Warning [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URL_Spider_Pro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)UtilMind [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URLGetFile [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)URI::Fetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)UCmore [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)UdmSearch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)UMBC [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)UniversalFeedParser [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)QuepasaCreep [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Vacuum [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)vayala [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)vobsub [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)vikspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)VCI [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)VoilaBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)voyager [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)VoidEYE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)w3mir [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Web\ Image\ Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Web\ Sucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Web2WAP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebaltBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)webalta [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebCollage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebAuto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebBandit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebCapture [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Webclipping.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebCopier [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebCopy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebEMailExtrac [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebEnhancer [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebFetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebFilter [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebFountain [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebGo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Web.Image.Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebLeacher [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebMiner [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebMirror [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebmasterWorldForumBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebReaper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebSauger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebSnake [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)webshag [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Website [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Webster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebStripper [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebVac [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebWhacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WebZIP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)webwalk [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Wells\ Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WEP\ Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WeRelateBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)wget [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WhosTalking [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Whacker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)whatweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Widow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Wildsoft\ Surfer [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WUMPUS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WinHTTrack [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)winhttp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WinHttpRequest [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WISENutbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WWW-Mechanize [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WWWOFFLE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)wwwster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)WWW-Collector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Xaldon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)xxxyy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)zermelo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Xenu [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)XGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)YahooYSMcm [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)YaDirectBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Zade [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)ZBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)zerxbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Zeus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)youda [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)Zyborg [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)zmeu [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(.*)zune
RewriteRule .* - [F]
# Block bad robot (add by SiteGround) / Block SEO crawler
</IfModule> Bloc les bots
Spam Bot Traffic
Cette deuxième liste permettait de bloquer des faux visiteurs provenant de certains site malicieux afin de générer de fausses visites. Nous pouvions les identifier sur google analytics et les reporter dans cette liste. Aujourd’hui je ne sais pas si cette liste a un intérêt, mais je la garde de côté au cas où.
# block visitors referred from bad website
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP_REFERER} iedit.ilovevitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} iloveitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} econom\.co [NC,OR]
RewriteCond %{HTTP_REFERER} semalt\.com [NC,OR]
RewriteCond %{HTTP_REFERER} iskalko\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} ilovevitaly\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} ilovevitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} o-o-8-o-o\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} o-o-6-o-o\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} semalt\.semalt\.com [NC,OR]
RewriteCond %{HTTP_REFERER} cenoval\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} priceg\.com [NC,OR]
RewriteCond %{HTTP_REFERER} darodar\.com [NC,OR]
RewriteCond %{HTTP_REFERER} cenokos\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} seoexperimenty\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} gobongo\.info [NC,OR]
RewriteCond %{HTTP_REFERER} vodkoved\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} adcash\.com [NC,OR]
RewriteCond %{HTTP_REFERER} websocial\.me [NC,OR]
RewriteCond %{HTTP_REFERER} cityadspix\.com [NC,OR]
RewriteCond %{HTTP_REFERER} luxup\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} ykecwqlixx\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} superiends\.org [NC,OR]
RewriteCond %{HTTP_REFERER} slftsdybbg\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} edakgfvwql\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} socialseet\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} screentoolkit\.com [NC,OR]
RewriteCond %{HTTP_REFERER} econom\.co [NC,OR]
RewriteCond %{HTTP_REFERER} semalt\.com [NC,OR]
RewriteCond %{HTTP_REFERER} savetubevideo\.com [NC,OR]
RewriteCond %{HTTP_REFERER} shopping\.ilovevitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} iedit\.ilovevitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} 7makemoneyonline\.com [NC,OR]
RewriteCond %{HTTP_REFERER} resellerclub\.com [NC,OR]
RewriteCond %{HTTP_REFERER} cenoval\.ru [NC,OR]
RewriteCond %{HTTP_REFERER} bestwebsitesawards\.com [NC,OR]
RewriteCond %{HTTP_REFERER} o-o-6-o-o\.com [NC,OR]
RewriteCond %{HTTP_REFERER} humanorightswatch\.org [NC,OR]
RewriteCond %{HTTP_REFERER} make-money-online\.7makemoneyonline\.com [NC,OR]
RewriteCond %{HTTP_REFERER} share\-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*free\-share\-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} click\.aliexpress\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*4webmasters\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*simple\-share\-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} googlsucks\.com [NC,OR]
RewriteCond %{HTTP_REFERER} best-seo-solution\.com [NC,OR]
RewriteCond %{HTTP_REFERER} get-free-traffic-now\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*Get\-Free\-Traffic\-Now\.com [NC,OR]
RewriteCond %{HTTP_REFERER} addons\.mozilla\.org [NC,OR]
RewriteCond %{HTTP_REFERER} googlsucks\.com [NC,OR]
RewriteCond %{HTTP_REFERER} theguardlan\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*torture\.ml [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*hulfingtonpost\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*offers\.bycontext\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*best\-seo\-offer\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*guardlink\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*-share-buttons.\com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*buy-cheap-online.info [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*best\-seo\-software\.xyz [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*free\-floating\-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^http://([^.]+\.)*get\-free\-social\-traffic\.com [NC,OR]
RewriteCond %{HTTP_REFERER} buttons\-for\-website\.com [NC]
RewriteRule .* – [F]
</IfModule> Bloc les faux visiteurs
Autres ressources
Si vous voulez rentrer un peu plus dans le détail du fichier htaccess, je vous conseille la lecture du fichier suivant:
https://github.com/h5bp/html5-boilerplate/blob/master/dist/.htaccess
Il est issu comme vous le voyez de html5-boilerplate, un template HTML et la configuration du fichier est très complet.
Bonjour
Comment puis je réécrire une url https://www.carolebonnard.fr/contact/contact.html pour n’afficher que https://www.carolebonnard.fr/contact quand j’arrive sur mon formulaire de contact ?
Bonjour,
Le but de cet article n’est pas la réécriture d’url.
C’est toutes les petites choses que l’on peut ajouter pour optimiser (compression, cache and co) et sécurisé le site via les instruction
.htaccess.Pour ma part, j’utilise souvent un CMS, donc je le délègue.
Sinon, je créé un répertoire et la page je la met en
index.html.Mais pour répondre à votre question je vous réoriente vers un tutoriel sur l’URL Rewriting.