Category Archives: server issues

Configuring Apache to use mod_expires for browser cache leverage

Posted on by
Reply

This document explains how to configure Apache 1.3x to send the Expires header using the mod_expires.c module.

To put the examples included in this document in context, please review the caching strategy described in Effective Use of Caching to Boost VoiceXML Application Performance.

  • 1. Verifying that mod_expires is installed
  • 2. Modifying httpd.conf
  • 3. Understanding the performance impact of .htaccess files
  • 4. ETags and Web server farms
1. Verifying that mod_expires is installed

Before you can configure Apache to output the Expires header with your content, you need to verify that the standard mod_expires module is installed. To see what Apache modules are currently installed, run the following command on the server running Apache:

 

% httpd -l

 

If mod_expires.c is not listed, consult your Apache distribution for instructions on how to enable the module.

2. Modifying httpd.conf

The most straightforward and high performance way to control the caching behavior of your documents is to edit your Apache configuration file (httpd.conf). Use the IfModule, LocationMatch, ExpiresActive, and ExpiresDefault directives to add the appropriate HTTP header to a file or directory of files.

In the following example, the first set of directives ensures that the Expires header is not sent for any content by default. The second set of directives configures Apache to send Expires and Cache-Control headers when content in any one of the six directories (plus any contained subdirectories) specified by the LocationMatch directive is requested. The content expires an hour after the request.

<Location />
   <IfModule mod_expires.c>
      ExpiresActive Off
   </IfModule>
</Location>
<IfModule mod_expires.c>
   <LocationMatch "/2002-03-23-1100/(vui|js|signin|quotes|portfolio|balances)/">
       ExpiresActive On
       ExpiresDefault "access plus 1 hour"
   </LocationMatch>
</IfModule>

 

The following example ensures that all files with a .wav extension are cached for one hour from the time they are requested.

<IfModule mod_expires.c>
  <FilesMatch "\.wav$">
       ExpiresActive On
       ExpiresDefault "access plus 1 hour"
   </FilesMatch>
</IfModule>

 

3. Understanding the performance impact of .htaccess files

Another mechanism for specifying expires headers is the .htaccess file. If .htaccess files are enabled, you can add the file to a directory to set expiration policies on the content in that directory and any contained subdirectories using the same syntax as described in the previous section.

Before resorting to .htaccess files, however, you should discuss the performance impact with your Web server administrator. Each time a file is requested from a directory containing an .htaccess file, Apache must process that .htaccess file as well as every .htaccess file that exists up the directory chain to the document root. The performance penalty can be signficant.

4. ETags and Web server farms

An ETag is an HTTP response header returned by an HTTP/1.1 compliant Web server such as Apache 1.3x. By default, Apache calculates an ETag for a requested file using a combination of the file’s location in the file system (I-Node number on Unix systems), its modification time, and its size. The proxy caches running on the Tellme Voice Application Network use the value of the ETag header in conjunction with the If-None-Match request header when validating an object with an origin server.

Because the ETag is calculated using the file’s I-Node, and an I-Node is machine-specific, administrators of Web server farms will experience unexpected requests if the ETag differs from machine to machine.

To work around this issue, use the FileETag directive to configure your Apache server to use only the file modification time and file size when calculating the ETag.

The following example configures Apache to only use the modification time (MTime) and size (Size) when calculating the ETag for any file contained in the /usr/local/httpd/htdocs directory or a subdirectory.

<Directory /usr/local/httpd/htdocs>
   FileETag MTime Size
</Directory>
Share

What’s an “Internal Server Error” and how do I fix it?

Posted on by
Reply

I’m all too familiar with this error. I see it all the time when setting up or making changes to web sites.

The good news? It’s not your fault.

The bad news? There’s nothing you can do.

•

Quite literally an “Internal Server Error” is an error on the web server that you’re trying to access. That server is misconfigured in some way that is preventing it from responding properly to what you’re asking for.

Aside from informing the site owner – which may, or may not, be the “webmaster” email address that’s included in the error message – there’s absolutely nothing you can do to resolve this problem, because it’s not your problem. The person who is responsible for the website needs to fix it.

So what if that’s you? What if you’re the webmaster?

Nine times out of 10, this error results from one of two very common errors:

  • An error in a cgi script that caused it to fail or output an error message before it started producing valid HTML. It happens to me all the time if I have a syntax error in one of my Perl cgi scripts.
  • A permissions issue attempting to access a cgi script. Depending on how your web server is configured, quite often it’s not enough for the script to have “execute” permission, but it must also be owned by the correct user, and belong to the correct group. I can’t tell you what that should be, since it varies widely from server to server (and even site to site on the same server). My best recommendation is to look at the attributes of a script that’s working, and copy that.
“Quite literally an ‘Internal Server Error’ is an error on the web server that you’re trying to access.”

The format of the error described here is common for Apache web servers. That means we can also look for common error logs. Once again, the location of these logs can vary a great deal based on the specific web server configuration.

  • access_log is the log of successful accesses. On a shared or virtual hosting server there may be many of these, one per web site, often with site-specific names, or in site-specific locations on the server.
  • error_log is the log of errors. One some hosts, there is an error log per site. On others, though, even though each site may have its own access_log, there may only be a single error_log for the system.
  • suexec_log is the log most people forget about. This log deals with the permissions used to execute cgi and other scripts. When a cgi fails to execute because its ownership is wrong, the generic “Internal Server Error” shows in the error log, but a more specific error detailing the permissions involved shows in this log.

And for the record, the message: “Additionally, a 500 Internal Server Error error was encountered while trying to use an ErrorDocument…” indicates a second configuration error. When the web server encounters an error, the page it displays to report the error is actually just another .html file stored on the server. In all likelihood, the website has been configured to use “custom” error pages (meaning that the website can provide its own custom error messages), but the custom error pages were never created or configured.

 

Share

Password Protecting Your Pages with .htaccess

Posted on by
Reply

Tutorial by Matt Doyle. Level: Intermediate. Published on 3 October 2005 in Apache.

Learn how to use Apache’s .htaccess files to protect pages on your site with a username and password.

Introduction

You may have visited a web page that pops up a dialog box similar to this one:

If you don’t know the username and password to enter, then you can’t access the page or site – it’s “password protected”. It’s sometimes handy to be able to password protect your pages like this – for example:

  • You’re building a new site, but you only want yourself (and maybe a select few) to be able to view the work-in-progress.

  • You have an area of your site that you never want the general public to have access to – for example, your web stats or private pages.

  • You have some paid (subscription) content on your site that only subscribers should be able to access.

Apache lets you password protect individual files, folders, or your entire site fairly easily. Read on to find out how it’s done.

How it works

To add password protection to your pages, you need to do the following two things:

  1. Create a text file on your server that will store your username and password.
  2. Create a special file called .htaccess in the folder you want to protect.

That’s it! Now let’s take a look at how to do each step.

Creating the password file

The first step is to create a simple text file that will store your username and password, separated by a colon (:). The small catch is that the password must be encrypted. Luckily, there are many free web-based utilities that will encrypt the password for you. Try one of these:

Simply enter your desired username and password in one of these pages and submit the form. You’ll get back a string similar to the following:


fred:p29cmnwl4a0et

Now, open up your favourite text editor (e.g. Notepad or TextEdit), then copy and paste the username/password string into the editor. Save the file and call it .htpasswd.

Next, upload this file to your website. Make sure you place it outside the Web root of your site if possible, as you don’t want just anyone to be able to view the file! For example, place it above your public_html or htdocs folder. (Having said this, Apache is often set up by default to block web-based access to files beginning with .ht. Better safe than sorry though!)

If you can’t place your .htpasswd file outside your Web root, name it something that’s not easily guessable – for example, .htxuymwp - so that people won’t be able to find it easily. (In addition, it helps to start the filename with .ht; as mentioned earlier, Apache usually blocks access to files starting with .ht.)

Alternative: Creating the password file using htpasswd

If you have SSH access to your web server (or you’re running Apache on a local machine), you can encrypt your password and add it to your password file in one go by using thehtpasswd utility that comes with Apache. Simply SSH to your server or open up a terminal window on your local machine, cd to the folder where you want to create your password file, and type:

htpasswd -c .htpasswd fred

(where fred is the username you want to use). You’ll be prompted to enter and retype your password, then the .htpasswd file will be created for you.

Creating the .htaccess file

Now that you have created and uploaded your password file, you need to tell Apache to use it to protect your page(s) or site. This is what your .htaccess file will do.

Open your text editor again, create a new file, and save it as .htaccess.

Protecting a folder

To password protect a folder on your site, you need to put the following code in your.htaccess file:


AuthUserFile /full/path/to/.htpasswd
AuthType Basic
AuthName "My Secret Folder"
Require valid-user

/full/path/to/.htpasswd should be the full path to the .htpasswd file that you uploaded earlier. The full path is the path to the file from the Web server’s volume root – for example,/home/username/.htpasswd or C:\wwwroot\username\.htpasswd. (If you’re not sure of the full path to your site or home directory, ask your Web hosting company for this info.)

The above .htaccess file will password protect all files in the folder that it is placed in, and all sub-folders under that folder too. So if you wanted to password protect your entire site, you would place the .htaccess file in your Web root folder.

Protecting a file

To password protect just a single file in a folder, use the following .htaccess file:


AuthUserFile /full/path/to/.htpasswd
AuthType Basic
AuthName "My Secret Page"
<Files "mypage.html">
  Require valid-user
</Files>

This will password protect just the mypage.html file in the folder where you put the.htaccess file.

Uploading the .htaccess file

Once you’ve created your .htaccess file, upload it to your website, placing it in the folder (or folder containing the file) that you want to protect.

Testing it out

Now use your Web browser to visit the folder or file that you’ve protected. You should see a password dialog like the one shown at the start of this tutorial. Type in the username and (unencrypted) password that you chose earlier, and you should be given access to your folder or file!

(By the way: with this type of password protection, you continue to have access to the password protected stuff until you restart your browser.)

Problems?

If you can’t access your stuff and the dialog keeps popping up, check that you entered the username and password correctly. If it still doesn’t work, check the path to your .htpasswdfile on the server – make sure the path specified in the AuthUserFile directive is correct. Also make sure that both the .htpasswd and .htaccess files are readable by the Web server user (chmod 644 should do the trick for UNIX/Linux/FreeBSD servers).

If the password protection isn’t working (i.e. you can still access your stuff without needing to enter a username/password), check that you uploaded your .htaccess file to the right folder. Also check that your web server supports .htaccess password protection (it needs to be an Apache server, and your server admin needs to have enabled the AuthConfigoverride for your site).

Password protecting more stuff

  • If you want to password protect other folders (that aren’t under the currently protected folder), simply copy your .htaccess file to the new folder to be protected.
  • To password protect more than one file in the same folder, just create more<Files></Files> blocks within the same .htaccess file – for example:

AuthUserFile /full/path/to/.htpasswd
AuthType Basic
AuthName "My Secret Page"
<Files "mypage.html">
  Require valid-user
</Files>
<Files "myotherpage.html">
  Require valid-user
</Files>

Adding more usernames and passwords

You’re not restricted to just one username/password. If you want to add other usernames and passwords, simply repeat the “Creating the password file” procedure above, but add each new username/password line to your existing .htpasswd file, e.g.:


fred:p29cmnwl4a0et
linda:vwp45xakfh89

Alternatively, if you’re using htpasswd to create your passwords, as described earlier, then you can add extra users with the command:

htpasswd .htpasswd linda

(where linda is the username you want to add). Make sure you don’t include the -c option when adding additional users, or htpasswd will attempt to create a new password file!

Further info

For full information on Apache’s mod_auth module (the module that does password protection, amongst other things), see the Apache mod_auth documentation.

Share

HTML:Iframe-inf wordpress Infection

Posted on by
Reply

Here is an article that really helped me getting my server back up and running.

HTML:Iframe-inf wordpress Infection

If your blog has been infected by the HTML:Iframe-inf  infection according to avast here are two scripts that can help you.

FirstWhat is the HTML:Iframe infection?– Its just a line of text that is inserted at the end of every index.php and/or index.htm in your website. Nothing to freak out about but you want to fix it. And Its probably due to wordpress not being secure.

Anyways, here is what you do : This is something you run on the commmand line – See the video below for an idea.

You will need to find infected files first.

find / -type f | xargs grep -l ‘<iframe’  2>/dev/null or you could print out a list of files possibly comprimised. by typing find / -type f | xargs grep -l ‘<iframe’  2>/dev/null >infectedFileslist.txt

The first step is figuring out what is going on with your virus infection.

If you know the time frame of when the virus ran then you could narrow the list of infected files even more by tweaking the find command.

Lets say you know it infected your website about 5 days ago.

Then you would modify the find command to search all files modified less than 10 days ago.

find / -type f -mtime -10 | xargs grep -l ‘<iframe’  2>/dev/null >infectedFileslist.txt

More info on the find command here

http://content.hccfl.edu/pollock/Unix/FindCmd.htm my short version find . -mtime +5 -mtime -10 # find files modifed between 5 and 10 days ago Ok so now you have a list of infected files … This is VERY HELPFUL as you are halfway there to cleaning up your server.

Remove infected text

find / -type f -mtime -10 | xargs grep -l ‘<iframe’| xargs perl -pi -e ‘s/^.*\<iframe.*$/ /g’ Here is an explanation of what the script does line by line so you can adjust per your situation. find / -type f -mtime -10 – looks all files that were modified in the last 10 days ( you adjust as needed) xargs grep -l ‘<iframe’ – of that list of files modified recently look for a line that says <iframe xargs perl -pi -e ‘s/^.*\<iframe.*$/ /g’ – search and replace that line with a blank space Understanding this last line – perl -pi -e is important — http://www.linux.org/lessons/short/perlpie/perl_pie.html You want to be sure that you know whats going on there because this is where the search and the replace happens – Check out this article — http://www.linux.org/lessons/short/perlpie/perl_pie.html You can modify the script line by line to

Here is a video explaining this:

My Contribution:

You can also use this linux command to find files that were changed in last 10 days

find /directory-path-to-search-files-from/ -type f -mtime -10 > infectedFileslist.txt
find . -name "*.js" > /home/star/public_html/infectedJSlist.txt
grep command: Recursively Search All Files For A String
cd /path/to/dir
 grep -r "word" .
 grep -r "string" .
 Ignore case distinctions:
 grep -ri "word" .
To display print only the filenames with GNU grep, enter:
 grep -r -l "foo" .
 You can also specify directory name:
 grep -r -l "foo" /path/to/dir/*.c
find command: Recursively Search All Files For A String
find command is recommend because of speed and ability to deal with filenames that contain spaces.
 cd /path/to/dir
 find . -type f -exec grep -l "word" {} +
 find . -type f -exec grep -l "seting" {} +
 find . -type f -exec grep -l "foo" {} +
 Older UNIX version should use xargs to speed up things:
 find /path/to/dir -type f | xargs grep -l "foo"
 It is good idea to pass -print0 option to find command that it can deal with filenames that contain spaces or other metacharacters:
 find /path/to/dir -type f -print0 | xargs -0 grep -l "foo"
 
I found this command to be really helpful if you are not certain if its an iframe attack or not.
Share