MySQL logging module for Apache
===============================

1.05:

* Removed some redundant code, after being noted by Vivek Khera that this code doesn't
  even get called with the current apache code.  It can be done in apache 1.3,
  but it works ok without it anyway.
* Added the necessary include file to make the module compile under Apache 1.3b6.  I
  haven't actually tested that it works, though.
* indent'd the code.


1.04:

* Rearranged some code to allow for a successful apache 1.3beta compilation.
  Please note that this is *untested*, I only got it to compile, I haven't
  actually tried to run apache 1.3.


1.03:

* Changed the check for 'mysql server has gone away' to be case insensitive,
  so that it works with MySQL 3.21
* Changed the behavior so that a link isn't established until it's necessary
  (e.g., if SQL logging is used for one virtual IP, a link won't be opened
  until there's an access to that IP).


1.02:

* Managed to track down that segmentation fault that occured once, and fixed it.
  No known bugs now exist.


1.01:

* Segmentation fault in case of certain parameters lacking fixed.
* Worked around the SIGPIPE signal that's sent in certain events from mysql_query().
* Minor modifications

----------

This module is a combination of the current mod_log_agent.c, mod_log_referer.c and mod_log_config.c 
(partially), hacked to save their information into MySQL tables instead of flat textfiles.

To compile it in, copy mod_log_mysql.c into your apache source directory, and add the line:

Module mysql_log_module	mod_log_mysql.o

to your Configuration file.  You must also compile in the config_log_module, since the mysql logging
module uses certain functions from it (it's enabled by default in the Configuration file). 
Also, if not there already, you would have to add

-L/usr/local/lib/mysql -lmysqlclient -lm

to your EXTRA_LIBS directive in the Configuration file (the location of your mysql client may be different,
you should obviously specify the correct location on your machine).
You would have to rerun ./Configure and rebuild the server.

I've tested this under Linux only, but there's nothing platform specific about it, so it should
work on just about any platform under which apache runs (and that has the mysql client library).
Also, this module is based on apache 1.2.x code, I haven't tested whether it works on earlier versions.


Approach
--------

In order to save speed and overhead, links are kept alive in between queries.  This module uses
one SQL link per httpd process.  Among other things, this means that this module supports logging
into only one MySQL server, and for now, also, only one SQL database (although the latter limitation
can be relatively easily removed).
Different data can be sent to different tables.  I.e., it's possible to define one table for TransferLog,
one for RefererLog, and a 3rd for AgentLog.  Virtual hosts are supported in the same manner they are in
the regular logging modules, as in, if you specify a different table for a virtual host it will be used,
otherwise the 'general' would be used.  Note since all 3 types of logs are implemented within the same
module, if you specify an overriding table for a virtual host for one type of log, it'll ignore any
previous 'general' defaults (see the example in the end).
SQL links are opened on demand (i.e., the first time each httpd needs to log something to SQL, the link
is opened).  In case the SQL server is down when trying to connect to it, the module remains silent
and logs no error (I didn't want thousands of error messages in the logfile).  In case the SQL link
is broken ("mysql server has gone away") a proper error message is kept to the error log (textual :),
and the module tries to reestablish the concact (and reports whether it succeeded or not in the error
log).  If the link cannot be reestablished, the module will, again, remain silent.
Technical note:  The SQL link is registered using apache's pool mechanism, so SQL links are properly
closed on any normal shutdown, kill -HUP or kill -TERM.


Supported directives
--------------------

LogMySQLInfo <host> <user> <password>	- Optional parameter (otherwise assumes NULLs, which will use
					  the current uid, now password and 'localhost').  If you 
					  wish to specify only certain parameters, specify '.'
					  for the ones you don't wish to specify (use once (atmost) per .conf file).
LogMySQLDB <db_name>			- Log into the database named 'db_name' (use once per .conf file)
TransferLogMySQLTable <table_name>	- Log transfer log into table named 'table_name'.
RefererLogMySQLTable <table_name>	- Log referer log into table named 'table_name'.
AgentLogMySQLTable <table_name>		- Log agent log into table named 'table_name'.
TransferIgnore <ign1> <ign2> ...	- Don't log transferlog entries that contain one of the specified strings
					  (uses strstr(), no regular expression/globbing of any kind done).
RefererIgnore <ign1> <ign2> ...		- Don't log refererlog entries that contain one of the specified strings
					  (ditto, and shared with the regular flat textfile RefererLog).
TransferLogMySQLFormat <format_string>	- Rather than actually being a format string, this specifies what to log
					  (unlike the text log, no actual formatting is done).  You can specify
					  which fields you'd like to log by specifying their corresponding
					  character (same ones as the regular TransferLogFormat directive,
					  you can see them in the log_mysql_item_keys[] table in mod_log_mysql.c).
					  If no such format is specified, the default string is "huSUsb", which
					  logs remote_host, remote_user, time_stamp, request_uri, status and
					  bytes sent.  You have to define the transferlog table so that it
					  contains (at-least) the fields that you wish to log!


Table structure
---------------

The logging module does NOT create table automatically, you have to create them yourself.
For the Agent log, you need a table with a string (char/varchar) field named 'agent', and numeric field
named 'time_stamp'.
For the Referer log, you need a table with string fields named 'url' and 'referer', and a numeric field
named 'time_stamp'.
For the Transfer log, in it's default format, you need a table with string fields named remote_host, remote_user
and request_uri, and numeric fields named 'time_stamp', 'status' and 'bytes_sent'.  Other specific formats would
require a different table structure (you can see what kind of fields you need by looking at the mod_mysql_item_keys[]
table, seeing what the field name is and whether it's a string or a value).

Notes
-----

* The 'time_stamp' field is kept in the standard unix 'seconds since 1/1/1970 12:00:00' format.  In MySQL 3.21
you can convert this to a human readable format using from_unixtime().
* String values can be char or varchar, at a length of your choice (longer strings would be truncated).
  As usual, space vs. speed, if you use varchar your log database would be much smaller, but if you use char
  speed would significantly increase.  You decide:)
* Most fields should probably be set to not-null to save space (and perhaps some speed?).  Only ones that
  shouldn't are ones that you don't intend the logging module to update (you can have other fields in the
  logging tables if you'd like, but if they're set to not-null, the logging module wouldn't be able to insert
  rows to these tables...).
* Numeric fields in the original logging module were often given the value '-' to mean 'not applicable'.
  Since this cannot be done in a numeric field in SQL, such fields are assigned the value -1.



Disclaimer
----------

It works for me (I've tested it on my '2 hits/busy day' home Linux box, and afterwards on our pretty busy
tucows mirror (>100K hits a day) and it appears to be working fine.

If it doesn't, and causes you damage of any sort, including but not limited to losing logs, losing money
or your girlfriend leaving you (read 'boyfriend' where applicable), I'm not liable to anything.  Bug reports
and constructive flame mail are ok, though (both about the code and this quickly-written README file).

Also, if the fact that this text file uses like 110 characters per line annoys you, I'm truly sorry, I'm bad
at formatting README files...


Author
------

The actual logging code was taken from the already existing flat file text modules, so all that credit goes
to the Apache Server group.  The MySQL routines and directives was added in by me,
Zeev Suraski <bourbon@netvision.net.il>
