Getting Started with PHP and MySQL on pair.com

Introduction

I'm an experienced C/C++/Perl/Java/SQL/HTML programmer. This page is a log of my experiences creating a database-driven web page, with an eye towards helping beginners (especially those not familiar with Unix) get past some stumbling blocks I noticed.

My web site is hosted at pair.com, which uses a free Unix operating system (FreeBSD), a free database (MySQL), and two free languages for server-side scripting: PHP4 and Perl.

Of the two scripting languages, I'll cover PHP4, which is more like Microsoft's Active Server Pages (ASP). (There's even a program, asp2php, that can help you convert old ASP pages to PHP. I haven't tried it, though.)

If you're new to Unix...

Pair.com uses Unix (FreeBSD, to be exact), so if you're not on good terms with the Unix commandline, you may want to brush up a bit by using some of the resources described on my page about learning Unix.

If your computer runs Microsoft Windows, you may wish to download Cygwin, which is a free and complete Unix environment that runs under Windows. This is an excellent way to practice Unix tools without the hassle of having to install Linux or BSD.

Scoping out PHP and MySQL support at Pair.com

I visited Pair's support page, and saw that they have a PHP tutorial and a MySQL tutorial which make using them seem fairly straightforward. Pair also has private support newsgroups, so I set up my browser's newsreader to access them, subscribed to the newsgroup pair.db, and read a few of the posts. There were plenty of discussions going on, and it looked like if I ran into trouble, I could probably get help from another user. Feeling like I knew how to start, and where to get help, I moved on to finding a demo app to try.

Demo #1: WebCalendar

To see how hard it was to get started developing active websites with PHP and MySQL, I decided to try installing an open source web calendar application that uses them. My favorite place for finding such software is Freshmeat.net. Searching there for 'php mysql calendar' yields about ten hits; the most promising one was WebCalendar, as it had a nice online demo, many users, and a very responsive author.

Anticipating Trouble

Whenever you install a new application, it pays to scope out the support situation, and see if there are common gotchas with known workarounds. That's why, even before I installed WebCalendar, I read through the WebCalendar support forum. I found that several people were getting an error message after installing because WebCalendar uses a feature of PHP3 which is now discouraged in PHP4. Searching for the word 'deprecated' (part of the error message), I found several people had posted solutions. So I was confident that if I ran into this common error, the fix would be easy.

Installing the demo application on my Pair.com account

  1. I followed the instructions in Pair's MySQL tutorial, for creating a database via their account administration page.
  2. Logged in to my pair account via telnet (ssh is better, if you have it)
  3. Downloaded WebCalendar by copying the download URL from my browser and giving it to a wget command in my pair ssh session, e.g.
    wget http://prdownloads.sourceforge.net/webcalendar/WebCalendar-0.9.34.tar.gz
  4. Unpacked it with the command
    tar -xzvf WebCalendar-0.9.34.tar.gz
  5. Read the installation instructions with the commands
    cd WebCalendar-0.9.34
    more INSTALL
    
  6. Followed the instructions to configure the app by editing a file:
    vi includes/config.inc
  7. Moved it to the desired location with the command
    mv WebCalendar-0.9.34 public_html/kegel/democal
I then tried it out by visiting it in my web browser at the url www.kegel.com/democal, and got the infamous error message
Warning: Call-time pass-by-reference has been deprecated...
I then went back to the WebCalendar support forum, found the thread that mentioned the problem and a fix. The fix was a replacement file, delawarewing.org/calendar/fix/php-dbi.inc.txt. I downloaded the file to my pair account with
wget http://delawarewing.org/calendar/fix/php-dbi.inc.txt
found the original file it was to replace, and compared them with the Unix command
diff php-dbi.inc{,.txt}
That looked awful until I realized the new file had MS-DOS line breaks. So I told diff to ignore whitespace:
diff -b php-dbi.inc{,.txt}
That looked much better; only a few lines were changed. Rather than make the edits by hand, I turned that into a patch with the command
diff -bau php-dbi.inc{,.txt} > fix.patch
and applied the patch with the command
patch -p0 < fix.patch
(I am such a geek). Visiting the calendar on the web then worked!

Total elapsed time from start of this log, including looking for a demo app: 3 hours. I could probably do it again in fifteen minutes now that I know what to do.

Demo #2: Phorm

I've twice been asked to set up an online survey by various small organizations I belong to. I could have used one of the free services out there, but they require registration, and subject users to advertising, so I decided to set up the survey the hard way.

After a bit of searching, we settled on Phorm as the package to use for the back-end that accepted the survey form POSTs, stored them in a MySQL database, and maintained a live count of results. It wasn't too hard to set up the first time following Phorm's documentation, but it wasn't trivial, so the second time around I took the following notes in hopes it helps other newbies get up and running.

Setting up Apache and PHP under Cygwin

The person I'm helping this time develops on a Windows box, so I decided to get Apache and PHP running on her machine so she could test everything locally. To do that, I downloaded and ran the Cygwin installer (be patient - the UI is a little funky!), picked my favorite mirror site (ftp.lug.udel.edu), selected Apache, PHP, and wget in the Web category, and clicked 'ok'. When it finished a few minutes later, the little "Setup Finished" dialog box was hidden behind something else, so I unburied it and clicked ok. I doubleclicked on the Cygwin icon to get a shell, edited /etc/apache/httpd.conf, set DocumentRoot to "/cygdrive/c/club/site" so Apache would find her documents, and ran apachectl start. That failed because it couldn't allocate memory for a DLL; the fix was to run 'rebaseall'. Then apachectl start worked, and I was able to browse to http://127.0.0.1 and see the site's files. (Note: this doesn't work on Windows ME, but it does on Windows XP.)

To make Apache easier to start, I created a batch file /startapache.bat containing

SET CYGWIN_ROOT=\cygwin
SET PATH=.;%CYGWIN_ROOT%\bin;%PATH%
cd \cygwin\usr\sbin
sh apachectl start &
made it executable (under XP, you really have to!) with
chmod +x /startapache.bat
and created a shortcut on the desktop to c:\cygwin\startapache.bat.

Creating a Database and Connecting to it from PHP

Now create the database on pair.com using their gui. Note that the nice page of passwords that creates can't be saved, and you'll never see those passwords again unless you print it. Fortunately, you can set the passwords anew via their gui, so even if you forget to print it out, you're not stuck. Select "remote access" so you can test the database from your home Apache before uploading.

Then to test database access via PHP from her home server, I created a small script 'test.php' in her document root directory, with the newly-created database's hostname, username, and password:

<?php
    mysql_connect("xxxx.pair.com", "xxxx_r", "xyZZy321") or
        die("Could not connect: " . mysql_error());
    printf ("MySQL server version: %s\n", mysql_get_server_info());
?>
and viewed it in her web browser via the url http://127.0.0.1/test.php (having already started Apache). After resetting the password at pair.com (I must have written it down wrong the first time), that worked fine.

Setting up Phorm

First, download phorm.tar.gz, unpack it in the local Apache's document root directory, and read the doc. For instance:
$ wget ftp://ftp.holotech.net/phorm/phorm.tar.gz
$ tar -xzvf phorm.tar.gz
$ cd distribution
$ more INSTALL*
$ more SUPPORT*
Then try reading the doc via the local Apache (why not?) at http://127.0.0.1/distribution/quickstart.rtf (requires a word processor) and http://127.0.0.1/distribution/docs/. Keep coming back to the doc; try cruising through all the topic in the left navbar, and the appendices. It's confusing, but everything you need is in there.
Geek moment: you can search the source code and doc with the Unix commands find and grep. For instance, to see every place it mentions MySQL, try
$ find . -type f | xargs grep -i mysql | more

PHP doesn't support mail under Cygwin :-(

Phorm comes with a handy test page that checks whether Phorm can send email; it's in testmail.php. It succeeds under Linux, but the Cygwin build of php doesn't support sending email. So we'll do the rest of this demo without worrying about email.

Quick test of Phorm without SQL

One quick way to try out Phorm is to get it to log to a file on the server. Here's a quick way to try that:
  1. Create a file named myconfig.php containing just the commands needed to tell Phorm to create a log file, and what form fields to put in it:
    <?php
      $PHORM_LOG = "phorm.log";
      $PHORM_LOGVAR = "Name PHORM_FROM Comment";
      // If using PHORM_* variables in PHORM_LOGVAR, must "declare" them
      // else they will be silently blanked out; see docs/configs.html
      $PHORM_FROM = ph_DECLARE;
    ?>
    
    (Be sure not to have anything before or after the php section, or you'll get a PHP error about "headers already started" later.)
  2. Edit the file x-comment.html (included with Phorm), and add the hidden form field PHORM_CONFIG with the name of that file, e.g.
     <INPUT TYPE=HIDDEN NAME="PHORM_CONFIG" VALUE="myconfig.php">
    
  3. Load the form in your browser from your home web server via the url http://127.0.0.1/distribution/x-comment.html, fill it out, and submit it. The web server's file phorm.log should have a line with the form's contents appended each time you click submit. (You can probably see it at http://127.0.0.1/distribution/phorm.log, which is not very secure; if you were doing this for real, you'd put the file somewhere more private.)

Quick test of Phorm, with SQL

Once the above is working, let's have Phorm log to a MySQL database instead of a text file.

First, design the SQL table to hold the form results. (Pair.com's MySQL tutorial may be helpful.) You'll need to know the field names of the form, and how big they can be, in order to figure out how to create the columns of the table. The x-comment.html example form has three fields. Two (Name and PHORM_FROM) are short, and can fit in, say, 32 chars; the third (Comment) is large, so make it as big as possible (255). You are free to make column names different from form field names, if you like. For example, I'll use "name" instead of "Name". The MySQL command to create a table with columns with those names and sizes is:

create table survey ( name varchar(32), email varchar(32), comment varchar(255) );
Then log in to your pair.com shell account via ssh, and create the table you've designed. Here's an example session, showing how to connect to the server via ssh from the cygwin commandline, and create the table. (The $, myname@zirx%, and mysql> below are the prompts Cygwin, your Pair.com server's shell, and MySQL with prompt you with -- don't type them. The arguments to mysql are your database server, username, password, and database name; you'll need to fill them in with your own.)
$ ssh myhost.com
myname@zirx% mysql -hxxxx.pair.com -uxxxx -pxxxxx xxxx_survey
mysql> create table survey ( name varchar(32), email varchar(32), comment varchar(255) );
mysql> quit
You can then check that the table is created ok with the MySQL command
mysql> select * from survey;
That should say "Empty set" (since there's no data yet).

Now set up Phorm to log to this table (see Phorm's docs/dblog.html.) To do this, edit myconfig.php, and add the lines inside the php section:

$PHORM_MYVARS = "name=Name, email=PHORM_FROM, comment=Comment";
// If using PHORM_* variables in PHORM_MYVARS, must "declare" them
// else they will be silently blanked out; see docs/configs.html
$PHORM_FROM = ph_DECLARE;
$PHORM_MYTABLE = "survey";
$DBase = "myaccount_survey";
$DHost = "xxxx.pair.com";
$DUser = "xxxx_w";
$DPass = "xyZZy987";
These define the form fields to log, the database table to log to, and how to connect to it. (You'll need to fill the last four in with the values for your database. Since pair has different usernames for different levels of access, be sure to use the 'write' access username here, i.e. the one ending in _w, and the associated password.)

Once you've done that, every time you click submit, a row should be added to that table in the database. You can verify this by logging in to your server via SSH, and using the MySQL command

mysql> select * from survey;
It should show one row for every time you click, and the rows should contain the data that was typed into the form.

Using PHP and MySQL to Tally Survey Results

Phorm will put the form results in the database, but it won't tally the results; you have to write your own PHP code to grab the results from MySQL, summarize them, and output the summary in HTML. Here's a simple example that just counts the number of responses:
<?php

// Connect To Database
// * mysql_connect takes the servername, user, and password
// * as arguments. mysql_selectdb takes the database name.
// * Together, they open a connection to your database.
mysql_connect("xxxx.pair.com","xxxx_r","xyZZy321");
mysql_selectdb("myaccount_survey");

// Execute Query
// * mysql_query takes as its argument the query you are
// * executing on the database. It should be assigned to
// * a variable -- the variable is used by other functions
// * to retrieve the results.
$QUERY = mysql_query("SELECT COUNT(*) from survey");

// How man rows in results?
// * mysql_num_rows takes the variable the query was
// * assigned to (referred to hereafter as the query
// * identifier) and returns the number of rows the query
// * resulted in.
$NUMROWS = mysql_result($QUERY, 0);

echo "Number of responses: $NUMROWS<br>";

?>

Doing it with your own form

Real forms are more complicated than x-comment.html, and it's a pain to copy field names into the various places they have to go. Here's fields.sh, a shell script that can take an HTML form and generate the PHORM_MYVARS value for your config file as well as an SQL statement to create your table (you may want to edit this a bit). It only works if the html tags aren't broken across lines, and don't have any funny characters or spaces in them.
#!/bin/sh

FILE=$1

# create field list for phorm
(
echo -n '$PHORM_MYVARS = "' 
grep -i name=._ $FILE | sed 's/.*"_/_/;s/".*//' | uniq | \
	sed 's/_\(.*\)/_\1=_\1, /' | tr '\012' ' '
echo '";'
) | sed 's/, *"/"/' > myvars.txt

# create table create statement for mysql
(
echo 'create table survey (' 
grep -i name=._ $FILE | sed 's/.*"_/_/;s/".*//' | uniq | \
	sed 's/_\(.*\)/_\1 CHAR(40), /' 
echo ');' 
) > fields.sql
To use it, run it with the filename of your form as the argument, e.g.
sh fields.sh ../survey.html
It saves the PHORM_MYVARS statement in myvars.txt; copy that into your myconfig.php file. It saves the SQL statement to create the table into a file fields.sql; you can copy and paste from that when creating your table.

A Nicer Report

The report script given earlier only gives the total number of responses. Here's a fragment you can add to that script to actually give summary counts for all questions. You have to edit it to embed the questions in it, which is a bit of a pain; I haven't bothered to automate that, as it's not quite worth it for just one or two surveys.

This script assumes that each question's field name in the form is an underscore followed by the question number -- with (for multiple checkbox questions only) a letter to make the name unique. (Phorm's design forces you to have all unique field names.)

It helps to be handy with grep and vi when plugging the questions into this report script. If you try to do it by hand, it takes a lot longer.

$questions = array(
"1. Gender",
"2. Age Range",
"3. Regarding potential weekend events:",
"4. I live in:",
"5. The location of meetings will affect my decision to attend meetings and get-togethers: ",
"6. My household income range is:",
"7. Other comments: ");

$fields = array(
"_1",
"_2",
"_3a",
"_3b",
"_3c",
"_3d",
"_4",
"_5",
"_6",
"_7");

$q = 1;
foreach ($fields as $field) {
	# if this answer matches the next question, print the new question
	if (preg_match("/_$q/", $field)) {
		echo "<p>Question ";
		echo $questions[$q-1];
		echo "<br>";
		$q++;
	}
	$QUERY = mysql_query("SELECT $field, COUNT(*) from survey group by $field");
	$NUMROWS = mysql_num_rows($QUERY);
	$I = 0;
	while ($I < $NUMROWS) {
		// Get Results
		// * mysql_result returns the value of a specific field
		// * in a specific row. It takes three arguments: the
		// * first is the query identifier, the second is the row
		// * number, and the third is the field name. In this
		// * example, a while loop is used to process all
		// * rows.
		$FIELD1 = mysql_result($QUERY,$I,0);
		$FIELD2 = mysql_result($QUERY,$I,1);
		if (strcmp($FIELD1, "")) {
			echo "$FIELD2 : $FIELD1<br>\n";
		}
		$I++;
	}
}

Securing the backend

Because a form processor presents a network service to the world, it is vulnerable to attack. Little web applications like Phorm are often installed without regard to security. This not only puts your own data at risk -- a little security hole in your application may be all the opening a cracker needs to take over the whole server. This could open you up to a lawsuit, or getting fired, or maybe just getting kicked off your server.

Scared yet? Good. Here are a few steps that might help you avoid becoming security road kill.

Updating the survey form

OK, so it's all online and ready to go live, but the folks who wrote the survey changed their minds, and sent you a new version of the survey form at the last minute. Here are the steps you have to go through to blow away the database, create a new one matching the new form, and update your report script:
  1. Edit the new form, and make sure its field names conform to the uniqueness rule (e.g. multi checkboxes and SELECT boxes need a letter suffix to make them unique, since more than one can be selected by the user at once).
  2. If you're going to use grep to extract the questions for the report script, make sure the questions aren't broken across lines in the HTML. Then use grep to pull out the questions, and save them in the report script. For instance, when preparing my last survey, these are the commands I ended up using:
    $ grep -i name=._ survey.html | sed 's/.*"_/"_/;s/" .*/",/' | uniq > fields.txt
    $ grep '>[0-9]*\.' survey.html | sed 's/.*>\([0-9]\)/\1/' > questions.txt
    
    (Looks cryptic, but once you're familiar with Unix text munging tools like grep and uniq, you can handle huge tasks without breaking a sweat.)
  3. Log in to the MySQL database and give the 'drop table survey' command.
  4. Run the fields.sh script shown above to generate new fields.sql and myvars.txt files, and use them as follows:
  5. Edit fields.sql to fix up the max field sizes (form fields that allow freeform comment should be VARCHAR(255), which is as big as is allowed). Then copy and paste that file's contents into your MySQL session to recreate the table with the new fields.
  6. Copy and paste the contents of myvars.txt into your myconfig.php file.
  7. Test, test, test...
If you get the error "Phorm is only for processing the results of forms...", that means you forgot the hidden field setting PHORM_CONFIG.

If you get an SQL error saying something like "unknown field name", you may have forgotten to update myconfig.php with the contents of myvars.txt.

Books on MySQL and PHP

For developing my own code, I'd like to have a good book handy, so I went looking on Amazon and at groups.google.com for recommendations, and also asked my local Linux users' group for advice.

I have yet to try any of these myself, so caveat emptor.

Related Sites