My first post of the year, and I should say Happy New Year I suppose. The year is already starting out better for you poor folks reading my non-sense blog, because I have an item I'm going to explain in detail for you, including videos and pictures, and this is how to set up a very simple anonymous chat system through a server to multiple browsers using HTML5's Server Side Eventing mechanisms to remove the need to write polling javascript code in your browser.
The reason I'm going to cover this in details is that though there are lots of examples and descriptions out there as to how to set about doing just this the example code given is sketchy in most cases, missing in many and does not actually accomplish much with the technology to plant the seed of actually putting it to use. So, in the hope of setting a trend for the year I'm going to put this post together, take video and explain it all and hopefully you'll all be able to get to the same stage I'm at and have a working chat exchange program.
Prerequisites
Lets cover the prerequisites, you need a web server set up on a machine, in my example here I'm going to use apache2 on Ubuntu 12.04 LTS, with PHP5 as my server side scripting language. The browser I'm going to use is Google's Chromium, because I like its debugging in the form of the javascript console.
The server I have has been set up as a blank server image on a VMWare virtual machine instance, and I've run the following commands to get it set up:
sudo apt-get install apache2 php5 libapache2-mod-php5
sudo /etc/init.d/apache2 restart
sudo apt-get install apache2 php5-mysql mysql-server
sudo /etc/init.d/apache2 restart
sudo apt-get install phpmyadmin
I allowed phpmyadmin to install and selected to install it into "apache2" at the prompt and I'm done, my passwords are all known to me and so whenever I've been asked for one I've typed it in, you will need do the same, and no, I don't want to know your passwords :)
I've gotten the server's IP (ifconfig) and from my desktop machine I've browses to its IP "http://192.168.2.101/phpmyadmin" and checked I can get onto the database.
Database
I'm using a database and a single table as an imtermediary, each browser connected to the chat page will insert lines of text "chat strings" and they will be inserted in time order into the table and then delivered to all the listening clients as they cycle around.
This processing of the chat potentially allows us to process and filter it for profanity, or prevent inappropriate discussions of any type we choose, before we insert it into the database.
We need to add a database with a table to the server, and so we're going to use this code:
(Don't worry this is an image, we're going to break down the SQL line by line).
CREATE DATBASE ChatSystem CHARACTER SET latin1 COLLATE latin1_general_ci;
This line actually creates the database for us, its called "ChatSystem" and uses the latin character set and sorts (collates) its columns in tables by index against the latin character set. If you're intenting to use a different character set you'll have to hit your chose SQL server's documentation for their available character sets. But latin1 is going to cover most of Western Europe, North and South America's and the Antipodes for this example.
USE ChatSystem;
This just selects the database we just created.
CREATE TABLE IF NOT EXISTS GeneralChat
(
Id BIGINT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (Id),
Tme BIGINT NOT NULL,
Text VARCHAR(128)
)
Engine = InnoDb;
This instruction does pretty much as you would guess reading it, it creates the table with the three fields "Id", "Tme" and "Text". Text is just the chat string we're exchanging, the field design allows for upto 128 characters so in our HTML presented to take chat strings in we must remember to limit the input box's length to allow only 128 or less characters.
Tme is actually time, I'm omitting the 'i' to avoid using a reserved word in anyones SQL implementation, the time we're going to take as the time the server receives the chat string. The client browser is not going to have to send it, and its just going to be a number becuase our server side scripting language (PHP) gives us the command "time()" which returns the time as a single integer.
Finally Id is the order the chat strings are inserted, we're going to leave this as an auto incremented integer because then we need not worry about it in our SQL insert statement later, again like Tme the Id need not be worried about nor generated by the client browser.
I take all this SQL and run it through phpmyadmin's SQL intepreter and I get my new database and table ready for use.
Client Side - Chat.html
On the client I'm going to have a very simple page, which is an outer page with an input box and a "send" button, and then an inner frame (iframe) which is showing the messages as they come back from the server... I could make this all one page, but I'm splitting things up to keep the demonstration as simple as possible.
The outer page first, we need to have a table (yes I like tables you can use divs and spans all you like) which has two rows, the top row is going to show us chat arriving and contain our iframe and below we're going to have our input text box:
<table>
<tr>
<td>
<iframe src="chatstream.html"></iframe>
</td>
</tr>
<tr>
<td>
<table width="100%" border=0 cellspacing=0 cellpadding=0>
<col width="85%">
<col width="15%">
<tr>
<td colspan=1>
<input id="ChatString"
type="text"
value=""
maxlength="48"
onkeypress="HandleKeyPress(event)">
</td>
<td colspan=1
onclick="UserSubmitChat()">
Send
</td>
</table>
</td>
</tr>
</table>
So we have several things we need now define, first the inner frame HTML "chatstream.html", which we'll cover in detail below. And then two javascript functions:
"HandleKeypress(event)" and "UserSubmitChat()", the first function just checks whether we're handling the enter key and calls the second function, it looks like this:
function HandleKeyPress (event)
{
var l_chCode = ('charCode' in event) ? event.charCode : event.keyCode;
if ( l_chCode == 13 )
{
UserSubmitChat();
}
}
So the mor importand function is "UserSubmitChat()" which looks like this:
function UserSubmitChat ()
{
var l_textBox = document.getElementById ("ChatString");
if ( l_textBox )
{
InsertChatString(l_textBox.value);
l_textBox.value = "";
}
}
Very simply the function looks up the element called "ChatString" which is our input box (see above in the HTML) and if found it inserts the chat string, blanking the input box.
Note: One could optimise the page to take the "document.getElementById" call once and store the "l_textBox" throughout!
So, I'm simply taking the string content of the input box and calling yet another function, what is this "InsertChatString(value)" function doing? Well, lets take a look:
function InsertChatString (p_ChatString)
{
m_ChatQueue.push(p_ChatString);
}
Well, it takes the string and pushes it onto the queue of strings going somewhere, this is important we want to order the chat we're posting and a queue is an easy way to achieve this, we push onto the back of the queue and pop off the front when we want to dispatch the string somewhere.
Few developers realise the queue functionality is built directly into the javascript array, so to allow the member "m_ChatQueue" to be a queue for us we simply have to create it as an empty array:
var m_ChatQueue = [];
So, all in all our javascript block now looks like this, with neat and tidy functions to handle key pressing, string extraction and pushing the string onto the queue.
<script language="javascript">
var m_ChatQueue = [];
function InsertChatString (p_ChatString)
{
m_ChatQueue.push(p_ChatString);
}
function UserSubmitChat ()
{
var l_textBox = document.getElementById ("ChatString");
if ( l_textBox )
{
InsertChatString(l_textBox.value);
l_textBox.value = "";
}
}
function HandleKeyPress (event)
{
var l_chCode = ('charCode' in event) ? event.charCode : event.keyCode;
if ( l_chCode == 13 )
{
UserSubmitChat();
}
}
</script>
But still the strings go no-where, what I have done is therefore rather than have an iframe or hidden on screen element posting (with an HTML form) the data I've decided to have a server side script page (php) which will take the chat string as a HTTP GET parameter.
This page is to be called "chatsubmission.php" and take the paramter "ChatString", so the URL for posting the chat string "HelloWorld" will be "chatsubmission.php?ChatString=HelloWorld".
When I open the chat posting page then I want to smooth out the posting of these strings to the server, as the user enters chat strings I don't want them to be paused out as it posts to the server I want things to flow. So the user seamlessly queues the chat strings they enter, and they may have posted many before the page submits them all, but the submission goes on as a background task very fast without seeming to interrupt the user or their browser.
To achieve this I'm going to use a timer, and when the HTML loads I'll start it:
<script language="javascript">
var m_ChatTimer = null;
function StartChatTimer ()
{
StopChatTimer ();
m_ChatTimer = setInterval(ChatTimerMatured, 500);
}
function StopChatTimer()
{
if ( m_ChatTimer != null )
{
clearInterval(m_ChatTimer);
m_ChatTimer = null;
}
}
</script>
Note: All the functions shown all go in the same single "<script>" block on the same page, I'm splitting them up for this post only, and will post the whole page code at the bottom.
The HTML to start the timer is then:
<body onload="StartChatTimer()">
Now we have a timer maturing every 0.5 seconds, and its going to submit to some URL we we've defined all the strings of text in the m_ChatQueue. What does this code look like?
function ChatTimerMatured ()
{
StopChatTimer ();
if ( m_ChatQueue.length > 0 )
{
while (m_ChatQueue.length > 0)
{
var l_value = m_ChatQueue.pop();
var l_URL = "chatsubmission.php?ChatString=" + l_value;
var l_HTTP = new XMLHttpRequest();
l_HTTP.open ("GET", l_URL, false);
l_HTTP.send (null);
}
}
StartChatTimer();
}
Let us just analyse this code, so we stop the chat timer, to prevent another timer mature happening whilst we're processing a potentially long list of strings of chat. And then check the queue, if there are queued items we take one item off (pop) at a time and build the submission URL.
Armed with the URL, we then cheat, we use the javascript XMLHttpRequest object to get that URL, effectively then our server side page gets the request and parameters. Much as if we had an HTML form and submitted it causing an HTTP Post.
Take a moment to look this code over, both above and in the full listing for "chat.html" at the bottom of the post.
Once all the strings are posted, or we're determined there's nothing to do, we start the chat timer again. During this interlude the user may have posted more chat strings, but we smooth out the operation. We could also temper the duration of the timer to better suit slow internet connections, or increase the responsiveness if necessary.
Server Side - ChatSubmission.php
So far we've taken the user input and cleverly posted it into the server, now we must see what the server side does with the chat string:
<?php
if ( !empty($_GET) )
{
require_once('dbFunctions.inc');
$l_ChatString = $_GET["ChatString"];
if ( InsertChat($l_ChatString) )
{
echo "Y";
}
else
{
echo "N";
}
}
?>
This is the complete "ChatSubmission.php" page called by the javascript XMLHttpRequest object we previously explored, the PHP page includes some functions (written by us) and then from the URL gets the "ChatString" parameter, if there is such a parameter it then calls the function "InsertChat".
The PHP function I have written for this looks like this:
function InsertChat ($p_ChatString)
{
$l_result = false;
$l_Connection = ConnectDatabase();
if ( $l_Connection != null )
{
$l_ServerTime = time();
$l_SQL = "INSERT INTO GeneralChat VALUES (null, $l_ServerTime, '" . $p_ChatString . "')";
mysql_query($l_SQL, $l_Connection);
CloseDatabase($l_Connection);
$l_result = true;
}
return $l_result;
}
It opens the database connection, builds the insert SQL and performs the insert, returning true upon success or otherwise false. Very simple, and you may pick over the bones in the full listing.
Before we move on, we need to just see the "$l_ServerTime" and use of "null" in the SQL we insert with, the null is for the Id, which as you should recall we set as AUTO INCREMENT, and the time is the server time at which we receive the message. This time will be of interest later.
So, there we have the first leg of our chat string's journey, from the text input box into a javascript queue and then on a timer maturation out to the server as a seamless posting and into the mysql database.
Note: You can use any database or server side language you want at this point! Change the script around however you want and experiment!
Server Side Event - Second Leg
The second leg of our chat strings journey is to be from the database out to the screen, this is performed with the "chatstream.html" page, which is going to set up the client browser to listen for the server side events being pushed back. This is the page which you'll see most demonstrated out there on the internet, most demo's cover this setting yp of the message reception, and many do it better than I.
There are basically three events you can handle with Server Side Events in HTML5, "open" which is the event of the connection to the server being established, "error" which is the event when something has gone wrong server side, and finally "message" which is a text based string of characters from the server.
What you choose to put into the event is up to you, there are examples of doing json, xml or just plain text. For our chat system the message sent is going to be the actual raw chat string. We're going to make sure the server sends one chat string per message.
So, lets take a look at our HTML, its very simple:
<html>
<head>
<script language="javascript">
...
</script>
</head>
<body onload="LoadPage()">
<table id="chatTable" border=1 width="100%">
<tr>
<td>Chat</td>
</tr>
</table>
</body>
</html>
So we have a table called "chatTable" and we call a function called "LoadPage" when we open the page in the browser, we can now define the javascript:
function LoadPage ()
{
m_ChatTable = document.getElementById("chatTable");
source = new EventSource('chatlist.php');
source.addEventListener ("message", ChatArrived, false);
}
Breaking this down, we grap the chatTable from the HTML and then set up the key SSE receiving object. This "EventSource" points to a new server side php page (chatlist.php).
And then registers for the "message" arriving to point to javascript function called "ChatArrived":
function ChatArrived (event)
{
console.log(event.data);
}
For debugging, this function will suffice, but my full function takes the data string and appends it to the chatTable:
<script language="javascript">
var source = null;
var m_LastTime = 0;
var m_ChatTable = null;
function LoadPage ()
{
m_ChatTable = document.getElementById("chatTable");
source = new EventSource('chatlist.php');
source.addEventListener ("message", ChatArrived, false);
}
function InsertChat (p_ChatString)
{
if ( m_ChatTable != null )
{
var l_newRow = m_ChatTable.insertRow(m_ChatTable.rows.length);
var l_newCell = l_newRow.insertCell(0);
l_newCell.innerHTML = p_ChatString;
}
}
function ChatArrived (event)
{
InsertChat (event.data);
}
</script>
This therefore completes the HTML5 SSE loop back, so long as the "chatlist.php" serves up SSE events each time it sends a message it'll arrive on the page and append to our chat table.
Note: Some readers have forgotten this "chatstream.html" page we've just defined is inside the iframe from the first page, so we can see on the table the chat listing in the iframe and then the input box below.
Server Side - chatlist.php
Our final page is the PHP page which lists the chat back to the clients, the design I've gone with is a loop which takes the time and then returns each line of chat between the time taken and "now" each loop. Then it sleeps and repeats, it looks like this:
<?php
header("Content-Type: text/event-stream\n\n");
header("Cache-Control: no-cache");
require_once('dbFunctions.inc');
$l_LastTime = 0;
while ( true )
{
$l_LastTime = ListChat($l_LastTime);
sleep(1);
}
?>
The important parts for the HTML5 SSE mechanism are the two header lines, we define the header type to be "text/event-stream" and that the page is to use no cache on the client side.
Note: The two new lines "\n\n" on the Content-Type string are very important and should not be ommitted.
Again the PHP page uses the include file we've defined, which has its complete listing below, but we'll just cover the "ListChat" function now:
function ListChat ($p_LastTime)
{
$l_LastTime = $p_LastTime;
$l_Connection = ConnectDatabase();
if ( $l_Connection != null )
{
if ( $l_LastTime == 0 )
{
$l_LastTime = time();
}
$l_Time = time();
$l_sql = "SELECT * FROM GeneralChat WHERE Tme > " . $l_LastTime . " AND Tme <= " . $l_Time;
$l_result = mysql_query($l_sql, $l_Connection);
if ( $l_result )
{
$l_numRows = mysql_numrows($l_result);
if ( $l_numRows > 0 )
{
for ($l_i = 0; $l_i < $l_numRows; ++$l_i)
{
$l_LastTime = mysql_result($l_result, $l_i, "Tme");
$l_Message = mysql_result($l_result, $l_i, "Text");
sendMsg ($l_LastTIme, $l_Message);
}
}
}
CloseDatabase($l_Connection);
}
return $l_LastTime;
}
Important to the operation of this function, it takes the last time we checked for chat and returns the "now" time for the last message dispatched. This lets us then query in the center for only those lines of chat we need to send.
The flow of the function simply opens the database, builds the SQL, performs the query and if there are results dispatches each row of results (each chat string) as an individual event to the client side. Our client handles this in its "ChatArrived" function and posts each row onto the onscreen table.
We also specially make sure if the time of the last query was zero (i.e. the script just started running) we correct to "now" to prevent the whole history of the text being stremed back to the client.
The key piece of code however is this snippet:
for ($l_i = 0; $l_i < $l_numRows; ++$l_i)
{
$l_LastTime = mysql_result($l_result, $l_i, "Tme");
$l_Message = mysql_result($l_result, $l_i, "Text");
sendMsg ($l_LastTIme, $l_Message);
}
Each result takes its time, which we use as a value to send the string against, and the string, the sendMsg function then looks like this:
function sendMsg($id, $msg)
{
echo "id: $id" . PHP_EOL;
echo "data: $msg" . PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
}
Again, many of the online examples of SSE give this style of function as their example, we add the Id to the PHP echo (which is going back down the text/event-stream to the client) and the "data: " line is the line which is passed to the "ChatArrived" function as the "event" parameter.
So if we put the code:
echo "id: 1" . PHP_EOL;
echo "data: HelloWorld" . PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
Then the "ChatArrived" function would receive a parameter of "HelloWorld", not "data: " leading it!
And this folks is the whole example.... You can follow the flow now:
- chat.html : Takes the input string, queues and then dispatches the strings
- chatsubmission.php : Accepts the dispatched strings into the database
- chatstream.html : Sets up the SSE event source and registers to handle messages from chatlist.php
- chatlist.php : Sends the messages as they arrive into the database
What follows is a live stream of my performing all these steps with a Ubuntu laptop and a Ubuntu server (Recorded 1st Jan 2013):
/// chat.sql full listing ----------------------------------
CREATE DATABASE ChatSystem
CHARACTER SET latin1
COLLATE latin1_general_ci;
USE ChatSystem;
CREATE TABLE IF NOT EXISTS GeneralChat
(
Id BIGINT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (Id),
Tme BIGINT NOT NULL,
Text VARCHAR(128)
) Engine = InnoDb;
/// chat.html full listing --------------------------------
<html>
<head>
<title>Chat</title>
<script language="javascript">
var m_ChatQueue = [];
var m_ChatTimer = null;
function InsertChatString (p_ChatString)
{
m_ChatQueue.push(p_ChatString);
}
function UserSubmitChat ()
{
var l_textBox = document.getElementById ("ChatString");
if ( l_textBox )
{
InsertChatString(l_textBox.value);
l_textBox.value = "";
}
}
function HandleKeyPress (event)
{
var l_chCode = ('charCode' in event) ? event.charCode : event.keyCode;
if ( l_chCode == 13 )
{
UserSubmitChat();
}
}
function StartChatTimer ()
{
StopChatTimer ();
m_ChatTimer = setInterval(ChatTimerMatured, 500);
}
function StopChatTimer()
{
if ( m_ChatTimer != null )
{
clearInterval(m_ChatTimer);
m_ChatTimer = null;
}
}
function ChatTimerMatured ()
{
StopChatTimer ();
if ( m_ChatQueue.length > 0 )
{
while (m_ChatQueue.length > 0)
{
var l_value = m_ChatQueue.pop();
var l_URL = "chatsubmission.php?ChatString=" + l_value;
var l_HTTP = new XMLHttpRequest();
l_HTTP.open ("GET", l_URL, false);
l_HTTP.send (null);
}
}
StartChatTimer();
}
</script>
<body onload="StartChatTimer()">
<table>
<tr>
<td>
<iframe src="chatstream.html"></iframe>
</td>
</tr>
<tr>
<td>
<table width="100%" border=0 cellspacing=0 cellpadding=0>
<col width="85%">
<col width="15%">
<tr>
<td colspan=1>
<input id="ChatString"
type="text"
value=""
maxlength="48"
onkeypress="HandleKeyPress(event)">
</td>
<td colspan=1
onclick="UserSubmitChat()">
Send
</td>
</table>
</td>
</tr>
</table>
</body>
</html>
/// dbcredentials.inc -- Full Listing ----------------------
<?
$g_User = "root";
$g_Password = "********"; // Place your password here
$g_Database = "ChatSystem";
?>
/// dbfunctions.inc -- Full Listing -------------------------
<?
function ConnectDatabase ()
{
$l_Connection = null;
include ('dbCredentials.inc');
$l_Connection = mysql_connect ('localhost', $g_User, $g_Password);
if ( $l_Connection )
{
if ( !mysql_select_db($g_Database) )
{
mysql_close($l_Connection);
$l_Connection = null;
}
}
return $l_Connection;
}
function CloseDatabase ($p_Connection)
{
if ( $p_Connection != null )
{
mysql_close($p_Connection);
$p_Connection = null;
}
}
function InsertChat ($p_ChatString)
{
$l_result = false;
$l_Connection = ConnectDatabase();
if ( $l_Connection != null )
{
$l_ServerTime = time();
$l_SQL = "INSERT INTO GeneralChat VALUES (null, $l_ServerTime, '" . $p_ChatString . "')";
mysql_query($l_SQL, $l_Connection);
CloseDatabase($l_Connection);
$l_result = true;
}
return $l_result;
}
function sendMsg($id, $msg)
{
echo "id: $id" . PHP_EOL;
echo "data: $msg" . PHP_EOL;
echo PHP_EOL;
ob_flush();
flush();
}
function ListChat ($p_LastTime)
{
$l_LastTime = $p_LastTime;
$l_Connection = ConnectDatabase();
if ( $l_Connection != null )
{
if ( $l_LastTime == 0 )
{
$l_LastTime = time();
}
$l_Time = time();
$l_sql = "SELECT * FROM GeneralChat WHERE Tme > " . $l_LastTime . " AND Tme <= " . $l_Time;
$l_result = mysql_query($l_sql, $l_Connection);
if ( $l_result )
{
$l_numRows = mysql_numrows($l_result);
if ( $l_numRows > 0 )
{
for ($l_i = 0; $l_i < $l_numRows; ++$l_i)
{
$l_LastTime = mysql_result($l_result, $l_i, "Tme");
$l_Message = mysql_result($l_result, $l_i, "Text");
sendMsg ($l_LastTIme, $l_Message);
}
}
}
CloseDatabase($l_Connection);
}
return $l_LastTime;
}
?>
/// ChatStream.html --- Full Listing ------------------------
<html>
<head>
<script language="javascript">
var source = null;
var m_LastTime = 0;
var m_ChatTable = null;
function LoadPage ()
{
m_ChatTable = document.getElementById("chatTable");
source = new EventSource('chatlist.php');
source.addEventListener ("message", ChatArrived, false);
}
function InsertChat (p_ChatString)
{
if ( m_ChatTable != null )
{
var l_newRow = m_ChatTable.insertRow(m_ChatTable.rows.length);
var l_newCell = l_newRow.insertCell(0);
l_newCell.innerHTML = p_ChatString;
}
}
function ProcessChat (p_ChatString)
{
InsertChat (p_ChatString);
}
function ChatArrived (event)
{
ProcessChat (event.data);
}
</script>
</head>
<body onload="LoadPage()">
<table id="chatTable" border=1 width="100%">
<tr>
<td>Chat</td>
</tr>
</table>
</body>
</html>
