Thursday, April 18, 2013

JavaScript: Password Validation using regular expressions and HTML5

. Guidelines for Secure Password Input

Use the "password" input type

Instead of <input type="text"> you should always use <input type="password"> as this lets the browser (and the user) know that the contents of that field need to be secured. Your password won't appear on the screen as you type and most browsers also won't 'remember' the values entered in password fields as they do with other form elements.

Confirm password input

Because the password input type obscures the text typed, you should let the user confirm that they haven't made a mistake. The simplest way to do this is to have the password entered twice, and then check that they are identical.
Another method is to display what they've entered as part of a 'confirmation page'. The problem here is that you're making the password visible in the browser, browser cache, proxy, etc. For security a password should never be displayed in HTML or even sent by email.

Enforce 'strong' passwords

If you're concerned about security you should have some policy on what constitutes a valid password. Some common restrictions are:
  • at least n characters
  • combination of upper- and lower-case characters
  • one or more digits
  • not related to other user data (name, address, username, ...)
  • not a dictionary word
Leaving the last requirement for now, as it requires a server-side script, let's see what's possible using just client-side HTML and JavaScript.

Server security

While having a strong password is a good first step, it needs to be backed up by additional measures on the server that prevent brute-force attacks. One popular approach is to install Fail2Ban to monitor log files and lock out repeat offendors. Of course that only works if your login system reports failed login attempts to a system log file. Otherwise your application needs to provide this function.

2. Basic Demonstration

The form below has three input fields: username, pwd1 and pwd2. When the form is submitted the checkForm script parses the input values and returns either true or false. If a false value is returned then the form submission is cancelled.
This code will work for browsers as far back as Netscape 4 (circa 1997).
Change Password
The code behind the form is as follows. If you're not sure how to place this on your page, you might need to read the preceding article on Form Validation, or view the HTML source of this page.
<script type="text/javascript"> function checkForm(form) { if(form.username.value == "") { alert("Error: Username cannot be blank!"); form.username.focus(); return false; } re = /^\w+$/; if(!re.test(form.username.value)) { alert("Error: Username must contain only letters, numbers and underscores!"); form.username.focus(); return false; } if(form.pwd1.value != "" && form.pwd1.value == form.pwd2.value) { if(form.pwd1.value.length < 6) { alert("Error: Password must contain at least six characters!"); form.pwd1.focus(); return false; } if(form.pwd1.value == form.username.value) { alert("Error: Password must be different from Username!"); form.pwd1.focus(); return false; } re = /[0-9]/; if(!re.test(form.pwd1.value)) { alert("Error: password must contain at least one number (0-9)!"); form.pwd1.focus(); return false; } re = /[a-z]/; if(!re.test(form.pwd1.value)) { alert("Error: password must contain at least one lowercase letter (a-z)!"); form.pwd1.focus(); return false; } re = /[A-Z]/; if(!re.test(form.pwd1.value)) { alert("Error: password must contain at least one uppercase letter (A-Z)!"); form.pwd1.focus(); return false; } } else { alert("Error: Please check that you've entered and confirmed your password!"); form.pwd1.focus(); return false; } alert("You entered a valid password: " + form.pwd1.value); return true; } </script>
expand code box
Remember that, as JavaScript isn't available in all browsers, you should user server-side scripting to validate all data before recording it in a database or text file. You might also want to spice up your forms using HTML5 Form Validation as we've done further down the page.

3. Advanced regular expressions

In the latest browsers - those that support JavaScript 1.5 (Firefox, Netscape 6/7, Mozilla, Safari and Opera 7) - you can use more powerful regular expressions. Older browsers will not recognise these patterns so the following is mostly useful for intranet rather than Internet applications.
The code presented above is fine in that it checks everything that we wanted to check, but uses a lot of code to test each requirement individually and present different error messages. We're going to show you now how to apply the password tests using a single regular expression.
Consider the following:
<script type="text/javascript"> // at least one number, one lowercase and one uppercase letter // at least six characters var re = /(?=.*\d)(?=.*[a-z])(?=.*[A-Z]).{6,}/; var validPassword = re.test(input); </script> The type of expression used here is called a 'look-ahead' which tries to match the contained regexp against the 'future' part of the string.
Translation:
  • matches a string of six or more characters;
  • that contains at least one digit (\d is shorthand for [0-9]);
  • at least one uppercase character; and
  • at least one lowercase character:
inputresult of testreason
abcABCfalseno numbers
abc123falseno uppercase characters
abAB1falsetoo short
abAB12true-
Aa123456true-
If you are using a supported browser you can use the form below to test the regular expression:
Password Regexp Test
(input must contain at least one digit/lowercase/uppercase letter and be at least six characters long)
If you want to restrict the password to ONLY letters and numbers (no spaces or other characters) then only a slight change is required. Instead of using . (the wildcard) we use \w:
<script type="text/javascript"> // at least one number, one lowercase and one uppercase letter // at least six characters that are letters, numbers or the underscore var re = /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}$/; var validPassword = re.test(input); </script> The \w is shorthand for 'any letter, number or the underscore character'.
Again, you can use the form below to test this regular expression:
Password Regexp Test 2
(as above, but this time ONLY letters and numbers are allowed)

4. Sample HTML and JavaScript code

You might implement this code on your own website as follows:
<script type="text/javascript"> function checkPassword(str) { var re = /^(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}$/; return re.test(str); } function checkForm(form) { if(form.username.value == "") { alert("Error: Username cannot be blank!"); form.username.focus(); return false; } re = /^\w+$/; if(!re.test(form.username.value)) { alert("Error: Username must contain only letters, numbers and underscores!"); form.username.focus(); return false; } if(form.pwd1.value != "" && form.pwd1.value == form.pwd2.value) { if(!checkPassword(form.pwd1.value)) { alert("The password you have entered is not valid!"); form.pwd1.focus(); return false; } } else { alert("Error: Please check that you've entered and confirmed your password!"); form.pwd1.focus(); return false; } return true; } </script> <form method="POST" action="form-handler.php" onsubmit="return checkForm(this);"> <p>Username: <input type="text" name="username"></p> <p>Password: <input type="password" name="pwd1"></p> <p>Confirm Password: <input type="password" name="pwd2"></p> <p><input type="submit"></p> </form> As you can see, it's well worth learning the intricacies of regular expressions. They can be used not just in JavaScript, but also PHP, Perl, Java and many other languages. Some text editors (not just vi) also allow them when searching for or replacing text.

5. HTML5 Form Validation

We earlier mentioned HTML5 form validation. This is a new technique available in modern browsers and definitely the way of the future. A few simple form attributes can have the same effect as reams of JavaScript code libraries.
Here we have an enhanced version of the above code where we've added HTML5 required and pattern elements to apply regular expression tests within the form itself in supporting browsers. Helpfully the regular expression syntax is identical with just the /^ and $/ removed.
We've also added a tricky little onchange handler to the first password field which updates the pattern required by the second password field - in effect forcing them to be identical:
Change Password
Here you can see a screen shot from Safari of the form being completed. The red/green markers have been implemented using CSS:
In this example it should be clear to the user that the form can only be submitted once all three green ticks appear. In any case browsers such as Firefox and Opera will enforce the HTML5 validation rules and present messages as shown here:
Presumably the browser messages will change according to the users language - something that would never be possible using only JavaScript.
All we have changed from the previous example is to add some extra attributes to the form input fields. The rest of the HTML and JavaScript remains unaltered:
... <p>Username: <input type="text" required pattern="\w+" name="username"></p> <p>Password: <input type="password" required pattern="(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}" name="pwd1" onchange="form.pwd2.pattern = this.value;"></p> <p>Confirm Password: <input type="password" required pattern="(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}" name="pwd2"></p> ... The best thing about HTML5 attributes is that they have no effect whatsoever on unsupported browsers, so Internet Explorer will act as if they are not present and simply run the JavaScript as before.
At this stage both Firefox and Opera enforce HTML5 validation attributes in the browser while Safari only lets you use them in combination with CSS effects.

6. Customised HTML5 browser alerts

As you can see from the screenshot above the alert message in Firefox for when the input doesn't match the pattern attribute is simply "Please match the requested format.". Not entirely helpful in this case where we have a number of different requirements.
Fortunately it is possible to customise this message using just a touch of JavaScript.
Change Password
The only change between this and the previous example is that we've modified the onchange handler for the Password input and added one for Confirm Password:
... <p>Password: <input type="password" required pattern="(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}" name="pwd1" onchange=" this.setCustomValidity(this.validity.patternMismatch ? 'Password must contain at least 6 characters, including UPPER/lowercase and numbers' : ''); if(this.checkValidity()) form.pwd2.pattern = this.value; "></p> <p>Confirm Password: <input type="password" required pattern="(?=.*\d)(?=.*[a-z])(?=.*[A-Z])\w{6,}" name="pwd2" onchange=" this.setCustomValidity(this.validity.patternMismatch ? 'Please enter the same Password as above' : ''); "></p> ... When the Password input is changed we check its validity.patternMismatch flag to see whether it matches the pattern attribute. If it doesn't match we set a custom error message to appear when the form is submitted.
Whenever setCustomValidity() has been used to set a custom message the field in question will be regarded as invalid and prevent the form from submitting (at least in Firefox and Opera). To reverse this it needs to be set to blank, which we do when the input matches our regex rules.
The custom message we have set appears in Firefox as shown here:
You can see from the code that we have applied a similar technique to the Confirm Password field so it will now display "Please enter the same Password as above" if there is a mismatch between the two fields. Basically what we've achieved here is to replicate our JavaScript validation script using HTML5.
Because we are only checking for patternMismatch we are not affecting the other default validation options - namely the required attribute - so submitting the form with blank fields will still display the generic "Please fill out this field" alert message. To override those errors you would need to check the validity.valueMissing flag.

7. Conclusions

As you can see there's a lot involved in providing a rich user experience even for something as simple as changing a password. To summarise what we've covered:
  • Always start by emailing new users a random password or unique activation link;
  • Use the password input type to prevent passwords appearing on screen;
  • Decide what constitutes a 'strong' password for your system and enforce it:
    • server-side for spambots and users with JavaScript disabled;
    • using JavaScript for browsers that don't support HTML5 validation;
    • using HTML5 for a more user-friendly experience;
  • Use CSS rules to highlight valid/invalid input for browsers that don't have alerts built-in; and
  • Customise the HTML5 error messages where appropriate for improved usability;
Most of all don't feel you have to install massive JavaScript or jQuery libraries just to validate a form. By taking advantage of new browser standards as show in this article you can save time and resources and at the same time provide a better user experience.

Passing variables to JavaScript usin php

 

 

 http://blog.outsourcing-partners.com/wp-content/uploads/2012/11/javascript.jpg

 

1. Escaping Quotes and Line Breaks

Suppose you start with a PHP variable that needs to be displayed in a JavaScript alert or confirmation dialog:
$message = "a short piece of text spanning more than one line and containing \"double\" & 'single' quotes"; The obvious way to turn this into an alert is:
<script type="text/javascript"> alert("<?PHP echo $message ?>"); </script> but that results in invalid JavaScript code: <script type="text/javascript"> alert("a short piece of text spanning more than one line and containing "double" & 'single' quotes"); </script>
There are two problems. Firstly, having line breaks inside a JavaScript variable is not an option. They'll need to be replaced with \n (character representation of a line break). Secondly, the double quotes inside the text conflict with the outer quotes of the alert call.
Learning from our mistakes, here's some code that actually works:
<script type="text/javascript"> alert("<?php echo preg_replace("/\r?\n/", "\\n", addslashes($message)); ?>"); </script> The above code relies on PHP being embedded within HTML code. If you're outputting the entire page from PHP then you need to go a step further:
$message = preg_replace("/\r?\n/", "\\n", addslashes($message)); echo "<script type=\"text/javascript\">\n"; echo " alert(\"$message\");\n"; echo "</script>\n\n"; The output is now:
<script type="text/javascript"> alert("a short piece of text\nspanning more than one line\nand containing \"double\" & \'single\' quotes"); </script> The same approach can be applied in pretty much any situation where PHP needs to pass variables to JavaScript.
When trying to debug this kind of code you should always check first what's valid in JavaScript (most modern browsers have a built-in debugger for this) and only then see how it can be generated from PHP.
Note: It's not necessary to encode characters such as & and " to their HTML entities as JavaScript is a separate language from HTML and can handle them without problem.

Exporting Data to Excel usin php

1. Preparing the data

The following examples use the dataset created for Sorting Arrays of Arrays which is defined as follows:
<?PHP $data = array( array("firstname" => "Mary", "lastname" => "Johnson", "age" => 25), array("firstname" => "Amanda", "lastname" => "Miller", "age" => 18), array("firstname" => "James", "lastname" => "Brown", "age" => 31), array("firstname" => "Patricia", "lastname" => "Williams", "age" => 7), array("firstname" => "Michael", "lastname" => "Davis", "age" => 43), array("firstname" => "Sarah", "lastname" => "Miller", "age" => 24), array("firstname" => "Patrick", "lastname" => "Miller", "age" => 27) ); ?> Note: Further down this page you can find an example on creating an export from an SQL query.
The first step is to output the data in a tab-delimited format (CSV can also be used but is slightly more complicated). To achieve this we use the following code:
<?PHP header("Content-Type: text/plain"); $flag = false; foreach($data as $row) { if(!$flag) { // display field/column names as first row echo implode("\t", array_keys($row)) . "\r\n"; $flag = true; } echo implode("\t", array_values($row)) . "\r\n"; } exit; ?> We set the content type to text/plain so that the output can more easily be viewed in the browser. Otherwise, because there is no HTML formatting, the output would appear as a single line of text.
The first line of output will be the column headings (in this case the field names are used). Values are separated with a tab \t and rows with a line break \n. The output should look something like the following:
firstname lastname age Mary Johnson 25 Amanda Miller 18 James Brown 31 Patricia Williams 7 Michael Davis 43 Sarah Miller 24 Patrick Miller 27 There's already a weakness in this code that may not be immediately obvious. What if one of the fields to be ouput already contains one or more tab characters, or worse, a newline? That's going to throw the whole process out as we rely on those characters to indicate column- and line-breaks.
The solution is to 'escape' the tab characters. In this case we're going to replace tabs with a literal \t and line breaks with a literal \n so they don't affect the formatting:
<?PHP function cleanData(&$str) { $str = preg_replace("/\t/", "\\t", $str); $str = preg_replace("/\r?\n/", "\\n", $str); } header("Content-Type: text/plain"); $flag = false; foreach($data as $row) { if(!$flag) { // display field/column names as first row echo implode("\t", array_keys($row)) . "\r\n"; $flag = true; } array_walk($row, 'cleanData'); echo implode("\t", array_values($row)) . "\r\n"; } exit; ?> Now, before each row is echoed any tab characters are replaced "\t" so that our columns aren't broken up. Also any line breaks within the data are replaced with "\n". Now, how to set this up as a download...

2. Triggering a download

What many programmers don't realise is that you don't have to create a file, even a temporary one, in order for one to be downloaded. It's sufficient to 'mimic' a download by passing the equivalent HTTP headers followed by the data.
If we create a PHP file with the following code then when it's called a file will be downloaded which can be opened directly using Excel.
<?PHP function cleanData(&$str) { $str = preg_replace("/\t/", "\\t", $str); $str = preg_replace("/\r?\n/", "\\n", $str); if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"'; } // filename for download $filename = "website_data_" . date('Ymd') . ".xls"; header("Content-Disposition: attachment; filename=\"$filename\""); header("Content-Type: application/vnd.ms-excel"); $flag = false; foreach($data as $row) { if(!$flag) { // display field/column names as first row echo implode("\t", array_keys($row)) . "\r\n"; $flag = true; } array_walk($row, 'cleanData'); echo implode("\t", array_values($row)) . "\r\n"; } exit; ?> Note that we've added an extra line to the cleanData function to detect double-quotes and escape any value that contains them. Without this an uneven number of quotes in a string can confuse Excel.

This should result in a file being downloaded and saved to your computer. If all goes well then the filename will be named "website_data_20130419.xls" and will open in Excel looking something like this:
screenshot showing data in Excel colums
How does it work? Setting the headers tells the browser to expect a file with a given name and type. The data is then echoed, but instead of appearing on the page it becomes the downloaded file.
Because of the .xls extension and the ms-excel file type, most computers will associate it with Excel and double-clicking will cause that program to open. You could also modify the file name and mime type to indicate a different spreadsheet package or database application.
There is no way to specify data or cell formatting, column widths, etc, using this method. We are only passing a tab-delimited text file. To include formatting try generating HTML code or a script that actually builds an Excel file. Or create your own macro in Excel that applies formatting after the import.
A similar technique can be used to allow users to download files that have been uploaded previously using PHP and stored with different names. More on that later...

3. Exporting from an SQL database

If your goal is to allow data to be exported from a query result then the changes are relatively simple:
<?PHP // Original PHP code by Chirp Internet: www.chirp.com.au // Please acknowledge use of this code by including this header. function cleanData(&$str) { $str = preg_replace("/\t/", "\\t", $str); $str = preg_replace("/\r?\n/", "\\n", $str); if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"'; } // filename for download $filename = "website_data_" . date('Ymd') . ".xls"; header("Content-Disposition: attachment; filename=\"$filename\""); header("Content-Type: application/vnd.ms-excel"); $flag = false; $result = pg_query("SELECT * FROM table ORDER BY field") or die('Query failed!'); while(false !== ($row = pg_fetch_assoc($result))) { if(!$flag) { // display field/column names as first row echo implode("\t", array_keys($row)) . "\r\n"; $flag = true; } array_walk($row, 'cleanData'); echo implode("\t", array_values($row)) . "\r\n"; } exit; ?> This would be the entire file required to query the database and trigger the file download. The database functions need to match the database you're using. MySQL users for example will need to use mysql_query and either mysql_fetch_assoc or mysqli_fetch_assoc in place of the PostgreSQL functions.
For other databases see under User Comments below or check the PHP documentation. If you are seeing duplicate columns (numbered as well as labeled) you need to change the fetch call to return only the associative (ASSOC) array.
If you're having trouble at this stage, remove the Content-Disposition header and change the Content-Type back to text/plain. This makes debugging a lot easier as you can see the output in your browser rather than having to download and open the generated file every time you edit the script.

4. Preventing Excel's ridiculous auto-format

When importing from a text file as we're essentially doing here, Excel has a nasty habit of mangling dates, timestamps, phone numbers and similar input values.
For our purposes, some simple additions to the cleanData function take care of most of the problems:
<?PHP // Original PHP code by Chirp Internet: www.chirp.com.au // Please acknowledge use of this code by including this header. function cleanData(&$str) { // escape tab characters $str = preg_replace("/\t/", "\\t", $str); // escape new lines $str = preg_replace("/\r?\n/", "\\n", $str); // convert 't' and 'f' to boolean values if($str == 't') $str = 'TRUE'; if($str == 'f') $str = 'FALSE'; // force certain number/date formats to be imported as strings if(preg_match("/^0/", $str) || preg_match("/^\+?\d{8,}$/", $str) || preg_match("/^\d{4}.\d{1,2}.\d{1,2}/", $str)) { $str = "'$str"; } // escape fields that include double quotes if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"'; } ?> The section that prevents values being scrambled does so by inserting an apostrophe at the start of the cell. When you open the resuling file in Excel you may see the apostrophe, but editing the field will make it disappear while retaining the string format. Excel is strange that way.
The types of values being escape this way are: values starting with a zero; values starting with an optional + and at least 8 consecutive digits (phone numbers); and values starting with numbers in YYYY-MM-DD format (timestamps). The relevant regular expressions have been highlighted in the code above.

5. Formatting - colours, numbers, dates

Again, this script generates and triggers the download of a tab-delimited text file with an .xls extension and Excel Mime-Type. We are not building an actual Excel document.
Defining styles, colours, column widths, etc, is not possible using this technique. You may be able to generate an HTML table with some formatted data that Excel will recognise, otherwise you need a much more complicated script.

6. Exporting to CSV format

The tab-delimited text options describe above may be a bit limiting if your data contains newlines or tab breaks that you want to preserve when opened in Excel or another spreadsheet application.
A better format then is comma-separated variables (CSV) which can be generated as follows:
<?PHP function cleanData(&$str) { if($str == 't') $str = 'TRUE'; if($str == 'f') $str = 'FALSE'; if(preg_match("/^0/", $str) || preg_match("/^\+?\d{8,}$/", $str) || preg_match("/^\d{4}.\d{1,2}.\d{1,2}/", $str)) { $str = "'$str"; } if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"'; } // filename for download $filename = "website_data_" . date('Ymd') . ".csv"; header("Content-Disposition: attachment; filename=\"$filename\""); header("Content-Type: text/csv;"); $out = fopen("php://output", 'w'); $flag = false; $result = pg_query("SELECT * FROM table ORDER BY field") or die('Query failed!'); while(false !== ($row = pg_fetch_assoc($result))) { if(!$flag) { // display field/column names as first row fputcsv($out, array_keys($row), ',', '"'); $flag = true; } array_walk($row, 'cleanData'); fputcsv($out, array_values($row), ',', '"'); } fclose($out); exit; ?> Normally the fputcsv command is used to write data in CSV format to a separate file. In this script we're tricking it into writing directly to the page by telling it to write to php://output instead of a regular file.

7. Exporting to CSV with Unicode intact

If like us your data contains UTF-8 characters you will notice that Excel doesn't handle them very well. Other applications can open UTF-8 content without problems, but for some reason Microsoft wants to keep you in the dark ages.
Fortunately, there is a trick you can use. Below you can see how we modify the script to convert everything from UTF-8 to UTF-16 Lower Endian (UTF-16LE) format which Excel, at least on Windows, will recognise.
Please Note: When you open the file in Excel you might find all the data bunched into the first column. This should be fixable using the "Text to Columns..." command in the Data menu.
<?PHP function cleanData(&$str) { if($str == 't') $str = 'TRUE'; if($str == 'f') $str = 'FALSE'; if(preg_match("/^0/", $str) || preg_match("/^\+?\d{8,}$/", $str) || preg_match("/^\d{4}.\d{1,2}.\d{1,2}/", $str)) { $str = "'$str"; } if(strstr($str, '"')) $str = '"' . str_replace('"', '""', $str) . '"'; $str = mb_convert_encoding($str, 'UTF-16LE', 'UTF-8'); } // filename for download $filename = "website_data_" . date('Ymd') . ".csv"; header("Content-Disposition: attachment; filename=\"$filename\""); header("Content-Type: text/csv; charset=UTF-16LE"); $out = fopen("php://output", 'w'); $flag = false; $result = pg_query("SELECT * FROM table ORDER BY field") or die('Query failed!'); while(false !== ($row = pg_fetch_assoc($result))) { if(!$flag) { // display field/column names as first row fputcsv($out, array_keys($row), ',', '"'); $flag = true; } array_walk($row, 'cleanData'); fputcsv($out, array_values($row), ',', '"'); } fclose($out); exit; ?> This script may not work for all versions of Excel. Please let us know using the Feedback form below if you encounter problems or come up with a better solution.

8. Specifying column headings

The above database download examples all use the database field names for the first row of the exported file which may not be what you want. If you want to specify your own more user-friendly headings you can modify the code as follow:
<PHP $colnames = array( 'memberno' => "Member No.", 'date_joined' => "Date joined", 'title' => "Title", 'firstname' => "First name", 'lastname' => "Last name", 'address' => "Address", 'postcode' => "Postcode", 'city' => "City", 'country' => "Country", 'phone' => "Telephone", 'mobile' => "Mobile", 'fax' => "Facsimile", 'email' => "Email address", 'notes' => "Notes" ); function map_colnames($input) { global $colnames; return isset($colnames[$input]) ? $colnames[$input] : $input; } // filename for download $filename = "website_data_" . date('Ymd') . ".csv"; ... if(!$flag) { // display field/column names as first row $firstline = array_map("map_colnames", array_keys($row)); fputcsv($outstream, $firstline, ',', '"'); $flag = true; } ... ?> The values in the first row will be mapped according to the $colnames associative array. If no mapping is provided for a fieldname it will remain unchanged. Of course you will want to provide your own list of suitable headings. They don't have to be in order.