Category: MySQL
Why you should NOT be using mysqli::prepare
If you're writing PHP code that works with a MySQL database, there's a good chance you'll have heard about the spiffy "prepared queries" functionality available with the mysqli library. "It's totally awesome!" some people will say – it makes queries easier to read in code, it handles all of the escaping for you (yay for easy security!), and it's way more efficient if you're running the same query a bunch of times.
Here's an example of how you might use the prepared query functionality in PHP (straight from the docs!):
if ($stmt = $mysqli->prepare("SELECT District FROM City WHERE Name=?")) { $stmt->bind_param("s", $city); $stmt->execute(); $stmt->bind_result($district); $stmt->fetch(); printf("%s is in district %s\n", $city, $district); $stmt->close(); }
It's a pretty reasonable-looking way of building a query. Here's how you would have to build that query normally:
$mysqli->query("SELECT District FROM City WHERE Name='" . $mysqli->real_escape_string($city) . "'");
Once you want to start using a lot of random static strings in your query, it can get pretty messy – having to concatenate all of those strings together, calling that escape function everywhere – using the bind_param function makes things a lot more readable.
IF YOU'RE AN IDIOT!
No wait, hear me out, I didn't mean it like that – what I meant to say is, the prepared query syntax is probably doing more work than you expect it to.
See, prepared statements are actually a SQL thing, not something that the PHP devs added to the mysqli wrapper just because they thought it would be cool. You can use prepared statements from the MySQL command line, too:
PREPARE my_statement FROM 'SELECT District FROM City WHERE Name=?'; SET @city := 'Amersfoort'; EXECUTE my_statement USING @city; DEALLOCATE PREPARE my_statement;
mysqli::prepare is just giving you access to this nifty feature, which is fine and dandy. However, you may have noticed something -
That's a lot of queries to run, just to run a single SELECT query!
Now, if you're going to be running a ton of INSERT or UPDATE queries, and you just want to swap the variables out each time, you're fine – letting MySQL hang on to the meat of a query you're going to be running a hundred times is downright reasonable.
BUT – if all you're doing is running a half dozen queries while you're building a web page to display to the user, the only thing you're doing is increasing the number of times you have to contact the server.
How many superfluous database-contacts will there be? As far as I can tell, you'll be contacting the server three times more than you need to:
- Once to prepare the statement
- Once to send the parameters to the server and get the results back
- Once to deallocate the statement
(I haven't looked at the source code, but I'm guessing that commenter Angus M is correct in his assessment that binding a parameter does not cause communication with the database.)
If your database server is running on the same host as your web server, the cost of contacting the database is probably pretty low. Still, after years of developing thinnish-desktop clients, my instinct is to avoid that overhead wherever possible.
Stuck between a rock and an inefficient place
I imagine that, among the people who took those last few paragraphs seriously, there are two camps – the coders who will say "well, whatever" and keep writing queries using the prepare syntax because it's more convenient and easier to read (which is an argument I can understand), and the efficiency advocates who will give a tortured sigh and go back to writing queries with lots of awkward concatenation, in order to avoid overhead.
Or you could write/copy some simple query-building tools that do the stuff that you originally thought mysqli::prepare was doing!
Here's a function I threw together last night, which I imagine would suit the needs of many developers using PHP/MySQL:
<?php function GetQueryWithData() { $Query = ""; $ParameterNumber = 0; if (func_num_args() && $Query = func_get_arg($ParameterNumber++)) { while ($ParameterNumber < func_num_args()) { $NextParameter = func_get_arg($ParameterNumber++); $PlaceToInsertParameter = strpos($Query, '?'); if ($PlaceToInsertParameter !== false) { $QuerySafeString = ''; if (is_bool($NextParameter)) { $QuerySafeString = $NextParameter ? 'TRUE' : 'FALSE'; } else if (is_float($NextParameter) || is_int($NextParameter)) { $QuerySafeString = $NextParameter; } else if (is_null($NextParameter)) { $QuerySafeString = 'NULL'; } else { $QuerySafeString = "'" . mysql_escape_string($NextParameter) . "'"; } $Query = substr_replace($Query, $QuerySafeString, $PlaceToInsertParameter, 1); } } } return $Query; } print GetQueryWithData("SELECT * FROM `ass` WHERE `butt` = ? AND `cheek_id` = ? AND ? " . "AND `nullable_field` IS ? ", "lol 'WUT'", 13, true, null) . "\n"; /* SELECT * FROM `ass` WHERE `butt` = 'lol \'WUT\'' AND `cheek_id` = 13 AND TRUE AND `nullable_field` IS NULL */ print GetQueryWithData('UPDATE `some_table` SET `some_column` = ?, `some_other_column` = ?, ' . '`some_id` = ? WHERE `yourface` = ?', '20', 21, 69, 'sucks') . "\n"; /* UPDATE `some_table` SET `some_column` = '20', `some_other_column` = 21, `some_id` = 69 WHERE `yourface` = 'sucks' */ ?>
The above function is a simple way to build a query with any number of parameters. Strings will be escaped, numbers will not be quoted, and the function never contacts the database server.
Using this function, you can redux the query from the original example while only contacting the database once:
$mysqli->query(GetQueryWithData("SELECT District FROM City WHERE Name=?", $city));
If you don't want to use that function, or roll your own query-builder, you can keep using mysqli's wrapper for prepared statements – just know that it was made for a different purpose, and it comes with a bit more cost.
Edit: PDO!
After some discussion with ss23 in #mysql, I thought I'd better add some comments about the PDO class.
On the surface (and according to much of the documentation), it appears to be a more generic (not just MySQL-specific) database wrapper with similar prepared-statement support. However, it turns out that by default, it only fakes prepared statements!
So, it is possible to use PDO to write queries using a bind-parameter syntax that can make your code a lot easier to read, and simultaneously avoid contacting the database too often. However, if that is how you decide to roll, I would recommend turning on fake-prepared-statements mode explicitly, just in case that default gets changed in the future:
$pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES, 1);
MySQL wrapper class (PHP)
I'm not saying that this is the best implementation of a MySQL wrapper class for PHP. I'm not advocating its use in production code.
However, this is what I've used in most scripts I've thrown together in the last few years – I hate having to throw a mysql_error call after every query, y'know?
<?php define('CLIENT_LONG_PASSWORD', 1); class MySQLWrapper { var $server_port = 'localhost:3306'; var $database = 'whatever'; var $username = 'a username'; var $password = 'a password'; var $force_long_password = false; var $db_object; function __construct($server_port = "", $database = "", $username = "", $password = "", $force_long_password = false) { $this->server_port = $server_port; $this->database = $database; $this->username = $username; $this->password = $password; $this->force_long_password = $force_long_password; if ($this->server_port != "") { $this->Connect(); } } function Connect() { if (isset($this->db_object)) { if (!mysql_ping($this->db_object)) { unset($this->db_object); $this->Connect(); } } else { $db_object = mysql_connect($this->server_port, $this->username, $this->password, false, $this->force_long_password ? CLIENT_LONG_PASSWORD : 0); if ($this->database != '' && !mysql_select_db($this->database)) { $result = mysql_error(); print "Select database failed for $this->database: $result\n"; } } } function Disconnect() { $return = true; if (isset($this->db_object)) { $return = mysql_close($this->db_object); unset($this->db_object); } return $return; } function Execute($query) { $this->Connect(); $result = mysql_query($query); if (!$result) { $result = mysql_error(); print "Query failed: $result\n"; print "Teh query itself: $query\n"; } return $result; } } ?>
You may find this slightly better than using PHP's MySQL functions straight out of the box. Or not. Who can say?
Run a query for every table in a database
Ever wished you could run a dangerous query like DROP TABLE or TRUNCATE against every table in a database in a single query?
No?
Well, how about something like CHECK or REPAIR table, then?
If you ever find yourself in the rare situation where you need to run the same query across every table in a database, this procedure might make your life easier:
DELIMITER $$ CREATE PROCEDURE `p_run_for_each_table`(IN strDatabase TEXT, IN strOperation TEXT) DETERMINISTIC BEGIN DECLARE strQuery TEXT; DECLARE strTable VARCHAR(255); DECLARE bDone INT DEFAULT 0; DECLARE TableCursor CURSOR FOR SELECT `TABLE_NAME` FROM `information_schema`.`TABLES` WHERE `TABLE_SCHEMA` = strDatabase AND TABLE_TYPE = 'BASE TABLE'; DECLARE CONTINUE HANDLER FOR NOT FOUND SET bDone = 1; OPEN TableCursor; REPEAT FETCH TableCursor INTO strTable; SET @QueryThatWasPassedIn := REPLACE( REPLACE(strOperation, '{?database}', strDatabase) , '{?table}', strTable); PREPARE Statement FROM @QueryThatWasPassedIn; EXECUTE Statement; UNTIL bDone END REPEAT; CLOSE TableCursor; DEALLOCATE PREPARE Statement; END$$ DELIMITER ;
Useage
The procedure takes two parameters. The first one is the name of the database whose tables you want to run the query for.
The second is the query you would like to run. The strings "{?database}" and "{?table}" will be replaced with the database and table names.
CALL p_run_for_each_table('databasename', 'SELECT * FROM {?database}.{?table}');
What queries you can run
You should be able to use any queries that can be run in a prepared statement – you can find the list about two-thirds of the way down this page.
As of MySQL 5.1, you can run these queries: "ALTER TABLE, CALL, COMMIT, CREATE INDEX, CREATE TABLE, DELETE, DO, DROP INDEX, DROP TABLE, INSERT, RENAME TABLE, REPLACE, SELECT, SET, UPDATE, and most SHOW statements."
Logical errors in queries: DO NOT WANT
During my career developing database-driven software (teehee, I'm a professional) I've noted that the most horrific query errors are the logical ones – queries that parse correctly, and return reasonable-looking data, but make wrong assumptions about how different parts of the query relate to each other.
One particular error that I've seen time and time again (even from people who have been writing queries for a while) can occur when summarizing data from multiple tables that have a one-to-many relationship.
…In other words, it could occur in queries written for most database-driven software.
Solution: fix the problem by writing about it!
I wrote a page documenting the cause of the logical error, doing my best to warn people against letting it slip into their own code.
I attempted to write it so it would be easy to read, possibly even entertaining (a lofty goal for a manual on writing database queries, perhaps) – there is some colorful language and plenty of juvenile humor mixed with the tech-speak.
The doc itself is part of the wiki of the company where I work. I don't write a ton of documentation for our developers or customers (certainly not as much as I should), but whenever I do, I get this awesome feeling of usefulness. Oh, and pride. Sometimes, I feel so proud, that I feel compelled to link other people (who have no relationship to my company) to what I wrote! Ridiculous, I know.
Remember: if you write queries, it is your responsibility to guarantee that they return true and accurate data!
Convert blocks of text to sentence case
You know what I hate? Paragraphs of capital (or all lowercase) letters.
The other day a coworker was looking to beautify a large quantity of data spread across some MySQL tables. I created this function to make his life easier:
DELIMITER $$ CREATE FUNCTION `f_sentence_case`(strInput TEXT, nMinimumLength INT) RETURNS TEXT DETERMINISTIC BEGIN DECLARE result TEXT; DECLARE LastSpace INT; DECLARE NextSpace INT; DECLARE NextSlash INT; DECLARE Word TEXT; DECLARE NewSentence INT; DECLARE PreviousCharacter CHAR(1); DECLARE TrimmedWord TEXT; DECLARE NumberOfSpaces INT; DECLARE Swap INT; SET strInput := CONCAT(strInput, ' '); SET result := ''; SET LastSpace := 1; SET NextSpace := LOCATE(' ', strInput, LastSpace + 1); SET NextSlash := LOCATE('/', strInput, LastSpace + 1); SET NextSpace := IF(LEAST(NextSlash, NextSpace) = 0, GREATEST(NextSlash, NextSpace), LEAST(NextSlash, NextSpace)); label1: WHILE NextSpace DO SET Word := SUBSTR(strInput, LastSpace, NextSpace - LastSpace); SET PreviousCharacter := SUBSTR(strInput, LastSpace - 1, 1); SET NewSentence := LastSpace = 1 OR (NewSentence AND PreviousCharacter = ' ') OR PreviousCharacter IN('.', '!', '?'); SET TrimmedWord := LTRIM(Word); SET NumberOfSpaces := LENGTH(Word) - LENGTH(TrimmedWord); SET Word := TrimmedWord; # Make it lowercase if it is all uppercase SET Word := IF(LENGTH(Word) >= nMinimumLength AND Word NOT REGEXP '[0-9]', IF(NewSentence, CONCAT(UCASE(SUBSTR(Word, 1, 1)), LCASE(SUBSTR(Word, 2, LENGTH(Word) - 1))), LCASE(Word) ), Word); SET result := CONCAT(result, REPEAT(' ', NumberOfSpaces), Word); SET Swap := LastSpace; SET LastSpace := NextSpace; SET NextSpace := LOCATE(' ', strInput, Swap + 1); SET NextSlash := LOCATE('/', strInput, Swap + 1); SET NextSpace := IF(LEAST(NextSlash, NextSpace) = 0, GREATEST(NextSlash, NextSpace), LEAST(NextSlash, NextSpace)); END WHILE label1; RETURN result; END$$ DELIMITER ;
What it does
It seems to perform generally as I hoped it would; which is to say that it formats text to be sentence case.
More specifically, it alters all the "words" (a set of non-numeric, non-whitespace characters) that are longer than the specified minimum length.
It changes the words to be all lowercase, unless they happen to be the first word after a punctuation mark (in which case the first character of the word is made uppercase).
Useage
To clean up a field so that it is formatted in sentence case (ignoring all words with less than 3 characters), simply run this query:
UPDATE `table` SET `field` = f_sentence_case(`field`, 3);
Other than that, my only specs for the query were for it to be functional and hopefully not break my brain when I went back to read it later. If anyone has any significant improvements to it, let me know!
My accomplishment for the day: a MySQL quine!
Some background: a "quine" is a program that outputs it's complete source code when run.
Some more background: I work at a company where I work with MySQL (a database engine) very often.
I was lounging around in the MySQL chat room on irc.freenode.net, and someone suggested that I try to write a MySQL quine – a database query that would return the text of the query itself.
After about half an hour of screwing around, I got it! It may not be a fantastic achievement, but I feel pretty leet about writing my first quine in a database query language.
SELECT REPLACE(@v:='SELECT REPLACE(@v:=\'2\',1+1,REPLACE(REPLACE(@v,\'\\\\\',\'\\\\\\\\\'),\'\\\'\',\'\\\\\\\'\'));',1+1,REPLACE(REPLACE(@v,'\\','\\\\'),'\'','\\\''));
Only 167 characters. Hah!