What is Q4M?

Q4M (Queue for MySQL) is a message queue licensed under GPL that works as a pluggable storage engine of MySQL 5.1, designed to be robust, fast, flexible. The development started in late December of 2007, and although it is very primitive, operates quite swiftly.
To start using Q4M, download either a binary or source distribution from the install page, and follow the installation instructions. A small tutorial is also avialable. You may use SQL to access Q4M queues, or there is a wrapper module available for perl (Queue::Q4M).
For more information, please read the developer's weblog (Kazuho@Cybozu Labs) or subscribe to the mailing list.

What's New

0.7 release (Jul 01 2008)
optimize SELECT count(*), update example, drop binlog compatibility flags
0.6 release (May 30 2008)
performance tuning and bug fixes
0.5.1 release (May 20 2008)
fix potential bugs that might block table compaction, or return of an empty result set when data exists
0.5 release (May 03 2008)
fix compile error with mysql-5.1.24-rc, improve examples, etc.
0.4 release (Apr 07 2008)
added support for conditional subscription, modified row_id generation scheme. Note that message relays used by 0.3 forwarders should be reset using the -v option.
0.3 release (Mar 18 2008)
added support for message relaying between different MySQL instances, prioritized subscription to multiple tables

Design Goals of Q4M

Robust
Q4M logs all message transfers to disk. A power failure or OS crash would never loose a message.
Fast
Although Q4M syncs every operation to disk, it runs fast by using group commits, and checksums for data recovery. Even with HDD write cache disabled, it is able to transfer thousands of messages per second on an ordinal linux server.
Flexible
Since Q4M works as a storage engine of MySQL, ordinally SQL can be used for administration. It is also possible to JOIN Q4M queues against other MySQL tables. Or you may use triggers for sending data into queue.

Limitations and Known Issues

Limitations, known issues under version 0.5.
  • row size is limited to 512MB
  • table size is limited to 2**63 bytes
  • no support for UPDATE and REPLACE
  • no support for indexes
  • removal of multiple rows from a single DELETE statement is not atomic
  • may cause server crash under extremely low memory condition
  • may lose data if OS crashes while Q4M is performing table compaction (since the directory is not fsynced)