What is Q4M?

Q4M (Queue for MySQL) is a message queue that works as a pluggable storage engine of MySQL 5.1, designed to be robust, fast, flexible. The development started in late December of 2007, and although it is very primitive, operates quite swiftly.

Design Goals of Q4M

Robust
Q4M logs all message transfers to disk. A power failure or OS crash would never loose a message.
Fast
Although Q4M syncs every operation to disk, it runs fast by using group commits, and checksums for data recovery. Even with HDD write cache disabled, it is able to transfer >1,000 messages per second on an ordinally Pentium 4 linux box.
Flexible
Since Q4M works as a storage engine of MySQL, it is possible to JOIN queues against ordinally MySQL tables. Or you may use triggers for sending data into queue.

License

Q4M is provided under GNU General Public License, version 2.

Using Q4M

To evaluate Q4M, download either a binary or source distribution from the install page, and follow the installation instructions. We also have a small tutorial. You may use SQL to access Q4M queues, or there is a wrapper module available for perl (Queue::Q4M).

Limitations and Known Issues

Limitations, known issues under version 0.2.
  • row size is limited to 1GB
  • table size is limited to 2**63 bytes
  • no support for UPDATE and REPLACE
  • no support for indexes
  • removal of multiple rows from a single DELETE statement is not atomic
  • may cause server crash under extremely low memory condition
  • may lose data if OS crashes while Q4M is performing table compaction (since the directory is not fsynced)