Hello. My name is Justin Swanhart and I am now the ‘MySQL Czar’ at Kickfire. In my previous position as the senior consultant in our ‘Technical Services’ team I worked directly with our initial customers and beta engagements. I hand delivered an appliance to one of our customers in Texas and have regularly consulted internally with our engineering and product groups about MySQL in general. In my new role I will be focused on examining all facets of the appliance from a MySQL transparency perspective.
As part of this new role, I will be posting to this blog on MySQL-related topics. As a first post, I thought I would give a high-level overview of our appliance and how it interacts with MySQL.
As you may know, Kickfire is focused on providing high-performance reporting and data warehousing capabilities for MySQL. The Kickfire MySQL Appliance utilizes specialized hardware (our SQL Chip) in combination with our high-performance vertical storage engine. Both of these advanced technologies are leveraged by our query optimizer, which automatically replaces the built-in optimizer when queries are run against tables created with our storage engine. We deliver an enhanced version of MySQL 5.1 which includes these technologies ready to go out of the box. This combination of hardware and software allows us to execute queries in parallel on compressed data, often without intermediate data decompression. This increases query performance and reduces resource consumption. It allows us to execute highly complex queries on vast amounts of data with near-linear scaling as the volume of data increases – up to 3TB at the highest end of the current generation Kickfire MySQL appliances. A Kickfire MySQL Appliance consumes up to 3RU of rack space and typically uses less power than a 600 watt college dormitory room microwave. In other words, as far as databases go, the Kickfire is as green as you can get, even at large data volumes.
As I’ve said, the Kickfire MySQL Appliance is more than merely a storage engine. Because Kickfire implements its own query optimizer and execution engine, there are some areas in which functionality differs somewhat from the Sun MySQL enterprise distribution. This is due largely in part to the idiosyncratic behavior of MySQL in certain functional areas. In other cases, Kickfire has chosen to approach problems in a different way than Sun/MySQL AB. This is usually to utilize the specialized hardware on which the appliance runs. For example, MySQL 5.1 has new partitioning options that were not available in prior releases of MySQL. KFDB does not support these table level partitioning features because internally the appliance performs a form of partitioning transparently. That said, compatibility and interoperability are very important to Kickfire and those areas that differ from regular MySQL are documented in our release notes. If you attempt to use an unsupported feature an error is raised, and an informational message about the nature of the problem is returned as a warning.
The Kickfire MySQL Appliance “accelerates” queries that are directed to MySQL when all of the underlying tables used in the query are backed by our KFDB storage engine. The Kickfire OS brokers communications between MySQL and the SQL Chip to deliver the query results. Accelerated queries are processed by the Kickfire Query Processing module (QPM). Query results are transparently passed back to the client through the MySQL protocol, meaning the client is unaware that anything “out of the ordinary” is happening during its request. If the appliance can’t accelerate the query for any reason, an error message and descriptive warnings are returned. We believe this approach is preferable to executing the query un-accelerated without any feedback.
Other storage engines will execute a query even if it cannot be optimized, which may result in performance problems without any explanation of the underlying cause. Kickfire also provides a “compatibility mode” which will allow you to execute an incompatible query on the Kickfire MySQL Appliance, but in general we suggest rewriting such queries to take advantage of our acceleration. We call this execution method “MySQL path” as opposed to Kickfire or “accelerated path”, and it can be enabled with a session variable. These queries will not perform as well as accelerated queries. When possible, rewrite such queries so that they may be accelerated.
We are constantly improving the amount of SQL syntax we can process internally. In fact, we support many native MySQL functions, far more than any other storage engine in our class. We also support complex expressions as operands to those functions so functional composition and calculation of formulas such as “SUM(price * qty)” is possible on Kickfire. Kickfire is also committed to expanding the number of functions we support and our service pack releases have consistently included new functions which our current customers have requested.
Returning to my new role, my focus is on working with our engineering and product groups to ensure we provide the absolute highest level of compatibility and transparency possible with “regular” MySQL going into the future. As I’ve said, there are certain aspects of the appliance which differ from regular MySQL behavior. We’ve documented those differences and suggested workarounds in our manual. We are committed to removing many of those differences in the future to ensure the highest level of interoperability with existing applications as possible. You can consider me the Kickfire ambassador to the MySQL community and I’ll be happy to answer your questions, and to help you understand the appliance better.
Sun/MySQL is currently encouraging new application development of data warehouse and data mart applications on MySQL 5.1 If you are interested in Kickfire, please fill out a technical request form here: http://www.kickfire.com/content/view/17/21/
