MySQL 5.7.7 Overview and Highlights

MySQL 5.7.7 was recently released (it is the latest MySQL 5.7, and is the first “RC” or “Release Candidate” release of 5.7), and is available for download here and here.

As for the fixes/changes, there are quite a few again, which is expected in an early RC release.

The main highlights for me were (though the enhancements, and potentially impactful changes, are definitely not limited to this list):

  • Optimizer Note: It is now possible to provide hints to the optimizer within individual SQL statements, which enables finer control over statement execution plans than can be achieved using the optimizer_switch system variable. Optimizer hints are specified as /*+ … */ comments following the SELECT, INSERT, REPLACE, UPDATE, or DELETE keyword of statements or query blocks. Hints are also permitted in statements used with EXPLAIN, enabling you to see how hints affect execution plans.
  • Security Note: The C client library now attempts to establish an SSL connection by default whenever the server is enabled to support SSL. This change affects these standard MySQL client programs: mysql, mysql_config_editor, mysql_install_db, mysql_plugin, mysql_secure_installation, mysql_upgrade, mysqladmin, mysqlbinlog, mysqlcheck, mysqldump, mysqlimport, mysqlshow, and mysqlslap. It will also affect new releases of MySQL Connectors that are based on the C client library: Connector/C, Connector/C++, and Connector/ODBC.
  • Spatial Data Support: The ST_Buffer(), ST_Difference(), ST_Distance(), ST_Intersection(), ST_IsSimple(), ST_SymDifference(), and ST_Union() functions have been reimplemented to use the functionality available in Boost.Geometry. The functions may raise an exception for invalid geometry argument values when the previous implementation may not have.
  • InnoDB: The innodb_file_format default value was changed to Barracuda. The previous default value was Antelope. This change allows tables to use Compressed or Dynamic row formats.
  • InnoDB: The innodb_large_prefix default value was changed to ON. The previous default was OFF. When innodb_file_format is set to Barracuda, innodb_large_prefix=ON allows index key prefixes longer than 767 bytes (up to 3072 bytes) for tables that use a Compressed or Dynamic row format.
  • InnoDB: The innodb_strict_mode default value was changed to ON. The previous default was OFF. When innodb_strict_mode is enabled, InnoDB raises error conditions in certain cases, rather than issuing a warning and processing the specified statement (perhaps with unintended behavior).

    The configuration parameter default changes described above may affect replication and mysqldump operations. Consider the following recommendations when using the new default settings:

    • When replicating or replaying mysqldump data from older MySQL versions to MySQL 5.7.7 or higher, consider setting innodb_strict_mode to OFF to avoid errors. Target settings should not be more strict than source settings.
    • When replicating from MySQL 5.7.7 or higher to older slaves, consider setting innodb_file_format=Barracuda and innodb_large_prefix=ON on the slave so that the target and source have the same settings.
  • InnoDB: To address a scalability bottleneck for some workloads where LOCK_grant is locked in read-mode, LOCK_grant locks are now partitioned. Read lock requests on LOCK_grant now acquire one of multiple LOCK_grant partitions. Write locks must acquire all partitions. To address another scalability bottleneck, the server no longer performs unnecessary lock acquisitions when creating internal temporary tables. (Bug #72829)
  • Replication: The XA implementation in MySQL has been made much more compatible with the XA specification.
  • Replication: The defaults of some replication related variables have been modified. The following changes have been made:
    • binlog_gtid_simple_recovery=TRUE
    • binlog-format=ROW
    • binlog_error_action=ABORT_SERVER
    • sync_binlog=1
    • slave_net_timeout=60

Again, there are numerous enhancements and many bug fixes, so please check out the full changelogs. If you’re running some 5.7 version, then you should definitely upgrade. (But this should not be used for production systems yet, of course.)

You can view the full 5.7.7 changelogs here:

Hope this helps.


MySQL 5.7.6 Overview and Highlights

MySQL 5.7.6 was recently released (it is the latest MySQL 5.7, and is the “m16” or “Milestone 16” release), and is available for download here and here.

As for the fixes/changes, there are quite a few (the official release was again split into 3 separate emails), which is expected in a “milestone” release.

The main highlights for me were (though the enhancements, and potentially impactful changes, are definitely not limited to this list):

  • Incompatible Change: The CREATE USER and ALTER USER statements have additional account-management capabilities. Together, they now can be used to fully establish or modify authentication, SSL, and resource-limit properties, as well as manage password expiration and account locking and unlocking. … A new statement, SHOW CREATE USER, shows the CREATE USER statement that creates the named user. The accompanying Com_show_create_user status variable indicates how many times the statement has been executed.
  • Configuration Note: mysqld now supports a –daemonize option that causes it to run as a traditional, forking daemon. This permits the server to work with operating systems that use systemd for process control.
  • Installation Note: The mysqld server and mysql_upgrade utility have been modified to make binary (in-place) upgrades from MySQL 5.6 easier without requiring the server to be started with special options. The server checks whether the system tables are from a MySQL version older than 5.7 (that is, whether the mysql.user table has a Password column). If so, it permits connections by users who have an empty authentication plugin in their mysql.user account row, as long as they have a Password value that is empty (no password) or a valid native (41-character) password hash.
  • Performance Schema Notes: The Performance Schema now allocates memory incrementally, scaling its memory use to actual server load, instead of allocating all the memory it needs during server startup. Consequently, configuration of the Performance Schema is easier; most sizing parameters need not be set at all. A server that handles a very low load will consume less memory without requiring explicit configuration to do so.
  • Incompatible Change: A new C API function, mysql_real_escape_string_quote(), has been implemented as a replacement for mysql_real_escape_string() because the latter function can fail to properly encode characters when the NO_BACKSLASH_ESCAPES SQL mode is enabled.
  • InnoDB: InnoDB system tablespace data is now exposed in the INNODB_SYS_TABLESPACES and INNODB_SYS_DATAFILES Information Schema tables.
  • InnoDB: Numerous (7) buffer pool flushing-related enhancements were added.
  • InnoDB: The default setting for the internal_tmp_disk_storage_engine option, which defines the storage engine the server uses for on-disk internal temporary tables, is now INNODB. With this change, the Optimizer uses the InnoDB storage engine instead of MyISAM for internal temporary tables.
  • InnoDB: InnoDB now supports native partitioning.
  • InnoDB: InnoDB now supports the creation of general tablespaces using CREATE TABLESPACE syntax. Tables are added to a general tablespace using CREATE TABLE tbl_name … TABLESPACE [=] tablespace_name or ALTER TABLE tbl_name TABLESPACE [=] tablespace_name syntax.
  • InnoDB: InnoDB now supports 32KB and 64KB page sizes. For both page sizes, the maximum record size is 16KB.
  • InnoDB: Replication-related support was added to InnoDB which enables prioritization of slave applier transactions over other transactions in deadlock scenarios. This transaction prioritization mechanism is reserved for future use.
  • InnoDB: The Performance Schema now instruments stage events for monitoring InnoDB ALTER TABLE and buffer pool load operations.
  • Replication: MySQL Multi-Source Replication adds the ability to replicate from multiple masters to a slave. MySQL Multi-Source Replication topologies can be used to back up multiple servers to a single server, to merge table shards, and consolidate data from multiple servers to a single server. See MySQL Multi-Source Replication. As part of MySQL Multi-Source Replication, replication channels have been added. Replication channels enable a slave to open multiple connections to replicate from, with each channel being a connection to a master. See Replication Channels.

Again, there are numerous enhancements and hundreds of bug fixes, so please check out the full changelogs. If you’re running some 5.7 version, then you should definitely upgrade. (But this should not be used for production systems yet, of course.)

You can view the full 5.7.6 changelogs here:

Hope this helps.


MySQL 5.7.5 Overview and Highlights

MySQL 5.7.5 was recently released (it is the latest MySQL 5.7, and is the “m15” or “Milestone 15” release), and is available for download here and here.

As for the fixes/changes, there are quite a few (the official release was split into 3 separate emails), which is expected in such an early milestone release.

The main highlights for me were (though the enhancements, and potentially impactful changes, are definitely not limited to this list):

  • InnoDB: The innodb_buffer_pool_size parameter is now dynamic, allowing you to resize the buffer pool without restarting the server. The resizing operation, which involves moving pages to a new location in memory, is performed chunks. Chunk size is configurable using the new innodb_buffer_pool_chunk_size configuration option. You can monitor resizing progress using the new Innodb_buffer_pool_resize_status status variable. For more information, see Resizing the InnoDB Buffer Pool Online.
  • Replication: When replicating from a master running a version earlier than MySQL 5.6.0 [Read “5.5” or “5.1”] to a slave running MySQL 5.6.0 or later, the slave requires the master_uuid value, which is the server_uuid value from the master. The master_uuid value is unsupported on the older master, and in such a replication situation could become invalid on the newer slave. A check for empty master_uuid now ensures that the slave uses an empty value for master_uuid. (Bug #18338203)
  • Incompatible Change: mysql_install_db has been rewritten from Perl into C++. This enables it to be provided as an executable binary and eliminates its dependency on having Perl installed.
  • MySQL builds on Windows using Visual Studio now require Visual Studio 2013 or later. The previous requirement was Visual Studio 2010 or later. (Bug #18404381)
  • Now, MYSQL_MAINTAINER_MODE is on by default when compiling debug builds with GCC, and MYSQL_MAINTAINER_MODE enbles -Werror regardless of whether GCC or Clang is used.
  • MySQL now includes DTrace support on Oracle Linux 6 or higher with UEK kernel. If DTrace is present, server builds will detect it with no special CMake options required.
  • Incompatible Change: A new log record type (MLOG_FILE_NAME) is used to identify file-per-table tablespaces that have been modified since the last checkpoint. This enhancement simplifies tablespace discovery during crash recovery and eliminates scans on the file system prior to redo log application. For more information about the benefits of this enhancement, see Tablespace Discovery During Crash Recovery. This enhancement changes the redo log format, requiring that MySQL be shut down cleanly before upgrading to or downgrading from MySQL 5.7.5.
  • Incompatible Change: The InnoDB storage engine can no longer be disabled. The –skip-innodb option is deprecated and has no effect, and its use results in a warning. It will be removed in a future MySQL release. This also applies to its synonyms (–innodb=OFF, –disable-innodb, and so forth). A new innodb_lock_no_retry flag for the –debug option is now available.
  • Incompatible Change: The Performance Schema now provides a user_variables_by_thread table that exposes user-defined variables. For more information, see Performance Schema Connection Attribute Tables. In consequence of this change, the server now limits user-defined variable names to a maximum of 64 characters, the length of the VARIABLE_NAME column in the table. Previously, the server did not enforce a limit.
  • The optimizer computes more accurate costs for semi-join materialization. (Bug #18558561)
  • To generate execution plans, the optimizer uses a cost model that is based on estimates of the cost of various operations that occur during query execution. The optimizer has a set of compiled-in default “cost constants” available to it to make decisions regarding execution plans. The optimizer now has in addition a database of cost estimates to use during execution plan construction. These estimates are stored in the server_cost and engine_cost tables in the mysql system database and are configurable at any time: Any non-NULL cost estimate stored in the cost model tables overrides the corresponding compiled-in default estimate. Any NULL estimate indicates to the optimizer to use the compiled-in default. Implementation and testing is ongoing to make it safe for DBAs to change these values. Currently, changing them should be considered at your own risk. If you upgrade to this release of MySQL from an earlier version, you must run mysql_upgrade (and restart the server) to incorporate these changes into the mysql database.
  • The optimizer now uses more exact index statistics. Currently, the improved values are used by InnoDB, with these effects: 1) In many cases, better execution plans result for queries for which previously a less optimal join index or table join order was chosen. 2) The row estimates in EXPLAIN output are more accurate, as well as the filter values in some cases. 3) Cardinality estimates in the index statistics displayed by SHOW INDEX are more accurate for InnoDB tables.
  • During query execution plan construction, the optimizer now uses condition filtering to make better use of all conditions on a table in determining the estimate of qualifying rows that will be joined to the next table. For example, even though there might be an index that can be used to select rows, there might also be additional conditions in the WHERE clause that can further restrict the estimate for qualifying rows. Use of additional conditions is controlled by the condition_fanout_filter flag of the optimizer_switch system variable. This flag is on by default but can be disabled to suppress use of condition filtering (for example, for a query that is found to perform better without it).
  • Security Note: Incompatible Change: MySQL 5.6 deprecated passwords that used the older pre-4.1 password hashing format. Support for these passwords is now removed, which involves the following changes. Applications that use any feature no longer supported must be modified. The mysql_old_password authentication plugin is removed. The –secure-auth option to the server and client programs is the default, but is now a no-op. It is deprecated and will be removed in a future MySQL release. The –skip-secure-auth option to the server and client programs is no longer supported and using it produces an error.
  • Incompatible Change: Strict SQL mode for transactional storage engines (STRICT_TRANS_TABLES) is now enabled by default.
  • InnoDB: SPATIAL indexes can now be used for InnoDB tables. InnoDB supports indexing of spatial data types, including use of ALTER TABLE … ALGORITHM=INPLACE for online operations (ADD SPATIAL INDEX). To support transaction isolation properties, InnoDB uses predicate locking. A predicate lock locks the minimum bounding rectangle (MBR) used for a query so that other transactions cannot insert or modify a row that would match the query condition.
  • Incompatible Change: Previously, mysql_upgrade performed an upgrade by invoking the mysql and mysqlcheck clients. mysql_upgrade has been reimplemented to generate the required SQL statements itself and execute them by communicating directly with server.
  • Incompatible Change: In MySQL 5.6.6, the YEAR(2) data type was deprecated. Support for YEAR(2) has now been removed. Once you upgrade to MySQL 5.7.5 or newer, any remaining YEAR(2) columns must be converted to YEAR(4) to become usable again. For conversion strategies, see YEAR(2) Limitations and Migrating to YEAR(4). For example, run mysql_upgrade after upgrading.
  • Incompatible Change: The GET_LOCK() has been reimplemented using the metadata locking (MDL) subsystem and its capabilities have been extended.
  • InnoDB: For optimal shutdown and recovery performance, shutdown and recovery phases are now supported by the multi-threaded page cleaner feature (innodb_page_cleaners) that was introduced in MySQL 5.7.4. (Bug #18805275)
  • InnoDB: Instead of inserting one index record at a time, InnoDB now performs a bulk load when creating or rebuilding indexes. This method of index creation is also known as a “sorted index build”. This enhancement, which improves the efficiency of index creation, also applies to full-text indexes.
  • InnoDB: InnoDB memory allocations now are instrumented for the Performance Schema and will appear in the memory summary tables.
  • InnoDB: You can now truncate undo logs that reside in undo tablespaces. This feature is enabled using the innodb_undo_log_truncate configuration option. For more information, see Truncating Undo Logs That Reside in Undo Tablespaces.
  • InnoDB: Work was done to introduce the notion of attachable transactions in InnoDB (for AutoCommit / ReadOnly / ReadCommitted / NonLocking transactions). This is used to read from InnoDB Data Dictionary tables. Along with this, attachable transactions were exposed to the server. Data Dictionary access code will use them to read Data Dictionary data.
  • Replication: Retrying of transactions is now supported when multi-threading is enabled on a slave. In previous versions, slave_transaction_retries was treated as equal to 0 when using multi-threaded slaves. (Bug #16390504, Bug #68465)
  • Replication: Global transaction identifiers (GTIDs) are now logged in a MySQL system table whenever they are enabled on the server, which lifts a previous requirement to use binary logging when replicating with GTIDs. If binary logging is disabled, the server stores the GTID for each transaction in the mysql.gtid_executed table as the transaction is executed. If binary logging is enabled, then, whenever the binary log is rotated or the server is shut down, the server also writes into the new binary log the GTIDs for all transactions from the previous binary log.
  • Replication: The new variable simplified_binlog_gtid_recovery can be used to change the way binary log files are searched for previous GTIDs during recovery, speeding up the process when a large number of binary log files exist. (Bug #69097, Bug #16741603, Bug #74071, Bug #19686914)
  • Replication: Multi-threaded slaves can use the new slave_preserve_commit_order variable to ensure that the order which transactions were committed on the master is preserved on the slave. This prevents the slave from entering a state that the master was not in and is well suited to using multi-threaded slaves for replication read scale-out.
  • Replication: The new options binlog_group_commit_sync_delay and binlog_group_commit_sync_no_delay_count provide a way to configure the synchronization of the binary log. This enables more transactions to be synchronized together to disk at once, reducing the overall time to commit a group of transactions because the larger groups require fewer time units per group.
  • Replication: To make monitoring of a replication setup easier, various replication related variables have been moved to the performance_schema tables. This is particularly helpful for monitoring multi-source replication.
  • The mysqladmin flush-logs command now permits optional log types to be given, to specify which logs to flush. Following the flush-logs command, you can provide a space-separated list of one or more of the following log types: binary, engine, error, general, relay, slow. These correspond to the log types that can be specified for the FLUSH LOGS SQL statement. Thanks to Daniël van Eeden for the patch. (Bug #60878, Bug #12368203)
  • Scalability for InnoDB tables was improved by avoiding THR_LOCK locks. As a result of this change, DML statements for InnoDB tables that previously waited for a THR_LOCK lock will wait for a metadata lock. (Bug #42147, Bug #11751331)
  • The Boost.Geometry library now is required to build MySQL.

And that pretty much just covers the highlights of the “Functionality Added or Changed” section. I’m not even getting into the “Bugs Fixed” section, of which there were 296 (many InnoDB & Replication)! So there has been a lot going on in this release. If you’re running some 5.7 version, then you should definitely upgrade. (But this should not be used for production systems yet, of course.)

You can view the full 5.7.5 changelogs here:

Hope this helps.


MySQL 5.7.4 Overview and Highlights

MySQL 5.7.4 was recently released (it is the latest MySQL 5.7, and is the “m14” or “Milestone 14” release), and is available for download here and here.

The 5.7.4 changelog begins with the following, so I felt it appropriate to include it here as well.

In Memoriam:

“This release is dedicated to the memory of two young engineers of the MySQL Engineering family, Astha and Akhila, whom we lost while they were in their early twenties. This is a small remembrance and a way to recognize your contribution to the 5.7 release. You will be missed.”

As for the fixes, there are quite a few, which is to be expected in such an early milestone release.

The main highlights for me were:

  1. The Performance Schema now instruments prepared statements (for both the binary and text protocols). Info is available in the prepared_statements_instances table, along with performance_schema_max_prepared_statements_instances system variable, and Performance_schema_prepared_statements_lost status variable.
  2. Incompatible Change: MySQL deployments installed using RPM packages now are secure by default (single root account, ‘root’@’localhost’, no anonymous-user accounts, no test database).
  3. Incompatible Change: MySQL now enables database administrators to establish a policy for automatic password expiration: Any user who connects to the server using an account for which the password is past its permitted lifetime must change the password.
  4. Performance; InnoDB: InnoDB now supports multiple page_cleaner threads for flushing dirty pages from buffer pool instances. A new system variable, innodb_page_cleaners, is used to specify the number of page_cleaner threads.
  5. Incompatible Change: The AES_ENCRYPT() and AES_DECRYPT() functions now permit control of the block encryption mode and take an optional initialization vector argument
  6. InnoDB: InnoDB now supports the Transportable Tablespace feature for partitioned InnoDB tables and individual InnoDB table partitions. This enhancement eases backup procedures for partitioned tables and enables copying of partitioned tables and individual table partitions between MySQL instances.

Of course, there were many, many more fixes/updates (InnoDB being #1, Replication #2, and Partitioning #3 with most fixed bugs), so be sure to read through the full changelog. And if you are running a previous version of *5.7*, then definitely plan on upgrading to this latest 5.7.4.

Hope this helps.


5.7 Upgrade and Resolving ERROR 1130 Host ‘localhost’ is Not Allowed to Connect

I recently upgraded an instance to 5.7.3 the other day, and ran into an error, so I wanted to share the resolution for it here.

In my case, I was upgrading 5.7.1 to 5.7.3. However, this will apply to anyone wanting to upgrade from pre-5.7.2 (including 5.6/5.5) to 5.7.2+.

I performed the upgrade, in-place, and restarted mysqld. This was fine. However, then I attempted to connect via the command-line, and received the following error:

shell> mysql -uroot -ppass -P3310
ERROR 1130 (HY000): Host 'localhost' is not allowed 
to connect to this MySQL server

Searching the net, you’ll mostly find RTM replies, which were all accurate as far as I could tell. In all of those prior reported cases, the issues were expected behavior and the issues were ultimately user error.

Of course I double-checked my config and data files. I knew I didn’t change anything in the user table, or any system table, for that matter. And I only upgraded from 5.7.1, which was a new instance at the time (i.e., the data and tables were not from a previous version).

I then ran mysql_upgrade thinking that surely would fix it. However, my initial mysql_upgrade attempt failed:

shell>mysql_upgrade -uroot -ppass -P3310
Looking for 'mysql.exe' as: C:\MySQL Server 5.7\bin\mysql.exe
Looking for 'mysqlcheck.exe' as: C:\MySQL Server 5.7\bin\mysqlcheck.exe
FATAL ERROR: Upgrade failed

I tried a couple more things, just ot make sure I wasn’t crazy, and then decided to checkout the changelogs (I know, I should have done this *beforehand* anyway). There wasn’t any mention of this in the 5.7.3 changelog, but aha!, there it was in the 5.7.2 changelog.

The full change entry is a bit long for me to post in full (which is a great thing – the detail is most appreciated), but I’ll post the most relevant part here:

“Incompatible Change: Previously, account rows in the mysql.user table could have an empty plugin column value. In this case, the server authenticated such an account using either the mysql_native_password or mysql_old_password plugin, depending on whether the password hash value in the Password column used native hashing or the older pre-4.1 hashing method. With the deprecation of old-format password hashes in MySQL 5.6.5, this heuristic for deciding which authentication plugin to use is unnecessary and it is desirable that user table rows always specify explicitly which authentication plugin applies.

To that end, the plugin column is now defined to be non-NULL with a default value of ‘mysql_native_password’, and associated server operations require the column to be nonempty. In conjunction with this plugin column definition modification, several other changes have been made…”

The page goes on to say that mysql_upgrade does fix this issue. However, you must start mysqld with the –skip-grant-tables. Now, mysql will start without the privilege tables being used, and thus you can connect with a client and you can run mysql_upgrade.

Note your server is unprotected while –skip-grant-tables is enabled, so you should also run it with the –skip-networking option, so outside connections cannot connect, and also disable any local apps that may attempt to access while you perform the upgrade.

As 5.7 becomes used more and more, and more versions are released over time, I suspect this will become a much more popular error, and there will be many looking for the fix.

Hope this helps.


Installing MySQL 5.7.1 (Milestone Release) on Windows 7

I wanted install MySQL 5.7.1 (1st Milestone Release) on Windows 7 and test it out a bit, so I did, and since things didn’t go as smooth as expected, I thought I’d share my experience, in case anyone else runs into the same issues.

I downloaded the .msi (mysql-5.7.1-m11-winx64.msi) from (then click the “Developmental Releases” tab) and installed it following the prompts. That seemed to complete fine, which was great. However, that was it – and not in a good way. I mean, the “installer” basically only unpacked the files to a location.

I was expecting the “configuration” tool to run, but it didn’t. It was not installed, and not an option.

I quickly read through the 5.7.1 changelog and found the config tool is not part of the .msi anymore. You must now use the “MySQL Installer” if you want to configure the MySQL instance.

So, I then downloaded the MySQL Installer (

I installed it, which went fine. I then ran it. It would not let me go straight to the “configuration” of my 5.7.1 instance. It made me start from the “beginning”, so-to-speak. It forced me to re-download 5.7.1. Then forced me to re-install 5.7.1 (via the same .msi installer I already downloaded and installed). Then, finally, I made to where I had the option to run the configuration tool. \o/

The configuration tool worked mostly (it created the service and started it), though it failed to create my password and complete the security settings, so I had some manual fixes to perform afterward.

After that, things were good to go. I realize this is just the first milestone release of 5.7 and the first release where the “config” tool is not “included” with the .msi installer, and so there are going to be some kinks to work out. But if you’re wanting to install 5.7.1 on Windows, I hope this helps. And I’d probably just go straight to the “MySQL Installer”, unless you want the no-install version.

For the sake of completeness, I did file the bugs I ran across here:

“Win MySQL Installer makes you re-download and re-install 5.7 before configuring”

“MySQL Win Installer tries to Install 5.5.28 and 5.1.66 when I install 5.7.1 only”

“MySQL Installer failed on User password & Security Settings for 5.7.1 on Win 7”

Hope this helps. 🙂