I recently wrote an article on dealing with an assertion failure in log/log0recv.c, specifically !page || (ibool)!!page_is_comp(page) == dict_table_is_comp(index->table).
I mention it because this occurred after a system outage, and I just encountered another system outage (either HDD power outage or some other serious HDD event), with a completely different assertion failure and error message. Similar to the one above, it’s also kind of obscure, so I wanted to post this so people searching for it will find this.
For reference, the first outage assertion failure was this:
InnoDB: Assertion failure in thread 139838283589376 in file log/log0recv.c line 1094 InnoDB: Failing assertion: !page || (ibool)!!page_is_comp(page) == dict_table_is_comp(index->table)
Here is the new assertion failure:
111201 16:45:00 InnoDB: Assertion failure in thread 4500 in file fut0lst.ic line 83 InnoDB: Failing assertion: addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA
As you can see, they are completely different, yet both due to similar, catastrophic events:
In fact, there are a number of relevant/useful snippets from this error log (in case you see one or more of these):
InnoDB: Starting crash recovery. InnoDB: Reading tablespace information from the .ibd files... InnoDB: Restoring possible half-written data pages from the doublewrite InnoDB: buffer... InnoDB: Warning: a page in the doublewrite buffer is not within space InnoDB: bounds; space id 0 page number 2248, page 22 in doublewrite buf. InnoDB: Warning: a page in the doublewrite buffer is not within space InnoDB: bounds; space id 0 page number 1775, page 31 in doublewrite buf. ... 111201 16:45:00 InnoDB: ERROR: We were only able to scan the log up to InnoDB: 670800896, but a checkpoint was at 670801309. InnoDB: It is possible that the database is now corrupt! 111201 16:45:00 InnoDB: Error: page 7 log sequence number 924466223 InnoDB: is in the future! Current system log sequence number 670801309. ... 111201 16:45:00 InnoDB: Assertion failure in thread 4500 in file fut0lst.ic line 83 InnoDB: Failing assertion: addr.page == FIL_NULL || addr.boffset >= FIL_PAGE_DATA ... 111201 16:45:00 - mysqld got exception 0xc0000005 ; ... 000000013F6A4C6F mysqld.exe!my_osmaperr() 000000013F69B9A0 mysqld.exe!my_osmaperr() 000000013F4727D6 mysqld.exe!?ha_initialize_handlerton@@ YAHPEAUst_plugin_int@@@Z() 000000013F46C172 mysqld.exe!?plugin_lock_by_name@@ YAPEAUst_plugin_int@@PEAVTHD@@PEBUst_mysql_lex_string@@H@Z() 000000013F4713E9 mysqld.exe!?plugin_init@@YAHPEAHPEAPEADH@Z() 000000013F45BDA7 mysqld.exe!handle_shutdown() 000000013F45C91A mysqld.exe!?win_main@@YAHHPEAPEAD@Z() 000000013F45CD90 mysqld.exe!?mysql_service@@YAHPEAX@Z() 000000013F45D0A3 mysqld.exe!?mysqld_main@@YAHHPEAPEAD@Z() 000000013F7FBB27 mysqld.exe!my_mb_ctype_mb() 00000000777F652D kernel32.dll!BaseThreadInitThunk() 0000000077A2C521 ntdll.dll!RtlUserThreadStart()
As you can see, the corruption was severe … so severe that InnoDB wouldn’t even start with innodb_force_recovery set to 6!
As discussed in the previous post, setting innodb_doublewrite=1, innodb_flush_log_at_trx_commit=1, sync_binlog=1, and having a battery backed cache can be the best bets against such issues.
However, I will say that in this case, innodb_flush_log_at_trx_commit=1 and innodb_doublewrite=1 were both set, and binary logging had been disabled, leaving only the battery backed cache to be desired.
Luckily, this was a test environment (and recovery from a backup was possible). Otherwise, using the InnoDB Recovery Tool would be necessary, which can be an undertaking.
Morals of the story: take regular backups (as always) and invest in a battery backed cache. 🙂