InnoDB does the following check for sequence table during check
table command:
- There should be only one index should exist on sequence table
- There should be only one row should exist on sequence table
- The leaf page must be the root page for the sequence table
- Delete marked record should not exist
- DB_TRX_ID and DB_ROLL_PTR of the record should be 0 and 1U << 55
To check the rows, the table needs to be opened. To that end, and like
MDEV-36038, we force COPY algorithm on ALTER TABLE ... SEQUENCE=1.
This also results in checking the sequence state / metadata.
The table structure was already validated before this patch.
(cherry picked from commit 6f8ef26885)
Adding a new column in INFORMATION_SCHEMA.COLLATIONS:
PAD_ATTRIBUTE ENUM('PAD SPACE','NO PAD')
and a new column Pad_attribute into SHOW COLLATION.
The new column has been added after SORTLEN but before COMMENT.
This order is compatible with MySQL-8.0 order,
with the exception that MariaDB has an extra last column COMMENT:
MariaDB [test]> desc information_schema.collations;
+--------------------+----------------------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------------+----------------------------+------+-----+---------+-------+
| COLLATION_NAME | varchar(64) | NO | | NULL | |
| CHARACTER_SET_NAME | varchar(32) | YES | | NULL | |
| ID | bigint(11) | YES | | NULL | |
| IS_DEFAULT | varchar(3) | YES | | NULL | |
| IS_COMPILED | varchar(3) | NO | | NULL | |
| SORTLEN | bigint(3) | NO | | NULL | |
| PAD_ATTRIBUTE | enum('PAD SPACE','NO PAD') | NO | | NULL | |
| COMMENT | varchar(80) | NO | | NULL | |
+--------------------+----------------------------+------+-----+---------+-------+
The new Pad_attribute in SHOW COLLATION has been added as the last column.
This is also compatible with MySQL:
MariaDB [test]> show collation like 'utf8mb4_bin';
+-------------+---------+------+---------+----------+---------+---------------+
| Collation | Charset | Id | Default | Compiled | Sortlen | Pad_attribute |
+-------------+---------+------+---------+----------+---------+---------------+
| utf8mb4_bin | utf8mb4 | 46 | | Yes | 1 | PAD SPACE |
+-------------+---------+------+---------+----------+---------+---------------+
In commit b6923420f3 (MDEV-29445)
some hash tables were accidentally created with the minimum size
(101 entries) instead of correctly deriving the size from the
initial innodb_buffer_pool_size. This led to very long hash bucket
chains, which are very slow to traverse.
ut_find_prime(): Assert that the size is nonzero in order to catch
this type of regression in the future.
innodb_init_params(): Do not bother reading buf_pool.curr_size()
when it is known to be 0,
srv_start(): Correctly initialize srv_lock_table_size to 5 times
buf_pool.curr_size(), that is, the buffer pool size in pages,
between invoking buf_pool.create() and lock_sys.create().
btr_search_enable(), dict_sys_t::create(), dict_sys_t::resize():
Correctly refer to buf_pool.curr_pool_size(), that is,
innodb_buffer_pool_size in bytes, when calculating the hash table size.
In MDEV-29445 the expressions buf_pool_get_curr_size() were
accidentally replaced with buf_pool.curr_size().
buf_buddy_shrink(): Properly cover the case when KEY_BLOCK_SIZE
corresponds to the innodb_page_size, that is, the ROW_FORMAT=COMPRESSED
page frame is directly allocated from the buffer pool, not via the
binary buddy allocator.
buf_LRU_check_size_of_non_data_objects(): Avoid a crash when the
buffer pool is being shrunk.
buf_pool_t::shrink(): Abort if over 95% of the shrunk buffer pool
would be occupied by the adaptive hash index or record locks.
In commit b6923420f3 (MDEV-29445)
we started to specify the MAP_POPULATE flag for allocating the
InnoDB buffer pool. This would cause a lot of time to be spent
on __mm_populate() inside the Linux kernel, such as 16 seconds
to pre-fault or commit innodb_buffer_pool_size=64G.
Let us revert to the previous way of allocating the buffer pool
at startup. Note: An attempt to increase the buffer pool size by
SET GLOBAL innodb_buffer_pool_size (up to innodb_buffer_pool_size_max)
will invoke my_virtual_mem_commit(), which will use MAP_POPULATE
to zero-fill and prefault the requested additional memory area, blocking
buf_pool.mutex.
Before MDEV-29445 we allocated the InnoDB buffer pool by invoking
mmap(2) once (via my_large_malloc()). After the change, we would
invoke mmap(2) twice, first via my_virtual_mem_reserve() and then
via my_virtual_mem_commit(). Outside Microsoft Windows, we are
reverting back to my_large_malloc() like allocation.
my_virtual_mem_reserve(): Define only for Microsoft Windows.
Other platforms should invoke my_large_virtual_alloc() and
update_malloc_size() instead of my_virtual_mem_reserve() and
my_virtual_mem_commit().
my_large_virtual_alloc(): Define only outside Microsoft Windows.
Do not specify MAP_NORESERVE nor MAP_POPULATE, to preserve compatibility
with my_large_malloc(). Were MAP_POPULATE specified, the mmap()
system call would be significantly slower, for example 18 seconds
to reserve 64 GiB upfront.
log_t::append_prepare_wait(): Do not attempt to read log_sys.write_lsn
because it is not protected by log_sys.latch but by write_lock, which
we cannot hold here. The assertion could fail if log_t::write_buf()
is executing concurrently, and it has not yet executed log_write_buf()
or updated log_sys.write_lsn.
Fixes up commit acd071f599 (MDEV-21923)
There were two issues with the test:
1. A race between a race_condition.inc and status variable, where the
status variable Rpl_semi_sync_master_status could be ON before the
semi-sync connection finished establishing, resulting in
Rpl_semi_sync_master_clients showing 0 (instead of 1). To fix this,
we simply instead wait for Rpl_semi_sync_master_clients to be 1
before proceeding.
2. Another race between a race_condition.inc and status variable,
where the wait_condition waited on a process_list command of
'BINLOG DUMP' to disappear to infer the binlog dump thread was
killed, to where we then verified semi-sync state was correct
using status variables. However, the 'BINLOG DUMP' command is
overridden with a killed status before the semi-sync tear-down
happens, and thereby we could see invalid values. The fix for
this is to change the wait_condition to instead wait for the
connection with the replication user is gone, because that stays
through the binlog dump thread tear-down life-cycle
Appliers need to verify foreign key constraints during normal
operation, in multi-active topologies, and for this reason appliers
are configured to enable FK checking.
However, during node joining, in IST and latter catch up period,
the node is still idle (from local connections), and only source
for incoming transactions is the cluster sending certified write
sets for applying. IST happens with parallel applying, and there
is a possibility that foreign key check cause lock conflicts between
appliers accessing FK child and parent tables. Also, the excessive
FK checking will slow down IST process somewhat.
For this reasons, we could relax FK checks for appliers during IST
and catch up periods. The relaxed FK check mode should, however, be
configurable e.g. by wsrep_mode flag: SKIP_APPLIER_FK_CHECKS_IN_IST.
When this operation mode is set, and the node is processing IST or
catch up, appliers should skip FK checking.
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
The patch for SYS_REFCURSOR (MDEV-20034) overrode these methods:
- Item_func_case_searched::check_arguments()
- Item_func_if::check_arguments()
to validate WHEN-style arguments (e.g. args[0] in case of IF) for being
able to return a boolean result.
However, this unintentionally removed the test for the THEN-style arguments
that they are not expressions of the ROW data type.
This led to a crash inside Type_handler_hybrid_field_type::aggregate_for_result
on a DBUG_ASSERT that arguments are not of the ROW data type.
Fix:
The fix restores blocking ROW expressions in the not supported cases,
to avoid the DBUG_ASSERT and to raise an SQL error instead.
Blocking ROW_RESULT expressions is done per Item_func_case_expression
descendant individually, instead of blocking any ROW_RESULT arguments
at the Item_func_case_expression level.
The fix is done taking into account the upcoming patch for associative arrays
(MDEV-34319). It should be possible to pass associative array expressions into
some hybrid type functions, where ROW type expressions are not possible.
As a side effect, some lecagy ER_OPERAND_COLUMNS changed to
a newer ER_ILLEGAL_PARAMETER_DATA_TYPE_FOR_OPERATION
Changes in the top affected class Item_func_case_expression:
- item_func.h:
Overriding Item_func_case_expression::check_arguments() to return false,
without checking any arguments. Descendant validate arguments
in a various different ways. No needs to block all non-scalar data type at
this level, to prevent disallowing associative arrays.
Changes in descendants:
- item_cmpfunc.cc:
Adding a test in Item_func_case_simple::aggregate_switch_and_when_arguments()
preventing passing ROW_RESULT expression in predicant and WHEN in a
simple CASE:
CASE predicant WHEN when1 THEN .. WHEN when2 THEN .. END;
This is not supported yet. Should be preferrably fixed before MDEV-34319.
- item_cmpfunc.cc:
Calling args[0]->type_handler()->Item_hybrid_func_fix_attributes()
from Item_func_nullif::fix_length_and_dec().
This prevents a ROW expression to be passed to args[0] of NULLIF().
But will allow to pass associative arrays.
args[1] is still only checked to be comparable with args[0].
No needs to add additional tests for it.
- item_cmpfunc.h:
Adding a call for Item_hybrid_func_fix_attributes() in
Item_func_case_abbreviation2::cache_type_info().
This prevents calling the descendant functions with
a ROW expression in combination with an explicit NULL
in the THEN-style arguments (but will allow to pass associative arrays):
IFNULL(row_expression, NULL)
IFNULL(NULL, row_expression)
IF(switch, row_expression, NULL)
IF(switch, NULL, row_expression)
NVL2(switch, row_expression, NULL)
NVL2(switch, NULL, row_expression)
Adding a THD* argument into involved methods.
- item_cmpfunc.h:
Overriding Item_func_case_abbreviation2_switch::check_arguments() to
check that the first argument in IF() and NVL2() can return bool.
Removing Item_func_if::check_arguments(), as it become redundant.
- sql_type.cc:
Fixing sql_type.cc not to disallow items[0] with ROW_RESULT.
This makes it call Item_hybrid_func_fix_attributes() at the end,
which block ROW arguments into THEN-style arguments of hybrid functions.
But this will allow to pass Type_handler_assoc_array expressions.
- sql_type.cc:
Changing Type_handler_row::Item_hybrid_func_fix_attributes to raise the
ER_ILLEGAL_PARAMETER_DATA_TYPE_FOR_OPERATION error instead of the DBUG_ASSERT.
Due to complications while parsing of INSERT..SELECT statements,
optimizer hints placed at the INSERT part are ignored.
At the same time, hints placed at the SELECT part of
INSERT..SELECT statements are fully supported.
When processing queries like
INSERT INTO t1 (..) SELECT .. FROM t1, t2 ...,
there is a single query block (i.e., a single SELECT_LEX) for both INSERT and
SELECT parts. During hints resolution, when hints are attached to particular
TABLE_LIST's, the search is performed by table name across the whole
query block.
So, if a table mentioned in an optimizer hint is present in the INSERT part,
the hint is attached to the that table. This is obviously wrong as
optimizer hints are supposed to only affect the SELECT part of
an INSERT..SELECT clause.
This commit disables possible attaching hints to tables in the INSERT part
and fixes some other bugs related to INSERT..SELECT statements processing
LooseScan strategy doesn't support join orders where tables from two
subqueries interleave. There is a check in LooseScan_picker::check_qep()
to prevent use of LooseScan for such join orders.
However for two subqueries with one table each a join order of
subquery1_table
subquery2_table
outer_table
was not rejected (although the POSITION object for the last table in the
join order clearly had pos->dups_producing_tables!=0).
Modified the no-interleaving check to catch more cases.
This commit:
- fixes a couple of bugs in check_join_cache_usage();
- separates a part of opt_hints.test to a new file opt_hints_join_cache.test;
- add a batch of test cases run against different join_cache_level settings.
With strict SQL_MODE warnings generated during DDL statements
are threated as errors. This is done to prevent potential data
corruption. However, optimizer hints cannot affect the integrity
of data, so warnings during their parsing or application should not
be escalated to the level of errors.
This commit introduces `push_warning_safe()` method that guarantees
that a warning is not treated as an error, and generation of warnings
during hints processing now uses this method instead of traditional
`push_warning_printf()`
After commit 408a637b87 for MDEV-29344
the mysql-test framework started to misinterpret slash/star sequences for
statements looking like this:
SELECT /* BNL(t1) */* FROM t1, t2
where there is no space after closing `*/` and the following `*`.
This commit fixes that misinterpretation
Optimizer hints parser analyzes `thd->is_error()` flag to detect a fatal
error occured during hints processing. However, this criteria is not
reliable and considered a bad practice.
This commit replaces the criteria with the parser's own `is_fatal_error()`
flag which is related strictly to hints parsing and not something else
This commit implements optimizer hints allowing to affect the order
of joining tables:
- JOIN_FIXED_ORDER similar to existing STRAIGHT_JOIN hint;
- JOIN_ORDER to apply the specified table order;
- JOIN_PREFIX to hint what tables should be first in the join;
- JOIN_SUFFIX to hint what tables should be last in the join.
- remove get_args_printer() from hints printing
- add append_hint_arguments(THD *thd, opt_hints_enum hint, String *str)
- add more comments
- rename st_opt_hint_info::hint_name to hint_type
- add pptimizer trace support for hints
- add dbug_print_hints()
- make print_warn() not be a template
- introduce Printable_parser_rule interface, make grammar rules that
emit warnings implement it and print_warn invokes its function)
- remove Parser::Hint::append_args() as it is not used anywhere
(it used to be necessary call print_warn(... (Parser::Hint*)NULL);
1. Disable opt_hint_timeout.test for embedded server. Make sure
the test does not crash even when started for embedded server.
Disable view-protocol since hints are not supported inside views.
2. Hints are designed to behave like regular /* ... */ comments:
`SELECT /*+ " */ 1;` -- a valid SQL query
However, the mysql client program waits for the closing doublequote
character.
Another problem is observed when there is no space character between
closing `*/` of the hint and the following `*`:
`SELECT /*+ some_hints(...) */* FROM t1;`
In this case the client treats `/*` as a comment section opening and
waits for the closing `*/` sequence.
This commit fixes all of these issues
- add a comment that opt_hints_global->check_unresolved() is never called
- improve comments
- rename everything with "resolved_children" to "fully_resolved_children"
- Opt_hints_table::adjust_key_hints() now returns value
- less "reach-back-to-parent" logic
- rename Hint "adjustment" and "resolution" (yes, both terms were used)
to "fixing". "Resolution" is already used for parse-tree objects
It places a limit N (a timeout value in milliseconds) on how long
a statement is permitted to execute before the server terminates it.
Syntax:
SELECT /*+ MAX_EXECUTION_TIME(milliseconds) */ ...
Only top-level SELECT statements support the hint.
BNL() hint effectively increases join_cache_level up to 4 if it is
set to value less than 4.
This commit also makes the BKA() hint override not only
`join_cache_bka` optimizer switch but `join_cache_level` as well.
I.e., BKA() hint enables BKA and BKAH join buffers both flat and
incremental despite `join_cache_level` and `join_cache_bka` setting.
join_cache_level=0 disables join cache buffers, but the hint
BNL() now allows to employ BNL(H) buffers for particular tables
or query blocks.
This commit also adds a number of test cases including
OUTER JOINs to make sure hints do not break the rules of
join buffers application
Forbid adding optimizer hints to view definitions.
In the case when optimizer hints are added to the view definition
at a `CREATE (OR REPLACE) VIEW`/`ALTER VIEW` statement, a warning is
generated and the hints are ignored.
This commit also disables ps-protocol for test cases where
`Unresolved table/index name` warnings are generated. The reason
for this is such warnings are generated during both PREPARE
and EXECUTE stages. Since opt_hints.test has `--enable_prepare_warnings`,
running it with `--ps-protocol` causes duplication of warning messages
- Using Lex_ident_sys to scan identifiers, like the SQL parser does.
This fixes handling of double-quote-delimited and backtick-delimited identifiers,
as well as handling of non-ASCII identifiers.
Unescaping and converting from the client character set to the system
character set is now done using Lex_ident_cli_st and Lex_ident_sys,
like it's done in the SQL tokenizer/parser.
Adding helper methods to_ident_cli() and to_ident_sys()
in Optimizer_hint_parser::Token.
- Fixing the hint parser to report a syntax error when an empty identifiers:
SELECT /*+ BKA(``) */ * FROM t1;
- Moving a part of the code from opt_hints_parser.h to opt_hints_parser.cc
Moving these method definitions:
- Optimizer_hint_tokenizer::find_keyword()
- Optimizer_hint_tokenizer::get_token()
to avoid huge pieces of the code in the header file.
- A Lex_ident_cli_st cleanup
Fixing a few Lex_ident_cli_st methods to return Lex_ident_cli_st &
instead of void, to use them easier in the caller code.
- Fixing the hint parser to display the correct line number
Adding a new data type Lex_comment_st
(a combination of LEX_CSTRING and a line number)
Using it in sql_yacc.yy
- Getting rid of redundant dependencies on sql_hints_parser.h
Moving void LEX::resolve_optimizer_hints() from sql_lex.h to sql_lex.cc
Adding a class Optimizer_hint_parser_output, deriving from
Optimizer_hint_parser::Hint_list. Fixing the hint parser to
return a pointer to an allocated instance of Optimizer_hint_parser_output
rather than an instance of Optimizer_hint_parser::Hint_list.
This allows to use a forward declaration of Optimizer_hint_parser_output
in sql_lex.h and thus avoid dependencies on sql_hints_parser.h.
This commit introduces:
- the infrastructure for optimizer hints;
- hints for join buffering: BNL(), NO_BNL(), BKA(), NO_BKA();
- NO_ICP() hint for disabling index condition pushdown;
- MRR(), MO_MRR() hint for multi-range reads control;
- NO_RANGE_OPTIMIZATION() for disabling range optimization;
- QB_NAME() for assigning names for query blocks.
The check go through the following steps:
1. Run check on the underlying engine. If not ok, then return.
2. Check that there's only one row in the table, and
2.1 warn if more than one row
2.2 return HA_ADMIN_CORRUPT if fewer than one row (i.e. 0 rows)
3. If the sequence is not initialised (e.g. after an ALTER TABLE ...
SEQUENCE=1), initialise the sequence by reading the sequence
metadata from the table. This will also flush the next_free_value,
i.e. set it to the next not cached value (SEQUENCE::reserved_until)
4. Check that the sequence metadata is valid, i.e. nothing out of
order e.g. minvalue < maxvalue etc. If invalid it reports
HA_ERR_SEQUENCE_INVALID_DATA
5. Check that the sequence has not been exhausted. It reports
ER_SEQUENCE_RUN_OUT as a warning if and only if a SELECT NEXTVAL
would do so
Limitations:
1. The check is independent of flags, so the vanilla check is the same
as CHECK ... EXTENDED or CHECK ... FOR UPGRADE etc.
2. When the check discovers invalid metadata from the table,
subsequent SELECT NEXTVAL will carry on (or fail) without this
piece of knowledge, independent of the CHECK. This is to ensure
consistency, i.e. CHECK does not modify behaviour of SELECT, and if
anything it makes more sense that SELECT reports
HA_ERR_SEQUENCE_INVALID_DATA in this case, regardless of prior
CHECK
