mirror of
https://github.com/MariaDB/server.git
synced 2025-08-01 07:45:44 +00:00
b66cdbd1ea
This makes it easier to compare different costs and also allows
the optimizer to optimizer different storage engines more reliably.
- Added tests/check_costs.pl, a tool to verify optimizer cost calculations.
- Most engine costs has been found with this program. All steps to
calculate the new costs are documented in Docs/optimizer_costs.txt
- User optimizer_cost variables are given in microseconds (as individual
costs can be very small). Internally they are stored in ms.
- Changed DISK_READ_COST (was DISK_SEEK_BASE_COST) from a hard disk cost
(9 ms) to common SSD cost (400MB/sec).
- Removed cost calculations for hard disks (rotation etc).
- Changed the following handler functions to return IO_AND_CPU_COST.
This makes it easy to apply different cost modifiers in ha_..time()
functions for io and cpu costs.
- scan_time()
- rnd_pos_time() & rnd_pos_call_time()
- keyread_time()
- Enhanched keyread_time() to calculate the full cost of reading of a set
of keys with a given number of ranges and optional number of blocks that
need to be accessed.
- Removed read_time() as keyread_time() + rnd_pos_time() can do the same
thing and more.
- Tuned cost for: heap, myisam, Aria, InnoDB, archive and MyRocks.
Used heap table costs for json_table. The rest are using default engine
costs.
- Added the following new optimizer variables:
- optimizer_disk_read_ratio
- optimizer_disk_read_cost
- optimizer_key_lookup_cost
- optimizer_row_lookup_cost
- optimizer_row_next_find_cost
- optimizer_scan_cost
- Moved all engine specific cost to OPTIMIZER_COSTS structure.
- Changed costs to use 'records_out' instead of 'records_read' when
recalculating costs.
- Split optimizer_costs.h to optimizer_costs.h and optimizer_defaults.h.
This allows one to change costs without having to compile a lot of
files.
- Updated costs for filter lookup.
- Use a better cost estimate in best_extension_by_limited_search()
for the sorting cost.
- Fixed previous issues with 'filtered' explain column as we are now
using 'records_out' (min rows seen for table) to calculate filtering.
This greatly simplifies the filtering code in
JOIN_TAB::save_explain_data().
This change caused a lot of queries to be optimized differently than
before, which exposed different issues in the optimizer that needs to
be fixed. These fixes are in the following commits. To not have to
change the same test case over and over again, the changes in the test
cases are done in a single commit after all the critical change sets
are done.
InnoDB changes:
- Updated InnoDB to not divide big range cost with 2.
- Added cost for InnoDB (innobase_update_optimizer_costs()).
- Don't mark clustered primary key with HA_KEYREAD_ONLY. This will
prevent that the optimizer is trying to use index-only scans on
the clustered key.
- Disabled ha_innobase::scan_time() and ha_innobase::read_time() and
ha_innobase::rnd_pos_time() as the default engine cost functions now
works good for InnoDB.
Other things:
- Added --show-query-costs (\Q) option to mysql.cc to show the query
cost after each query (good when working with query costs).
- Extended my_getopt with GET_ADJUSTED_VALUE which allows one to adjust
the value that user is given. This is used to change cost from
microseconds (user input) to milliseconds (what the server is
internally using).
- Added include/my_tracker.h ; Useful include file to quickly test
costs of a function.
- Use handler::set_table() in all places instead of 'table= arg'.
- Added SHOW_OPTIMIZER_COSTS to sys variables. These are input and
shown in microseconds for the user but stored as milliseconds.
This is to make the numbers easier to read for the user (less
pre-zeros). Implemented in 'Sys_var_optimizer_cost' class.
- In test_quick_select() do not use index scans if 'no_keyread' is set
for the table. This is what we do in other places of the server.
- Added THD parameter to Unique::get_use_cost() and
check_index_intersect_extension() and similar functions to be able
to provide costs to called functions.
- Changed 'records' to 'rows' in optimizer_trace.
- Write more information to optimizer_trace.
- Added INDEX_BLOCK_FILL_FACTOR_MUL (4) and INDEX_BLOCK_FILL_FACTOR_DIV (3)
to calculate usage space of keys in b-trees. (Before we used numeric
constants).
- Removed code that assumed that b-trees has similar costs as binary
trees. Replaced with engine calls that returns the cost.
- Added Bitmap::find_first_bit()
- Added timings to join_cache for ANALYZE table (patch by Sergei Petrunia).
- Added records_init and records_after_filter to POSITION to remember
more of what best_access_patch() calculates.
- table_after_join_selectivity() changed to recalculate 'records_out'
based on the new fields from best_access_patch()
Bug fixes:
- Some queries did not update last_query_cost (was 0). Fixed by moving
setting thd->...last_query_cost in JOIN::optimize().
- Write '0' as number of rows for const tables with a matching row.
Some internals:
- Engine cost are stored in OPTIMIZER_COSTS structure. When a
handlerton is created, we also created a new cost variable for the
handlerton. We also create a new variable if the user changes a
optimizer cost for a not yet loaded handlerton either with command
line arguments or with SET
@@global.engine.optimizer_cost_variable=xx.
- There are 3 global OPTIMIZER_COSTS variables:
default_optimizer_costs The default costs + changes from the
command line without an engine specifier.
heap_optimizer_costs Heap table costs, used for temporary tables
tmp_table_optimizer_costs The cost for the default on disk internal
temporary table (MyISAM or Aria)
- The engine cost for a table is stored in table_share. To speed up
accesses the handler has a pointer to this. The cost is copied
to the table on first access. If one wants to change the cost one
must first update the global engine cost and then do a FLUSH TABLES.
This was done to be able to access the costs for an open table
without any locks.
- When a handlerton is created, the cost are updated the following way:
See sql/keycaches.cc for details:
- Use 'default_optimizer_costs' as a base
- Call hton->update_optimizer_costs() to override with the engines
default costs.
- Override the costs that the user has specified for the engine.
- One handler open, copy the engine cost from handlerton to TABLE_SHARE.
- Call handler::update_optimizer_costs() to allow the engine to update
cost for this particular table.
- There are two costs stored in THD. These are copied to the handler
when the table is used in a query:
- optimizer_where_cost
- optimizer_scan_setup_cost
- Simply code in best_access_path() by storing all cost result in a
structure. (Idea/Suggestion by Igor)
208 lines
7.4 KiB
C++
208 lines
7.4 KiB
C++
/* Copyright (c) 2005, 2012, Oracle and/or its affiliates.
|
|
Copyright (c) 2009, 2017, MariaDB Corporation.
|
|
|
|
This program is free software; you can redistribute it and/or modify
|
|
it under the terms of the GNU General Public License as published by
|
|
the Free Software Foundation; version 2 of the License.
|
|
|
|
This program is distributed in the hope that it will be useful,
|
|
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
|
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
|
GNU General Public License for more details.
|
|
|
|
You should have received a copy of the GNU General Public License
|
|
along with this program; if not, write to the Free Software
|
|
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1335 USA */
|
|
|
|
#ifndef _sql_plugin_h
|
|
#define _sql_plugin_h
|
|
|
|
/*
|
|
the following #define adds server-only members to enum_mysql_show_type,
|
|
that is defined in plugin.h
|
|
*/
|
|
#define SHOW_always_last SHOW_KEY_CACHE_LONG, \
|
|
SHOW_HAVE, SHOW_MY_BOOL, SHOW_HA_ROWS, SHOW_SYS, \
|
|
SHOW_LONG_NOFLUSH, SHOW_LEX_STRING, SHOW_ATOMIC_COUNTER_UINT32_T, \
|
|
SHOW_OPTIMIZER_COST, \
|
|
/* SHOW_*_STATUS must be at the end, SHOW_LONG_STATUS being first */ \
|
|
SHOW_LONG_STATUS, SHOW_DOUBLE_STATUS, SHOW_LONGLONG_STATUS, \
|
|
SHOW_UINT32_STATUS
|
|
#include "mariadb.h"
|
|
#undef SHOW_always_last
|
|
|
|
#include "m_string.h" /* LEX_STRING */
|
|
#include "my_alloc.h" /* MEM_ROOT */
|
|
|
|
class sys_var;
|
|
enum SHOW_COMP_OPTION { SHOW_OPTION_YES, SHOW_OPTION_NO, SHOW_OPTION_DISABLED};
|
|
enum enum_plugin_load_option { PLUGIN_OFF, PLUGIN_ON, PLUGIN_FORCE,
|
|
PLUGIN_FORCE_PLUS_PERMANENT };
|
|
extern const char *global_plugin_typelib_names[];
|
|
|
|
extern volatile int global_plugin_version;
|
|
extern ulong dlopen_count;
|
|
|
|
#include <my_sys.h>
|
|
#include "sql_list.h"
|
|
|
|
#ifdef DBUG_OFF
|
|
#define plugin_ref_to_int(A) A
|
|
#define plugin_int_to_ref(A) A
|
|
#else
|
|
#define plugin_ref_to_int(A) (A ? A[0] : NULL)
|
|
#define plugin_int_to_ref(A) &(A)
|
|
#endif
|
|
|
|
/*
|
|
the following flags are valid for plugin_init()
|
|
*/
|
|
#define PLUGIN_INIT_SKIP_PLUGIN_TABLE 1U
|
|
#define PLUGIN_INIT_SKIP_INITIALIZATION 2U
|
|
|
|
#define INITIAL_LEX_PLUGIN_LIST_SIZE 16
|
|
|
|
typedef enum enum_mysql_show_type SHOW_TYPE;
|
|
typedef struct st_mysql_show_var SHOW_VAR;
|
|
|
|
#define MYSQL_ANY_PLUGIN -1
|
|
|
|
/*
|
|
different values of st_plugin_int::state
|
|
though they look like a bitmap, plugin may only
|
|
be in one of those eigenstates, not in a superposition of them :)
|
|
It's a bitmap, because it makes it easier to test
|
|
"whether the state is one of those..."
|
|
*/
|
|
#define PLUGIN_IS_FREED 1U
|
|
#define PLUGIN_IS_DELETED 2U
|
|
#define PLUGIN_IS_UNINITIALIZED 4U
|
|
#define PLUGIN_IS_READY 8U
|
|
#define PLUGIN_IS_DYING 16U
|
|
#define PLUGIN_IS_DISABLED 32U
|
|
|
|
struct st_ptr_backup {
|
|
void **ptr;
|
|
void *value;
|
|
void save(void **p) { ptr= p; value= *p; }
|
|
void save(const char **p) { save((void**)p); }
|
|
void restore() { *ptr= value; }
|
|
};
|
|
|
|
/* A handle for the dynamic library containing a plugin or plugins. */
|
|
|
|
struct st_plugin_dl
|
|
{
|
|
LEX_CSTRING dl;
|
|
void *handle;
|
|
struct st_maria_plugin *plugins;
|
|
st_ptr_backup *ptr_backup;
|
|
uint nbackups;
|
|
uint ref_count; /* number of plugins loaded from the library */
|
|
int mysqlversion;
|
|
int mariaversion;
|
|
bool allocated;
|
|
};
|
|
|
|
/* A handle of a plugin */
|
|
|
|
struct st_plugin_int
|
|
{
|
|
LEX_CSTRING name;
|
|
struct st_maria_plugin *plugin;
|
|
struct st_plugin_dl *plugin_dl;
|
|
st_ptr_backup *ptr_backup;
|
|
uint nbackups;
|
|
uint state;
|
|
uint ref_count; /* number of threads using the plugin */
|
|
uint locks_total; /* how many times the plugin was locked */
|
|
void *data; /* plugin type specific, e.g. handlerton */
|
|
MEM_ROOT mem_root; /* memory for dynamic plugin structures */
|
|
sys_var *system_vars; /* server variables for this plugin */
|
|
enum enum_plugin_load_option load_option; /* OFF, ON, FORCE, F+PERMANENT */
|
|
};
|
|
|
|
|
|
extern mysql_mutex_t LOCK_plugin;
|
|
|
|
/*
|
|
See intern_plugin_lock() for the explanation for the
|
|
conditionally defined plugin_ref type
|
|
*/
|
|
#ifdef DBUG_OFF
|
|
typedef struct st_plugin_int *plugin_ref;
|
|
#define plugin_ref_to_int(A) A
|
|
#define plugin_int_to_ref(A) A
|
|
#define plugin_decl(pi) ((pi)->plugin)
|
|
#define plugin_dlib(pi) ((pi)->plugin_dl)
|
|
#define plugin_data(pi,cast) ((cast)((pi)->data))
|
|
#define plugin_name(pi) (&((pi)->name))
|
|
#define plugin_state(pi) ((pi)->state)
|
|
#define plugin_load_option(pi) ((pi)->load_option)
|
|
#define plugin_equals(p1,p2) ((p1) == (p2))
|
|
#else
|
|
typedef struct st_plugin_int **plugin_ref;
|
|
#define plugin_ref_to_int(A) (A ? A[0] : NULL)
|
|
#define plugin_int_to_ref(A) &(A)
|
|
#define plugin_decl(pi) ((pi)[0]->plugin)
|
|
#define plugin_dlib(pi) ((pi)[0]->plugin_dl)
|
|
#define plugin_data(pi,cast) ((cast)((pi)[0]->data))
|
|
#define plugin_name(pi) (&((pi)[0]->name))
|
|
#define plugin_state(pi) ((pi)[0]->state)
|
|
#define plugin_load_option(pi) ((pi)[0]->load_option)
|
|
#define plugin_equals(p1,p2) ((p1) && (p2) && (p1)[0] == (p2)[0])
|
|
#endif
|
|
|
|
typedef int (*plugin_type_init)(struct st_plugin_int *);
|
|
|
|
extern I_List<i_string> *opt_plugin_load_list_ptr;
|
|
extern char *opt_plugin_dir_ptr;
|
|
extern MYSQL_PLUGIN_IMPORT char opt_plugin_dir[FN_REFLEN];
|
|
extern const LEX_CSTRING plugin_type_names[];
|
|
extern ulong plugin_maturity;
|
|
extern TYPELIB plugin_maturity_values;
|
|
extern const char *plugin_maturity_names[];
|
|
|
|
extern int plugin_init(int *argc, char **argv, int init_flags);
|
|
extern void plugin_shutdown(void);
|
|
void add_plugin_options(DYNAMIC_ARRAY *options, MEM_ROOT *mem_root);
|
|
extern bool plugin_is_ready(const LEX_CSTRING *name, int type);
|
|
#define my_plugin_lock_by_name(A,B,C) plugin_lock_by_name(A,B,C)
|
|
#define my_plugin_lock(A,B) plugin_lock(A,B)
|
|
extern plugin_ref plugin_lock(THD *thd, plugin_ref ptr);
|
|
extern plugin_ref plugin_lock_by_name(THD *thd, const LEX_CSTRING *name,
|
|
int type);
|
|
extern void plugin_unlock(THD *thd, plugin_ref plugin);
|
|
extern void plugin_unlock_list(THD *thd, plugin_ref *list, size_t count);
|
|
extern bool mysql_install_plugin(THD *thd, const LEX_CSTRING *name,
|
|
const LEX_CSTRING *dl);
|
|
extern bool mysql_uninstall_plugin(THD *thd, const LEX_CSTRING *name,
|
|
const LEX_CSTRING *dl);
|
|
extern bool plugin_register_builtin(struct st_mysql_plugin *plugin);
|
|
extern void plugin_thdvar_init(THD *thd);
|
|
extern void plugin_thdvar_cleanup(THD *thd);
|
|
sys_var *find_plugin_sysvar(st_plugin_int *plugin, st_mysql_sys_var *var);
|
|
void plugin_opt_set_limits(struct my_option *, const struct st_mysql_sys_var *);
|
|
extern SHOW_COMP_OPTION plugin_status(const char *name, size_t len, int type);
|
|
extern bool check_valid_path(const char *path, size_t length);
|
|
extern void plugin_mutex_init();
|
|
|
|
typedef my_bool (plugin_foreach_func)(THD *thd,
|
|
plugin_ref plugin,
|
|
void *arg);
|
|
#define plugin_foreach(A,B,C,D) plugin_foreach_with_mask(A,B,C,PLUGIN_IS_READY,D)
|
|
extern bool plugin_foreach_with_mask(THD *thd, plugin_foreach_func *func,
|
|
int type, uint state_mask, void *arg);
|
|
extern void sync_dynamic_session_variables(THD* thd, bool global_lock);
|
|
|
|
extern bool plugin_dl_foreach(THD *thd, const LEX_CSTRING *dl,
|
|
plugin_foreach_func *func, void *arg);
|
|
|
|
extern void sync_dynamic_session_variables(THD* thd, bool global_lock);
|
|
#endif
|
|
|
|
#ifdef WITH_WSREP
|
|
extern void wsrep_plugins_pre_init();
|
|
extern void wsrep_plugins_post_init();
|
|
#endif /* WITH_WSREP */
|