postgres-web/templates/pages/developer/backend.html

{%extends "base/page.html"%}
{%block title%}Backend Flowchart{%endblock%}
{%block contents%}

<h1>Backend Flowchart <i class="fas fa-code-branch"></i></h1>

<center>
<h3><em>Click on an item</em> to see more detail or look at the full
<a href="https://wiki.postgresql.org/wiki/Backend_flowchart">index.</a></h3>

<p><img src="../../../media/img/developer/backend/flow.gif" usemap="#flowmap" alt="flowchart" />

<!--
image size 529 820
clickable area size 145x45; to compute closing corner, use:
echo "0, 0," | awk -F, '{printf "%d, %d\n", $1 + 145, $2 + 45}'
 -->

<map name="flowmap" id="flowmap">
<area coords="50, 0, 195, 45" href="https://wiki.postgresql.org/wiki/Backend_flowchart#main" alt="main" />
<area coords="50, 65,  195, 110" href="https://wiki.postgresql.org/wiki/Backend_flowchart#postmaster" alt="postmaster" />
<area coords="50, 135, 195, 180" href="https://wiki.postgresql.org/wiki/Backend_flowchart#tcop" alt="tcop" />
<area coords="50, 225, 195, 270" href="https://wiki.postgresql.org/wiki/Backend_flowchart#parser" alt="parser" />
<area coords="50, 295, 195, 340" href="https://wiki.postgresql.org/wiki/Backend_flowchart#tcop" alt="tcop" />
<area coords="50, 365, 195, 410" href="https://wiki.postgresql.org/wiki/Backend_flowchart#rewrite" alt="rewrite" />
<area coords="50, 435, 195, 480" href="https://wiki.postgresql.org/wiki/Backend_flowchart#optimizer_path" alt="path" />
<area coords="50, 505, 195, 550" href="https://wiki.postgresql.org/wiki/Backend_flowchart#optimizer_plan" alt="plan" />
<area coords="50, 575, 195, 620" href="https://wiki.postgresql.org/wiki/Backend_flowchart#executor" alt="executor" />

<area coords="280, 35, 425, 80" href="https://wiki.postgresql.org/wiki/Backend_flowchart#libpq" alt="libpq" />
<area coords="280, 135, 425, 180" href="https://wiki.postgresql.org/wiki/Backend_flowchart#tcop" alt="tcop" />
<area coords="280, 285, 425, 330" href="https://wiki.postgresql.org/wiki/Backend_flowchart#commands" alt="commands" />

<area coords="0, 710, 145, 755" href="https://wiki.postgresql.org/wiki/Backend_flowchart#utils" alt="utils" />
<area coords="190, 710, 335, 755" href="https://wiki.postgresql.org/wiki/Backend_flowchart#catalog" alt="catalog" />
<area coords="365, 710, 510, 755" href="https://wiki.postgresql.org/wiki/Backend_flowchart#storage" alt="storage" />

<area coords="85, 780, 230, 825" href="https://wiki.postgresql.org/wiki/Backend_flowchart#access" alt="access" />
<area coords="280, 780, 425, 825" href="https://wiki.postgresql.org/wiki/Backend_flowchart#nodes" alt="nodes" />
</map>
</center>

<br />

<p>A query comes to the backend via data packets arriving through TCP/IP
or Unix Domain sockets. It is loaded into a string, and passed to the <a
href="https://wiki.postgresql.org/wiki/Backend_flowchart#parser">parser,</a>
where the lexical scanner, <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/parser/scan.l">scan.l,</a>
breaks the query up into tokens(words). The parser uses <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/parser/gram.y">gram.y</a>
and the tokens to identify the query type, and load the proper
query-specific structure, like <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/parsenodes.h">CreateStmt</a>
or <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/parsenodes.h">SelectStmt.</a></p>

<p>The statement is then identified as complex (<em>SELECT / INSERT /
UPDATE / DELETE</em>) or simple, e.g <em>CREATE USER, ANALYZE,</em>
etc.  Simple utility commands that do not require the executor are processed by statement-specific
functions in the <a href="https://wiki.postgresql.org/wiki/Backend_flowchart#commands">commands</a> module.
Complex statements require more handling.</p>

<p>The parser takes a complex query, and creates a <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/parsenodes.h">Query</a>
structure that contains all the elements used by complex queries.
Query.qual holds the <em>WHERE</em> clause qualification, which is filled
in by <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/parser/parse_clause.c">transformWhereClause().</a>
Each table referenced in the query is represented by a <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/parsenodes.h">RangeTableEntry,</a>
and they are linked together to form the <em>range table</em> of the
query, which is generated by <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/parser/parse_clause.c">transformFromClause().</a>
Query.rtable holds the query's range table.</p>

<p>Certain queries, like <em>SELECT,</em> return columns of data. Other
queries, like <em>INSERT</em> and <em>UPDATE,</em> specify the columns
modified by the query. These column references are converted to <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/primnodes.h">TargetEntry</a>
entries, which are linked together to make up the <em>target list</em> of
the query. The target list is stored in Query.targetList, which is
generated by <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/backend/parser/parse_target.c">transformTargetList().</a></p>

<p>Other query elements, like aggregates(<em>SUM()</em>), <em>GROUP
BY,</em> and <em>ORDER BY</em> are also stored in their own Query
fields.</p>

<p>The next step is for the Query to be modified by any
<em>VIEWS</em> or <em>RULES</em> that may apply to the query. This is
performed by the <a href="https://wiki.postgresql.org/wiki/Backend_flowchart#rewrite">rewrite</a>
system.</p>

<p>The <a
href="https://wiki.postgresql.org/wiki/Backend_flowchart#optimizer_path">optimizer</a>
uses the Query structure to determine the best table join order and join
type of each table in the RangeTable, using Query.qual(<em>WHERE</em>
clause) to consider optimal index usage.</p>  The <a
href="https://wiki.postgresql.org/wiki/Backend_flowchart#optimizer_path">path</a>
module  then generates an optimal <a
href="https://git.postgresql.org/gitweb/?p=postgresql.git;a=blob;f=src/include/nodes/plannodes.h">Plan,</a>
which contains the operations to be performed to execute the query.</p>

<p>The Plan is then passed to the <a
href="https://wiki.postgresql.org/wiki/Backend_flowchart#executor">executor</a> for execution, and the
result returned to the client. The Plan is actually as set of nodes,
arranged in a tree structure with a top-level node, and various
sub-nodes as children.</p>

<p>There are many other modules that support this basic
functionality. They can be accessed by clicking on the
flowchart.</p>

{%endblock%}