Jekyll2021-09-09T03:24:44+00:00http://support.sqlparser.com/atom.xmlGeneral SQL ParserDocuments and demos for General SQL Parser{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}General SQL Parser FAQ2021-07-22T00:00:00+00:002021-07-22T00:00:00+00:00http://support.sqlparser.com/gsp-faq<div id="entry-table-of-contents" class="toc-wrapper">
<h2 id="toc-toggle" class="no_toc">
Table of Contents <i class="toc-toggle-icon fas fa-chevron-down"></i>
</h2>
<ol id="markdown-toc">
<li><a href="#technical-support" id="markdown-toc-technical-support">Technical support</a> <ol>
<li><a href="#q-does-general-sql-parser-depend-on-any-third-party-librarysoftwaredlls" id="markdown-toc-q-does-general-sql-parser-depend-on-any-third-party-librarysoftwaredlls">Q: Does general SQL parser depend on any third party library/software/DLLs?</a></li>
<li><a href="#q-in-order-to-use-gsp-to-validate-sql-syntax-do-i-need-to-connect-to-a-database-instance-such-as-oracle" id="markdown-toc-q-in-order-to-use-gsp-to-validate-sql-syntax-do-i-need-to-connect-to-a-database-instance-such-as-oracle">Q: In order to use GSP to validate SQL syntax, do I need to connect to a database instance such as Oracle?</a></li>
<li><a href="#q-how-long-will-my-feature-request-or-bug-report-be-processed" id="markdown-toc-q-how-long-will-my-feature-request-or-bug-report-be-processed">Q: How long will my feature request or bug report be processed?</a></li>
<li><a href="#q-when-database-vendor-add-new-sql-syntax-how-long-will-those-sql-syntaxes-be-supported-in-general-sql-parser" id="markdown-toc-q-when-database-vendor-add-new-sql-syntax-how-long-will-those-sql-syntaxes-be-supported-in-general-sql-parser">Q: When database vendor add new SQL syntax, how long will those SQL syntaxes be supported in general SQL parser?</a></li>
<li><a href="#q-is-gsp-net-version-a-net-standard-library" id="markdown-toc-q-is-gsp-net-version-a-net-standard-library">Q: Is GSP .NET version a .NET Standard library?</a></li>
</ol>
</li>
<li><a href="#licensing-and-billing" id="markdown-toc-licensing-and-billing">Licensing and billing</a> <ol>
<li><a href="#q-whats-kind-of-general-sql-parser-license-do-i-need" id="markdown-toc-q-whats-kind-of-general-sql-parser-license-do-i-need">Q: What’s kind of General SQL Parser license do I need?</a></li>
<li><a href="#q-what-if-i-want-to-distribute-this-library" id="markdown-toc-q-what-if-i-want-to-distribute-this-library">Q: What if I want to distribute this library?</a></li>
<li><a href="#q-will-the-license-expire" id="markdown-toc-q-will-the-license-expire">Q: Will the license expire?</a></li>
<li><a href="#q-if-i-were-to-buy-support-for-one-database-to-start--can-i-add-additional-ones-at-a-later-date" id="markdown-toc-q-if-i-were-to-buy-support-for-one-database-to-start--can-i-add-additional-ones-at-a-later-date">Q: if I were to buy support for one database to start – can I add additional ones at a later date?</a></li>
<li><a href="#q-we-have-a-need-to-create-a-parsing-service-can-we-use-the-components-to-develop-a-parser-and-then-deploy-the-service-to-a-web-like-tomcat-for-other-to-consume-via-api-or-is-this-specific-to-a-specific-user-and-a--specific-users-machine" id="markdown-toc-q-we-have-a-need-to-create-a-parsing-service-can-we-use-the-components-to-develop-a-parser-and-then-deploy-the-service-to-a-web-like-tomcat-for-other-to-consume-via-api-or-is-this-specific-to-a-specific-user-and-a--specific-users-machine">Q: We have a need to create a parsing service. Can we use the components to develop a parser and then deploy the service to a web-like Tomcat for other to consume via API or is this specific to a specific user and a specific user’s machine?</a></li>
<li><a href="#q-may-i-use-gsp-in-more-than-one-product" id="markdown-toc-q-may-i-use-gsp-in-more-than-one-product">Q: May I use GSP in more than one product?</a></li>
<li><a href="#q-payment-term" id="markdown-toc-q-payment-term">Q: Payment term?</a></li>
</ol>
</li>
<li><a href="#sales-and-reseller" id="markdown-toc-sales-and-reseller">Sales and reseller</a> <ol>
<li><a href="#q-we-are-resellers-can-we-purchase-your-products-for-our-customers" id="markdown-toc-q-we-are-resellers-can-we-purchase-your-products-for-our-customers">Q: We are resellers. can we purchase your products for our customers?</a></li>
<li><a href="#q-can-we-purchase-via-emailed-po" id="markdown-toc-q-can-we-purchase-via-emailed-po">Q: Can we purchase via emailed PO</a></li>
</ol>
</li>
</ol>
</div>
<h3 id="technical-support">Technical support</h3>
<h5 id="q-does-general-sql-parser-depend-on-any-third-party-librarysoftwaredlls">Q: Does general SQL parser depend on any third party library/software/DLLs?</h5>
<blockquote>
<p>General SQL Parser(GSP) doesn’t depend on any third party library/software/DLLs. In order to run GSP Java version, JRE 1.5 is needed. In order to run GSP .NET version, .NET Framework 4.5 or higher is needed.</p>
</blockquote>
<h5 id="q-in-order-to-use-gsp-to-validate-sql-syntax-do-i-need-to-connect-to-a-database-instance-such-as-oracle">Q: In order to use GSP to validate SQL syntax, do I need to connect to a database instance such as Oracle?</h5>
<blockquote>
<p>GSP can validate SQL syntax without any connection to the database instance, no internet connection. GSP includes all SQL parser engines itself, no additional file or connection is needed.</p>
</blockquote>
<h5 id="q-how-long-will-my-feature-request-or-bug-report-be-processed">Q: How long will my feature request or bug report be processed?</h5>
<blockquote>
<p>We provide email-based tech support. Usually, Feature requests and bug reports will be processed in 2-3 weeks.
However, this is not guaranteed. According to the complexity of the issue, the processing time will be varied from weeks to months.
In addition to providing you with our free tech support,
we also offer customized services which fix the bug and implement the feature in time, please check info@sqlparser.com for the detailed information.</p>
</blockquote>
<h5 id="q-when-database-vendor-add-new-sql-syntax-how-long-will-those-sql-syntaxes-be-supported-in-general-sql-parser">Q: When database vendor add new SQL syntax, how long will those SQL syntaxes be supported in general SQL parser?</h5>
<blockquote>
<p>General SQL Parser supports both PL/SQL and SQL. Although we try to add support for all SQL syntax of the database, it’s quite difficult to make sure all SQL syntax of the database is supported especially keep up with the recent version.</p>
<p>The goal of General SQL Parser is NOT to support all SQL syntax of the database, but support the most used SQL syntax. So, our strategy is to add support for the new SQL syntax when it is requested by the user.</p>
</blockquote>
<h5 id="q-is-gsp-net-version-a-net-standard-library">Q: Is GSP .NET version a .NET Standard library?</h5>
<blockquote>
<p>Yes. General SQL Parser .NET version is .NET Standard compatible which means it can run on all .NET platforms that implement .NET Standard.</p>
</blockquote>
<h3 id="licensing-and-billing">Licensing and billing</h3>
<h5 id="q-whats-kind-of-general-sql-parser-license-do-i-need">Q: What’s kind of General SQL Parser license do I need?</h5>
<blockquote>
<p>General SQL Parser is licensed as per user/developer.</p>
<p>There are three developer licenses: single user license, team license(2-5 developers), and site license(more than five developers).</p>
<p>The Single License grants the one developer the right to install and use multiple copies of the Software during the development.</p>
<p>The Team License grants all those developers(less than 5) the right to install and use multiple copies of the Software during the development.<br />
The Team License cannot be shared or used concurrently by more than five developers.
The Team License is NOT a ‘floating’ license; that is, you cannot temporarily transfer access rights to users outside the team.</p>
<p>A Site License grants you the right to share or use the Software concurrently by multiple individual developers at the authorized site.</p>
<p>You also need to specify the database platforms that need to be included when you purchase the license.</p>
</blockquote>
<h5 id="q-what-if-i-want-to-distribute-this-library">Q: What if I want to distribute this library?</h5>
<blockquote>
<p>You can’t deploy this library together with your product/service to customers outside your organization.
Don’t hesitate to contact us for a distribution license if you need to distribute this library outside your organization as a part of your product/service.</p>
<p>The single license entitles you to deploy this library with your software that depends on the library to a single machine inside your organization.
This library can’t be deployed to more than one machine without purchase the additional distribution license.</p>
<p>The team license entitles you to deploy this library with your software that depends on the library to five machines inside your organization.
This library can’t be deployed to more than five machines without purchase the additional distribution license.</p>
<p>This site license entitles you to deploy this library with your software that depends on the library to unlimited machines inside your organization.</p>
<p>Please note that only the licensed developer can access General SQL Parser library; any third-party developer or program can’t access APIs of General SQL Parser even your program wraps it.</p>
</blockquote>
<h5 id="q-will-the-license-expire">Q: Will the license expire?</h5>
<blockquote>
<p>You can use the software without any time limitation. It never expired. Furthermore, you can upgrade to the latest version of the software within 12 months after purchase.
However, if you like to upgrade to the latest version after 12 months, you need purchase our yearly subscription which enables another 12 months upgrade and free tech support.
the price for annual subscription is 20% of the original purchase. You will be notified when it’s time to renew your license, You need to renew this annual subscription yourself.</p>
</blockquote>
<h5 id="q-if-i-were-to-buy-support-for-one-database-to-start--can-i-add-additional-ones-at-a-later-date">Q: if I were to buy support for one database to start – can I add additional ones at a later date?</h5>
<blockquote>
<p>Yes, of course. You only need to pay the price for the additional database when you need.</p>
</blockquote>
<h5 id="q-we-have-a-need-to-create-a-parsing-service-can-we-use-the-components-to-develop-a-parser-and-then-deploy-the-service-to-a-web-like-tomcat-for-other-to-consume-via-api-or-is-this-specific-to-a-specific-user-and-a--specific-users-machine">Q: We have a need to create a parsing service. Can we use the components to develop a parser and then deploy the service to a web-like Tomcat for other to consume via API or is this specific to a specific user and a specific user’s machine?</h5>
<blockquote>
<p>General SQL Parser is licensed as per user/developer. A developer license is needed If any user/developer/machine need to access API, even a wrapper is created and API of GSP is not accessed directly.</p>
</blockquote>
<h5 id="q-may-i-use-gsp-in-more-than-one-product">Q: May I use GSP in more than one product?</h5>
<blockquote>
<p>Yes. There is no limitation of how many products GSP can be used in.</p>
</blockquote>
<h5 id="q-payment-term">Q: Payment term?</h5>
<blockquote>
<p>The full licensed version will be available to download from the official site within 2 working days after we receive the payment. You need to send the payment first.</p>
</blockquote>
<h3 id="sales-and-reseller">Sales and reseller</h3>
<h5 id="q-we-are-resellers-can-we-purchase-your-products-for-our-customers">Q: We are resellers. can we purchase your products for our customers?</h5>
<blockquote>
<p>Yes, you can purchase software from our online shop on behalf of your customer. After purchasing software, email us detailed information about your customer. The full licensed version should be available to download from our site within 48 hours after we receive your order.
There is no discount for reseller within the first 50 licenses.</p>
</blockquote>
<h5 id="q-can-we-purchase-via-emailed-po">Q: Can we purchase via emailed PO</h5>
<blockquote>
<p>Yes, our award-winning payment processor support emailed PO</p>
</blockquote>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}Table of ContentsData lineage analysis from multiple SQL Files.2020-12-31T00:00:00+00:002020-12-31T00:00:00+00:00http://support.sqlparser.com/data-lineage-multi-sql-files<p>Data lineage analysis from multiple SQL Files</p>
<p>To get an accurate data lineage analysis result, we may provide
the definition of the database objects such as table, view, procedure to
the GSP(General SQL Parser).</p>
<h3 id="1-parse-sql-file-with-ambigious-tablecolumnn-relation">1. Parse SQL file with ambigious table/columnn relation</h3>
<p>Take this SQL (file1.sql) for example:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">CREATE</span> <span class="k">VIEW</span> <span class="n">test</span>
<span class="k">AS</span>
<span class="p">(</span><span class="k">SELECT</span> <span class="n">NAME</span><span class="p">,</span>
<span class="n">address</span>
<span class="k">FROM</span> <span class="n">manager</span><span class="p">,</span>
<span class="n">employee</span>
<span class="k">WHERE</span> <span class="n">manager</span><span class="p">.</span><span class="n">id</span> <span class="o">=</span> <span class="n">employee</span><span class="p">.</span><span class="n">id</span><span class="p">)</span>
</code></pre></div></div>
<p>Without more information. GSP doesn’t know the column <code class="language-plaintext highlighter-rouge">NAME</code>, <code class="language-plaintext highlighter-rouge">address</code> in
the select list belongs to which table in the from clause.</p>
<h3 id="2-provides-the-table-definition">2. Provides the table definition</h3>
<p>File2.sql</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">Create</span> <span class="k">table</span> <span class="n">employee</span> <span class="p">(</span><span class="n">id</span> <span class="n">number</span><span class="p">,</span> <span class="n">name</span> <span class="n">varchar2</span><span class="p">(</span><span class="mi">100</span><span class="p">),</span> <span class="n">address</span> <span class="n">varchar2</span><span class="p">(</span><span class="mi">100</span><span class="p">));</span>
</code></pre></div></div>
<p>File3.sql</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">Create</span> <span class="k">table</span> <span class="n">manager</span> <span class="p">(</span><span class="n">id</span> <span class="n">number</span><span class="p">,</span> <span class="n">age</span> <span class="n">varchar2</span><span class="p">(</span><span class="mi">100</span><span class="p">),</span> <span class="n">country</span> <span class="n">varchar2</span><span class="p">(</span><span class="mi">100</span><span class="p">));</span>
</code></pre></div></div>
<p>If you provide those 2 SQL files with the table definition to the GSP,
then column <code class="language-plaintext highlighter-rouge">NAME</code>, <code class="language-plaintext highlighter-rouge">address</code> will be linked to the table <code class="language-plaintext highlighter-rouge">employee</code> correctly.</p>
<h3 id="3-how-to-provides-multiple-sql-files-to-gsp">3. How to provides multiple SQL files to GSP</h3>
<p>In GSP, <code class="language-plaintext highlighter-rouge">gudusoft.gsqlparser.dlineage.DataFlowAnalyzer</code> class do the actual work of
data lineage analysis.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="kd">public</span> <span class="nf">DataFlowAnalyzer</span><span class="o">(</span><span class="nc">File</span> <span class="n">sqlFile</span><span class="o">,</span> <span class="nc">EDbVendor</span> <span class="n">dbVendor</span><span class="o">,</span> <span class="kt">boolean</span> <span class="n">simpleOutput</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">sqlFile</span> <span class="o">=</span> <span class="n">sqlFile</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">vendor</span> <span class="o">=</span> <span class="n">dbVendor</span><span class="o">;</span>
<span class="k">this</span><span class="o">.</span><span class="na">simpleOutput</span> <span class="o">=</span> <span class="n">simpleOutput</span><span class="o">;</span>
<span class="o">}</span>
</code></pre></div></div>
<p>As you can see here, the first parameter of <code class="language-plaintext highlighter-rouge">DataFlowAnalyzer</code> accept a <code class="language-plaintext highlighter-rouge">File</code> type
which will accept a directory that includes all SQL files that need to be processed.</p>
<p>You may also check the DataFlowAnalyzer demo under demos.lineage package shipped together
with the GSP library to find out how to feed multiple SQL files.</p>
<h3 id="4-pulling-all-objects-from-a-database-table-view-function-procedure-and-trigger-definitions">4. Pulling all objects from a database (table, view, function, procedure, and trigger definitions)</h3>
<p>Once you were pulling all objects from a database (table, view, function, procedure, and trigger definitions),
it is recommended to put the definition of a single object in a single SQL file, especially for
function, procedure, and trigger definitions. In this way, the processing error in one single SQL file
will not affect the other SQL files.</p>
<p>The order of those SQL files put under a directory doesn’t matter. GSP is smart enough to get the necessary
information accordingly.</p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}Data lineage analysis from multiple SQL Files and get accurate metadata result in JSON and CSV format.SQL parse tree node and underlying tokens2020-10-08T00:00:00+00:002020-10-08T00:00:00+00:00http://support.sqlparser.com/sql-parse-tree-node-and-underlying-tokens-en<p>This document explains how to use the GSP library to parse an existing SQL script, then modify the SQL parse tree, and rebuild the whole SQL using <strong>TParseTreeNode.toString()</strong> method.</p>
<blockquote>
<p>If you build a SQL parse tree from the scratch(Not from the existing SQL), and then generate SQL text from this parse tree, then <strong>TParseTreeNode.toScript()</strong> is the better choice.</p>
</blockquote>
<h2 id="1-tparsetreenode-setstring">1. TParseTreeNode setString()</h2>
<p>Change the text of a SQL clause or the whole SQL statement.</p>
<p>Take this SQL for example:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span>
<span class="k">WHERE</span> <span class="n">e</span><span class="p">.</span><span class="n">job_id</span> <span class="o">=</span> <span class="mi">1</span>
</code></pre></div></div>
<p>We like to change the condition in the where clause from <code class="language-plaintext highlighter-rouge">e.job_id = 1</code> to <code class="language-plaintext highlighter-rouge">e.salary > 1000</code>.</p>
<p>Below is the Java code illustrates how to achieve this.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT *\n"</span> <span class="o">+</span>
<span class="s">"FROM scott.employee\n"</span> <span class="o">+</span>
<span class="s">"WHERE e.job_id = 1"</span><span class="o">;</span>
<span class="n">sqlparser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="nc">TSelectSqlStatement</span> <span class="n">select</span> <span class="o">=</span> <span class="o">(</span><span class="nc">TSelectSqlStatement</span><span class="o">)</span><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="nc">TWhereClause</span> <span class="n">whereClause</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="na">getWhereClause</span><span class="o">();</span>
<span class="n">whereClause</span><span class="o">.</span><span class="na">getCondition</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="s">"e.salary > 1000"</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<p>After running the above Java code, the output is:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span>
<span class="k">WHERE</span> <span class="n">e</span><span class="p">.</span><span class="n">salary</span> <span class="o">></span> <span class="mi">1000</span>
</code></pre></div></div>
<h2 id="2-remove-a-node">2. remove a node</h2>
<h3 id="call-setxxx--method-from-the-parent-node-and-pass-null-as-input-parameter-will-remove-the-sql-clause-from-the-parent-node">Call <strong>setXXX()</strong> method from the parent node and pass <strong>null</strong> as input parameter, will remove the SQL clause from the parent node.</h3>
<p>This Java code will remove where clause from the select statement.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT * FROM TABLE_X where a>1 order by a"</span><span class="o">;</span>
<span class="n">sqlparser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="n">select</span><span class="o">.</span><span class="na">setWhereClause</span><span class="o">(</span><span class="kc">null</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<h3 id="call-removeitemint-index-of-tparsetreenodelist-will-remove-an-item-from-the-node-list">Call <strong>removeItem(int index)</strong> of <code class="language-plaintext highlighter-rouge">TParseTreeNodeList</code> will remove an item from the node list.</h3>
<p>This Java code will remove column <code class="language-plaintext highlighter-rouge">b</code> from the order by clause.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT * FROM TABLE_X order by a,b"</span><span class="o">;</span>
<span class="n">sqlparser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="n">select</span><span class="o">.</span><span class="na">getOrderbyClause</span><span class="o">().</span><span class="na">getItems</span><span class="o">().</span><span class="na">removeItem</span><span class="o">(</span><span class="mi">1</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<h2 id="3-update-a-node">3. update a node</h2>
<p>Please node’s <strong>setString()</strong> method.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TGSqlParser</span> <span class="n">parser</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TGSqlParser</span><span class="o">(</span><span class="nc">EDbVendor</span><span class="o">.</span><span class="na">dbvoracle</span><span class="o">);</span>
<span class="n">parser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT A.COLUMN1, B.COLUMN2 from TABLE1 A, TABLE2 B where A.COLUMN1=B.COLUMN1"</span><span class="o">;</span>
<span class="n">parser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="nc">TSelectSqlStatement</span> <span class="n">select</span> <span class="o">=</span> <span class="o">(</span><span class="nc">TSelectSqlStatement</span><span class="o">)</span><span class="n">parser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="n">select</span><span class="o">.</span><span class="na">getWhereClause</span><span class="o">.</span><span class="na">setString</span><span class="o">(</span><span class="s">"where a>2"</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span> <span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<h2 id="4-add-a-new-node">4. add a new node</h2>
<p>Call <strong>setXXX()</strong> method from the parent node and pass the new node as a paremeter.
In order to add a new node, we must know the parent node of this new added node.</p>
<p>Take this SQL for example, <code class="language-plaintext highlighter-rouge">TCustomSqlStatement.getWhereClause()</code> returns null.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span>
</code></pre></div></div>
<p>In order to add where clause for this SQL, below is the Java:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//create a new node</span>
<span class="nc">TWhereClause</span> <span class="n">whereClause</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TWhereClause</span><span class="o">();</span>
<span class="n">whereClause</span><span class="o">.</span><span class="na">setString</span><span class="o">(</span><span class="s">"where a>2"</span><span class="o">);</span>
<span class="c1">//link this new created node in the SELECT statement</span>
<span class="n">select</span><span class="o">.</span><span class="na">setWhereClause</span><span class="o">(</span><span class="n">whereClause</span><span class="o">);</span>
</code></pre></div></div>
<p>Then, we will get the new SQL like this:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span> <span class="k">where</span> <span class="n">a</span><span class="o">></span><span class="mi">2</span>
</code></pre></div></div>
<p>steps to add a new node:</p>
<ul>
<li>create a node, <code class="language-plaintext highlighter-rouge">new TParseTreeNode()</code>, then call <code class="language-plaintext highlighter-rouge">TParseTreeNode.setString()</code> to set the text of this node.</li>
<li>call <code class="language-plaintext highlighter-rouge">setXXX(TParseTreeNode)</code> from the parent node, and pass the new created node as parameter.</li>
</ul>
<h2 id="apis-available-to-modify-the-parse-tree">APIs available to modify the parse tree</h2>
<ul>
<li>TParseTreeNode.setString(String sqlSegment), update the text of a node.</li>
<li>TParseTreeNodeList.removeItem(int index), All decendant class of TParseTreeNodeList can use this method to remove a sub-node.</li>
<li>TCustomSqlStatement.setOutputClause(TOutputClause outputClause)</li>
<li>TCustomSqlStatement.setResultColumnList(TResultColumnList resultColumnList)</li>
<li>TCustomSqlStatement.setReturningClause(TReturningClause returningClause)</li>
<li>TCustomSqlStatement.setTargetTable(TTable targetTable)</li>
<li>TCustomSqlStatement.setTopClause(TTopClause topClause)</li>
<li>TCustomSqlStatement.setWhereClause(TWhereClause newWhereClause)</li>
<li>TCustomSqlStatement.setWhereClause()</li>
<li>TCustomSqlStatement.setWhereClause()</li>
<li>all TSelectSqlStatement.setXXX() method.</li>
</ul>
<h2 id="use-visitor-pattren-to-search-and-modify-node">use visitor pattren to search and modify node</h2>
<p>Since there are lots of nodes in a parse tree node, and you may only need to modify some specific node type.
So, use visitor to search and modify a specific type node is very convenient.</p>
<p>Please find how to search datatype, function, SQL statement and modify it here:
<a href="https://github.com/sqlparser/gsp_demo_java/tree/master/src/main/java/demos/visitors">Java demo</a></p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}A general review/summary of General SQL Parser: sql parser tree node and underlying tokensSQL parse tree node and expression modification2020-10-08T00:00:00+00:002020-10-08T00:00:00+00:00http://support.sqlparser.com/sql-parse-tree-node-expression-en<p>How to modify and rebuild expression.</p>
<h2 id="1-remove-a-sub-node-of-an-expression">1. remove a sub-node of an Expression</h2>
<p>After removing a sub-node of an expression, the whole expression maybe affected. Take this SQL for example:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>remove either d.cntrb_date1 or ‘$From_Date$’, the whole expression will be removed as well.</p>
<p>According to the different kind of expression, the result will be different after removing a sub-node.</p>
<ul>
<li>Math expression: +,-,*,/ and other expression with two operands, after removing one operand, the other will be remain unchanged.</li>
<li>Logical expression: and, or, after removing one operand, the other will be remain unchanged.</li>
<li>Comparison expression: <, > , after removing one operand, the whole expression will be removed.</li>
<li>in, between, () expression: after removing one operand, the whole expression will be removed.</li>
<li>Other kind of expression: after removing one operand, the whole expression will be removed.</li>
</ul>
<p>After the removal of the sub-node, if the whole parent expression is removed as well, the processing will be executed recursively until the top-level expression.</p>
<h3 id="11-api">1.1 API</h3>
<p>Call TExpression.removeMe() to remove an expression itself.</p>
<h3 id="12-using-tparsetreenodelistremoveitemint-index-to-remove-the-sub-expression-in-the-expression-list">1.2 using TParseTreeNodeList.removeItem(int index) to remove the sub-expression in the expression list</h3>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">)</span>
</code></pre></div></div>
<p>After calling</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expressionList</span><span class="o">.</span><span class="na">removeItem</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
</code></pre></div></div>
<p>The result is:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">)</span>
</code></pre></div></div>
<h2 id="2-modify-the-expression">2. Modify the expression</h2>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>After set ‘$From_Date$’ to 1 , the expression will be</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="mi">1</span>
</code></pre></div></div>
<p>The java code to achieve this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="s">"1"</span><span class="o">);</span>
<span class="n">assertTrue</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">equalsIgnoreCase</span><span class="o">(</span><span class="s">"d.cntrb_date1 >= 1"</span><span class="o">));</span>
</code></pre></div></div>
<h2 id="3-add-a-new-expression">3. Add a new expression</h2>
<p>If we like to change</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>to:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span> <span class="o">+</span> <span class="mi">1</span>
</code></pre></div></div>
<p>Here is the Java code:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">toString</span><span class="o">()+</span><span class="s">" + 1"</span><span class="o">);</span>
<span class="n">assertTrue</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">equalsIgnoreCase</span><span class="o">(</span><span class="s">"d.cntrb_date1 >= '$From_Date$' + 1"</span><span class="o">));</span>
</code></pre></div></div>
<h3 id="reference-java-code">Reference Java code</h3>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testExpression.java">testExpression</a></p>
<ol>
<li>public void testRemove1()</li>
<li>public void testRemoveExprList()</li>
</ol>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testModifyExpr.java">testModifyExpr</a>
<a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testModifySql.java">testModifySql</a></p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}A general review/summary of General SQL Parser: sql parser tree node and expression modificationSQL parse tree node and expression modification2020-09-10T00:00:00+00:002020-09-10T00:00:00+00:00http://support.sqlparser.com/sql-parse-tree-node-expression<p>表达式的修改及结果。</p>
<h3 id="一-删除表达式的子节点">一、 删除表达式的子节点</h3>
<p>删除表达式后,可能会对这个表达式所在的整个表达式产生影响,例如:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>以上比较表达式,删除 d.cntrb_date1 或 ‘$From_Date$’ 中的任意一个, 整个表达式也被删除。</p>
<p>根据表达式和其所在表达式的情况不同,处理的方式也不同,以下为主要情况</p>
<ul>
<li>数学表达式: +,-,*,/ 和其他含有两个 operand 的数学表达式,删除其中一个 operand 后, 还会留下另外一个。</li>
<li>逻辑表达式: and, or, 删除其中一个 operand 后, 还会留下另外一个。</li>
<li>比较表达式: <, > 等, 删除其中一个 operand 后, 整个表达式也被删除。</li>
<li>in, between, () 表达式, 删除其中一个 operand 后, 整个表达式也被删除。</li>
<li>其它表达式, 删除其中一个 operand 后, 整个表达式也被删除。</li>
</ul>
<p>删除一个表达式后,如果导致父表达式也被删除,会递归处理更高级别的表达式,直到最高层的表达式。</p>
<h4 id="1-api">1. API</h4>
<p>TExpression.removeMe()</p>
<h4 id="2-相关属性的变化">2. 相关属性的变化</h4>
<p>如果一个节点被删除, 那么该节点的属性:</p>
<ol>
<li>getNodeStatus() 为 ENodeStatus.nsRemoved</li>
<li>getStartToken(), getEndToken() 返回 null</li>
<li>toString(), 返回 null</li>
<li>expression.getExpressionType() == EExpressionType.removed_t</li>
</ol>
<p>判断一个节点是否被删除, 用 getNodeStatus() == ENodeStatus.nsRemoved</p>
<p>如果一个表达式的 left operann and right operand 都被删除, 那么该表达式的状态也处于被删除。
以上属性同样满足。</p>
<h4 id="3-利用-tparsetreenodelistremoveitemint-index-来移除-expression-list-中的-expresssion">3. 利用 TParseTreeNodeList.removeItem(int index) 来移除 expression list 中的 expresssion</h4>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">)</span>
</code></pre></div></div>
<p>调用</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expressionList</span><span class="o">.</span><span class="na">removeItem</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
</code></pre></div></div>
<p>结果为:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">4</span><span class="p">)</span>
</code></pre></div></div>
<h3 id="二更改表达式">二、更改表达式</h3>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>After set ‘$From_Date$’ to 1 , the expression will be</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="mi">1</span>
</code></pre></div></div>
<p>通过以下代码实现上面的更改功能:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="s">"1"</span><span class="o">);</span>
<span class="n">assertTrue</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">equalsIgnoreCase</span><span class="o">(</span><span class="s">"d.cntrb_date1 >= 1"</span><span class="o">));</span>
</code></pre></div></div>
<h3 id="三增加表达式">三、增加表达式</h3>
<p>增加表达式一般通过修改原有表达式来实现,例如:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span>
</code></pre></div></div>
<p>想变为:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">d</span><span class="p">.</span><span class="n">cntrb_date1</span> <span class="o">>=</span> <span class="s1">'$From_Date$'</span> <span class="o">+</span> <span class="mi">1</span>
</code></pre></div></div>
<p>通过以下代码实现上面的更改功能:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">getRightOperand</span><span class="o">().</span><span class="na">toString</span><span class="o">()+</span><span class="s">" + 1"</span><span class="o">);</span>
<span class="n">assertTrue</span><span class="o">(</span><span class="n">expression</span><span class="o">.</span><span class="na">toString</span><span class="o">().</span><span class="na">equalsIgnoreCase</span><span class="o">(</span><span class="s">"d.cntrb_date1 >= '$From_Date$' + 1"</span><span class="o">));</span>
</code></pre></div></div>
<h3 id="参考代码">参考代码</h3>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testExpression.java">testExpression</a>
中的</p>
<ol>
<li>public void testRemove1()</li>
<li>public void testRemoveExprList()</li>
</ol>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testModifyExpr.java">testModifyExpr</a>
<a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/test/java/common/testModifySql.java">testModifySql</a></p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}A general review/summary of General SQL Parser: sql parser tree node and expression modificationSQL parse tree node and underlying tokens2020-09-09T00:00:00+00:002020-09-09T00:00:00+00:00http://support.sqlparser.com/sql-parse-tree-node-and-underlying-tokens<p>本文目的是帮助用户掌握:<strong>通过操作 SQL 语句的 AST,输出新的 SQL 语句</strong>。具体而言, 是通过调用 <strong>TParseTreeNode.toString()</strong> 方法(拼接对应的 token list)来输出 SQL 语句。使用这种方法,只要是 GSP 能够解析的 SQL,
都可以正确输出 SQL 语句。这种方法的使用场景是: 解析 SQL 语句, 修改 SQL 对应的 AST, 输出新的 SQL 语句。</p>
<p>GSP 中另一个输出 SQL 语句的方法是 <strong>TParseTreeNode.toScript()</strong> , 它根据语法把 AST 中每个 node 转换为文本,
然后拼接成完整的 SQL 语句。主要的使用场景是:用户完全从头开始利用 GSP API 来构造一颗 SQL 语句的 AST 树,然后根据 AST 来输出 SQL 语句。 当然也可以利用 <strong>TParseTreeNode.toScript()</strong> 来输出 GSP 解析后的 SQL 语句,但如果 AST 中某个 node 转换文本功能没有支持,则整个 SQL 语句的输出将失败。</p>
<h2 id="一-sql-文本ast-node-及-tokens-的关系">一、 SQL 文本,AST Node 及 Tokens 的关系</h2>
<p>GSP 解析 SQL 语句,先由 lexer 把 SQL 文本分解成一系列 tokens, 然后由 parser 逐个处理这些 tokens, 生成语法树(AST)。AST 中的每个 node 对应 SQL 语句中的一部分文本,也对应 tokens 中的一段连续的 tokens.</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span>
</code></pre></div></div>
<p>以上SQL对应下面的 token list:</p>
<p><img src="/images/gsp/token_list1.png" alt="token_list1" /></p>
<p>每个 node 都含有一个起始 token(startToken) 和一个结束 token(endToken)。组成 node 的 token 由 startToken 开始,到 endToken 结束。node 中的所有 token 以双向链表方式建立关联。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getStartToken</span><span class="o">()</span>
<span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getEndToken</span><span class="o">()</span>
</code></pre></div></div>
<p>由 SQL 的语法决定, 一个 token 可以是一个或多个 node 的 startToken, 也可以是一个或多个 node 的 endToken.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nc">Stack</span><span class="o"><</span><span class="nc">TParseTreeNode</span><span class="o">></span> <span class="nf">getNodesStartFromThisToken</span><span class="o">()</span>
<span class="kd">public</span> <span class="nc">Stack</span><span class="o"><</span><span class="nc">TParseTreeNode</span><span class="o">></span> <span class="nf">getNodesEndWithThisToken</span><span class="o">()</span>
</code></pre></div></div>
<h4 id="例-1-token-emp_id">例 1, token: emp_id</h4>
<p>以它为 startToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:emp_id
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:emp_id
2: Node type:gudusoft.gsqlparser.nodes.TResultColumn, Node text:emp_id
</code></pre></div></div>
<p>以它为 endToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:emp_id
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:emp_id
2: Node type:gudusoft.gsqlparser.nodes.TResultColumn, Node text:emp_id
</code></pre></div></div>
<p><img src="/images/gsp/node_include_token_1.png" alt="node_include_token_1" /></p>
<p>可以发现,当 node 只有唯一一个 token 组成时, node 的 startToken 和 endToken 都为该 token。</p>
<h4 id="例-2-token-salary">例 2, token: salary</h4>
<p>以它为 startToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:salary
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:salary
2: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:salary+100
3: Node type:gudusoft.gsqlparser.nodes.TResultColumn, Node text:salary+100
</code></pre></div></div>
<p>以它为 endToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:salary
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:salary
</code></pre></div></div>
<p><img src="/images/gsp/node_include_token_2.png" alt="node_include_token_2" /></p>
<h4 id="例-3-token-100">例 3, token: 100</h4>
<p>以它为 startToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TConstant, Node text:100
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:100
</code></pre></div></div>
<p>以它为 endToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TConstant, Node text:100
1: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:100
2: Node type:gudusoft.gsqlparser.nodes.TExpression, Node text:salary+100
3: Node type:gudusoft.gsqlparser.nodes.TResultColumn, Node text:salary+100
</code></pre></div></div>
<h4 id="例-4-token-emp">例 4, token: emp</h4>
<p>Parser 以 LALR 的方式解析 SQL,因此以某个 token 开始的所有 node 被 parser 以创建的先后次序依次存放在栈中, 即子 node 比父 node 先进入栈中。但在 LALR 解析的前期或后期,以这个 token 为 startToken 或 endToken 的 node 可能还会被创建,因此在栈高层的 node 并不一定 都是底层 node 的父辈 node。 判断栈中两个 node 的包含关系, 通过比较它们包含 token 个数来决定, token 个数多的为父辈 node。</p>
<p>从 emp token 我们就可以观察到这种现象,特别是以它为 endToken 的 node。</p>
<p>以它为 startToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:emp
1: Node type:gudusoft.gsqlparser.nodes.TFromTable, Node text:emp
2: Node type:gudusoft.gsqlparser.nodes.TTable, Node text:emp
3: Node type:gudusoft.gsqlparser.nodes.TJoin, Node text:emp
</code></pre></div></div>
<p>以它为 endToken 的 node 有:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>0: Node type:gudusoft.gsqlparser.stmt.TSelectSqlStatement, Node text:SELECT emp_id,salary+100 FROM emp
1: Node type:gudusoft.gsqlparser.nodes.TObjectName, Node text:emp
2: Node type:gudusoft.gsqlparser.nodes.TFromTable, Node text:emp
3: Node type:gudusoft.gsqlparser.nodes.TSelectSqlNode, Node text:SELECT emp_id,salary+100 FROM emp
4: Node type:gudusoft.gsqlparser.nodes.TTable, Node text:emp
5: Node type:gudusoft.gsqlparser.nodes.TJoin, Node text:emp
</code></pre></div></div>
<p>其中, node0 是在 <code class="language-plaintext highlighter-rouge">getrawsqlstatements()</code> 时创建的。 node1, node2, node3 是在 Parser 以 LALR 的方式解析时创建的。 node4, node5 是在后期的语义处理阶段创建的。</p>
<h3 id="1-tparsetreenodelist-子类类型的-node">1, TParseTreeNodeList 子类类型的 node</h3>
<p>TParseTreeNodeList 子类类型的 node 维护着一个链表, 该链表包含多个相同类型的 node. TParseTreeNodeList 类型的 node 本身不直接包含 startToken 和 endToken。它的 startToken 为它链表中首个 node 的 startToken. 它的 endToken 为它链表中最后一个 node 的 endToken.</p>
<p>以 TParseTreeNodeList 子类 TResultColumnList 为例,TSelectSqlStatement.getResultColumnList() 返回下面 SELECT 语句的 <code class="language-plaintext highlighter-rouge">emp_id,salary+100</code> 部分。</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span>
</code></pre></div></div>
<p>因此, 该 TResultColumnList 的 startToken 为 emp_id, endToken 为 100, 同时从例1 和 例3 我们也可以知道, 以 emp_id 为 startToken 的 node 中并不包含 TResultColumnList, 以 100 为 endToken 的 node 中也不包含 TResultColumnList。</p>
<p>调用 node 的<code class="language-plaintext highlighter-rouge">ToString()</code>方法,即从 startToken 开始输出文本,遍历每一个 token,直到 endToken 结束。因此,<strong>使用 GSP 的 API 对 AST node 进行操作时,更新 node 结构也会同步跟新对应的 token</strong>,以保证 node 的 <code class="language-plaintext highlighter-rouge">ToString()</code> 输出正确的文本。</p>
<h3 id="2-节点与子节点的关系">2, 节点与子节点的关系</h3>
<p>语法树 (AST) 包含多个节点 (node),同时,节点 (node) 也可以包含多个子节点,因此,一个顶级的 node 就是一颗语法树。例如<code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code>.</p>
<p>不同数据库的相同 SQL 语句,例如 SELECT 语句,在 GSP 中用同一个节点 <code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code> 表示,它的子节点因为不同的数据库而可能会有不同,例如,Oracle 中就没有 <code class="language-plaintext highlighter-rouge">TTopClause</code> 这个子节点。visitor 访问代表不同数据库 SELECT 语句的 <code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code> 节点的方式是相同的。</p>
<p>节点可以包含多个子节点,同级子节点对应的 token list 不会重叠,节点的 token list 包含所有子节点的 token list,除此之外,可能还会包含节点自身独有的辅助 token,例如,<code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code> 就有 <code class="language-plaintext highlighter-rouge">SELECT</code> 这个token,它不属于任何子节点。</p>
<p>节点的 startToken, endToken 可能和它子节点的startToken, endToken重合,分为三种情况:</p>
<ul>
<li>节点和子节点的 startToken, endToken 都重合。</li>
<li>节点和子节点的 startToken 重合, 但节点的 endToken 在子节点的 endToken 之后。</li>
<li>节点和子节点的 endToken 重合, 但节点的 startToken 在子节点的 startToken 之前。</li>
</ul>
<p>因此,当某个节点的 startToken, endToken 发生变化时,共用这些 startToken, endToken 的节点也需要同步更新他们 startToken, endToken 的指向。</p>
<h2 id="二gsp-如何保证-ast-node-和-tokens-的同步">二、GSP 如何保证 AST Node 和 Tokens 的同步</h2>
<p>这个 SQL 表达式</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">fx</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span>
</code></pre></div></div>
<p>它的 token list 为:</p>
<p><img src="/images/gsp/token_list2.png" alt="token_list2" /></p>
<p>它的 node 关系图:</p>
<p><img src="/images/gsp/node_include_token_3.png" alt="node_include_token_3" /></p>
<p>从图中可以知道,<code class="language-plaintext highlighter-rouge">fx</code> 同时是 TObjectName, TFunctionCall, TExpression (3), TExpression (4) 的 startToken.
当我们用 <code class="language-plaintext highlighter-rouge">TFunctionCall.setString('gx(2)')</code> 把 <code class="language-plaintext highlighter-rouge">fx(2)</code> 更改为 <code class="language-plaintext highlighter-rouge">gx(2)</code> 时,子节点 <code class="language-plaintext highlighter-rouge">TFunctionCall</code> 的 startToken 变为 <code class="language-plaintext highlighter-rouge">gx</code>, 此时如果不做同步,其它三个节点的 startToken 仍然指向 <code class="language-plaintext highlighter-rouge">fx</code>,这是不对的。此时如果调用 <code class="language-plaintext highlighter-rouge">TExpression (3).toString()</code> , 它的结果是 <code class="language-plaintext highlighter-rouge">fx(2)</code>, 而不是已经变更后的 <code class="language-plaintext highlighter-rouge">gx(2)</code>.</p>
<p>有一点需要注意的是,如果从更高层级的node调用<code class="language-plaintext highlighter-rouge">toString()</code>方法,输出结果仍然是正确的,例如:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">WHERE</span> <span class="n">fx</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span><span class="o">+</span><span class="mi">1</span><span class="o">></span><span class="mi">1</span>
</code></pre></div></div>
<p>当用<code class="language-plaintext highlighter-rouge">TFunctionCall.setString('gx(2)')</code>把<code class="language-plaintext highlighter-rouge">fx(2)</code>更改为<code class="language-plaintext highlighter-rouge">gx(2)</code>后,<code class="language-plaintext highlighter-rouge">TWhereClause.toString()</code> 仍将输出正确的结果,原因是:</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">TFunctionCall.setString()</code>不会影响<code class="language-plaintext highlighter-rouge">TWhereClause</code>的startToken,它仍然是<code class="language-plaintext highlighter-rouge">WHERE</code>。</li>
<li>在<code class="language-plaintext highlighter-rouge">TFunctionCall.setString()</code>时,<code class="language-plaintext highlighter-rouge">gx</code>会取代<code class="language-plaintext highlighter-rouge">fx</code>加入到<code class="language-plaintext highlighter-rouge">TWhereClause</code>的token list中来。</li>
</ol>
<p>接下来, 我们主要讨论在修改 AST 的同时, 为了保证 node 和 token list 的同步, GSP 提供了哪些数据结构, 并且在利用 API 对 AST 进行操作时, 如何保证 node 和 token list 的同步。</p>
<h3 id="1-保证-ast-node-和-tokens-同步的数据结构">1、 保证 AST Node 和 Tokens 同步的数据结构</h3>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// TParseTreeNode:</span>
<span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getStartToken</span><span class="o">()</span>
<span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getEndToken</span><span class="o">()</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setStartToken</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">newStartToken</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setEndToken</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">newEndToken</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">removeTokens</span><span class="o">()</span> <span class="c1">// 从链表中移除该 node 对应的所有 token, 并确保 node 和 startToken, endToken状态的准确</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">appendNewNode</span><span class="o">(</span><span class="nc">TParseTreeNode</span> <span class="n">newNode</span><span class="o">,</span> <span class="kt">boolean</span> <span class="n">needCommaBefore</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">replaceWithNewNode</span><span class="o">(</span><span class="nc">TParseTreeNode</span> <span class="n">newNode</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setText</span><span class="o">(</span><span class="nc">String</span> <span class="n">nodeText</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setNewSubNode</span><span class="o">(</span> <span class="nc">TParseTreeNode</span> <span class="n">oldSubNode</span><span class="o">,</span> <span class="nc">TParseTreeNode</span> <span class="n">newSubNode</span><span class="o">,</span><span class="nc">TParseTreeNode</span> <span class="n">anchorNode</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setAnchorNode</span><span class="o">(</span><span class="nc">TParseTreeNode</span> <span class="n">anchorNode</span><span class="o">)</span>
<span class="kd">public</span> <span class="nc">ENodeStatus</span> <span class="nf">getNodeStatus</span><span class="o">()</span>
</code></pre></div></div>
<h4 id="设置-node-的-starttoken-endtoken">设置 node 的 startToken, endToken</h4>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">setStartToken</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">newStartToken</span><span class="o">)</span>
</code></pre></div></div>
<p>设置 node 的 startToken。如果该 node 原来已经有一个 startToken, 并且该 node 在原有 startToken 所维护的 <code class="language-plaintext highlighter-rouge">NodesStartFromThisToken</code> 栈的顶部, 那么把该 node 从原有 token 的 <code class="language-plaintext highlighter-rouge">NodesStartFromThisToken</code> 栈中弹出。然后检查 node 是否在新 token 的 <code class="language-plaintext highlighter-rouge">NodesStartFromThisToken</code> 中存在, 如果不存在, 压入该 node。</p>
<p>以上我们可以看出, 设置一个 node 的 startToken, 需同时维护 node 和 token 的双向关系。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">setEndToken</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">newStartToken</span><span class="o">)</span>
</code></pre></div></div>
<p>setEndToken() 的处理逻辑同 setStartToken()。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// TSourceToken:</span>
<span class="kd">public</span> <span class="nc">Stack</span><span class="o"><</span><span class="nc">TParseTreeNode</span><span class="o">></span> <span class="nf">getNodesStartFromThisToken</span><span class="o">()</span>
<span class="kd">public</span> <span class="nc">Stack</span><span class="o"><</span><span class="nc">TParseTreeNode</span><span class="o">></span> <span class="nf">getNodesEndWithThisToken</span><span class="o">()</span>
<span class="c1">// 双向链表中, 通过以下方法把 token 加入链表,或从链表中移除。</span>
<span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getNextTokenInChain</span><span class="o">()</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setNextTokenInChain</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">nextTokenInChain</span><span class="o">)</span>
<span class="kd">public</span> <span class="nc">TSourceToken</span> <span class="nf">getPrevTokenInChain</span><span class="o">()</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">setPrevTokenInChain</span><span class="o">(</span><span class="nc">TSourceToken</span> <span class="n">prevTokenInChain</span><span class="o">)</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">removeFromChain</span><span class="o">()</span>
</code></pre></div></div>
<p>利用 API 对 AST 进行操作后, 利用以上数据结构和方法, 同步 node 和 token list。主要实现这三个功能:</p>
<ol>
<li>在 token 双向链表中, 在指定的位置,把一个或多个 token 加入链表,或从链表中更新、移除。</li>
<li>token 被更新、移除后, 把该 token 作为 startToken 或 endToken 的 node 做更新。</li>
<li>node 被更新、移除后,它本身及子 node 的状态需要做更新,确保后续操作可以知道这些 node 所处的状态, 并作出合适的处理。</li>
</ol>
<h4 id="首次建立-token-间的双向链接">首次建立 token 间的双向链接</h4>
<p><code class="language-plaintext highlighter-rouge">TGSqlParser</code> parse SQL 语句时,所有 token 在 <code class="language-plaintext highlighter-rouge">dosqltexttotokenlist()</code> 中首次建立双向链接。</p>
<h3 id="2-利用-api-对-ast-进行的操作">2、 利用 API 对 AST 进行的操作</h3>
<h4 id="21-tparsetreenode-setstring">2.1 TParseTreeNode setString()</h4>
<p>给 node 设置 text 时,GSP 会把 text 转换成 tokens, 然后把该 node 原来在 AST 的 token list 中的 token 用这些新的 token 取代。node 及子 node 的结构并没有发生变化。</p>
<ol>
<li>把该 node 及子 node 的 nodeStatus 更新为 nsPartitial 或 nsDetached</li>
<li>更新和该 node 指向相同 startToken, endToken 父节点的 startToken, endToken</li>
<li>在 AST 的 token 链表中,把该 node 原有的 token 换成新的 token</li>
</ol>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span>
<span class="k">WHERE</span> <span class="n">e</span><span class="p">.</span><span class="n">job_id</span> <span class="o">=</span> <span class="mi">1</span>
</code></pre></div></div>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT *\n"</span> <span class="o">+</span>
<span class="s">"FROM scott.employee\n"</span> <span class="o">+</span>
<span class="s">"WHERE e.job_id = 1"</span><span class="o">;</span>
<span class="n">sqlparser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="nc">TSelectSqlStatement</span> <span class="n">select</span> <span class="o">=</span> <span class="o">(</span><span class="nc">TSelectSqlStatement</span><span class="o">)</span><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="nc">TWhereClause</span> <span class="n">whereClause</span> <span class="o">=</span> <span class="n">select</span><span class="o">.</span><span class="na">getWhereClause</span><span class="o">();</span>
<span class="n">whereClause</span><span class="o">.</span><span class="na">getCondition</span><span class="o">().</span><span class="na">setString</span><span class="o">(</span><span class="s">"e.salary > 1000"</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<p>执行上面Java代码后,SQL语句为:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="o">*</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span>
<span class="k">WHERE</span> <span class="n">e</span><span class="p">.</span><span class="n">salary</span> <span class="o">></span> <span class="mi">1000</span>
</code></pre></div></div>
<blockquote>
<p>如果需要删除该节点, 请在父节点中调用 setXXX(null) 方法。 调用 setString() 时传入一个长度为 0 的空字符串不起作用。</p>
</blockquote>
<blockquote>
<p>因为每种数据库的词法有差别,在把text转换成tokens时,需要明确是哪种数据库。为避免每次调用<code class="language-plaintext highlighter-rouge">setString()</code>时都额外指定数据库,引入一个静态全局变:<code class="language-plaintext highlighter-rouge">TGSqlParser.currentDBVendor</code>,当创建新的<code class="language-plaintext highlighter-rouge">TGSqlParser</code>实例时,设置<code class="language-plaintext highlighter-rouge">TGSqlParser.currentDBVendor</code>的值,该值总是和最近一次创建的<code class="language-plaintext highlighter-rouge">TGSqlParser</code>实例的数据库相同。如果想改变下一次<code class="language-plaintext highlighter-rouge">setString()</code>使用的数据库词法,可以更改该值。<strong>在多线程环境中这个设计可能导致问题</strong></p>
</blockquote>
<h4 id="22-删除-node">2.2 删除 node</h4>
<p>一 、 调用父节点 <strong>setXXX()</strong> 方法, 并且传入一个 null 参数,即把该 node 从父节点从删除。 GSP 的内部具体实现如下:</p>
<ol>
<li>调用 <code class="language-plaintext highlighter-rouge">TParseTreeNode.removeTokens()</code> 把对应的 tokens 从 AST 的 token list 中删除。</li>
<li>为保证 <code class="language-plaintext highlighter-rouge">toString()</code> 生成的 SQL 语法的正确,可能需要删除该 node 前后的一些辅助token。尤其是 <code class="language-plaintext highlighter-rouge">TParseTreeNodeList</code> 删除其中的某个元素时。</li>
<li>在父节点中把指向该 node 的引用设为 null。</li>
</ol>
<p>二、 <code class="language-plaintext highlighter-rouge">TParseTreeNodeList</code> 的子类节点移除其中的某个元素时,调用 <strong>removeItem(int index)</strong>, 它会自动调用 <code class="language-plaintext highlighter-rouge">removeAndSyncTokens(int index)</code>,如果被移除元素是 list 中的第一个节点,并且它之后有 comma token , 该 comma token 会被一起移除。
如果被移除元素不是 list 中的第一个节点,它之前有 comma token 时, 需要一起移除。</p>
<p>以该SQL为例</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">e</span><span class="p">.</span><span class="n">emp_id</span><span class="p">,</span><span class="n">e</span><span class="p">.</span><span class="n">fname</span><span class="p">,</span><span class="n">e</span><span class="p">.</span><span class="n">lname</span><span class="p">,</span><span class="n">j</span><span class="p">.</span><span class="n">job_desc</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span> <span class="k">AS</span> <span class="n">e</span><span class="p">,</span><span class="n">jobs</span> <span class="k">AS</span> <span class="n">j</span>
</code></pre></div></div>
<p>如果要从select list中删除<code class="language-plaintext highlighter-rouge">e.emp_id</code>, 则<code class="language-plaintext highlighter-rouge">e.emp_id</code>后面的<code class="language-plaintext highlighter-rouge">,</code>也必须一起删除。而删除<code class="language-plaintext highlighter-rouge">j.job_desc</code>时,则<code class="language-plaintext highlighter-rouge">j.job_desc</code>之前的<code class="language-plaintext highlighter-rouge">,</code>也必须一起删除。</p>
<h4 id="23-更新-node">2.3 更新 node</h4>
<p>调用父节点 <code class="language-plaintext highlighter-rouge">setXXX()</code> 方法设置新的 node。 <strong>这种情况一般建议使用原有 node 的 <code class="language-plaintext highlighter-rouge">setString()</code> 方法,效果是一样的,执行效率更高。</strong></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TGSqlParser</span> <span class="n">parser</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TGSqlParser</span><span class="o">(</span><span class="nc">EDbVendor</span><span class="o">.</span><span class="na">dbvoracle</span><span class="o">);</span>
<span class="n">parser</span><span class="o">.</span><span class="na">sqltext</span> <span class="o">=</span> <span class="s">"SELECT A.COLUMN1, B.COLUMN2 from TABLE1 A, TABLE2 B where A.COLUMN1=B.COLUMN1"</span><span class="o">;</span>
<span class="n">parser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="nc">TSelectSqlStatement</span> <span class="n">select</span> <span class="o">=</span> <span class="o">(</span><span class="nc">TSelectSqlStatement</span><span class="o">)</span><span class="n">parser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="mi">0</span><span class="o">);</span>
<span class="c1">//create a new node</span>
<span class="nc">TWhereClause</span> <span class="n">whereClause</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TWhereClause</span><span class="o">();</span>
<span class="n">whereClause</span><span class="o">.</span><span class="na">setText</span><span class="o">(</span><span class="s">"where a>2"</span><span class="o">);</span>
<span class="c1">//replace with the new created node</span>
<span class="n">select</span><span class="o">.</span><span class="na">setWhereClause</span><span class="o">(</span><span class="n">whereClause</span><span class="o">);</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span> <span class="o">(</span><span class="n">select</span><span class="o">.</span><span class="na">toString</span><span class="o">());</span>
</code></pre></div></div>
<h4 id="24-新增-node">2.4 新增 node</h4>
<p>父节点中原来指向该节点的指针为空, 新增 node 需要在父节点中调用对应的 <strong>setXXX()</strong> 方法。</p>
<p>例如这个 SELECT 语句, <code class="language-plaintext highlighter-rouge">TCustomSqlStatement.getWhereClause()</code> 是空的。</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span>
</code></pre></div></div>
<p>为给该语句增加 where clause, 我们可以这样:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">//create a new node</span>
<span class="nc">TWhereClause</span> <span class="n">whereClause</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TWhereClause</span><span class="o">();</span>
<span class="n">whereClause</span><span class="o">.</span><span class="na">setText</span><span class="o">(</span><span class="s">"where a>2"</span><span class="o">);</span>
<span class="c1">//link this new created node in the SELECT statement</span>
<span class="n">select</span><span class="o">.</span><span class="na">setWhereClause</span><span class="o">(</span><span class="n">whereClause</span><span class="o">);</span>
</code></pre></div></div>
<p>这样,我们就可以得到这个新的 SELECT 语句:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span> <span class="k">where</span> <span class="n">a</span><span class="o">></span><span class="mi">2</span>
</code></pre></div></div>
<p>增加node时,一般包含以下步骤:</p>
<ul>
<li>创建该node, <code class="language-plaintext highlighter-rouge">new TParseTreeNode()</code>, 然后调用 <code class="language-plaintext highlighter-rouge">TParseTreeNode.setString()</code> 设置该 node 的文本.</li>
<li>在父节点调用 <code class="language-plaintext highlighter-rouge">setXXX(TParseTreeNode)</code> 方法, 并传入 <code class="language-plaintext highlighter-rouge">TParseTreeNode</code> 参数.</li>
</ul>
<h5 id="确定插入位置">确定插入位置</h5>
<p>在 GSP 的内部, 需要在 AST 的 token list 中找到合适的位置 插入该 node 的 token。以上面的 SQL 为例, 需要找到 <code class="language-plaintext highlighter-rouge">emp</code> token, 然后在它后面把新的 token 插入。</p>
<p>我们以 TWhereClause 为例, 在 TCustomSqlStatement 中,</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">setWhereClause</span><span class="o">(</span><span class="nc">TWhereClause</span> <span class="n">newWhereClause</span><span class="o">)</span>
</code></pre></div></div>
<p>当使用这个方法时, token 的插入点默认为父节点的最后一个 token, 例如上面的 SQL 的例子。但有时,这种假设会产生不正确的结果。例如这个语句:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span> <span class="k">order</span> <span class="k">by</span> <span class="mi">1</span>
</code></pre></div></div>
<p>如果还是使用 <code class="language-plaintext highlighter-rouge">setWhereClause(TWhereClause newWhereClause)</code>, 那将产生下面错误的 SQL 语句。</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span> <span class="k">order</span> <span class="k">by</span> <span class="mi">1</span> <span class="k">where</span> <span class="n">a</span><span class="o">></span><span class="mi">2</span>
</code></pre></div></div>
<p>因为 SQL 语句的灵活性, GSP 无法自己辨别该把新 node 的 token 插入到哪个位置,因此,TParseTreeNode 提供这个方法,由调用者决定插入位置。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">void</span> <span class="nf">setAnchorNode</span><span class="o">(</span><span class="nc">TParseTreeNode</span> <span class="n">anchorNode</span><span class="o">)</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">TParseTreeNode anchorNode</code> 是和新 node 同级的node,并且在 AST 中已经存在。针对上例中的 SQL, 我们可以这样</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>select.setAnchorNode(select.joins);
select.setWhereClause(whereClause);
</code></pre></div></div>
<p>这样 where clause 为被插入到 anchor node: joins (即 from clause) 后, 得到以下正确的结果:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">emp_id</span><span class="p">,</span><span class="n">salary</span><span class="o">+</span><span class="mi">100</span> <span class="k">FROM</span> <span class="n">emp</span> <span class="k">where</span> <span class="n">a</span><span class="o">></span><span class="mi">2</span> <span class="k">order</span> <span class="k">by</span> <span class="mi">1</span>
</code></pre></div></div>
<h5 id="添加可能需要的辅助-token">添加可能需要的辅助 token</h5>
<ul>
<li>可能需要添加辅助token,以保证SQL语法的正确。(<strong>尚未有具体的实现</strong>)</li>
<li><code class="language-plaintext highlighter-rouge">TParseTreeNodeList.addElement(T ptn)</code> 插入子节点时,会对需要添加的辅助 token 做统一处理。</li>
</ul>
<p>以该SQL为例</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">e</span><span class="p">.</span><span class="n">emp_id</span><span class="p">,</span><span class="n">e</span><span class="p">.</span><span class="n">fname</span><span class="p">,</span><span class="n">e</span><span class="p">.</span><span class="n">lname</span>
<span class="k">FROM</span> <span class="n">scott</span><span class="p">.</span><span class="n">employee</span> <span class="k">AS</span> <span class="n">e</span><span class="p">,</span><span class="n">jobs</span> <span class="k">AS</span> <span class="n">j</span>
</code></pre></div></div>
<p>当在<code class="language-plaintext highlighter-rouge">e.lname</code>后加入<code class="language-plaintext highlighter-rouge">j.job_desc</code>时,必须在<code class="language-plaintext highlighter-rouge">j.job_desc</code>前同时加入<code class="language-plaintext highlighter-rouge">,</code>以确保语法正确。</p>
<h3 id="3gsp-中目前实现的对-ast-进行操作的-api">3、GSP 中目前实现的对 AST 进行操作的 API</h3>
<p>对 AST 进行的操作就是对 node 的新增、删除和更新(更新 node 自身,或更新 node 的文本)。</p>
<p>GSP API 已经完全支持 node 的删除和更新,但新增功能因为不同的 node 需要单独的 setXXX() 方法,需逐步添加支持,目前实现以下方法:</p>
<ol>
<li>TParseTreeNode.setString(String sqlSegment), 更新 node 文本。</li>
<li>TParseTreeNodeList.removeItem(int index), 删除 node</li>
<li>TCustomSqlStatement.setWhereClause()</li>
<li>所有 TSelectSqlStatement.setXXX() 方法</li>
</ol>
<h2 id="利用visitor来访问和修改node">利用visitor来访问和修改node</h2>
<p>利用 visitor 来找到指定类型的 node 是一种高效的方法。利用 visitor 遍历整颗语法树并对 node 进行操作时,需要注意以下几点:</p>
<ol>
<li>最小化原则,能够修改某个特定子节点,就不要修改整个父节点。同级节点的修改不会互相影响。</li>
<li>当用<code class="language-plaintext highlighter-rouge">setString()</code>修改某个节点后,它及其所有子节点都不再处于 <code class="language-plaintext highlighter-rouge">ENodeStatus.nsNormal</code> 状态,即不再属于整颗语法树,随后对这些子节点的改动也是无效的,不会反应在语法树中。</li>
<li>在 visitor 的 <code class="language-plaintext highlighter-rouge">postVisit()</code> 中处理节点时,可以保证先让子节点得到处理。</li>
<li>一个 visitor 可以根据实际业务需求,多次遍历同一个 node,处理不同的子节点。但要注意处理的节点必须处于 <code class="language-plaintext highlighter-rouge">ENodeStatus.nsNormal</code> 状态,否则改动不会反应到最后 <code class="language-plaintext highlighter-rouge">toString()</code> 的结果中。</li>
</ol>
<p>在把一颗代表Oracle的SELECT语句的<code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code>语法树转换成代表SQL Server的SELECT语句的<code class="language-plaintext highlighter-rouge">TSelectSqlStatement</code>时,我们可以采用上述方法。转换完成后,利用<code class="language-plaintext highlighter-rouge">toString()</code>就可以输出一个满足SQL Server语法的SELECT语句。</p>
<h2 id="输出修改-ast-后的-sql-语句-node-tostring">输出修改 AST 后的 SQL 语句: Node toString()</h2>
<p>增、删、改node时,node 的 token list 已经同步到整个 AST 中,那么,输出整个 SQL 语句的文本只要简单的
遍历 startToken 到 endToken 即可。</p>
<h3 id="node-toscript">Node toScript()</h3>
<p>从上面的介绍可知,利用<code class="language-plaintext highlighter-rouge">toString()</code>从语法树生成SQL文本时,对语法树上node做改动时,必须对底层对应的token做好同步。</p>
<p>利用<code class="language-plaintext highlighter-rouge">toScript()</code>从语法树生成SQL文本时,对语法树上node做改动,无需对底层对应的token做同步,但对语句中
的每一个node都要根据语法重新生成文本,即便这个node在本次操作中没有发生变化。由于GSP目前无法对所有的SQL
语法都支持重新生成文本,因此容易导致生成不正确的SQL文本。</p>
<p><code class="language-plaintext highlighter-rouge">toScript()</code>的优点在于改动语法树中的node时,无需同步更新底层的对应token,特别是一些辅助token。</p>
<h2 id="token的基本信息">Token的基本信息</h2>
<h4 id="1-token的类型">1. token的类型</h4>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nc">ETokenType</span> <span class="n">tokentype</span>
</code></pre></div></div>
<p>主要的类型有:</p>
<ol>
<li><code class="language-plaintext highlighter-rouge">ttkeyword</code>, 上例中的<code class="language-plaintext highlighter-rouge">SELECT</code>,<code class="language-plaintext highlighter-rouge">SELECT</code>,<code class="language-plaintext highlighter-rouge">FROM</code>,<code class="language-plaintext highlighter-rouge">WHERE</code>,<code class="language-plaintext highlighter-rouge">ORDER</code>,<code class="language-plaintext highlighter-rouge">BY</code>。</li>
<li><code class="language-plaintext highlighter-rouge">ttidentifier</code>,上例中的<code class="language-plaintext highlighter-rouge">e</code>,<code class="language-plaintext highlighter-rouge">emp_id</code>等。</li>
<li><code class="language-plaintext highlighter-rouge">ttwhitespace</code>, 空格和tab。</li>
<li><code class="language-plaintext highlighter-rouge">ttreturn</code>,换行符。</li>
<li>各种符号,<code class="language-plaintext highlighter-rouge">ttperiod</code>,<code class="language-plaintext highlighter-rouge">ttcomma</code>等。</li>
<li><code class="language-plaintext highlighter-rouge">ttsimplecomment</code>,<code class="language-plaintext highlighter-rouge">ttbracketedcomment</code>,注释。</li>
</ol>
<h4 id="2-token的code">2. token的code</h4>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">int</span> <span class="n">tokencode</span>
</code></pre></div></div>
<p>code用来表示token的编号。<code class="language-plaintext highlighter-rouge">ttkeyword</code>类型的token有唯一不同的编号。<code class="language-plaintext highlighter-rouge">ttidentifier</code>类型的token编号值相同,都为<code class="language-plaintext highlighter-rouge">264</code>。各种符号的编号就是它们的ASCII值。</p>
<h4 id="3-token的text">3. token的text</h4>
<p>token的文本。</p>
<h2 id="node">Node</h2>
<p>Node表示SQL语法中的各个元素,例如</p>
<ol>
<li>数据库对象名,<code class="language-plaintext highlighter-rouge">e.emp_id</code>,它包含三个token<code class="language-plaintext highlighter-rouge">e</code>,<code class="language-plaintext highlighter-rouge">.</code>,<code class="language-plaintext highlighter-rouge">emp_id</code>。</li>
<li>也可以是一个子句(clause),例如where子句,<code class="language-plaintext highlighter-rouge">WHERE e.job_id = j.job_id</code>,它包含<code class="language-plaintext highlighter-rouge">ttkeyword</code>,<code class="language-plaintext highlighter-rouge">ttwhitespace</code>,符号,<code class="language-plaintext highlighter-rouge">ttidentifier</code>等token。</li>
<li>也可以是一个语句,例如SELECT,包含各种SQL子句。</li>
</ol>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}A general review/summary of General SQL Parser: sql parser tree node and underlying tokensIterator interface implmented in TParseTreeNode and Iterable interface implmented in TParseTreeNodeList2020-08-20T00:00:00+00:002020-08-20T00:00:00+00:00http://support.sqlparser.com/gsp-api/gsp-sql-parse-tree-node-iterator<p>Iterator interface implmented in TParseTreeNode is used to iterates the all source tokens of the parse tree node.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">abstract</span> <span class="kd">class</span> <span class="nc">TParseTreeNode</span> <span class="kd">implements</span> <span class="nc">Visitable</span><span class="o">,</span><span class="nc">Iterator</span><span class="o"><</span><span class="nc">TSourceToken</span><span class="o">></span>
</code></pre></div></div>
<p>Iterable interface implmented in TParseTreeNodeList is used to iterates all the parse tree nodes included in this list.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TParseTreeNodeList</span><span class="o"><</span><span class="no">T</span> <span class="kd">extends</span> <span class="nc">TParseTreeNode</span><span class="o">></span> <span class="kd">extends</span> <span class="nc">TParseTreeNode</span> <span class="kd">implements</span> <span class="nc">Iterable</span><span class="o"><</span><span class="no">T</span><span class="o">></span>
</code></pre></div></div>
<p>Iterable interface implmented in TStatementList is used to iterates the all the sql statements included in this list.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">class</span> <span class="nc">TStatementList</span> <span class="kd">extends</span> <span class="nc">TParseTreeNode</span> <span class="kd">implements</span> <span class="nc">Iterable</span><span class="o"><</span><span class="nc">TCustomSqlStatement</span><span class="o">></span>
</code></pre></div></div>
<p>Let take this SQL for example:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">e</span><span class="p">.</span><span class="n">employee_id</span><span class="p">,</span>
<span class="n">e</span><span class="p">.</span><span class="n">last_name</span><span class="p">,</span>
<span class="n">e</span><span class="p">.</span><span class="n">department_id</span>
<span class="k">FROM</span> <span class="n">employees</span> <span class="n">e</span><span class="p">,</span>
<span class="n">departments</span> <span class="n">d</span>
<span class="p">;</span>
<span class="k">SELECT</span> <span class="n">e</span><span class="p">.</span><span class="n">employee_id</span><span class="p">,</span>
<span class="n">e</span><span class="p">.</span><span class="n">last_name</span><span class="p">,</span>
<span class="n">e</span><span class="p">.</span><span class="n">department_id</span>
<span class="k">FROM</span> <span class="n">employees</span> <span class="n">e</span>
<span class="k">JOIN</span> <span class="n">departments</span> <span class="n">d</span>
<span class="k">ON</span> <span class="n">e</span><span class="p">.</span><span class="n">department_id</span> <span class="o">=</span> <span class="n">d</span><span class="p">.</span><span class="n">department_id</span>
</code></pre></div></div>
<p>Print the type of all sql statements:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span> <span class="o">(</span><span class="nc">TCustomSqlStatement</span> <span class="nl">sqlStatement:</span><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">)</span> <span class="o">{</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">sqlStatement</span><span class="o">.</span><span class="na">sqlstatementtype</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Since TParseTreeNodeList is subclass of TParseTreeNode, so TParseTreeNodeList support both Iterable and Iterator interface.
Please aware that Iterator interface is used to get all source tokens belong to this node like this:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">while</span><span class="o">(</span><span class="n">sqlStatement</span><span class="o">.</span><span class="na">tables</span><span class="o">.</span><span class="na">hasNext</span><span class="o">()){</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">sqlStatement</span><span class="o">.</span><span class="na">tables</span><span class="o">.</span><span class="na">next</span><span class="o">().</span><span class="na">toString</span><span class="o">());</span>
<span class="o">}</span>
</code></pre></div></div>
<p>While Iterable interface is used to get parse tree node in the list:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">for</span><span class="o">(</span><span class="nc">TTable</span> <span class="nl">table:</span><span class="n">sqlStatement</span><span class="o">.</span><span class="na">tables</span><span class="o">){</span>
<span class="nc">System</span><span class="o">.</span><span class="na">out</span><span class="o">.</span><span class="na">println</span><span class="o">(</span><span class="n">table</span><span class="o">.</span><span class="na">getTableType</span><span class="o">());</span>
<span class="o">}</span>
</code></pre></div></div>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}Iterator interface implmented in TParseTreeNode is used to iterates the all source tokens of the parse tree node. public abstract class TParseTreeNode implements Visitable,Iterator<TSourceToken> Iterable interface implmented in TParseTreeNodeList is used to iterates all the parse tree nodes included in this list. public class TParseTreeNodeList<T extends TParseTreeNode> extends TParseTreeNode implements Iterable<T> Iterable interface implmented in TStatementList is used to iterates the all the sql statements included in this list. public class TStatementList extends TParseTreeNode implements Iterable<TCustomSqlStatement> Let take this SQL for example: SELECT e.employee_id, e.last_name, e.department_id FROM employees e, departments d ; SELECT e.employee_id, e.last_name, e.department_id FROM employees e JOIN departments d ON e.department_id = d.department_id Print the type of all sql statements: for (TCustomSqlStatement sqlStatement:sqlparser.sqlstatements) { System.out.println(sqlStatement.sqlstatementtype); } Since TParseTreeNodeList is subclass of TParseTreeNode, so TParseTreeNodeList support both Iterable and Iterator interface. Please aware that Iterator interface is used to get all source tokens belong to this node like this: while(sqlStatement.tables.hasNext()){ System.out.println(sqlStatement.tables.next().toString()); } While Iterable interface is used to get parse tree node in the list: for(TTable table:sqlStatement.tables){ System.out.println(table.getTableType()); }General SQL Parser and SQLFrog2020-07-15T00:00:00+00:002020-07-15T00:00:00+00:00http://support.sqlparser.com/sql-syntax/get-started-cn/gsp-sqlfrog<p>GSP在SQLFrog项目中的应用</p>
<h3 id="sqlfrog的两种工作模式">SQLFrog的两种工作模式</h3>
<ol>
<li>scan模式,仅找出需要转换的SQL语法和语义,给出报告,不做转换。</li>
<li>convert模式,找出需要转换的SQL语法和语义,并做转换。</li>
</ol>
<p>scan为默认模式。</p>
<h3 id="sqlfrog和gsp的关系">SQLFrog和GSP的关系</h3>
<p>SQLFrog的底层实现依赖GSP的解析能力、visitor模式、语法树改动、语法树到SQL文本的生成技术。</p>
<h3 id="使用gsp的visitor模式">使用GSP的visitor模式</h3>
<p>顶层SQL语句应用某种类型node的visitor后,可以快速高效的访问语句中所有该类型的node。</p>
<p>下面这段代码示例访问所有<code class="language-plaintext highlighter-rouge">TObjectName</code>类型的node。在同一个visitor中,我们可以同时处理多个类型的node,根据实际的业务需求决定。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kt">int</span> <span class="n">ret</span> <span class="o">=</span> <span class="n">sqlparser</span><span class="o">.</span><span class="na">parse</span><span class="o">();</span>
<span class="k">if</span> <span class="o">(</span><span class="n">ret</span> <span class="o">==</span> <span class="mi">0</span><span class="o">){</span>
<span class="n">objectNameVisitor</span> <span class="n">objectNameVisitor</span> <span class="o">=</span> <span class="k">new</span> <span class="n">objectNameVisitor</span><span class="o">();</span>
<span class="k">for</span><span class="o">(</span><span class="kt">int</span> <span class="n">i</span><span class="o">=</span><span class="mi">0</span><span class="o">;</span><span class="n">i</span><span class="o"><</span><span class="n">sqlparser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">size</span><span class="o">();</span><span class="n">i</span><span class="o">++){</span>
<span class="nc">TCustomSqlStatement</span> <span class="n">sqlStatement</span> <span class="o">=</span> <span class="n">sqlparser</span><span class="o">.</span><span class="na">sqlstatements</span><span class="o">.</span><span class="na">get</span><span class="o">(</span><span class="n">i</span><span class="o">);</span>
<span class="n">sqlStatement</span><span class="o">.</span><span class="na">acceptChildren</span><span class="o">(</span><span class="n">objectNameVisitor</span><span class="o">);</span>
<span class="o">}</span>
<span class="o">}</span>
<span class="kd">class</span> <span class="nc">objectNameVisitor</span> <span class="kd">extends</span> <span class="nc">TParseTreeVisitor</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">preVisit</span><span class="o">(</span><span class="nc">TObjectName</span> <span class="n">node</span><span class="o">){</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p><strong>GSP的visitor对所有node的深度访问可能会有遗漏,在开发中遇到此类问题需及时反馈。</strong></p>
<h4 id="利用visitor来进行sql语句中datatype的检查">利用visitor来进行SQL语句中datatype的检查</h4>
<p>例如,在netezza到snowflake的SQL转换过程中,我们需要检查datatype是否兼容,当发现create table语句中有使用ST_GEOMETRY datatype时, 我们就要标记出该datatype 需要被转换成snowflake的VARBINARY.</p>
<p>创建一个datatype visitor就非常容易实现以上功能。</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">class</span> <span class="nc">datatypeVisitor</span> <span class="kd">extends</span> <span class="nc">TParseTreeVisitor</span> <span class="o">{</span>
<span class="kd">public</span> <span class="kt">void</span> <span class="nf">preVisit</span><span class="o">(</span><span class="nc">TTypeName</span> <span class="n">node</span><span class="o">){</span>
<span class="c1">// 加入功能检查代码</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>类似的,我们可以对<strong>SQL函数</strong>进行检查。</p>
<h3 id="visitor配合gsp的语法树改动技术进行sql转换">visitor配合GSP的语法树改动技术,进行SQL转换</h3>
<p>当找到需要转换的语法或语义点后,需要进行转换,通过修改GSP生成的SQL语法树,我们可以做到这一点。GSP提供两种方法可以从语法树生成SQL文本:<code class="language-plaintext highlighter-rouge">toString()</code> and <code class="language-plaintext highlighter-rouge">toScript()</code>。</p>
<h4 id="tostring">toString()</h4>
<p>利用<code class="language-plaintext highlighter-rouge">toString()</code>从语法树生成SQL文本时,要求对语法树上node做改动时,必须对底层对应的token做好同步。
<strong>SQLFrog采用这种方法</strong>。</p>
<h4 id="toscript">toScript()</h4>
<p>利用<code class="language-plaintext highlighter-rouge">toScript()</code>从语法树生成SQL文本时,对语法树上node做改动,无需对底层对应的token做同步,但对语句中
的每一个node都要根据语法树重新生成文本,即便这个node在本次操作中没有发生变化。由于GSP目前无法对所有的SQL
语法都支持重新生成文本,因此容易导致生成不正确的SQL文本。</p>
<p>详细的说明可以看<a href="http://support.sqlparser.com/gsp-sql-parse-tree-to-query-cn.html">这篇文章</a>。
<em>还需要补充一篇文档对<code class="language-plaintext highlighter-rouge">toString()</code> and <code class="language-plaintext highlighter-rouge">toScript()</code>的工作原理做进一步的说明。</em></p>
<h3 id="visitor相关代码">visitor相关代码</h3>
<p>https://github.com/sqlparser/gsp_demo_java/tree/master/src/main/java/demos/visitors</p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}GSP在SQLFrog项目中的应用 SQLFrog的两种工作模式 scan模式,仅找出需要转换的SQL语法和语义,给出报告,不做转换。 convert模式,找出需要转换的SQL语法和语义,并做转换。 scan为默认模式。 SQLFrog和GSP的关系 SQLFrog的底层实现依赖GSP的解析能力、visitor模式、语法树改动、语法树到SQL文本的生成技术。 使用GSP的visitor模式 顶层SQL语句应用某种类型node的visitor后,可以快速高效的访问语句中所有该类型的node。 下面这段代码示例访问所有TObjectName类型的node。在同一个visitor中,我们可以同时处理多个类型的node,根据实际的业务需求决定。 int ret = sqlparser.parse(); if (ret == 0){ objectNameVisitor objectNameVisitor = new objectNameVisitor(); for(int i=0;i<sqlparser.sqlstatements.size();i++){ TCustomSqlStatement sqlStatement = sqlparser.sqlstatements.get(i); sqlStatement.acceptChildren(objectNameVisitor); } } class objectNameVisitor extends TParseTreeVisitor { public void preVisit(TObjectName node){ } } GSP的visitor对所有node的深度访问可能会有遗漏,在开发中遇到此类问题需及时反馈。 利用visitor来进行SQL语句中datatype的检查 例如,在netezza到snowflake的SQL转换过程中,我们需要检查datatype是否兼容,当发现create table语句中有使用ST_GEOMETRY datatype时, 我们就要标记出该datatype 需要被转换成snowflake的VARBINARY. 创建一个datatype visitor就非常容易实现以上功能。 class datatypeVisitor extends TParseTreeVisitor { public void preVisit(TTypeName node){ // 加入功能检查代码 } } 类似的,我们可以对SQL函数进行检查。 visitor配合GSP的语法树改动技术,进行SQL转换 当找到需要转换的语法或语义点后,需要进行转换,通过修改GSP生成的SQL语法树,我们可以做到这一点。GSP提供两种方法可以从语法树生成SQL文本:toString() and toScript()。 toString() 利用toString()从语法树生成SQL文本时,要求对语法树上node做改动时,必须对底层对应的token做好同步。 SQLFrog采用这种方法。 toScript() 利用toScript()从语法树生成SQL文本时,对语法树上node做改动,无需对底层对应的token做同步,但对语句中 的每一个node都要根据语法树重新生成文本,即便这个node在本次操作中没有发生变化。由于GSP目前无法对所有的SQL 语法都支持重新生成文本,因此容易导致生成不正确的SQL文本。 详细的说明可以看这篇文章。 还需要补充一篇文档对toString() and toScript()的工作原理做进一步的说明。 visitor相关代码 https://github.com/sqlparser/gsp_demo_java/tree/master/src/main/java/demos/visitorsSQL Function and TFunctionCall2020-07-13T00:00:00+00:002020-07-13T00:00:00+00:00http://support.sqlparser.com/sql-syntax/get-started-cn/sql-function-tfunctioncall<p>SQL function 在GSP中用TFunctionCall类表示。所有的function都用这个类表示。</p>
<h3 id="tfunctioncall中的基本信息">TFunctionCall中的基本信息</h3>
<p>一般的SQL function的语法如下:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">funcName</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span><span class="n">arg2</span><span class="p">)</span>
</code></pre></div></div>
<p>TFunctionCall中对应属性值:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>getFunctionName() = funcName
getArgs().size() = 2
getArgs().getExpression(0) = arg1
getArgs().getExpression(1) = arg2
getFunctionType() = unknown_t
</code></pre></div></div>
<p>目前 <code class="language-plaintext highlighter-rouge">getFunctionType()</code> 表示的函数类型<strong>并不完善</strong>,用它来判断函数并不一定准确,需小心。</p>
<h3 id="不规则参数的函数">不规则参数的函数</h3>
<p>一般情况下,函数的参数由<code class="language-plaintext highlighter-rouge">TExpressionList getArgs()</code>获得,这些函数的参数形如:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">funcName</span><span class="p">(</span><span class="n">arg1</span><span class="p">,</span><span class="n">arg2</span><span class="p">,</span><span class="n">arg3</span><span class="p">)</span>
</code></pre></div></div>
<p>其中,<code class="language-plaintext highlighter-rouge">arg1</code>,<code class="language-plaintext highlighter-rouge">arg2</code>,<code class="language-plaintext highlighter-rouge">arg3</code>的类型都是表达式:<code class="language-plaintext highlighter-rouge">TExpression</code>。</p>
<p>但有一些函数的参数并不能仅仅由表达式来表示,例如<code class="language-plaintext highlighter-rouge">cast</code>函数,</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="k">CAST</span><span class="p">(</span><span class="n">ytd_sales</span> <span class="k">AS</span> <span class="nb">CHAR</span><span class="p">(</span><span class="mi">5</span><span class="p">))</span> <span class="k">FROM</span> <span class="n">titles</span>
</code></pre></div></div>
<p>除了<code class="language-plaintext highlighter-rouge">ytd_sales</code>可以用<code class="language-plaintext highlighter-rouge">TExpression</code>,还有<code class="language-plaintext highlighter-rouge">AS</code>关键字和<code class="language-plaintext highlighter-rouge">CHAR(5)</code>数据类型,所以<code class="language-plaintext highlighter-rouge">cast</code>函数
的参数不能用<code class="language-plaintext highlighter-rouge">TExpressionList getArgs()</code>获得,它对应的参数分别为:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">getExpr1</span><span class="o">()</span> <span class="o">=</span> <span class="n">ytd_sales</span>
<span class="nf">getTypename</span><span class="o">()</span> <span class="o">=</span> <span class="no">CHAR</span><span class="o">(</span><span class="mi">5</span><span class="o">)</span>
</code></pre></div></div>
<p>函数的参数语法上的多样性,导致获取参数的API方式的不统一。以后API上可能需要改进以保证获取参数方法统一。</p>
<p><em>其它参数不规则的函数另行补充.</em></p>
<h3 id="判断一个函数是否为数据库的内置函数built-in-funciton">判断一个函数是否为数据库的内置函数(built-in funciton)</h3>
<ol>
<li>判断该函数是否为某一数据库的内置函数。<code class="language-plaintext highlighter-rouge">EDbVendor</code> 用来指定数据库厂商,例如<code class="language-plaintext highlighter-rouge">db2</code>,<code class="language-plaintext highlighter-rouge">oracle</code>等。
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kt">boolean</span> <span class="nf">isBuiltIn</span><span class="o">(</span><span class="nc">EDbVendor</span> <span class="n">pDBVendor</span><span class="o">)</span>
</code></pre></div> </div>
</li>
<li>静态函数,需指定函数名。功能同1.
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="kd">static</span> <span class="kt">boolean</span> <span class="nf">isBuiltIn</span><span class="o">(</span><span class="nc">String</span> <span class="n">pName</span><span class="o">,</span> <span class="nc">EDbVendor</span> <span class="n">pDBVendor</span><span class="o">)</span>
</code></pre></div> </div>
</li>
</ol>
<h3 id="aggregate-function">aggregate function</h3>
<h5 id="1-all--distinct">1. ALL | DISTINCT</h5>
<p><code class="language-plaintext highlighter-rouge">getAggregateType()</code> is used to determine the <code class="language-plaintext highlighter-rouge">ALL | DISTINCT</code> used in the aggregate function.</p>
<h5 id="2-within-group">2. WITHIN GROUP</h5>
<p><em>This information is not available in the TFunctionCall yet.</em></p>
<h3 id="window-function">window function</h3>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">FUNCTION_NAME</span><span class="p">(</span><span class="n">expr</span><span class="p">)</span> <span class="n">OVER</span> <span class="err">{</span><span class="n">window_name</span> <span class="o">|</span> <span class="p">(</span><span class="n">window_specification</span><span class="p">)</span><span class="err">}</span>
</code></pre></div></div>
<p>在Oracle and SQL Server中,<code class="language-plaintext highlighter-rouge">window_specification</code>也称为<code class="language-plaintext highlighter-rouge">window_clause</code>。</p>
<p>在GSP中,用<code class="language-plaintext highlighter-rouge">TWindowDef</code>表示<code class="language-plaintext highlighter-rouge">window_specification</code>, 在<code class="language-plaintext highlighter-rouge">TFunctionCall</code>中,用以下方法获得<code class="language-plaintext highlighter-rouge">window_specification</code></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">public</span> <span class="nc">TWindowDef</span> <span class="nf">getWindowDef</span><span class="o">()</span>
</code></pre></div></div>
<p>以这个包含window function的SQL为例:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">SELECT</span> <span class="n">NUM</span><span class="p">,</span> <span class="n">ODD</span><span class="p">,</span>
<span class="n">CUME_DIST</span><span class="p">(</span> <span class="p">)</span> <span class="n">OVER</span><span class="p">(</span><span class="n">PARTITION</span> <span class="k">BY</span> <span class="n">ODD</span> <span class="k">ORDER</span> <span class="k">BY</span> <span class="n">NUM</span><span class="p">)</span> <span class="n">cumedist</span>
<span class="k">FROM</span> <span class="n">test4</span>
</code></pre></div></div>
<p>GSP的输出为:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sstselect
--> function: CUME_DIST, type:unknown_t
window_specification
Parition value: ODD
Order by clause: NUM
</code></pre></div></div>
<h4 id="case-function">CASE FUNCTION</h4>
<p>用<code class="language-plaintext highlighter-rouge">TCaseExpression</code>表示。</p>
<h3 id="related-demo">Related demo</h3>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/main/java/demos/visitors/searchFunction.java">search SQL Function</a></p>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}SQL function 在GSP中用TFunctionCall类表示。所有的function都用这个类表示。 TFunctionCall中的基本信息 一般的SQL function的语法如下: funcName(arg1,arg2) TFunctionCall中对应属性值: getFunctionName() = funcName getArgs().size() = 2 getArgs().getExpression(0) = arg1 getArgs().getExpression(1) = arg2 getFunctionType() = unknown_t 目前 getFunctionType() 表示的函数类型并不完善,用它来判断函数并不一定准确,需小心。 不规则参数的函数 一般情况下,函数的参数由TExpressionList getArgs()获得,这些函数的参数形如: funcName(arg1,arg2,arg3) 其中,arg1,arg2,arg3的类型都是表达式:TExpression。 但有一些函数的参数并不能仅仅由表达式来表示,例如cast函数, SELECT CAST(ytd_sales AS CHAR(5)) FROM titles 除了ytd_sales可以用TExpression,还有AS关键字和CHAR(5)数据类型,所以cast函数 的参数不能用TExpressionList getArgs()获得,它对应的参数分别为: getExpr1() = ytd_sales getTypename() = CHAR(5) 函数的参数语法上的多样性,导致获取参数的API方式的不统一。以后API上可能需要改进以保证获取参数方法统一。 其它参数不规则的函数另行补充. 判断一个函数是否为数据库的内置函数(built-in funciton) 判断该函数是否为某一数据库的内置函数。EDbVendor 用来指定数据库厂商,例如db2,oracle等。 public boolean isBuiltIn(EDbVendor pDBVendor) 静态函数,需指定函数名。功能同1. public static boolean isBuiltIn(String pName, EDbVendor pDBVendor) aggregate function 1. ALL | DISTINCT getAggregateType() is used to determine the ALL | DISTINCT used in the aggregate function. 2. WITHIN GROUP This information is not available in the TFunctionCall yet. window function FUNCTION_NAME(expr) OVER {window_name | (window_specification)} 在Oracle and SQL Server中,window_specification也称为window_clause。 在GSP中,用TWindowDef表示window_specification, 在TFunctionCall中,用以下方法获得window_specification public TWindowDef getWindowDef() 以这个包含window function的SQL为例: SELECT NUM, ODD, CUME_DIST( ) OVER(PARTITION BY ODD ORDER BY NUM) cumedist FROM test4 GSP的输出为: sstselect --> function: CUME_DIST, type:unknown_t window_specification Parition value: ODD Order by clause: NUM CASE FUNCTION 用TCaseExpression表示。 Related demo search SQL FunctionSQL Datatype and TTypeName2020-07-06T00:00:00+00:002020-07-06T00:00:00+00:00http://support.sqlparser.com/sql-syntax/get-started-cn/sql-datatype-ttypename<p>SQL Datatype and TTypeName</p>
<p>SQL的数据类型,GSP中的对应类:TTypeName。</p>
<h3 id="sql-datatype的类型">SQL Datatype的类型</h3>
<p>TTypeName 表示SQL中的数据类型,例如:</p>
<div class="language-sql highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">char</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span>
<span class="nb">int</span><span class="p">,</span>
<span class="nb">float</span><span class="p">(</span><span class="mi">24</span><span class="p">),</span>
<span class="nb">decimal</span><span class="p">(</span><span class="mi">8</span><span class="p">,</span><span class="mi">2</span><span class="p">)</span>
</code></pre></div></div>
<h4 id="基本属性">基本属性</h4>
<p>以表示 <code class="language-plaintext highlighter-rouge">decimal(8,2)</code>为例,TTypeName的基本熟悉值如下:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>getDataType() = decimal_t
toString() = decimal(8,2)
getDataTypeName() = decimal
</code></pre></div></div>
<h4 id="扩展属性">扩展属性</h4>
<p>扩展属性仅适用于部分特定的datatype。</p>
<h5 id="getlength">getLength()</h5>
<p>以 <code class="language-plaintext highlighter-rouge">char(10)</code>为例</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>getLength() = 10
</code></pre></div></div>
<h5 id="getprecision-getscale">getPrecision(), getScale()</h5>
<p>以 <code class="language-plaintext highlighter-rouge">decimal(8,2)</code>为例</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>getPrecision() = 8
getScale() = 2
</code></pre></div></div>
<h3 id="参考资料">参考资料</h3>
<p>SQL2003 datatypes 的详细列表见 “SQL in a Nutshell, 3rd Edition” p30, Table 2-8. SQL2003 categories and datatypes</p>
<h3 id="related-demo">Related demo</h3>
<p><a href="https://github.com/sqlparser/gsp_demo_java/blob/master/src/main/java/demos/visitors/searchDatatype.java">search SQL Datatype</a></p>
<h3 id="edatatype的完整列表">EDataType的完整列表</h3>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="nn">gudusoft.gsqlparser</span><span class="o">;</span>
<span class="cm">/**
* @since v1.4.3.0
*/</span>
<span class="kd">public</span> <span class="kd">enum</span> <span class="nc">EDataType</span> <span class="o">{</span>
<span class="n">unknown_t</span><span class="o">,</span>
<span class="cm">/**
* user defined datetype
*/</span>
<span class="n">generic_t</span><span class="o">,</span>
<span class="n">bfile_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: bigint
* postgresql
*/</span>
<span class="n">bigint_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: blob
*/</span>
<span class="n">binary_t</span><span class="o">,</span>
<span class="n">binary_float_t</span><span class="o">,</span>
<span class="n">binary_double_t</span><span class="o">,</span>
<span class="cm">/**
* plsql binary_integer
*/</span>
<span class="n">binary_integer_t</span><span class="o">,</span>
<span class="cm">/**
* binary large object
* Databases: DB2, teradata
*/</span>
<span class="n">binary_large_object_t</span><span class="o">,</span>
<span class="n">bit_t</span><span class="o">,</span>
<span class="n">bit_varying_t</span><span class="o">,</span> <span class="c1">// = varbit</span>
<span class="n">blob_t</span><span class="o">,</span>
<span class="cm">/**
* bool, boolean, ansi2003: boolean
*/</span>
<span class="n">bool_t</span><span class="o">,</span>
<span class="n">box_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: byte
*/</span>
<span class="n">byte_t</span><span class="o">,</span>
<span class="n">bytea_t</span><span class="o">,</span> <span class="c1">//ansi2003 blob</span>
<span class="cm">/**
* teradata byteint
*/</span>
<span class="n">byteint_t</span><span class="o">,</span>
<span class="cm">/**
* char, character, ansi2003: character
*/</span>
<span class="n">character_t</span><span class="o">,</span>
<span class="n">char_t</span><span class="o">,</span>
<span class="n">char_for_bit_data_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: character large object
*/</span>
<span class="n">char_large_object_t</span><span class="o">,</span>
<span class="n">cidr_t</span><span class="o">,</span>
<span class="n">circle_t</span><span class="o">,</span>
<span class="n">clob_t</span><span class="o">,</span>
<span class="n">cursor_t</span><span class="o">,</span>
<span class="n">datalink_t</span><span class="o">,</span>
<span class="n">date_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: timestamp
*/</span>
<span class="n">datetime_t</span><span class="o">,</span>
<span class="n">datetimeoffset_t</span><span class="o">,</span><span class="c1">// ansi2003: timestamp</span>
<span class="n">datetime2_t</span><span class="o">,</span> <span class="c1">// ansi2003: timestamp with time zone</span>
<span class="cm">/**
* ansi2003: nclob
* Databases: DB2
*/</span>
<span class="n">dbclob_t</span><span class="o">,</span>
<span class="cm">/**
* dec,decimal, ansi2003: decimal
*/</span>
<span class="n">decimal_t</span><span class="o">,</span>
<span class="n">dec_t</span><span class="o">,</span>
<span class="cm">/**
* double, double precision, ansi2003: float
*/</span>
<span class="n">double_t</span><span class="o">,</span>
<span class="n">enum_t</span><span class="o">,</span>
<span class="n">float_t</span><span class="o">,</span><span class="c1">// ansi2003: double precision</span>
<span class="n">float4_t</span><span class="o">,</span><span class="c1">// ansi2003: float(p)</span>
<span class="n">float8_t</span><span class="o">,</span> <span class="c1">// ansi2003 float(p)</span>
<span class="cm">/**
* ansi2003 blob
*/</span>
<span class="n">graphic_t</span><span class="o">,</span>
<span class="n">geography_t</span><span class="o">,</span>
<span class="n">geometry_t</span><span class="o">,</span>
<span class="n">hierarchyid_t</span><span class="o">,</span>
<span class="n">image_t</span><span class="o">,</span>
<span class="n">inet_t</span><span class="o">,</span>
<span class="cm">/**
* int, integer, ansi2003: integer
*/</span>
<span class="n">integer_t</span><span class="o">,</span>
<span class="n">int_t</span><span class="o">,</span>
<span class="n">int2_t</span><span class="o">,</span> <span class="c1">// ansi2003: smallint</span>
<span class="n">int4_t</span><span class="o">,</span> <span class="c1">// ansi2003: int, integer</span>
<span class="cm">/**
* Postgresql
*/</span>
<span class="n">interval_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval day
*/</span>
<span class="n">interval_day_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval day to hour
*/</span>
<span class="n">interval_day_to_hour_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval day to minute
*/</span>
<span class="n">interval_day_to_minute_t</span><span class="o">,</span>
<span class="n">interval_day_to_second_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval hour
*/</span>
<span class="n">interval_hour_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval hour to minute
*/</span>
<span class="n">interval_hour_to_minute_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval hour to second
*/</span>
<span class="n">interval_hour_to_second_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval minute
*/</span>
<span class="n">interval_minute_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval minute to second
*/</span>
<span class="n">interval_minute_to_second_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: interval month
*/</span>
<span class="n">interval_month_t</span><span class="o">,</span>
<span class="cm">/**
* teradata:interval second
*/</span>
<span class="n">interval_second_t</span><span class="o">,</span>
<span class="cm">/**
* teradata interval year.
*/</span>
<span class="n">interval_year_t</span><span class="o">,</span>
<span class="n">interval_year_to_month_t</span><span class="o">,</span>
<span class="n">line_t</span><span class="o">,</span>
<span class="n">long_t</span><span class="o">,</span>
<span class="n">long_varchar_t</span><span class="o">,</span>
<span class="cm">/**
* long varbinary, mysql
* MySQL Connector/ODBC defines BLOB values as LONGVARBINARY and TEXT values as LONGVARCHAR.
*/</span>
<span class="n">long_varbinary_t</span><span class="o">,</span>
<span class="n">longblob_t</span><span class="o">,</span> <span class="c1">// ansi2003: blob</span>
<span class="cm">/**
* ansi2003: blob
*/</span>
<span class="n">long_raw_t</span><span class="o">,</span>
<span class="n">long_vargraphic_t</span><span class="o">,</span>
<span class="n">longtext_t</span><span class="o">,</span>
<span class="n">lseg_t</span><span class="o">,</span>
<span class="n">macaddr_t</span><span class="o">,</span>
<span class="n">mediumblob_t</span><span class="o">,</span>
<span class="cm">/**
* mediumint, middleint(MySQL) , ansi2003: int
*/</span>
<span class="n">mediumint_t</span><span class="o">,</span>
<span class="n">mediumtext_t</span><span class="o">,</span>
<span class="n">money_t</span><span class="o">,</span> <span class="c1">// = decimal(9,2),INFORMIX</span>
<span class="cm">/**
* national_char_varying,nchar_varying,nvarchar, ansi2003: national character varying
*/</span>
<span class="n">nvarchar_t</span><span class="o">,</span>
<span class="cm">/**
* nchar, national char, national character,ansi2003: national character
*/</span>
<span class="n">nchar_t</span><span class="o">,</span>
<span class="n">ncharacter_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: nclob
*/</span>
<span class="n">nclob_t</span><span class="o">,</span>
<span class="cm">/**
* ntext, national text, ansi2003: nclob
*/</span>
<span class="n">ntext_t</span><span class="o">,</span>
<span class="cm">/**
* nvarchar2(n)
*/</span>
<span class="n">nvarchar2_t</span><span class="o">,</span>
<span class="cm">/**
* number, num
*/</span>
<span class="n">number_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: numeric
*/</span>
<span class="n">numeric_t</span><span class="o">,</span>
<span class="n">oid_t</span><span class="o">,</span>
<span class="n">path_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: period(n)
*/</span>
<span class="n">period_t</span><span class="o">,</span>
<span class="cm">/**
* plsql pls_integer
*/</span>
<span class="n">pls_integer_t</span><span class="o">,</span>
<span class="n">point_t</span><span class="o">,</span>
<span class="n">polygon_t</span><span class="o">,</span>
<span class="n">raw_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: real
*/</span>
<span class="n">real_t</span><span class="o">,</span>
<span class="n">rowid_t</span><span class="o">,</span>
<span class="n">rowversion_t</span><span class="o">,</span>
<span class="n">serial_t</span><span class="o">,</span><span class="c1">// = serial4</span>
<span class="n">serial8_t</span><span class="o">,</span><span class="c1">// = bigserial</span>
<span class="n">bigserial_t</span><span class="o">,</span><span class="c1">//informix</span>
<span class="n">smallfloat_t</span><span class="o">,</span><span class="c1">//informix</span>
<span class="cm">/**
* MySQL: set
*/</span>
<span class="n">set_t</span><span class="o">,</span>
<span class="n">smalldatetime_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: smallint
*/</span>
<span class="n">smallint_t</span><span class="o">,</span>
<span class="n">smallmoney_t</span><span class="o">,</span>
<span class="n">sql_variant_t</span><span class="o">,</span>
<span class="n">table_t</span><span class="o">,</span>
<span class="n">text_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: time
*/</span>
<span class="n">time_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: time with time zone
*/</span>
<span class="n">time_with_time_zone_t</span><span class="o">,</span>
<span class="n">time_without_time_zone_t</span><span class="o">,</span>
<span class="n">timespan_t</span><span class="o">,</span> <span class="c1">// ansi2003: interval</span>
<span class="n">timestamp_t</span><span class="o">,</span> <span class="c1">// ansi2003: timestamp</span>
<span class="cm">/**
* timestamp with local time zone,
* Database: Oracle,SQL Server
*/</span>
<span class="n">timestamp_with_local_time_zone_t</span><span class="o">,</span>
<span class="cm">/**
* timestamp with time zone, timestamptz, ansi2003: timestamp with time zone
*/</span>
<span class="n">timestamp_with_time_zone_t</span><span class="o">,</span>
<span class="n">timestamp_without_time_zone_t</span><span class="o">,</span>
<span class="cm">/**
* time with time zone, ansi2003: time with time zone
* Databases: teradata
*/</span>
<span class="n">timetz_t</span><span class="o">,</span>
<span class="n">timentz_t</span><span class="o">,</span>
<span class="n">tinyblob_t</span><span class="o">,</span>
<span class="n">tinyint_t</span><span class="o">,</span>
<span class="n">tinytext_t</span><span class="o">,</span>
<span class="n">uniqueidentifier_t</span><span class="o">,</span>
<span class="n">urowid_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: blob
*/</span>
<span class="n">varbinary_t</span><span class="o">,</span>
<span class="cm">/**
* netezza, bit varying
*/</span>
<span class="n">varbit_t</span><span class="o">,</span>
<span class="cm">/**
* teradata: varbyte
*/</span>
<span class="n">varbyte_t</span><span class="o">,</span>
<span class="cm">/**
* varchar, char varying, character varying, ansi2003:character varying(n)
*/</span>
<span class="n">varchar_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: character varying
*/</span>
<span class="n">varchar2_t</span><span class="o">,</span>
<span class="n">varchar_for_bit_data_t</span><span class="o">,</span><span class="c1">// ansi2003: bit varying</span>
<span class="n">lvarchar_t</span><span class="o">,</span> <span class="c1">//informix,openedge</span>
<span class="n">idssecuritylabel_t</span><span class="o">,</span><span class="c1">//informix</span>
<span class="cm">/**
* ansi2003: nchar varying
*/</span>
<span class="n">vargraphic_t</span><span class="o">,</span>
<span class="n">row_data_types_t</span><span class="o">,</span> <span class="c1">//informix</span>
<span class="n">collection_data_types_collection_t</span><span class="o">,</span>
<span class="n">collection_data_types_set_t</span><span class="o">,</span>
<span class="n">collection_data_types_multiset_t</span><span class="o">,</span>
<span class="n">collection_data_types_list_t</span><span class="o">,</span>
<span class="cm">/**
* ansi2003: tinyint
*/</span>
<span class="cm">/**
* datatypeAttribute in cast function will be treated as a datatype without typename
* RW_CAST ( expr AS datatypeAttribute )
*/</span>
<span class="n">no_typename_t</span><span class="o">,</span>
<span class="n">year_t</span><span class="o">,</span>
<span class="n">xml_t</span><span class="o">,</span> <span class="c1">// ansi2003: xml</span>
<span class="n">xmltype_t</span><span class="o">,</span> <span class="c1">// ansi2003: xml</span>
<span class="n">natural_t</span><span class="o">,</span> <span class="c1">//plsql</span>
<span class="n">naturaln_t</span><span class="o">,</span><span class="c1">//plsql</span>
<span class="n">positive_t</span><span class="o">,</span>
<span class="n">positiven_t</span><span class="o">,</span>
<span class="n">signtype_t</span><span class="o">,</span>
<span class="n">simple_integer_t</span><span class="o">,</span>
<span class="n">double_precision_t</span><span class="o">,</span>
<span class="n">boolean_t</span><span class="o">,</span>
<span class="n">string_t</span><span class="o">,</span>
<span class="n">listType_t</span><span class="o">,</span> <span class="c1">//hive array <type></span>
<span class="n">structType_t</span><span class="o">,</span><span class="c1">//hive</span>
<span class="n">mapType_t</span><span class="o">,</span>
<span class="n">unionType_t</span><span class="o">,</span>
<span class="n">refcursor_t</span><span class="o">,</span><span class="c1">//postgresql</span>
<span class="n">json_t</span><span class="o">,</span> <span class="c1">//postgresql</span>
<span class="n">jsonb_t</span><span class="o">,</span><span class="c1">//postgresql</span>
<span class="n">self_t</span><span class="o">,</span><span class="c1">//oracle, constructor function</span>
<span class="n">seconddate_t</span><span class="o">,</span><span class="c1">//hana</span>
<span class="n">smalldec_t</span><span class="o">,</span><span class="c1">//hana</span>
<span class="n">array_t</span><span class="o">,</span><span class="c1">//hana,bigquery</span>
<span class="n">alphanum_t</span><span class="o">,</span><span class="c1">//hana</span>
<span class="n">shorttext_t</span><span class="o">,</span><span class="c1">//hana</span>
<span class="n">bintext_t</span><span class="o">,</span><span class="c1">//hana</span>
<span class="n">currency_t</span><span class="o">,</span><span class="c1">//dax</span>
<span class="n">int8_t</span><span class="o">,</span>
<span class="n">lvarbinary_t</span><span class="o">,</span><span class="c1">//openedge</span>
<span class="n">long_byte_t</span><span class="o">,</span><span class="c1">//mysql</span>
<span class="n">object_t</span><span class="o">,</span><span class="c1">//snowflake</span>
<span class="n">variant_t</span><span class="o">,</span><span class="c1">//snowflake</span>
<span class="n">unsigned_int_t</span><span class="o">,</span><span class="c1">//</span>
<span class="n">decfloat_t</span><span class="o">,</span><span class="c1">//db2</span>
<span class="n">struct_t</span><span class="o">,</span><span class="c1">//bigquery</span>
<span class="n">int64_t</span><span class="o">,</span><span class="c1">//bigquery</span>
<span class="n">float64_t</span><span class="o">,</span><span class="c1">//bigquery</span>
<span class="o">}</span>
</code></pre></div></div>{"name"=>nil, "picture"=>nil, "email"=>nil, "twitter"=>nil, "links"=>[{"title"=>nil, "url"=>nil, "icon"=>nil}]}SQL Datatype and TTypeName SQL的数据类型,GSP中的对应类:TTypeName。 SQL Datatype的类型 TTypeName 表示SQL中的数据类型,例如: char(10), int, float(24), decimal(8,2) 基本属性 以表示 decimal(8,2)为例,TTypeName的基本熟悉值如下: getDataType() = decimal_t toString() = decimal(8,2) getDataTypeName() = decimal 扩展属性 扩展属性仅适用于部分特定的datatype。 getLength() 以 char(10)为例 getLength() = 10 getPrecision(), getScale() 以 decimal(8,2)为例 getPrecision() = 8 getScale() = 2 参考资料 SQL2003 datatypes 的详细列表见 “SQL in a Nutshell, 3rd Edition” p30, Table 2-8. SQL2003 categories and datatypes Related demo search SQL Datatype EDataType的完整列表 package gudusoft.gsqlparser; /** * @since v1.4.3.0 */ public enum EDataType { unknown_t, /** * user defined datetype */ generic_t, bfile_t, /** * ansi2003: bigint * postgresql */ bigint_t, /** * ansi2003: blob */ binary_t, binary_float_t, binary_double_t, /** * plsql binary_integer */ binary_integer_t, /** * binary large object * Databases: DB2, teradata */ binary_large_object_t, bit_t, bit_varying_t, // = varbit blob_t, /** * bool, boolean, ansi2003: boolean */ bool_t, box_t, /** * teradata: byte */ byte_t, bytea_t, //ansi2003 blob /** * teradata byteint */ byteint_t, /** * char, character, ansi2003: character */ character_t, char_t, char_for_bit_data_t, /** * teradata: character large object */ char_large_object_t, cidr_t, circle_t, clob_t, cursor_t, datalink_t, date_t, /** * ansi2003: timestamp */ datetime_t, datetimeoffset_t,// ansi2003: timestamp datetime2_t, // ansi2003: timestamp with time zone /** * ansi2003: nclob * Databases: DB2 */ dbclob_t, /** * dec,decimal, ansi2003: decimal */ decimal_t, dec_t, /** * double, double precision, ansi2003: float */ double_t, enum_t, float_t,// ansi2003: double precision float4_t,// ansi2003: float(p) float8_t, // ansi2003 float(p) /** * ansi2003 blob */ graphic_t, geography_t, geometry_t, hierarchyid_t, image_t, inet_t, /** * int, integer, ansi2003: integer */ integer_t, int_t, int2_t, // ansi2003: smallint int4_t, // ansi2003: int, integer /** * Postgresql */ interval_t, /** * teradata: interval day */ interval_day_t, /** * teradata: interval day to hour */ interval_day_to_hour_t, /** * teradata: interval day to minute */ interval_day_to_minute_t, interval_day_to_second_t, /** * teradata: interval hour */ interval_hour_t, /** * teradata: interval hour to minute */ interval_hour_to_minute_t, /** * teradata: interval hour to second */ interval_hour_to_second_t, /** * teradata: interval minute */ interval_minute_t, /** * teradata: interval minute to second */ interval_minute_to_second_t, /** * teradata: interval month */ interval_month_t, /** * teradata:interval second */ interval_second_t, /** * teradata interval year. */ interval_year_t, interval_year_to_month_t, line_t, long_t, long_varchar_t, /** * long varbinary, mysql * MySQL Connector/ODBC defines BLOB values as LONGVARBINARY and TEXT values as LONGVARCHAR. */ long_varbinary_t, longblob_t, // ansi2003: blob /** * ansi2003: blob */ long_raw_t, long_vargraphic_t, longtext_t, lseg_t, macaddr_t, mediumblob_t, /** * mediumint, middleint(MySQL) , ansi2003: int */ mediumint_t, mediumtext_t, money_t, // = decimal(9,2),INFORMIX /** * national_char_varying,nchar_varying,nvarchar, ansi2003: national character varying */ nvarchar_t, /** * nchar, national char, national character,ansi2003: national character */ nchar_t, ncharacter_t, /** * ansi2003: nclob */ nclob_t, /** * ntext, national text, ansi2003: nclob */ ntext_t, /** * nvarchar2(n) */ nvarchar2_t, /** * number, num */ number_t, /** * ansi2003: numeric */ numeric_t, oid_t, path_t, /** * teradata: period(n) */ period_t, /** * plsql pls_integer */ pls_integer_t, point_t, polygon_t, raw_t, /** * ansi2003: real */ real_t, rowid_t, rowversion_t, serial_t,// = serial4 serial8_t,// = bigserial bigserial_t,//informix smallfloat_t,//informix /** * MySQL: set */ set_t, smalldatetime_t, /** * ansi2003: smallint */ smallint_t, smallmoney_t, sql_variant_t, table_t, text_t, /** * ansi2003: time */ time_t, /** * teradata: time with time zone */ time_with_time_zone_t, time_without_time_zone_t, timespan_t, // ansi2003: interval timestamp_t, // ansi2003: timestamp /** * timestamp with local time zone, * Database: Oracle,SQL Server */ timestamp_with_local_time_zone_t, /** * timestamp with time zone, timestamptz, ansi2003: timestamp with time zone */ timestamp_with_time_zone_t, timestamp_without_time_zone_t, /** * time with time zone, ansi2003: time with time zone * Databases: teradata */ timetz_t, timentz_t, tinyblob_t, tinyint_t, tinytext_t, uniqueidentifier_t, urowid_t, /** * ansi2003: blob */ varbinary_t, /** * netezza, bit varying */ varbit_t, /** * teradata: varbyte */ varbyte_t, /** * varchar, char varying, character varying, ansi2003:character varying(n) */ varchar_t, /** * ansi2003: character varying */ varchar2_t, varchar_for_bit_data_t,// ansi2003: bit varying lvarchar_t, //informix,openedge idssecuritylabel_t,//informix /** * ansi2003: nchar varying */ vargraphic_t, row_data_types_t, //informix collection_data_types_collection_t, collection_data_types_set_t, collection_data_types_multiset_t, collection_data_types_list_t, /** * ansi2003: tinyint */ /** * datatypeAttribute in cast function will be treated as a datatype without typename * RW_CAST ( expr AS datatypeAttribute ) */ no_typename_t, year_t, xml_t, // ansi2003: xml xmltype_t, // ansi2003: xml natural_t, //plsql naturaln_t,//plsql positive_t, positiven_t, signtype_t, simple_integer_t, double_precision_t, boolean_t, string_t, listType_t, //hive array <type> structType_t,//hive mapType_t, unionType_t, refcursor_t,//postgresql json_t, //postgresql jsonb_t,//postgresql self_t,//oracle, constructor function seconddate_t,//hana smalldec_t,//hana array_t,//hana,bigquery alphanum_t,//hana shorttext_t,//hana bintext_t,//hana currency_t,//dax int8_t, lvarbinary_t,//openedge long_byte_t,//mysql object_t,//snowflake variant_t,//snowflake unsigned_int_t,// decfloat_t,//db2 struct_t,//bigquery int64_t,//bigquery float64_t,//bigquery }