CVS Check-In: Faster cross node joins and support recent changes in the JDBC driver

This message was edited 2 times. Last update was at 1 Feb 2010 09:14:49

We added support for recent changes in the CopyManager of EnterpriseDB JDBC driver.
Added support for the CopyManager of the Postgres JDBC driver.
Added support for CopyOut: use CopyOut when sending intermediate results to avoid overhead of creating PesultSet and use CopyOut when running GridSQL's COPY TO command. This makes row shipping of intermediate results much faster.

IMPORTANT:

From this point forward, the new EnterpriseDB JDBC driver or new Postgres JDBC driver is needed to run GridSQL built from sources checked out from the CVS, as well as for subsequent releases.

Both drivers are required to build sources. The driver jars are available from the CVS server, if you are updating sources from the CVS, please make sure you are updated your lib subdirectory.

Also do not forget to update jars when you are deploying binaries.

If you prefer to use the Postgres JDBC driver instead of the EnterpriseDB one, you have to edit some config files.
1. Edit the .sh or .bat scripts to ensure the driver is on the class path
2. Add these lines into the config file:
xdb.default.jdbcdriver=org.postgresql.Driver
xdb.default.jdbcstring=jdbc:postgresql://{dbhost}:{dbport}/{database}

To use this new code with an older driver you have to disable JDBC COPY adding these lines to the config file:
xdb.loader.nodewriter.use_jdbc_copy_api=false
xdb.use_load_for_step=false
However this is not recommended, because of performance impact.

Files:

RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyManager.java,v
done
Checking in src/com/edb/gridsql/engine/copy/CopyManager.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyManager.java,v <-- CopyManager.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyManager.java,v
done
Checking in src/com/edb/gridsql/engine/copy/PgCopyManager.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyManager.java,v <-- PgCopyManager.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyIn.java,v
done
Checking in src/com/edb/gridsql/engine/copy/CopyIn.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyIn.java,v <-- CopyIn.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyIn.java,v
done
Checking in src/com/edb/gridsql/engine/copy/PgCopyIn.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyIn.java,v <-- PgCopyIn.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyIn.java,v
done
Checking in src/com/edb/gridsql/engine/copy/EdbCopyIn.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyIn.java,v <-- EdbCopyIn.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyOut.java,v
done
Checking in src/com/edb/gridsql/engine/copy/CopyOut.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/CopyOut.java,v <-- CopyOut.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyManager.java,v
done
Checking in src/com/edb/gridsql/engine/copy/EdbCopyManager.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyManager.java,v <-- EdbCopyManager.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyOut.java,v
done
Checking in src/com/edb/gridsql/engine/copy/PgCopyOut.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/PgCopyOut.java,v <-- PgCopyOut.java
initial revision: 1.1
done
RCS file: /cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyOut.java,v
done
Checking in src/com/edb/gridsql/engine/copy/EdbCopyOut.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/copy/EdbCopyOut.java,v <-- EdbCopyOut.java
initial revision: 1.1
done
Checking in src/com/edb/gridsql/optimizer/SqlExpression.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/optimizer/SqlExpression.java,v <-- SqlExpression.java
new revision: 1.23; previous revision: 1.22
done
Checking in src/com/edb/gridsql/planner/ExecutionPlan.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/planner/ExecutionPlan.java,v <-- ExecutionPlan.java
new revision: 1.6; previous revision: 1.5
done
Checking in src/com/edb/gridsql/planner/QueryPlan.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/planner/QueryPlan.java,v <-- QueryPlan.java
new revision: 1.11; previous revision: 1.10
done
Checking in src/com/edb/gridsql/planner/ExecutionStep.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/planner/ExecutionStep.java,v <-- ExecutionStep.java
new revision: 1.5; previous revision: 1.4
done
Checking in src/com/edb/gridsql/planner/StepDetail.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/planner/StepDetail.java,v <-- StepDetail.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/com/edb/gridsql/engine/loader/DataProcessorThread.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/DataProcessorThread.java,v <-- DataProcessorThread.java
new revision: 1.8; previous revision: 1.7
done
Checking in src/com/edb/gridsql/engine/loader/CopyOutReaderThread.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/CopyOutReaderThread.java,v <-- CopyOutReaderThread.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/com/edb/gridsql/engine/loader/Loader.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/Loader.java,v <-- Loader.java
new revision: 1.12; previous revision: 1.11
done
Removing src/com/edb/gridsql/engine/loader/EDBWriterFactory.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/EDBWriterFactory.java,v <-- EDBWriterFactory.java
new revision: delete; previous revision: 1.2
done
Checking in src/com/edb/gridsql/engine/loader/TextProcessorThread.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/TextProcessorThread.java,v <-- TextProcessorThread.java
new revision: 1.3; previous revision: 1.2
done
Removing src/com/edb/gridsql/engine/loader/EDBWriter.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/EDBWriter.java,v <-- EDBWriter.java
new revision: delete; previous revision: 1.3
done
Checking in src/com/edb/gridsql/engine/loader/PostgresWriter.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/PostgresWriter.java,v <-- PostgresWriter.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/com/edb/gridsql/engine/loader/PostgresWriterFactory.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/loader/PostgresWriterFactory.java,v <-- PostgresWriterFactory.java
new revision: 1.2; previous revision: 1.1
done
Checking in src/com/edb/gridsql/parser/SqlCopyData.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/parser/SqlCopyData.java,v <-- SqlCopyData.java
new revision: 1.6; previous revision: 1.5
done
Checking in src/com/edb/gridsql/parser/ExpressionType.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/parser/ExpressionType.java,v <-- ExpressionType.java
new revision: 1.5; previous revision: 1.4
done
Checking in src/com/edb/gridsql/util/XdbLoader.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/util/XdbLoader.java,v <-- XdbLoader.java
new revision: 1.6; previous revision: 1.5
done
Checking in src/com/edb/gridsql/engine/ProducerSender.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/ProducerSender.java,v <-- ProducerSender.java
new revision: 1.9; previous revision: 1.8
done
Checking in src/com/edb/gridsql/engine/NodeProducerThread.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/engine/NodeProducerThread.java,v <-- NodeProducerThread.java
new revision: 1.7; previous revision: 1.6
done
Checking in src/com/edb/gridsql/queryproc/QueryProcessor.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/queryproc/QueryProcessor.java,v <-- QueryProcessor.java
new revision: 1.17; previous revision: 1.16
done
Checking in src/com/edb/gridsql/common/util/Props.java;
/cvsroot/gridsql/apps/gridsql/src/com/edb/gridsql/common/util/Props.java,v <-- Props.java
new revision: 1.22; previous revision: 1.21
done
Checking in lib/edb-jdbc14.jar;
/cvsroot/gridsql/apps/gridsql/lib/edb-jdbc14.jar,v <-- edb-jdbc14.jar
new revision: 1.2; previous revision: 1.1
done