Release Notes - Quest® Data Connector for Oracle and Hadoop
Quest Data Connector for Oracle and Hadoop
Version 1.5
Release Notes
Wednesday, 4 April 2012
Welcome to Quest Data Connector for Oracle and Hadoop
Quest Data Connector for Oracle and Hadoop is an optional plugin to Sqoop. It facilitates the movement of data between Oracle and Hadoop. Quest Data Connector for Oracle and Hadoop inspects each Sqoop job and assumes responsibility for the ones it can perform better than the Oracle manager built into Sqoop.
Quest Data Transporter for Hive is distributed with Quest Data Connector for Oracle and Hadoop. Quest Data Transporter for Hive is a Java command-line utility that allows you to execute a Hive query and insert the results into an Oracle table. Quest Data Connector for Oracle and Hadoop and Quest Data Transporter for Hive together allow for all transfer of data between Oracle, HDFS and Hive.
| To Extract From ... | And Import Into... | Do ... |
|---|---|---|
| An Oracle Table | HDFS for analysis by Hadoop |
Execute Sqoop with Quest Data Connector for Oracle and Hadoop. |
| An Oracle Table | HDFS for analysis by Hive |
Execute Sqoop with Quest Data Connector for Oracle and Hadoop. |
| HDFS | An Oracle Table |
Execute Sqoop with Quest Data Connector for Oracle and Hadoop. |
| The Results of a Hive Query | An Oracle Table | Execute Quest Data Transporter for Hive |
New in This Release
Update to Quest Data Connector for Oracle and Hadoop 1.5
- Improvements to importing Oracle table partitions: You can now match Hadoop files to Oracle Table Partitions. You can filter the partitions to import.
Update to Quest Data Connector for Oracle and Hadoop 1.4
- Support for Oracle Optimizer Hints: An Oracle optimizer hint can be added to the SELECT statement for IMPORT jobs
- Updates available through Toad for Cloud Databases (Eclipse) 1.5: The graphical user interface to Quest Data Connector for Oracle and Hadoop in Toad for Cloud Databases (Eclipse) 1.5 supports this release and includes both import and export functionality.
Update to Quest Data Connector for Oracle and Hadoop 1.3
- Support for the Oracle RAW data type: Quest Data Connector for Oracle and Hadoop now supports import of the Oracle RAW data type.
- Quest Data Transporter for Hive 1.1: A new version of Quest Data Transporter for Hive is included which supports Hive 0.7.
- Graphical User Interface added to Toad for Cloud Databases (Eclipse) 1.3: A graphical user interface to Quest Data Connector for Oracle and Hadoop 1.3 import is provided in the Toad for Cloud Databases (Eclipse) 1.3 product.
Update to Quest Data Connector for Oracle and Hadoop 1.2
- Export using merge approach: Quest Data Connector for Oracle and Hadoop's Merge-Export is unique to Quest Data Connector for Oracle and Hadoop. There is no Sqoop equivalent. Merge-Export updates Oracle tables by modifying changed rows and inserting rows from the HDFS file that did not previously exist in the Oracle table. Use of this additional Quest Data Connector for Oracle and Hadoop functionality requires Sqoop version 1.2 and above.
Updates to Quest Data Connector for Oracle and Hadoop 1.1
- Support for Oracle RAC: Quest Data Connector for Oracle and Hadoop identifies active instances of the Oracle RAC and connects each Hadoop mapper to them in a round-robin manner. Quest Data Connector for Oracle and Hadoop also includes support for Oracle RAC Services.
- Export: Export HDFS data into an Oracle table with improved performance.
- Additional supported data types: The following data types are now supported in Quest Data Connector for Oracle and Hadoop: BINARY_DOUBLE, BINARY_FLOAT and URITYPE.
- Quest Data Transporter for Hive 1.0: Quest Data Transporter for Hive is a Java command-line utility that allows you to execute a Hive query and insert the results into an Oracle table. Quest Data Transporter for Hive is included in the Quest Data Connector for Oracle and Hadoop archive.
Known Issues
The following is a list of issues known to exist at the time of this release.
| Feature | Known Issue | Defect ID |
|---|---|---|
| Quest Data Connector for Oracle and Hadoop Export |
Quest Data Connector for Oracle and Hadoop cannot export data containing LOBs. The issue is: Sqoop export does not support LOB data due to a bug in the java code generated by Sqoop to read the data from HDFS. |
Jira: SQOOP-117 |
|
Quest Data Connector for Oracle and Hadoop export does not support the Oracle data type LONG. This is due to an Oracle issue. |
N/A | |
| Quest Data Connector for Oracle and Hadoop Import |
Column names that are explicitly listed for a Sqoop import, which are escaped cause an NullPointerException in Sqoop $ sqoop import ... --table customers --columns "\"\"first name\"\"" |
Jira: SQOOP-92 |
|
Null values from Oracle are imported into Hive (via Sqoop) as the string “null”. This is due to Sqoop serializing the value NULL as a string containing the four letters “null”. Executing the following query always returns zero, even with null values in Oracle: hive> select count(1) from customers where surname is null; To avoid this issue, query for: hive> select count(1) from customers where surname = 'null'; |
N/A | |
| NCLOB and NCHAR data that is UTF-16 encoded is converted to UTF-8 encoding during a Sqoop import. (You may or may not consider this a problem.) | N/A | |
| Quest Data Transporter for Hive |
java.sql.BatchUpdateException: Internal Error: Underflow Exception trying to bind <number> The Hive double value may be too small or large to be inserted into Oracle. For example, a Hive value of 2.225074E-308 would trigger this problem. Hive’s double type can hold min. 2.225074e-308 to max 1.797693e+308 but Oracle's NUMBER type can hold min 1.0e-127 to max 9.9999999999999999999999999999999999999e+121. Those values are translated to Hive’s double type as a range from min 1.0e-127 to max 9.99999e+121 This is the range you can transfer from Hive’s double to Oracle’s NUMBER by Quest Data Transporter for Hive. |
ST12527 |
|
Columns in the Oracle table are called "COL0, COL1, COL2…" instead of the column names given in the source table. This behavior occurs with all non simple HQL queries. For example: select customer_id, customer_name from customers or select customer_id as ID, customer_name as NAME from customers This problem does not occur with simple queries involving a asterisk like: Select * from <TABLE NAME> This problem is due to an issue with the JDBC driver. |
ST12554 | |
| Quest Data Transporter for Hive applying to Hive 0.5 | Quest Data Transporter for Hive cannot transfer data types tinyint, smallint and float because of a problem within the Hive JDBC driver |
Jira: HIVE-1859 (tiny int) Jira: HIVE-1860 (small int) Jira: HIVE-1861 (float) |
| Boolean data values in the Oracle table are recorded as true (1), false (0) or null (0). False and Null values are the same. It is not possible to distinguish between them. | Jira HIVE-1863 |
Upgrade and Compatibility
The Installation / Upgrade instructions for Quest Data Connector for Oracle and Hadoop are the same.
System Requirements
Before installing Quest Data Connector for Oracle and Hadoop and Quest Data Transporter for Hive, ensure your system meets the following minimum hardware and software requirements:
| Platform | 1 GHz Pentium 4 PC |
| Memory | 1 GB RAM |
| Hard Disk Space | 5 MB |
| Operating System | Linux or Linux-like environment |
| Database Server |
Oracle 10g or later (including RAC). |
| Additional Software |
Sqoop version 1.2 or later. (Cloudera SQL-to-Hadoop database import and export tool) for Quest Data Connector for Oracle and Hadoop Hive 0.5 or 0.7 or 0.7.1 (version 0.5.0+32 or 0.7.0 or 0.7.1) for Quest Data Transporter for Hive. Oracle Database 11g Release 2 JDBC driver Adobe Acrobat Reader 7.0 or later (for viewing the User Guide) |
Global Operations
This section contains information about installing and operating this product in non-English configurations, such as those needed by customers outside of North America. This section does not replace the materials about supported platforms and configurations found elsewhere in the product documentation.
This release was not enabled and tested for international operation. Only U.S. English configurations of this product and U.S. English datasets are fully supported. Support for non-ASCII characters or non-English configurations may work appropriately, but these configurations have not been tested and this release should be used with caution in these environments.
Getting Started
Contents of the Release Package
The Quest Data Connector for Oracle and Hadoop release package contains the following products:
- Quest Data Connector for Oracle and Hadoop 1.5
- Quest Data Transporter for Hive 1.1
- Product Documentation, including:
- Quest Data Connector for Oracle and Hadoop User Guide
- Quest Data Transporter for Hive User Guide
- Release Notes
Installation Instructions
Refer to the Quest Data Connector for Oracle and Hadoop User Guide and Quest Data Transporter for Hive User Guide for installation instructions.
For More Information
Get the latest product information and find helpful resources at http://www.quest.com/hadoop/.
For support please visit the Quest Data Connector for Oracle and Hadoop Forums: http://toadforcloud.com/forumindex.jspa?categoryID=735
Contact Quest Software
|
Email: |
|
|
Mail: |
Quest Software, Inc. |
|
Web site: |
Refer to our Web site for regional and international office information.
© 2012 Quest Software, Inc.
ALL RIGHTS RESERVED.
This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser’s personal use without the written permission of Quest Software, Inc.
The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document.
If you have any questions regarding your potential use of this material, contact:
Quest Software World Headquarters
LEGAL Dept
5 Polaris Way
Aliso Viejo, CA 92656
email: mailto:legal@quest.com
Refer to our Web site (www.quest.com) for regional and international office information.
Trademarks
Quest, Quest Software and the Quest Software logo are trademarks and registered trademarks of Quest Software, Inc in the United States of America and other countries. For a complete list of Quest Software’s trademarks, please see http://www.quest.com/legal/trademark-information.aspx. Other trademarks and registered trademarks are property of their respective owners.