Blog

06 Nov / Eligotech Harpoon Plugin for Oracle RDBMS

As part of the Eligotech Harpoon v2.5 release we have introduced the concept of plugins. Plugins allow you to extend the Eligotech Harpoon platform with any kind of functionality that you require for making your Big Data project a success.
Our customers are using plugins for importing specific data-formats, integrating analytical functions and machine learning algorithms.
One other requirement some of our customers are having is to integrate valuable datasets created by Eligotech Harpoon with their (existing) relational databases. This is often a need in order to protect investments they have done in existing BI-tools and to re-use the technical skills they have in-house.

Plugins can be easily created by using the native map/reduce Java APIs in Java or Scala or by using Apache Pig. But also functions from R, Python and other languages can be integrated.

Based on our experience we had with the Oracle Big Data Appliance (BDA) and the Oracle Loader for Hadoop, which are part of the software licenses that comes along with the Oracle BDA, we have created the Eligotech Harpoon Plugin for Oracle RDBMS.

 

Oracle Loader for Hadoop

 

From the Oracle Loader for Hadoop website: “Oracle Loader for Hadoop is an efficient and high-performance loader for fast movement of data from a Hadoop cluster into a table in an Oracle database. Oracle Loader for Hadoop prepartitions the data if necessary and transforms it into a database-ready format. It optionally sorts records by primary key or user-specified columns before loading the data or creating output files. Oracle Loader for Hadoop is a Map/Reduce application that is invoked as a command-line utility.”

As mentioned “Oracle Loader for Hadoop” is a Map/Reduce application and is easily supported by our plugin framework.

 

Eligotech Harpoon plugin

 

The Eligotech Harpoon plugin are based on 2 files packaged in a zip file. One file is a jar file containing the actual application, the other file is the plugin descriptor that tells Eligotech Harpoon what are the plugin parameters like the input table to be exported and additional parameters like the JDBC URL for connecting to the DB, the credentials, etc.

 

In the case of the Oracle Loader for Harpoon we made some minor adjustments to the Map/Reduce code*. The configuration file we created to go along is shown below.

Both are put in the .zip file, which then can be imported into Eligotech Harpoon.
Once successful imported, the plugin will appear in the list of available plugins.

plugin-post-1

Selecting the OraLoader will provide the input fields for selecting the Eligotech Harpoon table to be uploaded to your Oracle RDBMS.

plugin-post-2

For more information please provide your comments in the blog.

* The Oracle Loader for Hadoop license is part of the Oracle Big Data Appliance Licenses. See licensing information for more information.

By David Greco in Hadoop, Harpoon, Pig

Post a comment

  • trance

  • techno

  • synth-pop

  • soundtrack

  • smooth-jazz

  • rock

  • rap

  • r-b

  • psychedelic

  • pop-rock

  • pop

  • new-age

  • musicians

  • metal

  • melodic-metal

  • lounge

  • jazz-funk

  • jazz

  • index.php

  • house

  • hip-hop

  • heavy-metal

  • hard-rock

  • get.php

  • electronic

  • dubstep

  • drumbass

  • downtempo

  • disco

  • country

  • clubdance

  • classical

  • chillout

  • chanson

  • breakbeat

  • blues

  • ambient

  • alternative-rock