User Defined Function (UDF) : HP Vertica

UDF combines repeated (or complex) SQL or other task into one unit that can be reused. Lets explore this feature in Vertica.
There are five types of UDx (User Defined):

• UD SQL function: Can use built-in functions, arguments and operators in only one SQL expression and return result o the expression.
• UD Scalar function: Takes one row as input and produces one row as output. Can be written in C++ and R.
• UD Aggregate function: Takes one column data as input and produces one column output. Can be written in C++.
• UD Analytic function: Calculate one output row from multiple input row. Unlike transform function this take input rows one by one, so can read multiple rows and return one output row. Can be written in C++.
• UD Transform function: Takes multiple rows as input and output multiple rows. Can be written in C++ and R.

Fenced mode: Vertica has a separate process to execute UDx, so that they don't use significant resources of main Vertica process. 

All functions written in C++ can be run in fenced mode or not, but functions in R always run in fenced mode.

To install and setup Vertica see my previous post. It is recommended to have single node installation  of Vertica on developer machine or testing UDx.

Create Web Service SOAP request from WSDL and call service

In general, we generate stubs or use some XML builder to create SOAP request. This works for already known web services. Here, I am describing creating SOAP request (sample template) from any WSDL, like SOAP-UI does. After generating this request XML we can prompt user to fill data in request or, if values are known we can pass them to call the service. This can  be useful in creating SOAPUI like tool embedded in a web application or some integration solution.

Lets start with first scenario :

1. Creating a  request template.
2. Generate HTML form, from request XML to collect values to be passed.
3. Generate request from values collected call service.

In second scenario, generate request from known values of all parameters. In this no user integration required.

To do these I will use Membrane SOA (a Apache2 licence product) and develop web application using vertx 2 and JQuery. Vertx is node.js like framework for JVM.

My First Predictive Modeling With R : Using Caret Package

The caret package was developed to:

• create a unified interface for modeling and prediction (interfaces to 147 models)
• streamline model tuning using resampling
• provide a variety of “helper” functions and classes for day–to–day
• model building tasks
• increase computational efficiency using parallel processing

Caret package: http://topepo.github.io/caret/index.html