Example simple UDF function (StringUtilsUDF.java)
Step 1 : Wrote simple Java function - example (concat first name & lastName ) – which can be done via hive built in function
Step 2 : ADD JAR /home/gse/stringHiveUDF-1.0.jar;
Step 3 : CREATE TEMPORARY FUNCTION stringcat as 'com.test.udfs.StringUtilsUDF';
Step 4 : Use the function in the hive select query
select stringcat(billing_analyst_fname,billing_analyst_lname) from accounts
where account_number = 133708;
OK
Naoki,Ando
Time taken: 0.135 seconds, Fetched: 1 row(s)
StringUtilsUDF.java
package com.test.udfs;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
public class StringUtilsUDF extends UDF {
private Text result = new Text();
public Text evaluate(Text strFirst, Text strLast) {
if (strFirst != null && strLast != null) {
result.set(StringUtils.strip(strFirst.toString()) + "," + StringUtils.strip(strLast.toString()));
} else {
if (strFirst != null) {
result.set(StringUtils.strip(strFirst.toString()));
} else if (strLast != null) {
result.set(StringUtils.strip(strLast.toString()));
}else{
return null;
}
}
return result;
}
}
Step 1 : Wrote simple Java function - example (concat first name & lastName ) – which can be done via hive built in function
Step 2 : ADD JAR /home/gse/stringHiveUDF-1.0.jar;
Step 3 : CREATE TEMPORARY FUNCTION stringcat as 'com.test.udfs.StringUtilsUDF';
Step 4 : Use the function in the hive select query
select stringcat(billing_analyst_fname,billing_analyst_lname) from accounts
where account_number = 133708;
OK
Naoki,Ando
Time taken: 0.135 seconds, Fetched: 1 row(s)
StringUtilsUDF.java
package com.test.udfs;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
public class StringUtilsUDF extends UDF {
private Text result = new Text();
public Text evaluate(Text strFirst, Text strLast) {
if (strFirst != null && strLast != null) {
result.set(StringUtils.strip(strFirst.toString()) + "," + StringUtils.strip(strLast.toString()));
} else {
if (strFirst != null) {
result.set(StringUtils.strip(strFirst.toString()));
} else if (strLast != null) {
result.set(StringUtils.strip(strLast.toString()));
}else{
return null;
}
}
return result;
}
}
Really Good blog post.provided a helpful information.I hope that you will post more updates like this Big data hadoop online Course Hyderabad
ReplyDelete