Talend Useful Links

1) 100 - Best Videos
http://meta-guide.com/videography/100-best-talend-videos/

2) DataBase Connection
https://www.youtube.com/watch?v=FezmHmYm-4c

3) MongoDB Connection
https://www.youtube.com/watch?v=zRoIiLlcnQE
https://www.talendforge.org/forum/viewtopic.php?id=27589

4) dynamic schema job
http://bekwam.blogspot.sg/2011/06/dynamic-schemas-in-talend-open-studio.html
https://www.youtube.com/watch?v=bLq8ip6nH5g&gl=SG&hl=en-GB

5) compare two tables via query
http://anilmca4u.blogspot.sg/2012/12/how-to-compare-two-tables-for.html

6) Insert into MongoDB 
https://www.youtube.com/watch?v=BX9ncCCmN8Y
https://satishchennupati.wordpress.com/2015/01/20/mongo-db-operations-using-talend/ (tMap)

7) MongoDB to postgrey sql
http://copyrightdev.tumblr.com/post/109311096968/using-talend-big-data-to-move-data-from-mongodb-to

8) Converting DataType 
http://talendhunter.blogspot.sg/2013/06/tutorial-converting-data-types-with.html

9) Pass flow data from Child to Parent Job in Talend
https://www.youtube.com/watch?v=msk3HedymMI

10) Pass parameters and variables to child jobs in Talend Open Studio
https://www.youtube.com/watch?v=D6BrfnFKSLQ
http://bekwam.blogspot.sg/2011/05/passing-parameters-and-variables-to.html

11) Comparing data in an RDBMS to data in MongoDB (Sonar Diff)
https://www.youtube.com/watch?v=REy6oT9bSw8

12) Add Hyperlink to Cells - Excel Java POI Example Program
http://thinktibits.blogspot.sg/2012/12/Java-POI-Create-Hyperlink-Excel-Cell-Example-Program.html

13) create Bar Chart in Excel - Java POI Example Program
http://thinktibits.blogspot.sg/2012/12/Create-Bar-Chart-Excel-Java-Example-Program-POI-JFreeChart-Tutorial.html
https://help.talend.com/display/TalendOpenStudioComponentsReferenceGuide55EN/tBarChart

14) Combining Excel Spreadsheets with tUnite in Talend Open Studio
http://talendhunter.blogspot.sg/2013/06/combining-excel-spreadsheets-with.html

15) Error Handling
http://bekwam.blogspot.sg/2011/04/three-error-handling-strategies-in.html

16) Parametrized Queries for MongoDB in Jaspersoft
https://www.youtube.com/watch?v=fVeYyLCLph4

17) Talend - Cluster
https://help.talend.com/display/KB/How+to+create+a+clustered+server+in+Talend+Administration+Center+-+Talend+v5.2+features
https://help.talend.com/display/TalendAdministrationCenterUserGuide55EN/6.3+Deploying+a+Job+or+a+Route+in+cluster+mode

http://bekwam.blogspot.sg/2011/04/three-error-handling-strategies-in.html

18) Pass context parameters via command line
http://www.vikramtakkar.com/2013/04/how-to-pass-context-parameters-to.html

19) How to pass Context parameters to Talend Job through command line
http://www.vikramtakkar.com/2013/04/how-to-pass-context-parameters-to.html

20) Mongodb bulk Load/Read
https://www.youtube.com/watch?v=QTzVetSm28s
http://talendhunter.blogspot.sg/2013/06/handling-empty-json-object-in-talend.html

21) Reading MongoDB Subdocuments
http://community.jaspersoft.com/blog/dealing-nested-documents-mongodb-and-talend-aka-baking-cake
http://youtu.be/82p_FnAPGb4
https://help.talend.com/display/TalendComponentsReferenceGuide54EN/tMongoDBOutput

22) Handling JSON
http://bekwam.blogspot.sg/2011/06/handling-empty-json-object-in-talend.html
https://www.talendforge.org/forum/viewtopic.php?id=25943

23) Write Document as Array
https://www.talendforge.org/forum/viewtopic.php?id=44448

Spoon CRASH ERROR java: cairo-misc.c:380: _cairo_operator_bounded_by_source

Spoon CRASH ERROR "java: cairo-misc.c:380: _cairo_operator_bounded_by_source


Solution : 

Add the below line in the spoon.sh OPT variable will resolve the problem

"-Dorg.eclipse.swt.internal.gtk.cairoGraphics=false"

Step by Step Hive User Defined functions (UDF)

Example simple UDF function  (StringUtilsUDF.java)

Step 1  :  Wrote simple Java function -  example (concat first name & lastName ) – which can be done via hive built in function
Step 2 : ADD JAR /home/gse/stringHiveUDF-1.0.jar;
Step 3 : CREATE TEMPORARY FUNCTION stringcat as 'com.test.udfs.StringUtilsUDF';
Step 4 :  Use the function in the hive select query            
                                select stringcat(billing_analyst_fname,billing_analyst_lname) from accounts


where account_number = 133708;
OK
Naoki,Ando
Time taken: 0.135 seconds, Fetched: 1 row(s)



StringUtilsUDF.java

package com.test.udfs;

import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;

public class StringUtilsUDF extends UDF {
private Text result = new Text();
  public Text evaluate(Text strFirst, Text strLast) {
 if (strFirst != null && strLast != null) {
 result.set(StringUtils.strip(strFirst.toString()) + "," + StringUtils.strip(strLast.toString()));
} else {
if (strFirst != null) {
result.set(StringUtils.strip(strFirst.toString()));
} else if (strLast != null) {
result.set(StringUtils.strip(strLast.toString()));
}else{
return null;
}
}
 return result;
    }
}

invalid LOC header (bad signature)

 some binary file being broken, most likely one of the dependencies.  just delete the respective maven repository and build again
rm -rf ~/.m2/repository/

example
       rm -rf ~/.m2/repository/org

Pagination in MongoDB - can be achieved via skip and limit

Pagination in MongoDB - can be achieved via skip and limit

     skip - NUMBER_OF_ITEMS * (PAGE_NUMBER - 1)
.limit(NUMBER_OF_ITEMS )

MongoDB aggregation result exceeds maximum document size (16MB)

MongoDB aggregation result exceeds maximum document size (16MB)
 
{
    "errmsg" : "exception: aggregation result exceeds maximum document size (16MB)",
    "code" : 16389,
    "ok" : 0
}

Solution : allowDiskUse to true or limit critera to max elements


Example

AggregationOptions aggregationOptions = new AggregationOptions(true,false,null);
Aggregation aggregation = newAggregation(
match(criteria),
limit(10),
sort(Sort.Direction.ASC,  "OrderSubmissionDate")
).withOptions(aggregationOptions);

In mongoDB - newAggregation throws match compilation error

Aggregation aggregation = newAggregation (
match(criteria)
)

solution
Import static newAggregation instead of nonstatic newAggregation
import static org.springframework.data.mongodb.core.aggregation.Aggregation.newAggregation;

Limitations In mongoDB - Cannot add two conditons with same key

Limitations In mongoDB

within same query you cannot have orderstaus in ("In Progress" , "Closed") and orderstatus not in { "$nin" : [ "Closed"]}'

ADFC-0619: Authorization check failed: User 'weblogic' does not have 'VIEW' permission on 'jsf.resourcesPageDef'

Issue : oracle.adf.controller.security.AuthorizationException: ADFC-0619: Authorization check failed: User 'weblogic' does not have 'VIEW' permission on 'jsf.resourcesPageDef'.
 at oracle.adf.controller.internal.security.AuthorizationEnforcer.handleFailure(AuthorizationEnforcer.java:198) at oracle.adf.controller.internal.security.AuthorizationEnforcer.internalCheckPermission(AuthorizationEnforcer.java:166) at oracle.adf.controller.internal.security.AuthorizationEnforcer.checkPermission(AuthorizationEnforcer.java:120) at oracle.adfinternal.controller.state.ControllerState.checkPermission(ControllerState.java:913) at oracle.adfinternal.controller.state.ControllerState.synchronizeStatePart2(ControllerState.java:723) at oracle.adfinternal.controller.application.SyncNavigationStateListener.afterPhase(SyncNavigationStateListener.java:83) at oracle.adfinternal.controller.lifecycle.ADFLifecycleImpl$PagePhaseListenerWrapper.afterPhase(ADFLifecycleImpl.java:537) at oracle.adfinternal.controller.lifecycle.LifecycleImpl.internalDispatchAfterEvent(LifecycleImpl.java:120) at oracle.adfinternal.controller.lifecycle.LifecycleImpl.dispatchAfterPagePhaseEvent(LifecycleImpl.java:168) at oracle.adfinternal.controller.faces.lifecycle.ADFPhaseListener$PhaseInvokerImpl.dispatchAfterPagePhaseEvent(ADFPhaseListener.java:302) at oracle.adfinternal.controller.faces.lifecycle.ADFPhaseListener.afterPhase(ADFPhaseListener.java:88) at oracle.adfinternal.controller.faces.lifecycle.ADFLifecyclePhaseListener.afterPhase(ADFLifecyclePhaseListener.java:53) at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl._executePhase(LifecycleImpl.java:489) at oracle.adfinternal.view.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:225)


 Solution 

 Oracle SOA Suite 12c needs JDK Minium version JDK 1.6 but it does not support JDK 1.8. I Installed JDK 1.7 and it resolves the Issue.

Missing artifact org.aspectj:aspectjweaver:jar:1.8.0.M1 for spring 4.0.0

Problem When you upgrade pom version to Spring 4.0.0.RELEASE, it is prompting missing artifact org.aspectj:aspectjweaver:jar:1.8.0.M1

Solution

upgrade aspectjweaver 1.7.4 as 1.8.0 is not available.
1) org.aspectj aspectjweaver 1.7.4
2) Add the Aspect in the configuration false src/main/java src/main/webapp/schemas **/*.xsd src/main/resources/bindings false org.springframework spring-aspects true

nested exception is java.lang.incompatibleclasschangeerror org.springframework.asm.classvisitor

Problem BeanDefinitionStoreException Failed to read candidate component class nested exception is java.lang.incompatibleclasschangeerror org.springframework.asm.classvisitor Solution upgrade Spring all version to 4.0 Check the dependency version by the following command. mvn dependency:tree Change the Spring version by adding the dependenncy in the pom.xml org.springframework spring-core ${spring.version} org.springframework spring-web ${spring.version} org.springframework spring-webmvc ${spring.version} org.springframework spring-context ${spring.version} org.springframework spring-beans ${spring.version} org.springframework spring-aop ${spring.version} org.springframework spring-aspects ${spring.version} commons-collections commons-collections 3.2 org.springframework spring-oxm ${spring.version} org.springframework.ws spring-ws-core ${spring.ws.version} org.apache.ws.commons.schema XmlSchema 1.4.3 org.springframework spring-jms ${spring.version} org.aspectj aspectjweaver 1.7.4 org.hibernate hibernate-core 4.3.5.Final

ERROR 2003 (HY000): Can't connect to MySQL server on localhost (10061)

Problem : ERROR 2003 (HY000): Can't connect to MySQL server on localhost (10061) Solution : Start the mysql as service. "C:\Program Files\MySQL\MySQL Server 5.1\bin\mysqld" --install-manual go to services.msc start the mysql as automatic

Format DateTime in Tibco BW

WSDL 2.0 VS WSDL 1.1

WSDL 2.0

1  1) WSDL 2.0 was designed to solve the interoperability issues found in WSDL 1.0/1.1 by using Improved SOAP v1.2 bindings

2   2) WSDL 2.0 permits fault messages to be declared within the interface element in order to facilitate reuse of faults across operations. If a fault occurs, it terminates whatever message sequence was indicated by the message exchange pattern of the operation,

     3) WSDL2.0 provides Support for additional message patterns (ex:  “Out – Multi In” could correspond to Register for Events, Receive 0-n Events 

     4) WSDL 2.0 Provides Better and more powerful fault handling notation, allowing clearer and easier reuse of WSDL segments within the WSDL file

     5) WSDL2.0 Support for interface inheritance.

      6)      RESTful services

RESTful services can be described fairly with the structure supported by WSDL 2.0. WSDL 2.0 provides a              better model for describing RESTful services than WSDL 1.1

However with WSDL 1.1,

RESTful services could also be defined using WADL (Web Application Description Language.

7) WS-I Basic Profile
            WS-I Basic Profile v1.1 provides guidance for using SOAP 1.1, WSDL 1.1, and UDDI 2.0

            WS-I Basic Profile v2.0 provides guidance for using SOAP 1.2, WSDL 1.1, UDDI 2.0, WS-Addressing, and               MTOM.

8)      Changes from WSDL 1.1 Specification

1)      target Namespace a required attribute of the definitions element in WSDL 2.0

2)      Message constructs are specified using the XML schema type system in the types element in WSDL 2.0

3)      No support for operator overloading in WSDL 2.0

4)      PortTypes renamed to interfaces. Support for interface inheritance is achieved by using the extends attribute in the interface element.

5)      Ports renamed to endpoints




    9) Differences between WSDL1.1 and WSDL 2.0:

tokenize() Usage to split the String in Tibco BW

Tokenize Usage in Tibco


Example

Input String  :  PS00007.ELEM~1,PS00012.ELEM~1

Output Expected
<ns0:POFLine>
<ns0:ProductPartNumber>PS00007.ELEM</ns0:ProductPartNumber>
<ns0:Count>1</ns0:Count>
</ns0:POFLine>
<ns0:POFLine>
<ns0:ProductPartNumber>PS00012.ELEM</ns0:ProductPartNumber>
<ns0:Count>1</ns0:Count>

</ns0:POFLine>

Steps
tib:tokenize($Start/root/param, ',')

output
 PS00007.ELEM~1
 PS00012.ELEM~1

Pass dynamic file URL in document in XSLT

XSLT - How to pass dynamic file URL in document.

<xsl:for-each select="document($fileUrl)/SXXXXX/*/Field">

1) Use Parameter name instead of variable will resolve the issue

2) Pass Absolute path to a variable
    <xsl:variable name="fileLocation" select="'C:/XXX/'"/>

3) Pass Relative Path to a variable
     <xsl:variable name="fileUrl">../staging/<xsl:value-of  select="$producttype"/>.xml</xsl:variable>

Concatenate query values with comma separated with quote IN SQL usage in Tibco SQL Direct

Concatenate query values with comma and quote Separated in Tibco SQL Direct is  passed as IN parameter

concat("'",tib:concat-sequence-format($Start/pfx:GetOrdersByXXXXRequest/pfx:OrderXXX,"','"),"'")

org.springframework.beans.InvalidPropertyException: Invalid property Index of out of bounds in property path

When we add dynamically adding table rows and columns via javascript and trying to binding to the spring path variable it throws InvalidPropertyException: stating that
Index: 1, Size: 1

org.springframework.beans.InvalidPropertyException: Invalid property 'intnlTrnsVORcvlist[1]' of bean class [com.citi.aml.cadd.webapp.accountprofileform.Accou
at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:585)
at org.springframework.beans.BeanWrapperImpl.getNestedBeanWrapper(BeanWrapperImpl.java:415)
at org.springframework.beans.BeanWrapperImpl.getBeanWrapperForPropertyPath(BeanWrapperImpl.java:392)
at org.springframework.beans.BeanWrapperImpl.setPropertyValue(BeanWrapperImpl.java:597)
at org.springframework.beans.AbstractPropertyAccessor.setPropertyValue(AbstractPropertyAccessor.java:49)
Truncated. see log file for complete stacktrace
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
at java.util.ArrayList.RangeCheck(ArrayList.java:547)
at java.util.ArrayList.get(ArrayList.java:322)
at org.springframework.beans.BeanWrapperImpl.getPropertyValue(BeanWrapperImpl.java:534)
at org.springframework.beans.BeanWrapperImpl.getNestedBeanWrapper(BeanWrapperImpl.java:415)
at org.springframework.beans.BeanWrapperImpl.getBeanWrapperForPropertyPath(BeanWrapperImpl.java:392)
Truncated. see log file for complete stacktrace
>


Solution is
Instead of ArrayList of VO class. use GrowList.decorate(ArrayList .. along with VO class.

Example

Instead of List rcvnewList = new ArrayList(WireCountryVO.class);

Use this
List rcvnewList =GrowthList.decorate(LazyList.decorate(new
ArrayList(),FactoryUtils.instantiateFactory(WireCountryVO.class)));

Hibernate Interview Questions

What is “N+1 Select problem”

In my last two posts, I mentioned that immediate fetching or lazy fetching can cause ‘N+1 select problem’. If you are wondering what exactly is this, read on.

Consider the example of Department and Employees. When you call EntityLoad(”Department”), following sqls will be executed.

SELECT * FROM department;

SELECT * FROM employees WHERE deptId = ?

The first query will be executed once (1) and the second query will be executed as many times as the department (N). Thus the above entityLoad call results into ‘N+1′ sql execution and thus can be a cause of performance bottleneck. Because of N+1 sqls, this is known as ‘N+1 select problem’. This will almost always happen when the fetching is “Immediate” (using fetch=”select”) or can happen with lazy loading.

With immediate fetching it is obvious why this would happen. When lazy=’true”, this can happen when the association is accessed immediately on each of the owning object (department in this case).

If you think this could be happening in your application, use either of these two options.

set lazy=”false” and use fetch=”join” so that the departments and its employees get loaded together. (Eager fetch)

Keep lazy=”true” but load the department using hql with join. So instead of using EntityLoad(”Department”), use

ORMExecuteQuery("from Department dept left join fetch dept.employees")

· The classic N+1 problem in encountered in a simple lazy load scenario in any general application.

1query to get all department objects with PK

SELECT deptID FROM department;

And then query to get all other details of a particular dept

SELECT * FROM department where deptID=?;

The second query is executed as many times (N) as the number of full records you want to fetch. Its when N is high, eager can be a good strategy.

What is Hibernate Proxy

1) Class can be mapped to a proxy instead to a table. When you actually call load on session it returns you proxy. This proxy may contain actual method to load the data.

2) By default Hibernate creates a proxy for each of the class you map in mapping file. This class contain the code to invoke JDBC. This class is created by hibernate using CGLIB

What is the difference between and merge and update

Update():- if you are sure that the session does not contains an already persistent instance with the same identifier,then use update to save the data in hibernate

Merge():-if you want to save your modificatiions at any time with out knowing abot the state of an session, then use merge() in hibernate.

Second Level Cache

Hibernate Session is a transaction-level cache of persistent data. It is possible to configure a cluster or JVM-level (SessionFactory-level) cache on a class-by-class and collection-by-collection basis.

usage="transactional|read-write|nonstrict-read-write|read-only"

region="RegionName"

include="all|non-lazy"

/>

you can specify and elements in hibernate.cfg.xml.

Whenever you pass an object to save(), update() or saveOrUpdate(),

and whenever you retrieve an object using load(), get(), list(), iterate() or scroll(), that object is added to the internal cache of the Session.

evict () method can be used to remove the object and its collections from the first-level cache

ScrollableResult cats = sess.createQuery("from Cat as cat").scroll(); //a huge result set

while ( cats.next() ) {

Cat cat = (Cat) cats.get(0);

doSomethingWithACat(cat);

sess.evict(cat);

}

What are the id generator classes in hibernate?
A: increment: It generates identifiers of type long, short or int that are unique only when no other process is inserting data into the same table. It should not the used in the clustered environment.
identity: It supports identity columns in DB2, MySQL, MS SQL Server, Sybase and HypersonicSQL. The returned identifier is of type long, short or int.
sequence: The sequence generator uses a sequence in DB2, PostgreSQL, Oracle, SAP DB, McKoi or a generator in Interbase. The returned identifier is of type long, short or int

hilo: The hilo generator uses a hi/lo algorithm to efficiently generate identifiers of type long, short or int, given a table and column (by default hibernate_unique_key and next_hi respectively) as a source of hi values. The hi/lo algorithm generates identifiers that are unique only for a particular database. Do not use this generator with connections enlisted with JTA or with a user-supplied connection.

seqhilo: The seqhilo generator uses a hi/lo algorithm to efficiently generate identifiers of type long, short or int, given a named database sequence.

uuid: The uuid generator uses a 128-bit UUID algorithm to generate identifiers of type string, unique within a network (the IP address is used). The UUID is encoded as a string of hexadecimal digits of length 32.

guid: It uses a database-generated GUID string on MS SQL Server and MySQL.
native: It picks identity, sequence or hilo depending upon the capabilities of the underlying database.

assigned: lets the application to assign an identifier to the object before save() is called. This is the default strategy if no element is specified.
select: retrieves a primary key assigned by a database trigger by selecting the row by some unique key and retrieving the primary key value.
foreign: uses the identifier of another associated object. Usually used in conjunction with a primary key association

Java - Garbage Collection

How is the generational collector implemented in HotSpot(tm)?

The default collector in HotSpot has two generations:

1) young generation and the

2) tenured generation.

è Most allocations are done in the young generation. The young generation is optimized for objects that have a short lifetime relative to the interval between collections.

è Objects that survive several collections in the young generation are moved to the tenured generation.

è young generation is typically smaller and is collected more often. The tenured generation is typically larger and collected less often.

young generation collector is a copying collector.

young generation is divided into 3 spaces:

ü eden-space,

ü to-space, and

ü from-space

Allocations are done from eden-space and from-space. When those are full a young generation is collection is done. The expectation is that most of the objects are garbage and any surviving objects can be copied to to-space. If there are more surviving objects than can fit into to-space, the remaining objects are copied into the tenured generation. There is an option to collect the young generation in parallel.

What is the relevance of -XX:MaxNewSize? Where will the differences between -XX:NewSize and -XX:MaxNewSize grow, Eden or Survivor Spaces?

The young generation is set by a policy that bounds the size from below by NewSize and bounds it from above by MaxNewSize. As the young generation grows from NewSize to MaxNewSize, both eden and the survivor spaces grow.

How should the permanent generation be sized?

The permanent generation is used to hold reflective of the VM itself such as class objects and method objects. These reflective objects are allocated directly into the permanent generation, and it is sized independently from the other generations. Generally, sizing of this generation can be ignored because the default size is adequate. However, programs that load many classes may need a larger permanent generation.

How can I increase the permanent generation size?

command line option -XX:MaxPermSize=

How do I know what classes are being loaded or unloaded?

command line options -XX:+TraceClassloading and -XX:+TraceClassUnloading

What is the best size for the young generation?

The young generation should be sized large enough so that short-lived objects have a chance to die before the next young generation collection. This is a tradeoff since a larger young generation will allow more time for objects to die but may also take longer to collect. Experiment with the size of the young generation to optimize the young generation collection time or the applicationthroughput.

What should I do if my application has mid- or long-lived objects?

Objects that survive a young generation collection have a copying cost (part of the algorithm for a young generation collection is to copy any objects that survive). Mid- or long-lived objects may be copied multiple times.

Use the -XX option MaxTenuringThreshold to determine the copying costs.

Use -XX:MaxTenuringThreshold=0 to move an object that survives a young generation collection immediately to the tenured generation. If that improves the performance of the application, the copying of long-lived objects is significant. Note that the throughput collector does not use the MaxTenuringThreshold parameter.

What type of collection does a System.gc() do?

An explicit request to do a garbage collection does a full collection (both young generation and tenured generation).

A full collection is always done with the application paused for the duration of the collection.

What is the Concurrent Mark Sweep (CMS) collector?

The Concurrent Mark Sweep (CMS) collector (or concurrent low pause collector) collects the tenured generation. It attempts to minimize the pauses due to garbage collection by doing most of the garbage collection work concurrently with the application threads.

What are the phases of the concurrent low pause collector?

There are six phases involved in the collection:

Phase 1 (Initial Checkpoint) involves stopping all the Java threads, marking all the objects directly reachable from the roots, and restarting the Java threads.

Phase 2 (Concurrent Marking) starts scanning from marked objects and transitively marks all objects reachable from the roots. The mutators are executing during the concurrent phases 2, 3, and 5 below and any objects allocated in the CMS generation during these phases (including promoted objects) are immediately marked as live.

Phase 3: During the concurrent marking phase mutators may be modifying objects. Any object that has been modified since the start of the concurrent marking phase (and which was not subsequently scanned during that phase) must be rescanned. Phase 3 (Concurrent Precleaning) scans objects that have been modified concurrently. Due to continuing mutator activity the scanning for modified cards may be done multiple times.

Phase 4 (Final Checkpoint) is a stop-the-world phase. With mutators stopped the final marking is done by scanning objects reachable from the roots and by scanning any modified objects. Note that after this phase there may be objects that have been marked but are no longer live. Such objects will survive the current collection but will be collected on the next collection.

Phase 5 (Concurrent Sweep) collects dead objects. The collection of a dead object adds the space for the object to a free list for later allocation. Coalescing of dead objects may occur at this point. Note that live objects are not moved.

Phase 6 (Resetting) clears data structures in preparation for the next collection.

What is the Parallel Garbage collector (-XX:+UseParallelGC)?

ü The new parallel garbage collector is similar to the young generation collector in the default garbage collector but uses multiple threads to do the collection.

ü By default on a host with N CPUs, the parallel garbage collector uses N garbage collector threads in the collection.

ü The number of garbage collector threads can be controlled with a command line option (see below).

ü On a host with a single CPU the default garbage collector is used even if the parallel garbage collector has been requested.

ü On a host with 2 cpus the Parallel garbage collector generally performs as well as the default garbage collector and a reduction in the young generation garbage collector pause times can be expected on hosts with more than 2 cpus.

What is the Parallel Young Generation collector (-XX:+UseParNewGC)?

The parallel young generation collector is similar to the parallel garbage collector (-XX:+UseParallelGC) in intent and differs in implementation.

Most of the above description for the parallel garbage collector (-XX:+UseParallelGC) therefore applies equally for the parallel young generation collector.

Unlike the parallel garbage collector (-XX:+UseParallelGC) this parallel young generation collector can be used with the concurrent low pause collector that collects the tenured generation.

Java Performance Tuning

What to monitor in OS Level

1) CPU Utilization

2) Network Traffic

3) Disk I/O

4) Virtual Usage Memory usage

5) Process and Kernal Locks

CPU Monitoring

1) High SYS /Kernal CPU time

2) Idle Cpu

- In Multi threaded applications and muticore system idle cpu can be an indicator of an application’s inability to scale.

3) High VCX ( voluntary Context Switching)

Tools for monitoring

1) Vmstat – (solaris & Linux) - Used to obtain Summaries of CPU’s Usage.

2) Task Manager (windows)

light-weight process (LWP)

is a means of achieving multitasking.

LWP runs in user space on top of a single kernel thread and shares its address space and system resources with other LWPs within the same process.

Network I/O

1) Network Utilization in terms of TCP statistics and established connections

Tools for monitoring

Netstat (solaris and linux)

Monitoring Disk I/o

1) Number of Disk accesses and average Latencies

2) Tools for monitoring

iostat (solaris and linux)

Virtual Memory :Fixing the swapping problem

1) Smaller Java Heap Size

2) Add Physical Memory

3) Reduce number of applications Running on the machine