Wednesday, December 26, 2012

Pentaho Data Integration Repositories & Commands to run transformations and Jobs

Pentaho Data Integration Repositories & Commands to run transformations and Jobs :

We can save kettle jobs and transformations in three type of repositories.
 1. File Repository
 2. Database Repository
 3. Enterprise Repository (We can create this repository in Pentaho Enterprise Edition )
             
          I have mentioned the commands to run the transformations and jobs which are saved in all three types of repositories for windows as well as Linux.
 
    To run the jobs, we can use kitchen.bat or kitchen.sh command.
    To run the transformations, we can use pan.bat or pan.sh command

 Do the following steps to run the commands.

 1. Open the command prompt
 2. Go to the tool home directory.
   $> cd <data-integration-home>
      for me, it is a c:\pentaho\design-tools\data-integration.
      ex : cd c:\pentaho\design-tools\data-integration 
 3. Now run the command

Windows :

 File Repository: 
      pan.bat /file:"C:\Users\Suribabu\Desktop\Pentaho\repository\Trans\Example.ktr" /level:Basic 
      kitchen.bat /file:"C:\Users\Suribabu\Desktop\Pentaho\repository\Jobs\Example.kjb" /level:Basic 

 Database Repository :
        pan.bat /rep:"DB_Rep" /trans:"Example" /dir:/Trans/ /user:admin /pass:admin /level:Basic 
        kitchen.bat /rep:"DB_Rep" /job:"Example" /dir:/Jobs/ /user:admin /pass:admin /level:Basic 

 Enterprise Repository:
        pan.bat /rep:"DB_Rep" /trans:"Example" /dir:/public/Trans/ /user:joe /pass:password /level:Basic
        kitchen.bat /rep:"DB_Rep" /job:"Example" /dir:/public/Jobs/ /user:joe /pass:password /level:Basic


 Linux : 
File Repository:
     pan.sh -file="/home/suri/repository/Trans/Example.ktr" -level=Minimal
     kitchen.sh -file="/home/suri/repository/Jobs/Example.ktr" -level=Minimal 

 Database Repository : 
       pan.bat -rep:"DB_Rep" -trans:"Example" -dir:/Trans/ -user:joe -pass:password -level:Basic 
       kitchen.bat -rep:"DB_Rep" -job:"Example" -dir:/Jobs/ -user:joe -pass:password -level:Basic 

 Enterprise Repository:
       pan.bat -rep:"DB_Rep" -trans:"Example" -dir:/public/Trans/ -user:joe -pass:password -level:Basic
       kitchen.bat -rep:"DB_Rep" -job:"Example" -dir:/public/Jobs/ -user:joe -pass:password -level:Basic 

 You can schedule the above commands or You can write wrapper script on top of the commands as follows and schedule it:

example.bat or example.sh 
# cd <data-integration-home>
cd c:\pentaho\design-tools\data-integration 
pan.bat /file:"C:\Users\Suribabu\Desktop\Pentaho\repository\Trans\Example.ktr" /level:Basic


 Now You can run and schedule example.bat or example.sh file for your ETL Trans.