Sunday, 4 November 2018

Expression Transformation in Informatica


Expression transformation is a connected, passive transformation used to calculate values on a single row.
Expression transformations are used for row-wise manipulation.
For any manipulation, you wish to perform on an individual record, use an Expression transformation.
The Expression transformation accepts the row-wise data, manipulates it, and passes it to the target.
The transformation receives the data from the input port and sends the data out from output ports.

An expression transformation is created with following types of ports.


  1. Input port (I)
  2. Output Port (O)
  3. Variable Port (V)

  • Data Manipulation      : concatenation( CONCAT or || ) , Case change (UPPER,LOWER) truncation, InitCap (INITCAP)
  • Datatype conversion :  (TO_DECIMAL, TO_CHAR, TO_DATE)
  • Data cleansing – check nulls (ISNULL) , replace chars, test for spaces (REPLACESTR) , test for number
  • Manipulate dates – convert, add, test (GET_DATE_PART, IS_DATE, DIFF_DATES)
  • Scientific calculations and numerical operations – exponential, power, log, modulus (LOG, POWER, SQRT)
  • ETL specific – if, lookup, decode (IIF, DECODE).



Open folder where we want to create the mapping.

Click Tools -> Mapping Designer.
Click Mapping -> Create -> Give mapping name. Ex: m_totalsal
Drag EMP from source in mapping.
Click Transformation -> Create -> Select Expression from list. Give name and click Create. Now click done.




Now create a new port out_Total_SAL. Make it as output port only.
Click the small button that appears in the Expression section of the dialog box and enter the expressi
on in the Expression Editor.
Enter expression SAL + 10000 . You can select SAL and 10000 from Ports tab in expression editor.
Check the expression syntax by clicking Validate.
Click OK -> Click Apply -> Click Ok.
Now connect the ports from Expression to target table.
Click Mapping -> Validate
Repository -> Save









What is Filter Transformation?


Filter transformation is an active, connected transformation.
The filter transformation is used to filter out rows in a mapping.
It may change the number of rows passed through it.
You have to specify a filter condition in the filter transformation.
The rows that meet the specified filter condition are passed to other transformations.
The rows that do not meet the filter condition are dropped.


 For example, for loading the employee records having deptno equal to 20 only, we can put filter transformation in the mapping with the filter condition deptno=20.

Step 1 – Create a mapping having source "EMP" and target "EMP_TAR"





source "EMP"



 target "EMP_TAR"

  


 mapping having source "EMP" and target "EMP_TAR"






Step 2 – Then in the mapping
  1. Select Transformation menu
  2. Select create option Filter Transformation from the list


Step 3 – The filter transformation will be created, Select "Done" button in the create transformation window.



Step 4 – In the mapping
  1. Drag and drop  the Source qualifier columns to the filter transformation.


Step 5 – Double click on the filter transformation to open its properties, and then


  1. Select the properties menu.
  2. Click on the Filter condition editor.
  3. Then in the filter condition expression editor.
  4. Enter filter condition – deptno=20
  5. Select OK button.



Step 6 – Now again in the edit transformation window in Properties tab you will see the filter condition, select OK button.





Step 7 – Link the columns from filter transformation to the target table.






Step 8 – Save the mapping.
1). create new workflow




Step 9 –   Create the connection ,link tasks and session.



Step 10 –   Save the workflow.

1) Right click and start workflow.
2) check the workflow monitor.










Step 10 –   Connect target database and check the emp_tar table
.





Sunday, 28 October 2018

List of Active and Passive Transformations in Informatica?

Active and Passive Rule



1. Active Transformation:- An active transformation can perform any of the following actions:

(a) Change the number of rows that pass through the transformation
(b) Change the transaction boundary:.
(c) Change the row type:


2. Passive Transformation:- An Passive transformation which will satisfy all below conditions:

(a) Do not Change the number of rows that pass through the transformation
(b) Maintains the transaction boundary
(c) Maintains the row type.



3. Connected Transformations:- Transformations which are connected to other transformation in the data flow are called Connected Transformations

4. Unconnected Transformations:- Transformations which are not connected to other transformation in the data flow are called as Unconnected Transformations.An unconnected transformation is called within another transformation, and returns a value to that transformation.


   


Informatica Transformations – List






Sl No Transformations Name

1 Aggregator Transformation
2 Application Source Qualifier Transformation
3 Custom Transformation
4 Data Masking Transformation
5 Expression Transformation
6 External Procedure Transformation
7 Filter Transformation
8 HTTP Transformation
9 Input Transformation
10 Java Transformation
11 Joiner Transformation
12 Lookup Transformation
13 Normalizer Transformation
14 Output Transformation
15 Rank Transformation
16 Reusable Transformation
17 Router Transformation
18 Sequence Generator Transformation
19 Sorter Transformation
20 Source Qualifier Transformation
21 SQL Transformation
22 Stored Procedure Transformation
23 Transaction Control Transaction
24 Union Transformation
25 Unstructured Data Transformation
26 Update Strategy Transformation
27 XML Generator Transformation
28 XML Parser Transformation
29 XML Source Qualifier Transformation
30 Advanced External Procedure Transformation
31 External Transformation

Sunday, 14 October 2018

Source/Target Designer in Informatica – Import Source/Target table from database

Source Analyzer - The first tool in PowerCenter Designer,  is used to identify the sources that will be used to build the data mart or warehouse and create definitions for those sources. Source definitions describe the sources that are going to be providing data to the warehouse. There are different ways to create source definitions.


The source or target created/imported in Informatica can be reused any no of times in different mappings. Every mapping must have at least on loadable target. Otherwise mapping will be invalid.

Note- When we create source/target in source analyzer/target designer, structures are only created in Informatica. At the database level, there is no object created. So, you have to create Database objects having the same structure as you have created in Informatica.

You can import following type of sources using source analyzer

Relation tables ( database tables), views and synonyms
Flat files
Cobol files
XML files

How to open Source Analyzer

Step 1 - Open Informatica PowerCenter designer tool





Step 2 – In next screen
  1. Double click on the repository which you want to connect
  2. Enter username
  3. Enter Password
  4. Click on Connect button





After Successful login, the folders of the user will be listed under the repository name.

  1. If login is not successful, check if the user has privileges of connecting to repository
  2. If no folder is visible under repository name, check if the folder is created.

Step 3 – In the next step
  1. Right click on the folder
  2. Select open option




When a folder is open, sub folders of that folder will be listed.




Step 4 – Click on the Source analyzer menu, as shown in the figure.


How to import Source table in Source Analyzer

Step 1 - In source analyzer
  1. Click on tab "sources" from the main menu
  2. Select import from database option, after this ODBC Connection box will open.

Step 2 – We will now create ODBC connection (If you already have ODBC connection, move to step 3)
  1. Click on the button next to ODBC data Source.



  1. On the next page, Select user DSN tab and click Add button.


When you click on the add button, you will see a list of drivers for various databases (Oracle, SQL Server, Sybase, Microsoft Access, Excel, etc.) The driver which you select depends on what database you want to connect. Select the driver for the data source. 



On the next page, select the general tab and enter database details. Then Click on test connect.


Step 3 -  you will be using oracle's Scott/Tiger schema tables. If you do not have these tables in the database, create those using this script.

Step 4 – In the import tables window.
  1. Select ODBC data source for the oracle database.
  2. Enter database username
  3. Enter database password
  4. Click on connect/reconnect button. This will show tables for the database user.
  5. Expand the tree under tables folder and select EMP table
  6. Select OK button.

Step 5- The table will be imported in the Informatica Source Analyzer. 



Similarly, you can import any other database tables in source analyzer.

How to import target in target designer


In the Informatica Designer, click target designer icon to launch target, designer.




import target in Target Designer of Informatica Powercenter



you have imported sources and targets. Now, you are all set to create your first mapping.












Saturday, 13 October 2018

Informatica power center

Informatica power center 
constitutes of three main components:

Client tools: installed on developer machines.
Power Centre repository: place to store metadata for an application
Power center server: server to perform data executions
With growing customer base, Informatica is continuously trying to leverage its data integration solutions. This tool has in built powerful mapping templates to help manage data in an efficient manner.



There are mainly 4 steps in the Informatica ETL process,


  1. Extract or Capture
  2. Scrub or Clean
  3. Transform
  4. Load and Index

Features of Informatica ETL: 


For all the Data integration and ETL operations, Informatica has provided us with Informatica PowerCenter. Let us now see some key features of Informatica ETL:
  • Provides facility to specify a large number of transformation rules with a GUI.
  • Generate programs to transform data.
  • Handle multiple data sources.
  • Supports data extraction, cleansing, aggregation, reorganisation, transformation, and load operations.
  • Automatically generates programs for data extraction.
  • High-speed loading of target data warehouses.

List of ETL Tools

List of ETL tools -

1 Oracle Warehouse Builder (OWB)
2 SAP Data Services
3 IBM Infosphere Information Server
4 SAS Data Management
5 PowerCenter Informatica
6 Elixir Repertoire for Data ETL
7 Data Migrator (IBI)
8 SQL Server Integration Services (SSIS)
9 Talend Studio for Data Integration
10 Sagent Data Flow
11 Actian DataConnect
12 Open Text Integration Center
13 Oracle Data Integrator (ODI)
14 Cognos Data Manager
15 CloverETL
16 Centerprise Data Integrator
17 IBM Infosphere Warehouse Edition
18 Pentaho Data Integration
19 Adeptia Integration Server
20 Syncsort DMX
21 QlikView Expressor
22 Relational Junction ETL Manager (Sesame Software)

What is Informatica?

What is Informatica?

Informatica is a Software development company, 
which offers data integration products. 
It offers products for ETL, data masking, data Quality, data replica, data virtualization, master data management, etc.


What is ETL?

ETL is a type of data integration and involves an architecture that extracts, transforms, and then loads data in target database or file.  It is the foundation of data warehouse.



Extracts data from source systems
Tracks changes made to the source data required for the warehouse
Restructures keys
Maintains the metadata
Refreshes the warehouse with updated data
Transforms and cleans up the data
Indexes data
Summarizes data

Loads data into the warehouse


What do I need to know about ETL?

Data must be properly formatted and normalized in order to be loaded into these types of data storage systems, 
and ETL is used as shorthand to describe the three stages of preparing data. ETL also describes the commercial software category that automates the three processes. 

What is the Future of ETL?
Informatica -ETL products and services are provided to improve business operations, reduce big data management, provide high security to data, data recovery under unforeseen conditions and automate the process of developing and artistically design visual data. They are broadly divided into-

1. ETL with Big Data
2. ETL with Cloud
3. ETL with SAS
4. ETL with HADOOP
5. ETL with Meta data etc. 
6. ETL as Self-service Access

7. Mobile optimized solution etc


Expression Transformation in Informatica

Expression transformation is a connected, passive transformation used to calculate values on a single row. Expression transformations ar...