- There is no default group in Filter Transformation.
- The records which does not satisfy filter condition are discarded and not written to reject file or session log file
where does the records goes which does not satisfy condition in filter transformation?
find out duplicate records using aggregator
1) When we are removing duplicate rows using aggregator we have to check all the ports by group by ports.
Without using summation we can eliminate duplicates by using aggregator.
Without using summation we can eliminate duplicates by using aggregator.
How can you improve the performance of Aggregate transformation?
- create group by condition only on numaric columns
- use sorter transimition before aggregator
- give sorter input to aggreator.
- increate the cashe size of aggreator.
how can we delete the cache files while we using aggregator transformaation?
After the session executes DTM removes the cache memory and deletes
cachefiles
Incase of persistant cache and incremtal aggregation cache files are
saved.
Generally cache files are created in Informatica sever no default
memory is allocated for this once memory exceeded than these (index
cache) files You can see this in cache directory.
cachefiles
Incase of persistant cache and incremtal aggregation cache files are
saved.
Generally cache files are created in Informatica sever no default
memory is allocated for this once memory exceeded than these (index
cache) files You can see this in cache directory.
what is the diffrence between SCD and INCREMENTAL Aggregation?
HIRE DIMENSIONAL DATA IS STORED. NO AGGRIGATE CALCULATIONS
SCD WE R USING 3 WAYS
1.TYPE-1: IT'S MAINTAIN CURRENT DATA ONY
2.TYPE-2: IT'S MAINTAIN CURRENT DATA + COMPLET HOSTROY RECORDS
THESE R 3 WAYS :1.FLAG DATA
2.VERSION NO MAPPING
3.EFFECTIVE DATE RANGE
3.TYPE-3: IT'S MAINTAIN CURRENT DATA +ONE TIME HISTROY
INCREMENTAL AGGRATION ARE STORED AGGRAGATE VALUES ACCORDING TO THE USER REQUIREMENTS
scd means 'slowly changing dimentions'since dimention table
maintains master data the column values occationally changed .so
dimention tables are called as scd tables and the fields in the scd
tables are called as slowly changing dimentions .in order to
maintain those changes we are following three types of methods.1.
SCD TYPE1 this method maintains only current data2. SCD TYPE2 this
method maintains whole history of the dimentions here three methods
to identify which record is current one . 1> flag current data 2>
version number mapping 3> effective date range3. SCD TYPE3 this
method maintains current data and one time historical
data.INCREMENTAL AGGRIGATIONsome requirements (daily weekly every 15
days quartly..........) need to aggrigate the values of certain
colums. HERE U have to do the same job every time (according to
requirement) and add the aggrigate value to the previous aggrigate
value(previous run value) of those column.THE PROCESS CALLED AS
INCREMENTAL AGGRIGATION.
SCD WE R USING 3 WAYS
1.TYPE-1: IT'S MAINTAIN CURRENT DATA ONY
2.TYPE-2: IT'S MAINTAIN CURRENT DATA + COMPLET HOSTROY RECORDS
THESE R 3 WAYS :1.FLAG DATA
2.VERSION NO MAPPING
3.EFFECTIVE DATE RANGE
3.TYPE-3: IT'S MAINTAIN CURRENT DATA +ONE TIME HISTROY
INCREMENTAL AGGRATION ARE STORED AGGRAGATE VALUES ACCORDING TO THE USER REQUIREMENTS
scd means 'slowly changing dimentions'since dimention table
maintains master data the column values occationally changed .so
dimention tables are called as scd tables and the fields in the scd
tables are called as slowly changing dimentions .in order to
maintain those changes we are following three types of methods.1.
SCD TYPE1 this method maintains only current data2. SCD TYPE2 this
method maintains whole history of the dimentions here three methods
to identify which record is current one . 1> flag current data 2>
version number mapping 3> effective date range3. SCD TYPE3 this
method maintains current data and one time historical
data.INCREMENTAL AGGRIGATIONsome requirements (daily weekly every 15
days quartly..........) need to aggrigate the values of certain
colums. HERE U have to do the same job every time (according to
requirement) and add the aggrigate value to the previous aggrigate
value(previous run value) of those column.THE PROCESS CALLED AS
INCREMENTAL AGGRIGATION.
How to use incremental aggregation in real time?
The first time you run a session with incremental aggregation enabled the server process the entire source.
At the end of the session the server stores aggregate data from that session ran in two files the index file and data file.
The server creates the file in local directory.The second time you run the session use only changes in the source as source data for the session.
The server then performs the following actions:For each input record the session checks the historical information in the index file for a corresponding group then:If it finds a corresponding group – The
server performs the aggregate operation incrementally using the aggregate data for that group and saves the incremental changes.
Else Server create a new group and saves the record dataWhen writing to the target the server applies the changes to the existing target.
Updates modified aggregate groups in the targetInserts new aggregate dataDelete removed aggregate dataIgnores unchanged aggregate dataSaves modified aggregate data in Index/Data files to be used as historical data the next time you run the session.
Each Subsequent time you run the session with incremental aggregation you use only the incremental source changes in the session.
If the source changes significantly and you want the server to continue saving the aggregate data for the future incremental changes configure the server to overwrite existing aggregate data with new aggregate data.
At the end of the session the server stores aggregate data from that session ran in two files the index file and data file.
The server creates the file in local directory.The second time you run the session use only changes in the source as source data for the session.
The server then performs the following actions:For each input record the session checks the historical information in the index file for a corresponding group then:If it finds a corresponding group – The
server performs the aggregate operation incrementally using the aggregate data for that group and saves the incremental changes.
Else Server create a new group and saves the record dataWhen writing to the target the server applies the changes to the existing target.
Updates modified aggregate groups in the targetInserts new aggregate dataDelete removed aggregate dataIgnores unchanged aggregate dataSaves modified aggregate data in Index/Data files to be used as historical data the next time you run the session.
Each Subsequent time you run the session with incremental aggregation you use only the incremental source changes in the session.
If the source changes significantly and you want the server to continue saving the aggregate data for the future incremental changes configure the server to overwrite existing aggregate data with new aggregate data.
Subscribe to:
Posts (Atom)