Discard Rows in a flat file

Say I have a flat file with the form

FHRxxxxxxxxxxxxxxxxxxxxx
SHR1xxxxxxxxxxxxxxxxxxxx
DataRow
DataRow
STR1xxxxxxxxxxxxxxxxxxxx
SHR2xxxxxxxxxxxxxxxxxxxx
DataRow
DataRow
DataRow
STR2xxxxxxxxxxxxxxxxxxxx
FTRxxxxxxxxxxxxxxxxxxxxx

Where FHR=File Header, SHR=Section Header, STR=Section Trailer & FTR=File Trailer

How can I discard all the header and trailer rows?

I have the manager defined as ragged right and have defined the fixed length columns.

I know I can tell the flat file connection manager to discard n header records. I can also tell it to redirect rows which are truncated to "nowhere" but some headers & trailers still get through (they're longer than data rows)

Is there any other way I can discard these rows? Something like "if this rows starts with SHR/STR/etc. ignore it"?

Ta.

Greg.

Greg,

You can use a conditional split transform to filter out these rows but this is AFTER they have been loaded into the pipeline. if you want to discard them BEFORE they get into the pipeline then you're probably into script source component or custom source adapter territory.

Donald Farmer has a fantastic example of how to load "irregular" files such as these using the script component in his book - you can find it on Amazon.

-Jamie

|||

Hey Greg,

the easiest way is the to use the conditional split component.

Your condition looks like this example:

SUBSTRING(Input,1,3) == "SHR"

For more power take a look at the scripting book of Donald Farmer.

I like the scripting component!!!

Loom

|||Thanks guys. Great stuff

Greg.