Memory Usage of SSIS 2005

[Webmonkey26]

We have written a simple app that executes a simple DTS package constantly to
monitor memory. We were expecting to see memory usage maintained at the same
level as the package is not actually doing anything with the data, however we
are seeing a small, steady increase in memory consumed by the executing
process.

.NET code that calls the execution looks like...
XmlDocument packageXml = new XmlDocument();
packageXml.Load("c:\\test.dtsx");

Package package = new Package();
package.LoadFromXML(packageXml, null);

while (1 > 0)
{
package.Execute();
}

We run this in a console app and memory seems to creap up at the rate of
approx 12kb's per second.

If we re-run the test with the package.Execute(); commented out then memory
usage remains fixed and does not increase.

The DTSX package is as simple as they come, its just a single dataflow task
that reads from a small table with < 20 rows and does nothing with the data.

Anyone seen anything like this before? or any clues what might be causing
the leak?

Thanks

WebMonkey

[Charles Wang [MSFT]]

Hi,
Why do you make a dead-loop to execute the package?

Everytime you execute a package, the process need to allocate the space for
the package execution. However the memory release of the package may not be
immediately in .NET application for which GC determines when to release
memory. Normally you need not worry about this.

Best regards,
Charles Wang

[Webmonkey26]

The dead-loop if purely to demonstrate the problem we have.

The when monitoring the memory usage then it continues to climb and is never
free'd. Over a long period of time the service will crash due to lack of
available memory.

We monitored the execution of a simple package that read 20 rows of data but
did not do anything with it. Execution of the package using this code
resulted in the process consiming 1.5 Gbs of memory after 2 days.

If we remove the Execute call then memory does not increase at all.

If we new up a Package object in the loop so that a new Package object is
created on each iteration then we are presented with the same problem of
memory increasing idenfinatly.

We have experimented with calling GC Collect but this appears to make no
difference to the memory consumption of the process.

Execute in the loop

Post by Charles Wang [MSFT]
Hi,
Why do you make a dead-loop to execute the package?
Everytime you execute a package, the process need to allocate the space for
the package execution. However the memory release of the package may not be
immediately in .NET application for which GC determines when to release
memory. Normally you need not worry about this.
Best regards,
Charles Wang
.

[Charles Wang [MSFT]]

Hi,
Thank you for your clarification.

I performed a test at my side and reproduced this issue. After looked at
the Performance Monitor, I found that the "Bytes in all heaps" counter
value is constant for the process, but the working set private bytes value
is continously increased. This indicates that the memory on the manged heap
of your process is normal and that the problem should be related to
unmanaged resources. Since the internals of the .NET SSIS library is
undocumented, to track the root cause of this issue, you need to contact
Microsoft Customer Support Services (CSS) for dump analysis. Please be
advised that contacting phone support will be a charged call.

To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS

If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.

As a workaround, I recommend that you use DTEXEC utility to execute your
SSIS packages. You can call the process to execute the package and exit the
process after the execution. By this way, you should not face this memory
issue.

For more information, you can refer to:
dtexec utility
http://msdn.microsoft.com/en-us/library/ms162810.aspx

Process.Start Method
http://msdn.microsoft.com/en-us/library/system.diagnostics.process.start.asp
x

How to wait for a shelled application to finish by using Visual C#
http://support.microsoft.com/kb/305369

Best regards,
Charles Wang

[Webmonkey26]

Thanks for the clarification Charles, I will progress this with a support
case as you suggest.

Post by Charles Wang [MSFT]
Hi,
Thank you for your clarification.
I performed a test at my side and reproduced this issue. After looked at
the Performance Monitor, I found that the "Bytes in all heaps" counter
value is constant for the process, but the working set private bytes value
is continously increased. This indicates that the memory on the manged heap
of your process is normal and that the problem should be related to
unmanaged resources. Since the internals of the .NET SSIS library is
undocumented, to track the root cause of this issue, you need to contact
Microsoft Customer Support Services (CSS) for dump analysis. Please be
advised that contacting phone support will be a charged call.
To obtain the phone numbers for specific technology request please take a
look at the web site listed below.
http://support.microsoft.com/default.aspx?scid=fh;EN-US;PHONENUMBERS
If you are outside the US please see http://support.microsoft.com for
regional support phone numbers.
As a workaround, I recommend that you use DTEXEC utility to execute your
SSIS packages. You can call the process to execute the package and exit the
process after the execution. By this way, you should not face this memory
issue.
dtexec utility
http://msdn.microsoft.com/en-us/library/ms162810.aspx
Process.Start Method
http://msdn.microsoft.com/en-us/library/system.diagnostics.process.start.asp
x
How to wait for a shelled application to finish by using Visual C#
http://support.microsoft.com/kb/305369
Best regards,
Charles Wang
.