sql-serverssisbidsssis-2008

Why does child SSIS package's .dtsx file get opened in BIDS when running parent package in debug mode?


SSIS 2008 R2 + BIDS

I'm in the process of rewriting an ETL job. The job is composed of about 50 steps, some of which are SSIS packages, and some of which are just stored proc calls. I'm writing a wrapper proc (Gatekeeper) that checks which of these "steps" are ready to be executed, and then executes the appropriate .dtsx package or stored proc.

When the Gatekeeper starts up, calls a stored proc to get a list of "steps" for the current run. It then goes through a Foreach Loop, checks whether a "step" is a package or a proc, based on a flag column, and then dynamically runs the appropriate task type for the step. After the step completes, I log either the success or the failure, and iterate the Foreach Loop.

So far, I've gotten the business logic and the dynamic execution of packages and of stored procs working correctly. For stored proc calls I build the query in a string variable, and use an Execute SQL Task with SqlStatementSourceType set to Variable. For packages, I use an Execute Package Task that uses a File Connection Manager that uses an expression to set the ConnectionString to the path to the .dtsx file on the file system.

This all working how I want it to, and the appropriate stored procs and packages are being executed at the appropriate times.

So what's the problem? I've noticed that when the Gatekeeper has to execute 2 or more packages in a row, when it hits the Execute Package Task each consecutive time, BIDS attempts to open the previously completed child package. All the packages we use are encrypted with a password (same for all files), and I get the "Package Password" prompt when BIDS tries to open the file.

What's weird is that if the order is Package -> proc -> package, the strange file opening behavior doesn't happen. I've tried changing the various DelayValidation values in the child packages and in the Gatekeeper, I've tried running inside and outside of the parent process, but these don't seem to have any bearing on the problem.

Here's a screenshot of the Gatekeeper's Control Flow: enter image description here

Here's the configuration of the File Connection Manager: enter image description here

And here's the configuration of the Execute Package Task: enter image description here

EDIT:

On a hunch, I tried preemptively opening two child packages in BIDS before running a test execution of the Gatekeeper. I watched as BIDS hooked into the first child package and went through its tasks in debug mode. "Cool, maybe it was just trying to open the file to run it in debug mode" I thought. Nope. Once it hit the second child package, BIDS tried to open another copy of the first child package.

EDIT 2:

Right after I posted my previous edit, I realized that I may not have had the exact same files already open that the Gatekeeper was trying to run. I had the source files from my TFS local folder open, instead of the packages actually deployed to the file system. So I closed the source files for the children, and left the first deployed child file open and reran the Gatekeeper. It still hooked into the open child file on the first iteration, and to my surprise, it did not try to open another copy of child #1 when it hit the second iteration.

So my thinking now is that this is some kind of weird debug issue with BIDS. My next test is going to be deploy Gatekeeper to the Test server and see if I run into any problems when executing it via a SQL Agent Job.


Solution

  • I never did figure out the issue with BIDS, but I found a workaround. Instead of trying to execute the child package directly from Gatekeeper, I have a SQL Agent Job that has all the children as individual steps. Each step is configured to Quit on Success or on Failure.

    The Gatekeeper calls a stored proc that provides the job_id and step_id of the child step that is ready to be executed. It then uses those values to execute msdb.dbo.sp_start_job in an Execute SQL Task. The code I'm using to that is:

    DECLARE @step VARCHAR(100)
    ,@Job VARCHAR(100)
    
    SET @step = ?
    SET @Job = ?
    
    EXEC msdb.dbo.sp_start_job @job_id = @job, @step_name = @step
    

    I had to change the Control Flow a bit. I no longer care about if a child is a proc or an SSIS package, but I need to check that the "child steps" job isn't running before starting the child job step, but I had to add some waits so SSIS didn't try to execute sp_start_job too soon.

    enter image description here