How to programmatically render DataFormWebPart?

I have a publishing website build with MOSS 2007 and I need to get the page content programmatically. I know that I can use WebClient to sent a request to the page and then parse the response text. But I want to do it in SharePoint model since the scale is quiet large.

The web parta I want to render are DataFormWebPart and they display fine if viewed in a browser. But I'm getting exceptions when trying to render them programmatically.

The code:

var partMgr = siteCollection.RootWeb.GetLimitedWebPartManager(pageUrl, PersonalizationScope.Shared);

var sr = new StreamWriter(@"d:\temp\test.txt", false);
var htr = new HtmlTextWriter(sr);

foreach (WebPart part in partMgr.WebParts)
{
    if (part.GetType() == new DataFormWebPart().GetType())
    {
        try
        {
            htr.WriteLine("");
            part.RenderBeginTag(htr);
            htr.WriteLine("");
            part.RenderControl(htr);
            htr.WriteLine("");
            part.RenderEndTag(htr);
            htr.WriteLine("");
        }
        catch (Exception exc)
        {
            htr.WriteLine("Message: " + exc.Message);
            htr.WriteLine("StackTrace: " + exc.StackTrace);
            htr.WriteLine("InnerException: " + (exc.InnerException == null).ToString());
        }
    }
}
sr.Close();
htr.Close();

The output:

<div id="g_1722aa69_d0d7_4804_83fa_c8f4a250080a">
    Message: Value cannot be null. Parameter name: page
    StackTrace:    at System.Web.UI.WebControls.WebParts.WebPartManager.GetCurrentWebPartManager(Page page)
   at Microsoft.SharePoint.WebPartPages.WebPart.Render(HtmlTextWriter writer)
   at ExtractPageData.Program.Main(String[] args) in D:\DNR-Playground\ExtractPageData\ExtractPageData\Program.cs:line 49
    InnerException: False

    <div id="g_1115045e_b0d7_41ce_ad19_15aab0f3871d">
        Message: Value cannot be null. Parameter name: page
        StackTrace:    at System.Web.UI.WebControls.WebParts.WebPartManager.GetCurrentWebPartManager(Page page)
   at Microsoft.SharePoint.WebPartPages.WebPart.Render(HtmlTextWriter writer)
   at ExtractPageData.Program.Main(String[] args) in D:\DNR-Playground\ExtractPageData\ExtractPageData\Program.cs:line 49
        InnerException: False

As you can see in the output, the begin tag is rendered but the control can't be rendered due to part.Page is null and it it's a readonly attribute.

This just won't work. Web parts require proper SPContext which is setup during HTTP request handling in the ASP.NET pipeline. There's no easy answer how to overcome it. You would essentially have to simulate an HTTP request to the page that you need to render in-memory.

Also, this expression part.GetType() == new DataFormWebPart().GetType() is far from ideal. Rather use part.GetType() == typeof(DataFormWebPart) to prevent instantiation and possible unwanted side effects.