How to get access token from ACS via SAML assertion?

I was able to retrieve, decode the security token (SWT) from various identity providers configured in ACS. Now I should be - according to example - able to do this:

String headerValue = string.Format("WRAP access_token=\"{0}\"", securityToken);

WebClient client = new WebClient();
client.Headers.Add("Authorization", headerValue);

using (Stream stream = client.OpenRead(@"http://xxx.cloudapp.net/xxx.svc/users"))
using (StreamReader reader = new StreamReader(stream))
{
    String response = reader.ReadToEnd();
}

It works in a sense it fails for a non-existing endpoint for example. So service is there (secured), token module and token validator on the server side are called, and token passes through. So it's not that. But anyway the problem is that the response contains HTML of a login page (that one with identity provider list in it). It seem like as though the token validation was OK, it still isn't enough for security.

What should I do now, to receive my data from a service? Any hints?

Scenario: http://tinyurl.com/WcfRestSaml

Update: I've included link to a picture of scenario I'm trying to achieve.

Update 2: OK, I've switched to Saml2, but same error has occured. Then I've found out I need assertion to receive access token. So I did:

WebClient client = new WebClient { BaseAddress = string.Format("https://{namespace}.accesscontrol.windows.net") };
NameValueCollection parameters = new NameValueCollection
{
    { "wrap_assertion_format", "SAML" },
    { "wrap_assertion", securityToken },
    { "wrap_scope", "http://{our}.cloudapp.net/" }
};

Byte[] responseBytes = client.UploadValues("WRAPv0.9", parameters);
String response = Encoding.UTF8.GetString(responseBytes);

This returns yet another error as well though:

Error:Code:401:SubCode:T0:Detail:ACS50008: SAML token is invalid.:TraceID:1d3774fa-a5e6-3e3b-a5e5-5a0bde6e0771:TimeStamp:2013-06-06 16:18:05Z

But it seems that this should return my desired access token.

Update 3: Nothing helps, nowhere to gather the information, damn. I'm posting a full token on a blind chance somebody will notice something is wrong at least (I've removed sensitive informations though).

<Assertion ID="_541a71ba-1e00-478c-8d2b-0beac3a35d35" IssueInstant="2013-06-07T11:38:31.741Z" Version="2.0" xmlns="urn:oasis:names:tc:SAML:2.0:assertion">
  <Issuer>https://{removed}.accesscontrol.windows.net/</Issuer>
  <ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#">
    <ds:SignedInfo>
      <ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
      <ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256" />
      <ds:Reference URI="#_541a71ba-1e00-478c-8d2b-0beac3a35d35">
        <ds:Transforms>
          <ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature" />
          <ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#" />
        </ds:Transforms>
        <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256" />
        <ds:DigestValue>{removed}</ds:DigestValue>
      </ds:Reference>
    </ds:SignedInfo>
    <ds:SignatureValue>{removed}</ds:SignatureValue>
    <KeyInfo xmlns="http://www.w3.org/2000/09/xmldsig#">
      <X509Data>
        <X509Certificate>{removed}</X509Certificate>
      </X509Data>
    </KeyInfo>
  </ds:Signature>
  <Subject>
    <NameID>https://www.google.com/accounts/o8/id?id={removed}</NameID>
    <SubjectConfirmation Method="urn:oasis:names:tc:SAML:2.0:cm:bearer" />
  </Subject>
  <Conditions NotBefore="2013-06-07T11:38:31.694Z" NotOnOrAfter="2013-06-07T12:38:31.694Z">
    <AudienceRestriction>
      <Audience>http://{removed}.cloudapp.net/</Audience>
    </AudienceRestriction>
  </Conditions>
  <AttributeStatement>
    <Attribute Name="http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress">
      <AttributeValue>{removed}</AttributeValue>
      <AttributeValue>https://www.google.com/accounts/o8/id?id={removed}</AttributeValue>
      <AttributeValue>{removed}</AttributeValue>
    </Attribute>
    <Attribute Name="http://schemas.microsoft.com/accesscontrolservice/2010/07/claims/identityprovider">
      <AttributeValue>Google</AttributeValue>
    </Attribute>
  </AttributeStatement>
</Assertion>

Solution

As noted above:

We've solved it with Microsoft Support, it was caused by the fact, the service was behind passive federation, and the process doesn't like obviously. I've solved it by creating a <RequestSecurityToken> envelope for the SAML2 assertion token, this way it "simulated" browser activity, and it works well and also plays well with passive federation (due to WS-TRUST envelope).

It needed to pass it first and then it connected to ACS.The problem wasn't in access token or SAML assertion at all. It was caused - as I've pointed out - by the fact that part of website was behind the custom STS federation. The WS Federation module was blocking reception of this token. When I enveloped it in RequestSecurityToken for WS Federation to chew on first. It then passed to ACS as presented in my question.