I have a test environment like this:
The subnet mask is always 255.255.0.0
When I use these fixed ip addresses, i can ping from my W7 clients to the TMG. Now I want to use DHCP for my W7 clients, so the TMG needs to do DHCP Relaying to the DHCP Server in VLAN 101.
I use wireshark to see the DHCP Discover originate from the client, and wireshark on the TMG server also sees it arriving. However, I don't see this in the Logging section of TMG.
The request gets no response, I suspect because the TMG doesn't do anything with it.
I followed this article on Configuring the DHCP Relay Agent on ISA Server 2004 by Microsoft: https://technet.microsoft.com/en-us/library/cc302680.aspx
I am totally stuck so any help would be much appreciated.
I finally figured out what I need to do. It is a combination of problems and this is how I resolved them:
I’m assuming you already added the VLANS on the host and created virtual switches for them.
Shut down the virtual machine.
In Hyper-V: add hardware, add network adapter, enable vlan identification ‘104’, use virtual switch ‘Switch VLAN 104’.
Start the TMG VM.
Go to your network adapters and find the new one. Rename it to ‘VLAN 104’.
Disable ipv6, set static ip address. (I use 10.104.0.254, subnet mask 255.255.0.0)
Leave default gateway and dns servers empty.
Open Routing and Remote Access (RRAS). You will see it is still running (if it was running before).
We want to add our new adapter under ipv4 DHCP Relay Agent, but the new adapter is not listed.
We need to disable rras and enable it again to make it appear, so we will first save the RRAS configuration.
Open cmd as administrator and run ‘netsh routing ip relay dump > RRASBackUp.txt’
Go back to RRAS, right click the server and choose ‘disable routing and remote access’.
Right click again and choose ‘configure and enable Routing and Remote Access’.
I chose custom configuration, select services ‘VPN access’, ‘Dial-up access’, ‘Demand-dial conections’ and ‘LAN routing’. No NAT, since TMG will be doing that.
Start the service. An error will occur. This is because ipv6 has to be removed.
TMG and ipv6 don't go well together.
Run regedit and go delete this entry and everything under it:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\RemoteAccess\RouterManagers\Ipv6
Refresh the RRAS screen and ipv6 wil disappear.
Right-click the server, all tasks, start server. It will start now.
Go back to your cmd as administrator and run this command: ‘netsh -f RRASBackUp.txt’ to restore the configuration. On the RRAS screen, right-click the server, all tasks, restart server.
Go to ipv4, DHCP Relay Agent, right click, new interface and select VLAN 104 (it is there now).
Try to get an ip from a test client.
Refresh the DHCP Relay Agent screen to see if you receive requests.
If all requests get discarded, right click DHCP Relay Agent and open properties. Check if your DHCP Server address is in the list.
Also remember that there’s still a TMG that will block DHCP traffic! (I allow all DHCP protocols from all networks to all networks)
Don’t forget to add the new adapter to your TMG:
Go to networking, tab networks, select internal, edit selected network, tab addresses, add adapter, select ‘VLAN 104’.
When you restart the TMG server, you might see that RRAS does not start.
This is because the service starts delayed. So just have patience!
If at some point, TMG breaks RRAS, and it does not start anymore, check if the ipv6 entry is back, and delete it again from the registry to fix it.