Kicking the Tires on the TMG 2010 RC ISP Redundancy

[LEFT][CODE]http://www.isaserver.org/tutorials/Kicking-Tires-TMG-2010-RC-ISP-Redundancy-Part1.html[/CODE]

[B][SIZE=3]PART-1[/SIZE][/B]

[B]TMG 2010 RC ISP redundancy; how to configure the virtual infrastructure and the TMG firewall interfaces. [/B]


[B]Introduction[/B]

One of the features that I have been looking most forward to in the new TMG firewall is the ISP load balancing feature. If you have been in the ISA firewall space for a while, you might know that support for multiple ISPs is a feature that has been requested since ISA 2004 came out. Good things are worth waiting for, and now we are going to have it with the upcoming TMG firewall!
Before getting into the details of the TMG multi-ISP feature, there are a few things you should know about it in advance:

[LIST][*] While the term for the feature is “multi-ISP” support, it more accurately could be called “dual ISP” support, since you are limited to two ISPs[*] There must be a NAT relationship between the source and destination Networks – so if you are using a route relationship on any of the TMG firewall Protected Networks, they are not going to be able to take advantage of multiple ISPs[*] Each ISP connection needs to connect to a default gateway that is on a different network ID as the other ISP – both default gateways can not be on the same network ID (which means the external IP addresses on the TMG firewall can not be on the same network ID)[*] Your external interfaces can not use DCHP to get their addresses – if you are using a home-user type ISP connection, then multi-ISP support is not for you[*] You can host both ISP connections on one NIC, or two NICs – this article covers a two-NIC configuration where each ISP connection is represented by its own external interface[*] Network offload processing needs to be set either on or off on both NICs – if one of the NICs has it on and the other has it off, then offloading processing will be disabled on the NIC that has it on.[/LIST]
Those are some of the basic things you should know about multi-ISP support before we get started.
Multi-ISP support allows you to use your two ISPs in either of two ways:

[LIST][*] [B]Failover only[/B] – in this mode one ISP is used at all times [I]until[/I] it becomes unavailable. When that happens, connections are forwarded to the secondary ISP. This is a good choice when you have one high speed link and the other is slow – but good enough to get you by until the primary link comes back up again – also, you do not have to pay for bandwidth used on a link that not being used until required[*] [B]Failover and load balancing[/B] – in this mode both links are used. You have the option to set a weighting for each of the links, so if you do not want to use both of them equally, you don’t have to. If one of the links fails, all the connections go to the one that’s online[/LIST]
All that stuff sounds good on paper, but does it really work? Yes it works. But, do you really want to believe me, or do you want to set it up yourself and see that [I]you[/I] can make it work. I suspect the latter. Thus, I am going to show you how to make it work in a virtual environment.
As you might know, my preferred dev environment for articles on ISAserver.org is VMware Workstation (I’m currently using version 6.5). Why? Because that is what I have always used, it works for me, I understand it and its networking model. There is no reason you have to use VMware Workstation to test this configuration yourself. If you like Windows Virtual PC, ESX Server, or Microsoft Hyper-V, those will work great too. The only difference is going to be in the terminology, but all the same principles apply to each product.
Let us start with the basic virtual networking scheme for this test lab. We are going to use four virtual networks or virtual switches, each of which belongs to a different physical or virtual Ethernet broadcast segment.

[LIST][*] [B]Bridged[/B] – this is the live production network in my office. The virtual NICs are bridged to this network, have valid IP addresses on the live network, and connect to the Internet on this network. [*] [B]VMNet3[/B] – This is a virtual switch that represents the Ethernet segment that connects the TMG firewall to the first ISP[*] [B]VMNet4[/B] – this is the virtual switch that represents the Ethernet Segment that connects the TMG firewall to the second ISP[*] [B]VMNet2[/B] – this is the virtual switch that presents that Ethernet segment connecting the internal interface of the TMG firewall to the default Internet Network[/LIST]
The figure below shows each of the VMNets and the devices that are connected to them:

[LIST][*] [B]RRAS1[/B] – this is a Windows Server 2003 VM with the RRAS Service configured as a NAT server. The external interface of this VM is connected to the Bridged network and the internal interface is connected to VMNet3 – which connects the NIC on the TMG firewall dedicated to RRAS1 ISP to the RRAS Windows 2003 NAT server[*] [B]RRAS2 [/B]– this is a Windows Server 2003 VM with the RRAS Service configured as a NAT server. The external interface is connected to the Bridged network and the internal interface is connected to VMNet4 – which connects the NIC on the TMG firewall dedicated to the RRAS2 ISP to the RRAS Windows NAT server[*] [B]TMG Firewall[/B] – the TMG firewall has three NICs. One connected to VMNet3, which connects it to the RRAS1 ISP; one connected to VMNet4, which connects it to the RRAS2 ISP; and one connected to VMNet2, which connects it to the default Internet Network[*] [B]DC[/B] – is a Windows Server 2003 domain controller for the msfirewall.org domain; the TMG firewall belongs to this domain and it is connected to VMNet2[/LIST]
Some things to note about the configuration:

[LIST][*] The RRAS1 and RRAS2 nodes represent the default gateways you would use when configuring ISPs for your production configuration. So, internal IP address of RRAS1 represents the default gateway of your first ISP, and the internal IP address of RRAS2 represents the default gateway of your second ISP. Our test environment is different, in that the actual Internet connection is on the bridged network; because of this the external interfaces on RRAS1 and RRAS2 use the same default gateway [*] I am using dedicated NICs on the TMG firewall for each of the ISPs – this is not required, and in the next article, I will go over the alternate configuration where you do not have dedicated NICs for your ISP connections[*] You can achieve the same network segmentation with other virtualization solutions – Windows Virtual PC, ESX and Hyper-V all support similar methods of networks segmentation[/LIST]
[IMG]http://www.isaserver.org/img/upl/image0021256567971703.gif[/IMG]
[B]Figure 1[/B]
Now that we have the virtual network infrastructure defined, we can take a look at the IP addressing scheme. IP addressing information used in this lab is shown in the figure below. Notice that the TMG RRAS1 ISP NIC uses the internal interface of RRAS1 as its default gateway, and that the TMG RRAS2 ISP NIC uses the internal interface of RRAS2 as its default gateway. Also, notice that the RRAS1 network segment is on network ID 10.0.1.0/24 and that the RRAS2 network segment is on network ID 10.0.2.0/24.
The TMG firewall’s default Internal Network is on network ID 10.0.0.0/24, and the DC on the default Internal Network uses the TMG firewall as its default gateway.
[B]Note:
[/B]Notice the capitalization used for the term “network”. When “network” is spelled in lower case, it represents the term in its generic sense. However, the term “Network” with a capital “N” refers to a TMG firewall Network – which has to be defined on the TMG firewall. The TMG firewall’s default Internal Network is an example of a TMG firewall Network.
[IMG]http://www.isaserver.org/img/upl/image0041256567971703.jpg[/IMG]
[B]Figure 2[/B]
In this article we are going to go over the ISP load balancing feature. Because of this, I thought you might like to know how the load balancing decisions are made by the TMG firewall. Essentially what happens is that the TMG firewall looks at the source address (the client) and the destination address (the server) and creates a hash value, which is then represented as a value somewhere between 1-100. All possible hash values are equally distributed across this range. After calculating this value, the TMG firewall then looks at the weighting assigned to each ISP.
For example, suppose ISP1 was assigned 75% of the load and ISP2 was assigned 25% of the load. If the hash value based number is 30 – the connection would go to ISP1, since that hash based value is lower than value assigned to the preferred connection. If the hash based value were 80, then the connection would be forwarded to ISP2, since the value is higher than the value assigned to the preferred connection.
So, to summarize:

[LIST][*] Check the load number on the preferred connection – this is presented as a percentage[*] If the hash based value is lower than that number, then it goes to the preferred connection[*] If the hash based value is higher than that number, then it goes to the secondary connection[/LIST]
The figure below shows you an example. ISP RRAS1 is assigned 75% of the connections and ISP RRAS2 is assigned 25% of the connections. You can see the configuration interfaces in the figure. When PC1 connects to WEB-1, a hash based value is 60 is calculated. Since this is lower than the percentage assigned to the preferred connection, the connection is forwarded through the preferred connection, which in this case is ISP RRAS1. When PC1 connects to WEB-2, the hash based value is calculated as 80, which is higher than the value for the percentage assigned the preferred connection, so it’s forwarded through the secondary (non-preferred) connection.
Of course, if you set both of the ISP connections to 50%, half the hash based values would be lower than 50 and half would be higher than 50 – so all connections would be equally distributed between the ISPs.
[IMG]http://www.isaserver.org/img/upl/image0061256567971719.jpg[/IMG]
[B]Figure 3[/B]
What about network interface configuration on the TMG firewall? This was something that was not very well understood before a recent posting on the TMG Firewall Team blog, and they did a great job explaining what had to be done – something that was not included in the Help File (hint!).
In the figure below you can see the network interfaces configured on the TMG firewall used in this example. The LAN interface is connected to VMNet2, WAN interface is connected to VMNet3 and WAN2 is connected to VMNet4.
[IMG]http://www.isaserver.org/img/upl/image0081256567971719.jpg[/IMG]
[B]Figure 4[/B]
The WAN interface was the initial NIC configured on the firewall, and therefore was given a default gateway before getting started on the TMG firewall configuration. In the figure below you can see that it has been assigned an IP address valid for ISP RRAS1 and given the default gateway that is the internal IP address of the RRAS1 virtual machine (this would be the default gateway of your actual ISP1 in a production environment).
There is one more thing you need to do though. You need to get into the [B]Advanced[/B] TCP/IP settings and turn off the [B]Automatic metric[/B] feature. Microsoft recommends this so that the ISP redundancy feature works correctly – maybe there is a contention between the Windows and TMG routing mechanisms – they did not mention the details. However, you do need to turn this off and set a manual metric. The value is not important from what I can tell – the only requirement is that you set your preferred interface with a low metric than the non-preferred interface. In the figure below, you can see that I set the preferred interface to use a metric of [B]1[/B].
[IMG]http://www.isaserver.org/img/upl/image0101256568063250.jpg[/IMG]
[B]Figure 5[/B]
The figure below shows the IP addressing information for the non-preferred ISP, which is the RRAS2 ISP. I set the metric for this interface to be [B]2[/B].
[IMG]http://www.isaserver.org/img/upl/image0121256568063250.jpg[/IMG]
[B]Figure 6[/B]
Whoa! What’s this? Windows is not happy with me assigning two default gateways on the same machine. I can not say that I blame Windows for this, since we know that we are not supposed to do that. However, the TMG firewall’s ISP redundancy feature allows us to break this rule, so you can safely click [B]Yes[/B] because you know that in this instance, it’s going to be OK.
[IMG]http://www.isaserver.org/img/upl/image0141256568063265.jpg[/IMG]
[B]Figure 7[/B]
Just for fun, and before setting the ISP redundancy feature up on the TMG firewall, I wanted to see what interface would be used. I had already configured the TMG firewall and created an “all open” rule (famous for testing – but infamous and detested in production environments). Using [B]tracert[/B], I found that the preferred interface was used. That is good – and probably expected because it has the lower metric? Or maybe because this was the first interface configured on this computer that had a default gateway? If I have time later, I will see what happens when I switch the metrics.
[IMG]http://www.isaserver.org/img/upl/image0161256568063265.jpg[/IMG]
[B]Figure 8[/B]
I guess that should bring up a discussion of when to configure the interfaces and in what order. I’m afraid this is going to be a little like “shaking the skulls and bones and then pitching them onto the floor to read the signs”, since I haven’t found any information on what order to do things. However, this is what I did:

[LIST][*] Created the initial virtual machine with only two NICs – internal and external[*] Assigned IP addressing information to the internal and external interfaces[*] Installed the TMG firewall software[*] Confirmed that the TMG firewall installation was successful[*] Turned off the TMG VM and installed a third virtual NIC to support the second ISP link[*] Configured the 3rd virtual NIC after restarting the VM[*] Restarted the VM after configuring the IP addressing information on the 3rd interface[/LIST]
I need to be clear that this is not a recommended approach, it is not a preferred approach, and it is not something I got from Microsoft or anyone else. This is what I did and it worked. In the future I will test it and see what happens with a more simple approach that does not require machine restarts.
[B]Summary
[/B]

In this, Part 1 of my series on TMG firewall ISP redundancy, we went over the virtual networking infrastructure and some details on how which ISP is selected. We then talked about interface configuration and some of the things you need to be aware of when configuring the network interfaces that connect to each ISP. In the next article, we’ll actually configure the ISP load balancing feature and see if it actually works by testing from a client and checking the TMG firewall logs and some Netmon traces. See you then! –Tom.


[/LEFT]

[LEFT][CODE]http://www.isaserver.org/tutorials/Kicking-Tires-TMG-2010-RC-ISP-Redundancy-Part2.html[/CODE]

[B][SIZE=3]PART-2[/SIZE][/B]

How to enable ISP redundancy on TMG 2010 RC ISP.

[B]Introduction[/B]

In the first part of this article series, we went over the lab network and what you need to do with the interfaces to get them ready for multi-ISP support. Now that we have got the basic groundwork covered, let us get to the fun part!
To start, open the TMG firewall console and click the [B]Networking[/B] node in the left pane of the console. In the Task Pane, click the Tasks Tab. On the Tasks Tab, click the [B]Configure ISP Redundancy[/B] link, as seen in the figure below.
[IMG]http://www.isaserver.org/img/upl/image0021257787641234.jpg[/IMG]
[B]Figure 1[/B]
This brings up the [B]Welcome to the ISP Redundancy Configuration Wizard[/B] page. Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0041257787641265.jpg[/IMG]
[B]Figure 2[/B]
On the [B]ISP Redundancy Mode[/B] page, you can choose one of two options:

[LIST][*] [B]Load balancing with failover capability[/B]. This option allows you to use both ISPs at the same time. You can set a preferred ISP where most of the traffic goes out of the preferred ISP, or you can have traffic use both ISPs equally. Use this option if you want to get more total bandwidth and don’t have to worry about paying for bandwidth for both ISPs. If one of the ISPs fails, all traffic will go out the other ISP connection.[*] [B]Failover only[/B]. Use this option if you only want to use a single ISP, but want the other to be used if the first ISP fails. Use this option if you don’t want to pay for bandwidth for both ISPs, but want to make sure that you can stay online if the primary ISP fails. This is a good option if you have to pay bandwidth charges for the second ISP.[/LIST]
In this example we’ll choose the [B]Load balancing with failover capability[/B] option. Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0061257787641265.jpg[/IMG]
[B]Figure 3[/B]
On the [B]ISP Connection 1[/B] page you configure the settings for the first ISP link. In this example, we will name the connection [B]RRAS1[/B], since this will be the connection through the RRAS1 NAT server that’s simulating the first ISP. Since we are using separate NICs for each of the ISP connections, we can select the NIC that connects us to RRAS1 in the [B]Network adapter (optional)[/B] drop down list. Notice that after we select the NIC, the subnet address that defines the default gateway for that NIC, which is the internal address of the RRAS1 NAT server, is listed in the [B]Subnet[/B] text box. Remember, each ISP must be on a different network ID, which means that each ISP connection is on a different subnet.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0081257787641281.jpg[/IMG]
[B]Figure 4[/B]
On the [B]ISP Connection 1 – Configuration[/B] page, confirm the [B]Gateway address[/B] and [B]Mask[/B]. Also confirm the [B]Subnet[/B] text box has the correct subnet mask. You can enter a [B]Primary DNS server[/B] and a [B]Alternate DNS server[/B] if you like, but since it is highly recommended that you do not configure the firewall to use external DNS servers, I suggest that you never enter any addresses in these text boxes. There will be times when you need to enter external IP address for DNS servers on the TMG firewall, but this is not one of them.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0101257787760937.jpg[/IMG]
[B]Figure 5[/B]
On the [B]ISP Connection 1 – Dedicated Servers[/B] page, you enter the IP addresses of servers that you always want to always use this ISP connection. These are typically servers that are on the ISP’s network that aren’t accessible from outside networks, such as DNS and time servers. SMTP servers are also often placed on the ISP network for outbound messages that are not available from outside networks. Since we are not using forwarders in this example, and we are using Internet time servers, we would not enter any IP addresses for the dedicated servers.
Note that if you do enter IP addresses for dedicated servers, if that ISP goes down, the connections won’t be forwarded out the other ISP. However, this should not be much of an issue, since these IP addresses would not be accessible from external networks anyway.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0121257787760937.jpg[/IMG]
[B]Figure 6[/B]
On the [B]ISP Connection 2[/B] page, you do the same thing you did on a similar page for ISP 1. In this example, ISP2 is going to be the one that connects through the RRAS2 NAT server. Note that the [B]Subnet[/B] is on a different network ID than the first ISP connection.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0141257787760953.jpg[/IMG]
[B]Figure 7[/B]
Confirm the settings on the [B]ISP Connection 2 – Configuration[/B] page and click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0161257787760953.jpg[/IMG]
[B]Figure 8[/B]
On the [B]ISP Connection 2 – Dedicated Servers[/B] page, you enter the IP addresses of servers that are accessible from the second ISP connection. The same principles and limitations apply here as they did with the first ISP connection.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0181257787984672.jpg[/IMG]
[B]Figure 9[/B]
On the [B]Load Balancing Configuration[/B] page, you decide how you want to weigh the connections. If the connections are the same speed, you typically will set it up to use both ISPs equally. However, if one ISP is faster than the other, you will want to give me weight to the faster ISP. In this example, RRAS2 is faster than RRAS1, so I will give it 75% of the connections and RRAS1 25% of the connections. Connections are based on the method I described in the first paper of this series, so if you want to know how the TMG firewall assigns connections, check out part 1 of this series.
Click [B]Next[/B].
[IMG]http://www.isaserver.org/img/upl/image0201257787984672.jpg[/IMG]
[B]Figure 10[/B]
Check your settings on the [B]Completing the ISP Redundancy Configuration Wizard[/B] page, and then click [B]Finish[/B].
[IMG]http://www.isaserver.org/img/upl/image0221257787984687.jpg[/IMG]
[B]Figure 11[/B]
After you click [B]Finish[/B] a dialog box appears informing you that you must add a persistent static route for each DNS IP address configured on the external network adapters on every Forefront TMG server. This is required to ensure that DNS requests are routed through the proper network adapter.
The reason you need to manually create static routes for the ISPs is that the automatic routing that works with ISP Redundancy only works when there is a NAT relationship between source and destination. Since the DNS connections are coming from the TMG firewall itself, the connection is going to have a route relationship, since all connections from the Local Host Network to any other network use a route relationship.
Click [B]OK[/B].
[IMG]http://www.isaserver.org/img/upl/image0241257787984703.jpg[/IMG]
[B]Figure 12[/B]
Click [B]Apply[/B] to save the firewall configuration. Enter a change description in the [B]Configuration Change Description[/B] dialog box if you like, then click [B]Apply[/B] in that dialog box. Click [B]OK[/B] in the [B]Saving Configuration Changes[/B] dialog box.
Now let’s take a look at how some of the ISP Redundancy feature works. First, let us take a look at the Dashboard. Here you can see information about [B]Network Status[/B] for the ISP connections. In the figure below, you can see the [B]Status[/B], the [B]Uptime[/B] and the [B]Bytes/Sec[/B] being used by each ISP. Notice that RRAS2 is using a lot of bandwidth because at the time I took the screenshot, I was downloading Windows XP SP2 through the TMG firewall.
[IMG]http://www.isaserver.org/img/upl/image0261257788074687.jpg[/IMG]
[B]Figure 13[/B]
What if you want to change the configuration of the ISP Redundancy settings? Just click on the [B]Networking [/B]node in the left pane of the TMG firewall console and double click on the ISP connection you want to change. In the figure below, you can see the [B]General[/B] tab of the [B]RRAS1 Properties[/B] dialog box. Here you can change the [B]Name[/B], [B]Gateway IP address[/B], [B]Mask[/B], [B]Subnet[/B], [B]Connectivity detection[/B], and [B]Load Balancing ratio[/B].
Notice that the [B]Connectivity detection[/B] option was not exposed in the wizard. Here you have three options:

[LIST][*] [B]Disable, connection is down. [/B]This disables detection of whether the ISP is up or down and disables the ISP connection. You might want to set this if you want to disable this ISP for a while for administrative or testing purposes.[*] [B]Disabled, connection is up[/B]. This disables detection of whether the ISP is up or down but leaves the link up. Again, you might want to enable this option if you want the ISP connection to be used all the time, regardless of the status of the link.[*] [B]Enabled[/B]. This is the default setting.[/LIST]
Now you might wonder “how does the TMG firewall detect whether an ISP link is up or down?” That’s a good question. What TMG does is send connection requests to the root DNS servers on the Internet. If it connects, then the connection is up – if it does not connect, then the connection is down.
[IMG]http://www.isaserver.org/img/upl/image0281257788074703.jpg[/IMG]
[B]Figure 14[/B]
We can confirm this by looking a piece of a NetMon trace in the figure below. IP addresses [B]10.0.2.3[/B] and [B]10.0.1.3[/B] are the external addresses on the TMG firewall connecting to each of the ISP connections. The destination address [B]192.33.4.12[/B] is an IP address of one of the root DNS servers on the Internet. What’s interesting here is that these are not actually DNS queries – they are just TCP port 53 connections to the root DNS servers. If you look at the decode, you will find that there is no DNS protocol information. You only see a three-way handshake. I’m sure there’s a good reason for this, but I do not know what it is.
[IMG]http://www.isaserver.org/img/upl/image0301257788074703.jpg[/IMG]
[B]Figure 15[/B]
Now, how does the TMG firewall decide that a connection is down, and how does it decide that the connection is up again?
Multiple servers are polled to determine if there are any connectivity problems through a particular ISP. If multiple Root DNS servers fail to respond through a specific ISP, TMG retries the connection two more times (for three total attempts, which includes the first one) at an interval of 60 seconds each before switching over to the secondary ISP connection and marking the other connection as “down”.
So, if RRAS1 actually went down at 12:58PM, it will check connectivity to the root DNS servers at 12:59PM and 1:00PM. If both connection attempts fail, then the link is marked “down”. During the period between 12:58PM to 1:00PM, connections will still be routed through the dead ISP.
After the ISP connection is marked “down”, the TMG firewall will test the “downed” ISP every 5 minutes (300 seconds) and when the “downed” connection responds for the first time, two more consecutive requests at an interval of 60 seconds each must be successful before the primary link is marked as working again. Once the primary link is considered operational, TMG creates new connections using the primary ISP link.
So, if RRAS1 was marked “down” at 1:00PM, it won’t be tested again until 1:05PM. Then a test at 1:05PM is done. If successful, it will test again at 1:06PM and again at 1:07PM. If both of those tests are successful, then the link will be marked “up” and connections will be routed through that connection again.
To see what happens when an ISP goes down, I turned off RRAS2. If you wait 2-3 minutes, you’ll see in the [B]Dashboard[/B] node in the left pane of the TMG firewall console, so that [B]Status[/B] changes to [B]Local[/B].
[IMG]http://www.isaserver.org/img/upl/image0321257788151937.jpg[/IMG]
[B]Figure 16[/B]
However, there is no problem! I went to my client and downloaded a file and it automatically move to RRAS1. Keep in mind that if the client was using the downed ISP to reach a particular site, it’s going to take about 2 minutes before that site is routed to the online ISP.
If you check the Alerts, you’ll see one generated that informs you that the ISP connection is unavailable, as seen in the figure below.
[IMG]http://www.isaserver.org/img/upl/image0341257788151937.jpg[/IMG]
[B]Figure 17[/B]
Speaking of alerts, there are several of them related to ISP Redundancy, as seen in the figure below.
[IMG]http://www.isaserver.org/img/upl/image0361257788151937.jpg[/IMG]
[B]Figure 18[/B]
[B]Summary
[/B]

In this two part series I went through how ISP Redundancy works, how to set up your NICs to prepare for ISP Redundancy, and how to configure the feature. Note that this article series referred to a specific use case, where you have a dedicated NIC for each ISP. This is not a requirement. You can put both ISP IP and gateway addresses on a single NIC. If anyone is interested, I can do a separate article on that configuration, but it’s pretty straightforward and the same principles apply. Have fun with TMG ISP Redundancy and let me know what you think of it and how it works for you! Thanks! –Tom.

[/LEFT]