The Hunt For HTTP Signatures - ISA 2006 Firewall HTTP Filter

Printable View

2010-02-02, 12:56 PM
patris1

The Hunt For HTTP Signatures - ISA 2006 Firewall HTTP Filter

[LEFT][CODE]http://www.carbonwind.net/ISA/HTTPSig/HTTPSig1.htm[/CODE]

[SIZE=3][B]Part 1: Playing With Wireshark[/B][/SIZE]

You've bought an ISA 2006 Firewall and you want to use it to block applications like [URL="http://messenger.yahoo.com/"]Yahoo Messenger[/URL] or [URL="http://get.live.com/messenger/overview"]Windows Live Messenger[/URL] just to name a few.
Currently you cannot afford investing in an advanced web filtering solution like the one offered by [URL="http://www.websense.com/"]WebSense[/URL] or [URL="http://www.gfi.com/"]GFI[/URL]. So you must manually block such applications(not the most nice job in the world).
You are aware of the fact that these applications have a "signature". Thus you must try and identify this signature and then configure the [URL="http://www.microsoft.com/technet/isa/2004/help/FW_HTTPFilter.mspx?mfr=true"]HTTP filter[/URL] in ISA to block it.
Note that the HTTP signatures you can apply on ISA are found in the Request URL or in the HTTP Headers or Bodies. Both Request or Response Headers or Bodies can be inspected.
Be careful with the Request or Response body, if you enable ISA to search "deep" into them(increase the maximum number of bytes inspected by ISA) you will suffer from performance degradation.
We will search for signatures within the Request URL or within the HTTPHeaders or Bodies.
A great tool for the hunt of signatures is our favourite network protocol analyzer, [URL="http://www.wireshark.org/"]Wireshark[/URL].
For the ones not used with Wireshark, the use of it combined with the need to identify signatures can be intimidating. Therefore let's spend a few minutes playing with Wireshark first.
You can skip this part if you are familiar with Wireshark and jump to [URL="http://www.carbonwind.net/ISA/HTTPSig/HTTPSig2.htm"]Part 2[/URL].
Open Wireshark.
Click "Edit" and then click "Preferences". See [B]Figure1[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Pref.JPG[/IMG]
[B]Figure1: Wireshark Preferences[/B]
On the "User Interface" click "Capture". See [B]Figure2[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Pref_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Pref.jpg[/IMG][/URL][B]
Figure2: Wireshark Capture Preferences[/B]
Here you can set the default interface so you do not have to specify every time which interface you want to use for capturing packets. Also you can capture packets in promiscuous mode(the adapter will capture all frames, not just frames addressed to that adapter) or scroll the live capture or hide the capture info dialog([B]Figure8[/B] shows the capture info dialog) or decide if the update list of packets in real time is checked or not. If you not take an "Update list of packets in real time" capture, Wireshark should not consume memory as it captures packets. However Wireshark will consume memory when you stop the capture and it reads it.
On the"User Interface" click "Name Resolution". See [B]Figure3[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Name_Pref_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Name_Pref.jpg[/IMG][/URL][B]
Figure3: Wireshark Name Resolution Preferences[/B]
Wireshark is able to perform name resolution. MAC name resolution attempts to provide a more human readable MAC address. Network name resolution will convert an IP address to the hostname associated with it. Transport name resolution will transform TCP/UDP ports to something more "human readable(say TCP Port 80 to HTTP). See [B]Figure4[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_NameRes_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_NameRes.jpg[/IMG][/URL][B]
Figure4: Wireshark Name Resolution[/B]
Enabling network name resolution may slow down Wireshark.
On the Menu click "Capture". See [B]Figure5[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt.JPG[/IMG]
[B]Figure5: Wireshark Capture[/B]
If you click on "Interfaces" you have the chance to select on which interface you want to start the capture. See [B]Figure6[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Int_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Int.jpg[/IMG][/URL][B]
Figure6: Wireshark Capture Interfaces[/B]
If you click on "Options" you can specify some settings for the capture. See [B]Figure7[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Opt_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capt_Opt.jpg[/IMG][/URL][B]
Figure7: Wireshark Capture Options[/B]
If you click "Start", Wireshark will begin capturing packets. See [B]Figure8[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capturing_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Capturing.jpg[/IMG][/URL][B]
Figure8: Wireshark Capture Started
[/B]
From Wireshark Menu, click "View" and "Time Display Format". From here you can select the time format. For example you may want to see the date and the time of the day when a packet was captured. See [B]Figure9[/B] and [B]Figure10[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Time_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Time.jpg[/IMG][/URL][B]
Figure9: Wireshark Select Time Format[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Time_Format_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Time_Format.jpg[/IMG][/URL][B]
Figure10: Wireshark Time Format[/B]

After you have captured some traffic you may want to analyze only certain things. For example you are insterested in HTTP traffic only(TCP port 80). You can apply a filter. You can enter an expression into the Filter. See [B]Figure11[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter.jpg[/IMG][/URL][B]
Figure11: Wireshark Filter[/B]
And then Apply this filter. See [B]Figure12[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter2_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter2.jpg[/IMG][/URL][B]
Figure12: Wireshark "tcp.port eq 80" Filter[/B]
The "eq" expression is also equal with "==" (Equality).
The "and" expression is also equal with "&&" (Concatenation).
The "or" expression is also equal with "||" (Alternation).
The "not" expression is also equal with "!" (Negation).
You may want to see HTTP and HTTPS traffic only. Note that we do not use the "and" expression, "or" is used instead. See [B]Figure13[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter3_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter3.jpg[/IMG][/URL][B]
Figure13: Wireshark "tcp.port eq 80 or tcp.port eq 443" Filter[/B]
Since probably there are plenty of HTTP packets, it would be useful to view only interesting HTTP traffic, like packets destined to TCP Port 80 (HTTP requests). Click the "Expression" button and in the "Field Name" scroll for "TCP", select "tcp.dstport == 80". See [B]Figure14[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter4_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter4.jpg[/IMG][/URL][B]
Figure14: Wireshark Building a TCP filter[/B]
And then apply this expression. See [B]Figure15[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter5_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter5.jpg[/IMG][/URL][B]
Figure15: Wireshark "tcp.dstport == 80" Filter[/B]
You can view only packets destined to TCP Port 80 (HTTP requests) or to TCP Port 443 (HTTPS requests). See [B]Figure16[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter6_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter6.jpg[/IMG][/URL][B]
Figure16: Wireshark "tcp.dstport eq 80 or tcp.dstport eq 443" Filter[/B]
You can filter TCP traffic based on source port or on flags too for example. Wireshark has plenty of firepower.
You may be interested in cleaning up the capture a little bit by excluding certain ports for example. You can do that with the "!" expressions. In [B]Figure17[/B] Wireshark will not display packets destined to TCP port 80.
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter7_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter7.jpg[/IMG][/URL][B]
Figure17: Wireshark "!tcp.dstport eq 80" Filter[/B]
While is useful to filter traffic based on ports, in certain situations you need to view only HTTP requests containing methods like "POST" for example. Click the "Expression" button and in the "Field Name" scroll for "HTTP", select "http.request.method == POST". See [B]Figure18[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter8_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter8.jpg[/IMG][/URL][B]
Figure18: Wireshark Building an HTTP filter[/B]
And then apply this expression. See [B]Figure19[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter9.jpg[/IMG]
[B]Figure19: Wireshark "http.request.method == "POST" " Filter[/B]
Or you can include other methods like "GET", thus to view HTTP requests containing the "POST" or "GET" methods. See [B]Figure20[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter10_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Wireshark_Filter10.jpg[/IMG][/URL][B]
Figure20: "http.request.method eq "POST" or http.request.method eq "GET" " Filter[/B]
Also you can filter based on IP protocol(IP address, IP Protocol, IP flags and so on). See[B]Figure21[/B] and [B]Figure22[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/WS_IP_Expr_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/WS_IP_Expr.jpg[/IMG][/URL][B]
Figure21: Wireshark IP Expression[/B]
[IMG]http://www.carbonwind.net/ISA/HTTPSig/WS_IP_Filter.jpg[/IMG]
[B]Figure22: Wireshark "ip.addr == 192.168.10.110" Filter[/B]
Wireshark has plenty of options. If you want to find out more about Wireshark make sure you do not miss its excellent [URL="http://wiki.wireshark.org/FrontPage"]Wiki[/URL].

[/LEFT]
2010-02-02, 12:58 PM
patris1

[LEFT][CODE]http://www.carbonwind.net/ISA/HTTPSig/HTTPSig2.htm[/CODE]

[SIZE=3][B]Part 2: IM Blocking[/B][/SIZE]

1. The Signature of Yahoo Messenger
Starting with a popular application like [URL="http://messenger.yahoo.com/"]Yahoo Messenger[/URL]. While it's a nice chat tool(and much more) it can affect work productivity (and your network's health and safety). Therefore you may want to prevent its use in your office. On ISA 2006 I have created a test rule allowing HTTP, HTTPS and FTP for All Users (unauthenticated rule). See [B]Figure23[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Access_rule.JPG[/IMG]
[B]Figure23: ISA 2006 Access Rule[/B]
Always allow through ISA only needed traffic. An "Allow All" rule makes every firewall pretty dumb. I did not use authentication on this rule because I want to obtain, as much as possible, a clean and simple Wireshark capture. The first tests will be made with a machine that it's only a SecureNAT client. ISA is a domain member. In a production environment you should use authentication on your rules. For that, depending on the traffic required, make your clients Web Proxy Clients and/or Firewall Clients(SecureNAT clients cannot authenticate). As we will see later in this article, switching between ISA clients and using or not authentication on the above rule, will have certain effects on various applications.
For the internal DNS server I have a separate rule. The clients query this internal DNS server which uses ISP DNS servers as Forwarders.
The problem with Yahoo Messenger is the fact that it can be configured to use only HTTP requests thus if we had allowed only HTTP and HTTPS on our firewall it will be able to connect. However the HTTP Filter can help us blocking it.
On ISA 2006 Firewall I had installed the [URL="http://support.microsoft.com/kb/939455"]Supportability Update Package[/URL] in order to benefit from the improved log viewing functionality. The Firewall Client is disabled on the machine which runs Yahoo Messenger.
Start the live log on ISA and a Wireshark capture(on the correct interface) on the client on which Yahoo Messenger is running.
And then start Yahoo Messenger and login.
Stop the live log on ISA and the Wireshark capture.
Time to analyze them. We need to follow the HTTP flow, starting with the first client request and first server reply packet.
A quick look at ISA's log. See [B]Figure24[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Yms_Allowed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Yms_Allowed.jpg[/IMG][/URL][B]
Figure24: ISA Log Yahoo Messenger Allowed[/B]
We can see that ISA has allowed a packet using the "POST" method. The Client Agent(User Agent) does not tell us much.
So no joy with ISA's log.
In [URL="http://www.ietf.org/rfc/rfc2616.txt"]RFC2616 Hypertext Transfer Protocol -- HTTP/1.1[/URL], an [I]"Internet Official Protocol Standards" (STD 1)[/I], the User Agent is defined as:
- "[I]the client which initiates a request. These are often browsers, editors, spiders (web-traversing robots), or other end user tools[/I]", "[I]The User-Agent request-header field contains information about the user agent originating the request.[/I]".
- "[I]The field can contain multiple product tokens and comments identifying the agent and any subproducts which form a significant part of the user agent. By convention, the product tokens are listed in order of their significance for identifying the application.[/I]".
So the User-Agent string can indicate for example which browser you are using, its version number and details about your system, such as operating system and version. Therefore the Web server will be aware of the browser you use and it can provide content that is tailored for that specific browser. You can find out more about the User-Agent reading the following Microsoft doc: [URL="http://msdn2.microsoft.com/en-us/library/ms537503%28VS.85%29.aspx"]Understanding User-Agent Strings[/URL].
Let's analyze the corresponding packet from the Wireshark capture. Maybe there is another "helpful" field within the Request Header([URL="http://www.ietf.org/rfc/rfc2616.txt"]RFC2616[/URL] Section 5.3). See [B]Figure25[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Yms_Allowed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Yms_Allowed.jpg[/IMG][/URL][B]
Figure25: Wireshark Capture Yahoo Messenger Allowed[/B]
From this capture we can spot something interesting in the Host header field, the "shttp.msg.yahoo.com" pattern. Actualy if you scroll the capture you will see that this patern repeats. The "*.msg.yahoo.com" appears very often("shttp" can be replaced with "insider" for example). It seems that Yahoo Messenger "needs" this host string.
We can also check the DNS queries from this capture. See [B]Figure26[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/DNS_Query_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/DNS_Query.jpg[/IMG][/URL][B]
Figure26: DNS Queries[/B]
So "msg.yahoo.com" can represent the signature for Yahoo Messenger, found in the Request Headers, the Host header field. If we look at the [URL="http://www.microsoft.com/technet/isa/2004/plan/commonapplicationsignatures.mspx"]Common Application Signatures[/URL] Microsoft doc, we can see that the same Yahoo Messenger signature is shown.
Time to see if it works. Right click on your ISA web access access rule and click Configure HTTP. See [B]Figure27[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Configure_HTTP.JPG[/IMG]
[B]Figure27: Configure HTTP[/B]
Click the Signatures tab and add the signature we've spoted for Yahoo Messenger. See[B]Figure28[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Yms_Sig_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Yms_Sig.jpg[/IMG][/URL][B]
Figure28: Yahoo Messenger Signature[/B]
Apply your configuration.
Again start the live log on ISA and a Wireshark capture(on the correct interface) on the client on which Yahoo Messenger is running.
Start Yahoo Messenger and try to login. You will not be able to do that and Yahoo Messenger will be left fuming.
If we analyze the log on ISA we will notice that now ISA has blocked the packet containing the POST method. See [B]Figure29[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Yms_Blocked_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Yms_Blocked.jpg[/IMG][/URL][B]
Figure29: ISA Log Yahoo Messenger Blocked[/B]
Within the corresponding Wireshark capture we can spot the denied attempts of Yahoo Messenger. See [B]Figure30[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Yms_Blocked_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Yms_Blocked.jpg[/IMG][/URL][B]
Figure30: Wireshark Capture Yahoo Messenger Blocked[/B]
As you might know you can now chat directly within the browser from your Yahoo mail account. However the above signature should block this chat type too, since it is not dependent on a specific User-Agent, it depends on a host.
I have taken a Wireshark trace while I'm using my Yahoo mail account from browser(from the login phase). However this resulted in a big Wireshark capture. We need to quickly filter that. A good start will be to see packets denied by ISA(Response Codes 502 and 500). My machine is a Web Proxy client this time. Click the "Expression" button and create the required filter. See [B]Figure31[/B] and [B]Figure32[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Req_Code_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Req_Code.jpg[/IMG][/URL][B]
Figure31: Wireshark "http.response.code == 502"[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Req_Code_Filter_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Req_Code_Filter.jpg[/IMG][/URL][B]
Figure32: Wireshark "http.response.code == 502 or http.response.code == 500"[/B]
Write down the frame number. Clear the filter by clicking the "Clear" button.
You can quickly navigate to the desired packet by using the "Go" menu and select "Go to Packet". See [B]Figure33[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Go_to_Packet.JPG[/IMG]
[B]Figure33: Wireshark Go to Packet[/B]
Type the packet number and then click "Jump to". See [B]Figure34[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Packet_No.JPG[/IMG]
[B]Figure34: Go to Packet number 1747[/B]
And take a look at this packet. See [B]Figure35[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/The_Packet_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/The_Packet.jpg[/IMG][/URL][B]
Figure35: Packet number 1747[/B]
We can view all the packets containing the msg.yahoo.com signature(first clear the current filter). Click the "Expression" button and create the required filter. See [B]Figure36[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_Host_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_Host.jpg[/IMG][/URL][B]
Figure36: Wireshark "http.host - Host (HTTP Host) contains msg.yahoo.com" Filter[/B]
And the corresponding packets to the ones from [B]Figure32[/B] appear. See [B]Figure37[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_Host_Filter_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_Host_Filter.jpg[/IMG][/URL][B]
Figure37: Wireshark "http.host - Host (HTTP Host) contains msg.yahoo.com" Filter[/B]
And we can clearly see the packet denied by ISA because it contains our signature pattern.See [B]Figure38[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_1297_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Filter_1297.jpg[/IMG][/URL][B]
Figure38: Wireshark "tcp.port eq 1297" Filter[/B]
2. The Signature of Windows Live Messenger
Next let's try to find out the signature of another popular IM application, [URL="http://get.live.com/messenger/overview"]Windows Live Messenger[/URL]. We have the same problem as with Yahoo Messenger, although Windows Live Messenger's default protocol is blocked on ISA(we allow only HTTP and HTTPS with our test access rule), Windows Live Messenger is able to use only HTTP and HTTPS.
Repeat the procedures(start ISA live logging, start a Wireshark capture and login with Windows Live Messenger).
Now if we analyze the log on ISA we can spot within the Request Headers the Client Agent field containing the "Windows Live Messenger 8.5.1302" pattern. See [B]Figure39[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Msn_Allowed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Msn_Allowed.jpg[/IMG][/URL][B]
Figure39: ISA Log Windows Live Messenger Allowed[/B]
Within the corresponding Wireshark capture look at the same packet. See [B]Figure40[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Msn_Allowed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Msn_Allowed.jpg[/IMG][/URL][B]
Figure40: Wireshark Capture Windows Live Messenger Allowed[/B]
We can conclude that the "Windows Live Messenger" pattern looks like a good possible signature. Obviously we should not include 8.5.1302 because that's the version of the Windows Live Messenger application and versions are changing.
So let's give it a go and configure ISA to block this signatures. See [B]Figure41[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/LiveMsg_Sig_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/LiveMsg_Sig.jpg[/IMG][/URL][B]
Figure41: Windows Live Messenger Messenger Signature[/B]
And try to connect with Windows Live Messenger. And Windows Live Messenger fails to connect. See [B]Figure42[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Live_Msg_Blocked.jpg[/IMG]
[B]Figure42: Windows Live Messenger Error[/B]
ISA's log tells us that ISA blocked the packet containing the "POST" method. See [B]Figure43[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Msn_Blocked_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Msn_Blocked.jpg[/IMG][/URL][B]
Figure43: ISA Log Windows Live Messenger Blocked[/B]
The same thing can be noticed from the corresponding Wireshark Capture. See [B]Figure44[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Msn_Blocked_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Msn_Blocked.jpg[/IMG][/URL][B]
Figure44: Wireshark Capture Windows Live Messenger Blocked[/B]
So far so good.
3. The Signature of Google Talk
Another IM client: [URL="http://www.google.com/talk/"]Google Talk[/URL]. As you will see, Google Talk is a different kind of beast and might escape through ISA. Google Talk uses an open protocol, XMPP(Extensible Messaging and Presence Protocol), see [URL="http://www.ietf.org/rfc/rfc3920.txt"]RFC3920[/URL] and [URL="http://www.ietf.org/rfc/rfc3921.txt"]RFC3921[/URL], both [I]Internet Official Protocol Standards" (STD 1)[/I], for more details about XMPP. XMPP can use HTTP binding for users behind restricted firewalls.
First thing to be noted is the fact that Google Talk is by default set to automatically discover the proxy. See [B]Figure45[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/GoogleTalk.JPG[/IMG]
[B]Figure45: Google Talk[/B]
So even if you did not set your machine as web proxy client, if Web Proxy Auto-Discovery is enabled on your network Google Talk will use ISA as its proxy.
Time to find out the signature of Google Talk.
Repeat the process used till now on this article an connect with Google Talk.
As you start Google Talk, it will issue a version check. See [B]Figure46[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Allowed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Allowed.jpg[/IMG][/URL][B]
Figure46: ISA Log Google Talk Version Check[/B]
Although we did not enter yet our credentials within Google Talk, we already can spot within the User-Agent field something useful: "Google Talk".
Analyzing the corresponding Wireshark we can notice the same thing. If we look at the following "GET" request we will see the same User-Agent. See [B]Figure47[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Allowed_Vers_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Allowed_Vers.jpg[/IMG][/URL][B]
Figure47: Wireshark Google Talk Version Check[/B]
As said before, since I have enabled Web Proxy Auto-Discovery(anyway this machine was set as SecureNAT Client and Web Proxy Client for this test), Google Talk will use ISA as its proxy. Scrolling down through the Wireshark capture we will notice a successful "Connect www.google:443" request from Google Talk. According to [URL="http://www.ietf.org/rfc/rfc2616.txt"]RFC2616[/URL]: "[I]this specification reserves the method name CONNECT for use with a proxy that can dynamically switch to being a tunnel (e.g. SSL tunneling)[/I]". See [B]Figure48[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Allowed_Connect_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Allowed_Connect.jpg[/IMG][/URL][B]
Figure48: Wireshark Google Talk CONNECT[/B]
The same User-Agent is used.
Let's give it a go and configure ISA to block this signature. See [B]Figure49[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/GT_Sig_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/GT_Sig.jpg[/IMG][/URL][B]
Figure49: Google Talk Signature[/B]
My machine is configured as a SecureNAT Client(has as its DG ISA) and Web Proxy Client (using the Automatic Configuration Script).
And surprise: Google Talk is able to connect. What went wrong ?
At a first glance the logs on ISA tell us that something functioned. The "Connect www.google:443" request was blocked. See [B]Figure50[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Really_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Really.jpg[/IMG][/URL][B]
Figure50: ISA Log Google Talk Blocked, Really ?[/B]
However there are some HTTPS requests allowed by our rule. See [B]Figure51[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_HTTPS_Initiated_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_HTTPS_Initiated.jpg[/IMG][/URL][B]
Figure51: ISA Log HTTPS Initiated[/B]
Analyzing the Wireshark capture we can find out more. First we can notice that ISA blocks the "Connect www.google:443" request because it contains the User-Agent "Google Talk". See [B]Figure52[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Blocked_Connect_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Blocked_Connect.jpg[/IMG][/URL][B]
Figure52: Wireshark "Connect www.google:443" Blocked[/B]
However if we scroll down we find the explanation. Since I do not require authentication on my rule, Google Talk has resumed to be only a SecureNAT client, thus it initiates directly the SSL tunnel(without the "CONNECT" method). ISA cannot inspect its requests since they are encrypted. See [B]Figure53[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_TLS_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_TLS.jpg[/IMG][/URL][B]
Figure53: Wireshark Google Talk SecureNAT Client[/B]
If I go and modify my rule to require authentication(replace "All Users" with "All Authenticated Users", ISA is a domain member) Google Talk cannot connect anymore because it must use the proxy on ISA 2006 Firewall in order to successfully authenticate. See [B]Figure54[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/GT_Conn_Error.JPG[/IMG]
[B]Figure54: Google Talk Connection Error[/B]
Let's analyze the corresponding Wireshark capture. As expected, ISA requires authentication, Google Talk obeys and then the "Connect www.google:443" request is blocked by ISA because it contains the User-Agent "Google Talk". See [B]Figure55[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth.jpg[/IMG][/URL][B]
Figure55: Wireshark Google Talk Blocked Indeed[/B]
Then Google Talk attempts to become a simple SecureNAT client. However this time it cannot do so since SecureNAT clients cannot authenticate, thus ISA blocks its attempt to directly initiate the SSL tunnel(without the "CONNECT" method). See [B]Figure56[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth2_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth2.jpg[/IMG][/URL][B]
Figure56: Wireshark Google Talk Blocked Indeed[/B]
This scenario repeats a couple of times. See [B]Figure57[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth3_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_GT_Proxy_Auth3.jpg[/IMG][/URL][B]
Figure57: Wireshark Google Talk Blocked Indeed[/B]
And finally Google Talk gives up with the error from [B]Figure54[/B].
This time ISA's log shows the denied HTTPS attempts. See [B]Figure58[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Indeed_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Indeed.jpg[/IMG][/URL][B]
Figure58: ISA Log Google Talk HTTPS Blocked[/B]
Also the SSL-tunnel is denied because of the User-Agent "Google Talk". See [B]Figure59[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Indeed2_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_GT_Blocked_Indeed2.jpg[/IMG][/URL][B]
Figure59: ISA Log Google Talk SSL-tunnel Blocked[/B]
So our signature worked. However, if your machine is a Firewall Client too, then Google Talk will be able to connect again. One solution for this is to disable FWC for Google Talk(not very efficient, if one user changes Google Talk's filename, then this method will not function anymore).
Another method, a real, elegant and very efficient one is to enable ISA to inspect outbound SSL connections with the help of the excellent add-on for ISA made by [URL="http://collectivesoftware.com/"]Collective Software[/URL], called [URL="http://collectivesoftware.com/Products/#ClearTunnel"]ClearTunnel[/URL]. Doing so you do not need anymore to require authentication on your rule(if desired or needed, however is recommended to force authentication) since [URL="http://collectivesoftware.com/Products/#ClearTunnel"]ClearTunnel[/URL] can inspect outbound SSL connections made by SecureNAT clients too.

[/LEFT]
2010-02-02, 01:00 PM
patris1

[LEFT][CODE]http://www.carbonwind.net/ISA/HTTPSig/HTTPSig3.htm[/CODE]

[SIZE=3][B]Part 3: Other Signatures[/B][/SIZE]

1. Block the Download of Attachments from Web Mail
Another thing you may want to block is the download of attachments from Web Mail(like Yahoo Mail, Windows Live Mail or Google Mail). Before we proceed you must be aware of the fact that if your Web Mail connection is protected from end-to-end with SSL, ISA will not be able to inspect packets. Thus you really need the already mentioned add-on for ISA made by [URL="http://collectivesoftware.com/"]Collective Software[/URL] called [URL="http://collectivesoftware.com/Products/#ClearTunnel"]ClearTunnel[/URL].
Typically only the credentials are protected with SSL by popular Web Mail services like Yahoo Mail or Windows Live Mail(as writing this article). However Google Mail is able to use SSL after login if you use [url]https://mail.google.com[/url] instead of [url]http://mail.google.com[/url]. Doing so, after you had login, you will notice that the connection remains protected by SSL.
And obviously if your users use a payed Web Mail solution, they will have the option to protect from end-to-end with SSL their connection.
Let's try to identify a signature for blocking the download of attachments from Web Mail.
We can start the Wireshark capture before we login. But doing so it will result in a big and hard to analyze capture.
Login and navigate to the received email message. Once you have done that, start the Wireshark capture and the live logging on ISA. And now download your attachment.
I will use Yahoo Mail, the new version of Yahoo Mail(not the classic one). I had received a doc.zip attachment.
Time to analyze the Wireshark capture. When I click on the attachment, a virus scan is performed and finally a download link appears. See [B]Figure57[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Down_attach.JPG[/IMG]
[B]Figure57: Dowload Attachment[/B]
This is the critical moment for finding our signature, when we click the "Download Attachment" button.
If we look at the Wireshark capture we will notice a couple of "GET" requests. But none of them contains something for useful for us. See [B]Figure58[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Get_Down_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Get_Down.jpg[/IMG][/URL][B]
Figure58: Wireshark Capture[/B]
Till now we had only searched through HTTP Request Headers. What about the Response Headers ?
There is a segmented server response to the packet selected in [B]Figure58[/B]. What we can do is to right-click on that packet and click "Follow TCP Stream". See [B]Figure59[/B].
[IMG]http://www.carbonwind.net/ISA/HTTPSig/Follow_TCP_Stream.JPG[/IMG]
[B]Figure59: Wireshark "Follow TCP Stream"[/B]
And something very interesting can be observed within the Reponse Header. See [B]Figure60[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/TCP_Stream_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/TCP_Stream.jpg[/IMG][/URL][B]
Figure60: Wireshark The TCP Stream[/B]
First there is a "Content-Type" field containing "application/zip". This can be useful if we want to block the download of .zip files from Web Mail. But not very efficient, because we can change the extension of it (say .rar).
Also there is a "Content-Disposition" field containing the "attachment" string. Please refer to [URL="http://www.ietf.org/rfc/rfc2183.txt"]RFC2183 Communicating Presentation Information in Internet Messages: The Content-Disposition Header Field[/URL], an [I]"Internet Official Protocol Standards" (STD 1)[/I]for more details about this header field. This might be the signature we are looking for.
Let's quickly enter this signature on ISA and see what's happening. See [B]Figure61[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Attach_Down_Sig_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Attach_Down_Sig.jpg[/IMG][/URL][B]
Figure61: ISA "Block The Download of Attachments" Signature[/B]
And success. When I click on the "Download Attachment" button(after the virus scan), ISA blocks this time the attempt to download the attachment. See [B]Figure62[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Yahoo_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Yahoo.jpg[/IMG][/URL][B]
Figure62: ISA Log The Download of Attachment Was Blocked Yahoo Mail[/B]
And the correspoding Wireshark capture. See [B]Figure63[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Yahoo.Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Yahoo.jpg[/IMG][/URL][B]
Figure63: Wireshark The Download of Attachment Was Blocked Yahoo Mail[/B]
Same thing can be noticed with Windows Live Mail. See [B]Figure64[/B] and [B]Figure65[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Hotmail_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Hotmail.jpg[/IMG][/URL][B]
Figure64: ISA Log The Download of Attachment Was Blocked Windows Live Mail[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Hotmail_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Hotmail.jpg[/IMG][/URL][B]
Figure65: Wireshark The Download of Attachment Was Blocked Windows Live Mail[/B]
Same thing happens with Google Mail(as long as the session is not protected from end-to-end with SSL). See [B]Figure66[/B] and [B]Figure67[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Google_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Attach_Down_Blocked_Google.jpg[/IMG][/URL][B]
Figure66: ISA Log The Download of Attachment Was Blocked Google Mail[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Google_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Attach_Down_Blocked_Google.jpg[/IMG][/URL][B]
Figure67: Wireshark The Download of Attachment Was Blocked Google Mail[/B]
2. Block the Upload of Files through Web Mail(and not only)
You may also want to block the upload of files through Web Mail.
I will use Yahoo Mail for the first test. I will not start the capture from the login phase. I will login, create a new message and start the Wireshark capture before hitting the "Attach" button. Doing this way I will obtain a nice and clean Wireshark capture.
Let's analyze it. See [B]Figure68[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Upload_Allowed_Yahoo_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Upload_Allowed_Yahoo.jpg[/IMG][/URL][B]
Figure68: Wireshark The Upload of a File Yahoo Mail[/B]
We can spot something interesting, within a packet containing the "POST" method. This packet can be easily observed because the Request URL contains the "upload" string. Within the Request Headers there is the "Content-Type" field containing the "multipart/form-data" pattern. We can find out more about this pattern reading the following Microsoft doc: [URL="http://www.microsoft.com/technet/isa/2004/plan/block_forms.mspx"]How to Block the Signature for HTML Forms File Upload[/URL].
As it is stated there, by blocking this signature with ISA, we will block file submission with HTML forms. Therefore this signature is not useful only to block the upload of files through Web Mail, it will apply to all file submissions with HTML forms. So if you need this feature for a particular destination, you should carefully define your ISA access rules.
Let's configure ISA to block this signature. See [B]Figure69[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Block_File_Up_Sig_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Block_File_Up_Sig.jpg[/IMG][/URL][B]
Figure69: ISA Block the Signature for HTML Forms File Upload[/B]
And try again to upload the file. This time ISA blocks it. See [B]Figure70[/B] and [B]Figure71[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Sig_Blocked_Yahoo_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Sig_Blocked_Yahoo.jpg[/IMG][/URL][B]
Figure70: ISA Log File Submissions with HTML Forms Blocked Yahoo Mail[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Yahoo_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Yahoo.jpg[/IMG][/URL][B]
Figure71: Wireshark File Submissions with HTML Forms Blocked Yahoo Mail[/B]
As expected, this signature works for Windows Live Mail too. See [B]Figure72[/B] and [B]Figure73[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Blocked_Hotmail_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Blocked_Hotmail.jpg[/IMG][/URL][B]
Figure72: ISA Log File Submissions with HTML Forms Blocked Windows Live Mail[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Hotmail_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Hotmail.jpg[/IMG][/URL][B]
Figure73: Wireshark File Submissions with HTML Forms Blocked Windows Live Mail[/B]
And it will apply to Google Mail as long as the session is not protected from end-to-end with SSL . See [B]Figure74[/B] and [B]Figure75[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Blocked_Google_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_File_Up_Blocked_Google.jpg[/IMG][/URL][B]
Figure74: ISA Log File Submissions with HTML Forms Blocked Google Mail[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Google_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_File_Up_Blocked_Google.jpg[/IMG][/URL][B]
Figure75: Wireshark File Submissions with HTML Forms Blocked Google Mail[/B]
3. Talking about Download Managers
Before we proceed, note that the download managers were installed on machines that are SecureNAT and Web Proxy clients.
Well, the title has changed now. Instead of blocking, "talking" appear.
Why is that ?
The answer is very simple. Because they are hard to block as we will see later.
ISA is actually capable to kill popular download managers(as we will see later) because they are unable to use NTLM authentication. So unless you check basic authentication on ISA's Proxy, they will never make it through ISA(Integrated Authentication is checked by default, meaning that NTLM or Kerberos are used). Actually some of them or not capable to handle even the basic authentication.
Till now we have analyzed various signatures.
The User-Agent for example. In a perfect world, download managers will have a specific User-Agent. But in reality they don't. They identify themselves with common patterns like Mozilla/4.0, MSIE 6.0, MSIE 7.0, Windows NT 5.1 and so on. Blocking one of these strings will result in the block of many browsers for example.
While download managers are very useful applications because they permit us to take full advantage of our Internet connection speed, users can install them without authorization on the corporate network and therefore exhaust the available bandwidth for the corporate Internet connection.
If you have a capable bandwidth manager installed on ISA(ISA 2006 Firewall does not have bandwidth management features by default, you need a third-party add-on) and ISA is properly configured, although you do not specifically block them(like in case of IM applications), you actually annihilate them. That's due to the fact that users will have a limited and controlled bandwidth. You can identify easily(real time monitoring) wasteful users and limit their traffic usage accordingly. Or you can limit the number of simultaneous connections(as we will see later download mangers can open multiple connections to the same URL and might have p2p capabilities).
Let's analyze one such client. Probably one of the best download manager is [URL="http://www.metaproducts.com/MD.html"]Mass Downloader[/URL](shareware).
Let's take a look at it. Actually, we can set its User-Agent. See [B]Figure76[/B], [B]Figure77[/B] and[B]Figure78[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident.jpg[/IMG][/URL][B]
Figure76: Mass Downloader Options: Default User-Agent[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident2_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident2.jpg[/IMG][/URL][B]
Figure77: Mass Downloader Options: More User-Agents[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident3_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Ident3.jpg[/IMG][/URL][B]
Figure78: Mass Downloader Options: More User-Agents[/B]
As can be seen from the above pictures we won't have any luck in blocking Mass Downloader by its User-Agent, unless the users are dumb enough to use the "Mass Downloader 3.3" User-Agent. You can change the number of retries, increase/decrease the connection time-out for HTTP and FTP.
Also it can use a Proxy Server that requires authentication(including NTLM). So it does not have any problems with the authentication required by ISA. See [B]Figure79[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Proxy_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Proxy.jpg[/IMG][/URL][B]
Figure79: Mass Downloader Proxy Server Settings[/B]
When you start a new Project you can set the number of threads(downloading channels). The default value is 10. This value specifies in how many parts the file will be split when it's downloaded. All the parts will be loaded simultaneously to increase speed. Actually 10 simultaneous connections will be used. See [B]Figure80[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Project_Big.JPG"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Mass_Downloader_Project.jpg[/IMG][/URL][B]
Figure80: Mass Downloader Project Properties[/B]
Other popular download managers support multiple connections to the same URL too(in order to increase the download speed). [URL="http://www.orbitdownloader.com/"]Orbit Downloader[/URL] for example. See [B]Figure81[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Orbit_Conn_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Orbit_Conn.jpg[/IMG][/URL][B]
Figure81: Orbit Downloader[/B]
I've made a quick test to visualise this behaviour. Limit the number of threads to two. The Wireshark capture shows two "GET" requests made when the file is downloaded. See[B]Figure82[/B] and [B]Figure83[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Two_Threads_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Two_Threads.jpg[/IMG][/URL][B]
Figure82: Two Simultaneous Connections: First from Source Port 1477[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Two_Threads2_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Two_Threads2.jpg[/IMG][/URL][B]
Figure83: Two Simultaneous Connections: Second from Source Port 1478[/B]
Orbit Downloader has an option to accelerate downloads by the use of Orbinet(p2p(dht) and auto mirror search). See [B]Figure84[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Orbit_p2p_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Orbit_p2p.jpg[/IMG][/URL][B]
Figure84: Orbit Accelerate[/B]
Note that, as writing, Orbit Downloader cannot authenticate against a Proxy Server. Thus if you require authentication on your web access rules(as you should), Orbit Downloader cannot pass through ISA.
Also if you start a Wireshark capture on the client on which Orbit Downloader is idle running, you will see a lot of UDP packets send to numerous destination hosts due to Orbinet and p2p. See [B]Figure85[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Orbit_UDP_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Orbit_UDP.jpg[/IMG][/URL][B]
Figure85: Orbinet Traffic[/B]
Unless you use an "Allow All" rule from "Your Client Network" to "External", these packets will be blocked by ISA. Remember: "Allow All" rules can make your firewall useless!
[URL="http://www.flashget.com/en/download.htm"]FlashGet[/URL] is another popular download manager. Like Mass Downloader, it lets us specify its User-Agent. See [B]Figure86[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Flash_Get_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Flash_Get.jpg[/IMG][/URL][B]
Figure86: FlashGet User-Agent[/B]
Although it supports an authentication proxy, as writing this article it does not handle NTLM authentication, only basic authentication.
It also supports the use of mirrors. See [B]Figure87[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Flash_Get_Mirrors_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Flash_Get_Mirrors.jpg[/IMG][/URL][B]
Figure87: FlashGet Mirrors[/B]
If you require authentication on your web access rules(as you should), FlashGet cannot make it through ISA(referring to the version currently available as writing this article) unless you have enabled Basic authentication(by default only Integrated authentication is enabled) on ISA's Web Proxy.
Another popular application is [URL="http://www.freedownloadmanager.org/"]Free Download Manager[/URL]. This one supports an authentication proxy and actually it can use Kerberos. See [B]Figure88[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man.jpg[/IMG][/URL][B]
Figure88: Free Download Manager Proxy Server Options[/B]
It also provides support for mirrors and BitTorrent. See [B]Figure89[/B] and See [B]Figure90[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Mirrors_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Mirrors.jpg[/IMG][/URL][B]
Figure89: Free Download Manager Mirrors[/B]
[URL="http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Tor_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Tor.jpg[/IMG][/URL][B]
Figure90: Free Download Manager Bit Torrent Support[/B]
It also supports multiple connections to the same original URL.
However, as default installed, it has a specific User-Agent(version 2.5 is the latest as currently writing). See [B]Figure91[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Ws_Free_Down_Man_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Ws_Free_Down_Man.jpg[/IMG][/URL][B]
Figure91: Wireshark Free Download Manager Capture[/B]
Thus we can configure ISA to block this signature. See [B]Figure92[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man_Sig_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man_Sig.jpg[/IMG][/URL][B]
Figure92: ISA Free Download Manager Signature[/B]
And as expected, ISA blocks it. See [B]Figure93[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Free_Down_Man_Blocked_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/ISA_Log_Free_Down_Man_Blocked.jpg[/IMG][/URL][B]
Figure93: ISA Log Free Download Manager Blocked[/B]
However if your users are smart, this detail can be "fixed", they can specify another User-Agent and Free Download Manager will escape though ISA. See [B]Figure94[/B].
[URL="http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man_UserAg_Big.jpg"][IMG]http://www.carbonwind.net/ISA/HTTPSig/Free_Down_Man_UserAg.jpg[/IMG][/URL][B]
Figure94: Specify the User-Agent for Free Download Manager[/B]
As said before, with the correct bandwidth manager installed on ISA, we can easily annihilate these download managers. Also you should always allow only needed traffic to needed destinations.

[/LEFT]