From grunion@NRAO.EDU Mon Jan 14 11:06:43 2002 Date: Sat, 6 Jan 2001 11:40:23 -0500 (EST) Subject: DNS requests from 209.67.50.203 From: Gene Runion To: unisog@sans.org Hi, Have others seen a steady stream of DNS requests ( ~220/min/DNS server) from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for this address, who said they are not the source but the target of a DoS attack. He claimed they are seeing 60 - 90 Mb data stream toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder routers. After ~ 48 hours these DNS requests are still coming. Gene Runion Ph: 804 296 0327 National Radio Astronomy Observatory 520 Edgemont Road Fax: 804 296 0278 Charlottesville, Va 22903 From USTS034@UABDPO.DPO.UAB.EDU Mon Jan 14 11:06:43 2002 Date: Sat, 06 Jan 01 15:41:21 CST Subject: Re: DNS requests from 209.67.50.203 From: Landy Manderson To: Gene Runion , unisog@sans.org Yes, as a matter of fact we (Univ Alabama Birmingham) are seeing several megs of DNS queries from this address per hour. We are also seeing occasional web traffic to/from it. On Sat, 6 Jan 2001 11:40:23 -0500 (EST) you said: >Have others seen a steady stream of DNS requests ( ~220/min/DNS server) >from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for >this address, who said they are not the source but the target of a DoS >attack. He claimed they are seeing 60 - 90 Mb data stream >toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder >routers. After ~ 48 hours these DNS requests are still coming. From robin@umbc.edu Mon Jan 14 11:06:43 2002 Date: Sat, 6 Jan 2001 19:05:33 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Robin To: Gene Runion Cc: unisog@sans.org, Global Incident Analysis Center On Sat, 6 Jan 2001, Gene Runion wrote: > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > this address, who said they are not the source but the target of a DoS > attack. He claimed they are seeing 60 - 90 Mb data stream > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > routers. After ~ 48 hours these DNS requests are still coming. While we don't normally check for DNS requests in our NIDS, we added a rule to look for this address. We're also receiving a veritable flood of this traffic. We're going to ask our network folks to block this also. Thanks for the heads up. --- Robin Anderson UNIX SysAdmin, Specialist Office of Informational Technology University of MD, Baltimore County (UMBC) PGP fingerprint: (resumbc99) 1024/5B5A87A DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 "Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace From julie@eos.csustan.edu Mon Jan 14 11:06:43 2002 Date: Sun, 7 Jan 2001 08:09:13 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Julie D. Gorman To: Robin Cc: Gene Runion , unisog@sans.org, Global Incident Analysis Center Hi, We are getting hit also. I've asked our network folks to block this IP. Julie On Sat, 6 Jan 2001, Robin wrote: >On Sat, 6 Jan 2001, Gene Runion wrote: > >> Have others seen a steady stream of DNS requests ( ~220/min/DNS server) >> from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for >> this address, who said they are not the source but the target of a DoS >> attack. He claimed they are seeing 60 - 90 Mb data stream >> toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder >> routers. After ~ 48 hours these DNS requests are still coming. > >While we don't normally check for DNS requests in our NIDS, we added a >rule to look for this address. We're also receiving a veritable flood of >this traffic. > >We're going to ask our network folks to block this also. Thanks for >the heads up. > >--- >Robin Anderson UNIX SysAdmin, Specialist >Office of Informational Technology University of MD, Baltimore County (UMBC) > >PGP fingerprint: (resumbc99) 1024/5B5A87A >DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 > >"Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace > > |Julie D. Gorman | julie@eos.csustan.edu | |Computer Science | " I will ride the wave | |California State University, Stanislaus | where it | |Turlock, CA 95382 (209) 667-3273 | takes me ....." | From tkw@southwestern.edu Mon Jan 14 11:06:43 2002 Date: Sun, 07 Jan 2001 02:32:10 -0600 Subject: Re: DNS requests from 209.67.50.203 From: Todd K. Watson To: unisog@sans.org Yes, We are seeing it as well. We have just implemented a NetEnforcer system from Allot which verified that as one of our top bandwidth sites this evening. Thanks for posting, we are blocking it too now. -- Todd K. Watson Senior System & Network Administrator Southwestern University, Georgetown, TX tkw@southwestern.edu || TEL:512.863.1508 || FAX:512.863.1605 Robin wrote: > > On Sat, 6 Jan 2001, Gene Runion wrote: > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > > this address, who said they are not the source but the target of a DoS > > attack. He claimed they are seeing 60 - 90 Mb data stream > > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > > routers. After ~ 48 hours these DNS requests are still coming. > > While we don't normally check for DNS requests in our NIDS, we added a > rule to look for this address. We're also receiving a veritable flood of > this traffic. > > We're going to ask our network folks to block this also. Thanks for > the heads up. > > --- > Robin Anderson UNIX SysAdmin, Specialist > Office of Informational Technology University of MD, Baltimore County (UMBC) > > PGP fingerprint: (resumbc99) 1024/5B5A87A > DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 > > "Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace From glratt@rice.edu Mon Jan 14 11:06:43 2002 Date: Sat, 6 Jan 2001 23:35:33 -0600 (CST) Subject: Re: DNS requests from 209.67.50.203 From: Glenn Forbes Fleming Larratt To: unisog@sans.org We've seen this in a steady stream since Jan 3rd, pointing at our primary nameserver. We blocked them about 30 hours ago, reasoning that at best something was broken, and at worst it was a DoS or DDoS. Glenn Forbes Fleming Larratt Rice University Network Management glratt@rice.edu On Sat, 6 Jan 2001, Gene Runion wrote: > Hi, > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > this address, who said they are not the source but the target of a DoS > attack. He claimed they are seeing 60 - 90 Mb data stream > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > routers. After ~ 48 hours these DNS requests are still coming. > > > Gene Runion Ph: 804 296 0327 > National Radio Astronomy Observatory > 520 Edgemont Road Fax: 804 296 0278 > Charlottesville, Va 22903 > > Glenn Forbes Fleming Larratt Rice University Network Management glratt@rice.edu From vanepp@sfu.ca Mon Jan 14 11:06:43 2002 Date: Sat, 6 Jan 2001 19:08:19 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Peter Van Epp To: unisog@sans.org, intrusion@sans.org Cc: CompServ@Exodus.net > > On Sat, 6 Jan 2001, Gene Runion wrote: > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > > this address, who said they are not the source but the target of a DoS > > attack. He claimed they are seeing 60 - 90 Mb data stream > > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > > routers. After ~ 48 hours these DNS requests are still coming. > > While we don't normally check for DNS requests in our NIDS, we added a > rule to look for this address. We're also receiving a veritable flood of > this traffic. > > We're going to ask our network folks to block this also. Thanks for > the heads up. > > --- > Robin Anderson UNIX SysAdmin, Specialist > Office of Informational Technology University of MD, Baltimore County (UMBC) Looks like you can add us to the list as well (and the list that is about to block this site to reduce the load). The argus logs indicate a couple of hits per minute from at least this morning (as far back as I looked in the short term) and continuing now. Tcpdump indicates this for all of them (pretty clearly a DOS): 19:01:25.064832 futuresite.register.com.6200 > whistler.sfu.ca.domain: 29119+ M X? aol.com. (25) (DF) 19:01:25.068338 whistler.sfu.ca.domain > futuresite.register.com.6200: 29119 13 /2/11 MX yc.mx.aol.com. 15, MX yd.mx.aol.com. 15, MX ye.mx.aol.com. 15, MX yg.mx .aol.com. 15, MX yh.mx.aol.com. 15, MX xa.mx.aol.com. 15, MX xb.mx.aol.com. 15, MX xd.mx.aol.com. 15, MX za.mx.aol.com. 15, MX zb.mx.aol.com. 15, MX zc.mx.aol.c om. 15, MX zd.mx.aol.com. 15, MX yb.mx.aol.com. 15 (496) (DF) For the Exodus folks (or the user of the netblock since I expect it is a customer) please email us when the problem clears and I'll remove the block in the router. Peter Van Epp / Operations and Technical Support Simon Fraser University, Burnaby, B.C. Canada abuse@sfu.ca From robin@umbc.edu Mon Jan 14 11:06:43 2002 Date: Sat, 6 Jan 2001 20:04:21 -0500 Subject: Re: DNS requests from 209.67.50.203 (fwd) From: Robin To: unisog@sans.org Cc: Global Incident Analysis Center Sorry to follow-up to my own message, but we've just gotten more info from our Networking folks. They said that traffic was not appreciably affecting network or DNS performance for our site, so we are not in fact going to block them out. --- Robin Anderson UNIX SysAdmin, Specialist Office of Informational Technology University of MD, Baltimore County (UMBC) PGP fingerprint: (resumbc99) 1024/5B5A87A DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 "Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace ---------- Forwarded message ---------- Date: Sat, 6 Jan 2001 19:05:33 -0500 From: Robin To: Gene Runion Cc: unisog@sans.org, Global Incident Analysis Center Subject: Re: DNS requests from 209.67.50.203 On Sat, 6 Jan 2001, Gene Runion wrote: > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > this address, who said they are not the source but the target of a DoS > attack. He claimed they are seeing 60 - 90 Mb data stream > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > routers. After ~ 48 hours these DNS requests are still coming. While we don't normally check for DNS requests in our NIDS, we added a rule to look for this address. We're also receiving a veritable flood of this traffic. We're going to ask our network folks to block this also. Thanks for the heads up. --- Robin Anderson UNIX SysAdmin, Specialist Office of Informational Technology University of MD, Baltimore County (UMBC) PGP fingerprint: (resumbc99) 1024/5B5A87A DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 "Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace From tb40@cornell.edu Mon Jan 14 11:06:43 2002 Date: Sun, 07 Jan 2001 15:56:02 -0500 Subject: Re: DNS requests from 209.67.50.203 (fwd) From: Thomas P. Braun To: Robin , unisog@sans.org Cc: Global Incident Analysis Center We (Cornell University) are seeing quite a bit of this traffic as well. But just in your case it doesn't really affect us. The only reason to put up a block would be to protect 209.67.50.203 - who claim to be a victim of this activity (our dns server answers every request from this address). If the owner of this address (Mathew Zito?) requests a block we could do that for him. Posting this requests to the list would probably encourage others to block traffic to/from this IP as well. Thomas >Sorry to follow-up to my own message, but we've just gotten more info from >our Networking folks. They said that traffic was not appreciably >affecting network or DNS performance for our site, so we are not in fact >going to block them out. > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > > this address, who said they are not the source but the target of a DoS > > attack. He claimed they are seeing 60 - 90 Mb data stream > > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > > routers. After ~ 48 hours these DNS requests are still coming. -- Dr. Thomas P. Braun Systems & Network Infrastructure Security Cornell University Information Technology 719 Rhodes Hall phone: (607) 255.1346 Cornell University fax: (607) 255.8521 Ithaca, NY 14853 email: tb40@cornell.edu http://www.people.cornell.edu/pages/tb40/ From Dick_Jacobson@ndsu.nodak.edu Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 08:00:59 -0600 (CST) Subject: Re: DNS requests from 209.67.50.203 From: Dick Jacobson To: unisog@sans.org Cc: John Grosen , Greg Wettstein On Sat, 6 Jan 2001, Gene Runion wrote: We are still seeing this this morning .. at the rate of one hit every ~10 minutes. The connections are bing refused. > Hi, > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > this address, who said they are not the source but the target of a DoS > attack. He claimed they are seeing 60 - 90 Mb data stream > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > routers. After ~ 48 hours these DNS requests are still coming. > > > Gene Runion Ph: 804 296 0327 > National Radio Astronomy Observatory > 520 Edgemont Road Fax: 804 296 0278 > Charlottesville, Va 22903 > > *********************************************************************** Please Note my new address: Dick_Jacobson@ndsu.nodak.edu ----------------------------------------------------------------------- Dick Jacobson e-mail : Dick_Jacobson@ndsu.NoDak.edu ND HECN MultiUser Host SysAd office : IACC 206, NDSU ND HECN Security Officer phone : 701-231-7385 ----------------------------------------------------------------------- From brockj@hope.edu Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 10:22:30 -0500 Subject: RE: DNS requests from 209.67.50.203 From: Jonathan R Brockmeier To: unisog@sans.org We are also seeing the same to both of our DNS servers. At some where between 266-450/sec (NetEnforcer from Allot show that many connections and the time out is 1 minute for each connection.) Jon Brockmeier Computing and Information Technology Hope College >===== Original Message From Dick Jacobson ===== >On Sat, 6 Jan 2001, Gene Runion wrote: > >We are still seeing this this morning .. at the rate of one hit every ~10 >minutes. The connections are bing refused. > >> Hi, >> >> Have others seen a steady stream of DNS requests ( ~220/min/DNS server) >> from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for >> this address, who said they are not the source but the target of a DoS >> attack. He claimed they are seeing 60 - 90 Mb data stream >> toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder >> routers. After ~ 48 hours these DNS requests are still coming. >> >> >> Gene Runion Ph: 804 296 0327 >> National Radio Astronomy Observatory >> 520 Edgemont Road Fax: 804 296 0278 >> Charlottesville, Va 22903 >> >> > >*********************************************************************** > >Please Note my new address: Dick_Jacobson@ndsu.nodak.edu > >----------------------------------------------------------------------- >Dick Jacobson e-mail : Dick_Jacobson@ndsu.NoDak.edu >ND HECN MultiUser Host SysAd office : IACC 206, NDSU >ND HECN Security Officer phone : 701-231-7385 >----------------------------------------------------------------------- From lee@mailhost.sju.edu Mon Jan 14 11:06:43 2002 Date: Mon, 08 Jan 2001 11:01:45 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Stephen Lee To: unisog@sans.org Gene Runion wrote: > We are getting these also. I have asked the Network Manager to block that address for now. Steve > Hi, > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > this address, who said they are not the source but the target of a DoS > attack. He claimed they are seeing 60 - 90 Mb data stream > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > routers. After ~ 48 hours these DNS requests are still coming. > > Gene Runion Ph: 804 296 0327 > National Radio Astronomy Observatory > 520 Edgemont Road Fax: 804 296 0278 > Charlottesville, Va 22903 -- Stephen J. Lee Saint Joseph's University Systems Administrator 5600 City Avenue Networking & Telecommunications Philadelphia, PA 19131-1395 E-mail: lee@sju.edu Voice: (610) 660-1679 Fax: (610) 660-1536 From irwin@Princeton.EDU Mon Jan 14 11:06:43 2002 Date: Mon, 08 Jan 2001 11:11:55 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Irwin Tillman To: unisog@sans.org [ Moderator's note: it looks like most of us are seeing the traffic. ] [ This is the last message saying so that I'll pass through. If ] [ someone is interested in knowing just who's seeing it, post to the ] [ list volunteering to summarize and then post a summary. ] We're seeing it too, about 3 requests/second to our published DNS server. Have blackholed it. Irwin Tillman Network Systems Computing and Information Technology Princeton University irwin@princeton.edu (U.S.A.) 609-258-6062 From dittrich@cac.washington.edu Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 09:33:13 -0800 (PST) Subject: RE: DNS requests from 209.67.50.203 From: Dave Dittrich To: Jonathan R Brockmeier Cc: unisog@sans.org On Mon, 8 Jan 2001, Jonathan R Brockmeier wrote: > We are also seeing the same to both of our DNS servers. At some where between > 266-450/sec (NetEnforcer from Allot show that many connections and the time > out is 1 minute for each connection.) > > Jon Brockmeier > Computing and Information Technology > Hope College FYI. These are being seen all over the place, according to a number of people on a DDoS related list I'm on (we are even seeing them at the UW, as is someone in the UK). Many sites are blocking at their borders, just to do their part in shutting this down. It does appear to be a DoS attack. -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From pete@shadows.uottawa.ca Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 13:58:25 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Pete Hickey To: unisog@sans.org OK, lots of us are seeing traffic 'from' this site, however is anyone seeing traffic leaving their site with a source address of 209.67.50.203 (or ... if you block source addresses which are not yours, have you seen it in the log of what has blocked?) While I am seeing requests coming in, I see no attempts at forged packets going out. Blocking traffic 'from' 209.67.50.203 does not eliminate the source of the problem. -- Pete Hickey | | VEIWIT Communication Services | Pete@mudhead.uottawa.CA | Makers of transparent University of Ottawa | | mirrors for Ottawa,Ont. Canada K1N 6N5| (613) 562-5800x1008 | dyslexics. From vanepp@sfu.ca Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 12:29:00 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Peter Van Epp To: unisog@sans.org > > > OK, lots of us are seeing traffic 'from' this site, however > is anyone seeing traffic leaving their site with a source > address of 209.67.50.203 (or ... if you block source addresses > which are not yours, have you seen it in the log of what has > blocked?) While I am seeing requests coming in, I see no > attempts at forged packets going out. > > Blocking traffic 'from' 209.67.50.203 does not eliminate the source > of the problem. > > -- > Pete Hickey | | VEIWIT > Communication Services | Pete@mudhead.uottawa.CA | Makers of transparent > University of Ottawa | | mirrors for > Ottawa,Ont. Canada K1N 6N5| (613) 562-5800x1008 | dyslexics. > > I'm seeing the packets come in to me from our Internet connection and no indication that it is being generated internally (which the antispoof filters on the border would supress going out). I could try and trace the MAC address back through our provider's transit routers (not a trivial task, but at least I'm part of the technical team for our provider as well so I could go that far), but past experience tells me that it will die there because the transit providers aren't prepared to trace the MAC address back through their networks (too much work) and it is very likely coming in via one of the transit routes. Assuming (which certainly used to be too much of an assumption, but perhaps things have improved?) that the transit links are doing antispoof filtering on the borders, this traffic should be orginating near the site being attacked and that would probably be the place to catch it. If one or more transit providers (which used to be the case) depend on their customers to filter spoofed source addresses it will likely be pretty much untracable unless someone has a new idea I haven't heard yet. Peter Van Epp / Operations and Technical Support Simon Fraser University, Burnaby, B.C. Canada From rackow@mcs.anl.gov Mon Jan 14 11:06:43 2002 Date: Mon, 08 Jan 2001 14:43:24 -0600 Subject: Re: DNS requests from 209.67.50.203 From: Gene Rackow To: unisog@sans.org Cc: rackow@mcs.anl.gov 209.67.50.203 is a "register.com" site that appears to be the base web server for quite a number of sites. It appears to be the "a real web page/server is coming soon" page. The are also offering a "Free Domain Name Monitoring" package that will tell you what other domains are like yours and/or may be infringing on your name space. I wonder if something hasn't gone south? Considering the reports that are showing up here, I'm amazed that their web server is still able to operate. Maybe they are just blocking named replies for that machine at an upstream location. --Gene Pete Hickey made the following keystrokes: > >OK, lots of us are seeing traffic 'from' this site, however >is anyone seeing traffic leaving their site with a source >address of 209.67.50.203 (or ... if you block source addresses >which are not yours, have you seen it in the log of what has >blocked?) While I am seeing requests coming in, I see no >attempts at forged packets going out. >Blocking traffic 'from' 209.67.50.203 does not eliminate the source >of the problem. > >-- >Pete Hickey | | VEIWIT >Communication Services | Pete@mudhead.uottawa.CA | Makers of transparent >University of Ottawa | | mirrors for >Ottawa,Ont. Canada K1N 6N5| (613) 562-5800x1008 | dyslexics. > From matt@sans.org Mon Jan 14 11:06:43 2002 Date: Mon, 08 Jan 2001 19:15:25 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Matt Fearnow To: Robin , Gene Runion Cc: unisog@sans.org, Global Incident Analysis Center I'm just curious if anyone can tell me any more information about what is going on here? Is the traffic still happening? Gene, you said you contacted Mathew Zito, have you heard any more from him? Any information would be great. Thanks Matt Fearnow SANS GIAC Incident Handler matt@sans.org At Saturday 1/6/2001 07:05 PM, Robin wrote: >On Sat, 6 Jan 2001, Gene Runion wrote: > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > > this address, who said they are not the source but the target of a DoS > > attack. He claimed they are seeing 60 - 90 Mb data stream > > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > > routers. After ~ 48 hours these DNS requests are still coming. > >While we don't normally check for DNS requests in our NIDS, we added a >rule to look for this address. We're also receiving a veritable flood of >this traffic. > >We're going to ask our network folks to block this also. Thanks for >the heads up. > >--- >Robin Anderson UNIX SysAdmin, Specialist >Office of Informational Technology University of MD, Baltimore County >(UMBC) > >PGP fingerprint: (resumbc99) 1024/5B5A87A >DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 > >"Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace From grunion@NRAO.EDU Mon Jan 14 11:06:43 2002 Date: Mon, 8 Jan 2001 22:13:16 -0500 (EST) Subject: Re: DNS requests from 209.67.50.203 From: Gene Runion To: Matt Fearnow Cc: Robin , unisog@sans.org, Global Incident Analysis Center I too have been curious. Here is the run down from my end. I still have network 209.67.50.0 blocked at our three routers with different internet access. I am still seeing about the same number of denies. (I am no longer logging the denies so I am assuming that they are still all coming from 209.67.50.203 (or spoofed) and that they are DNS requests). We first discovered this because we have one DNS server with the newer bind that is configured not resolve names for hosts that are not in our domain when the request comes from the internet. We were logging such requests which resulted in an abnormally large log file which got our attention. Then I noticed a steady stream of DNS requests from 209.67.50.203 to our five DNS servers. At that point I decided something was wrong, other than someone trying to use our DNS server, and blocked that network. We then sent an email to abuse@exodus.net. Then I received a telephone call from Mathew Zito (212-798 9205) who said they were not the source but the victim and they, for the last 72 hours or so, have been trying to put an end to it. This all took place on 4 Jan from ~3-9pm est. We have had no further correspondence with them. Late Friday afternoon I checked with a sister organization who, after checking their logs, saw the same behavior. Saturday, after checking to see if this traffic was still present, I sent a message to unisog. That's it from my end. Gene Runion Ph: 804 296 0327 National Radio Astronomy Observatory 520 Edgemont Road Fax: 804 296 0278 Charlottesville, Va 22903-2475 On Mon, 8 Jan 2001, Matt Fearnow wrote: > I'm just curious if anyone can tell me any more information about what is > going on here? Is the traffic still happening? Gene, you said you > contacted Mathew Zito, have you heard any more from him? Any information > would be great. > > Thanks > > Matt Fearnow > SANS GIAC Incident Handler > matt@sans.org > > At Saturday 1/6/2001 07:05 PM, Robin wrote: > >On Sat, 6 Jan 2001, Gene Runion wrote: > > > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS server) > > > from 209.67.50.203? On Thursday I talked to Mathew Zito, the admin for > > > this address, who said they are not the source but the target of a DoS > > > attack. He claimed they are seeing 60 - 90 Mb data stream > > > toward 209.67.50.203. On Thursday I blocked 209.67.50.0 at our boarder > > > routers. After ~ 48 hours these DNS requests are still coming. > > > >While we don't normally check for DNS requests in our NIDS, we added a > >rule to look for this address. We're also receiving a veritable flood of > >this traffic. > > > >We're going to ask our network folks to block this also. Thanks for > >the heads up. > > > >--- > >Robin Anderson UNIX SysAdmin, Specialist > >Office of Informational Technology University of MD, Baltimore County > >(UMBC) > > > >PGP fingerprint: (resumbc99) 1024/5B5A87A > >DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 > > > >"Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace > > From jwj@umich.edu Mon Jan 14 11:06:43 2002 Date: Mon, 08 Jan 2001 22:12:03 -0500 Subject: Re: DNS requests from 209.67.50.203 From: James W. Jeffries To: Peter Van Epp , unisog@sans.org Interestingly, in watching for all traffic from that machine, I'm seeing it on TCP port 80. We too have anti-spoofing filters on our router, so it is most likely coming from "outside" (unless someone "inside" is playing games - unlikely). Jim Jeffries Director, Office of Computing Services (OCS) jwj@umich.edu Physics Department (734)936-2426 (voice) University of Michigan (734)817-6356 (page) 2428-C Randall Labs, Ann Arbor, MI 48109-1120 (734)763-9694 (fax) "If we could read the secret history of our enemies, we should find in each man's life sorrow and suffering enough to disarm any hostility." - Henry Wadsworth Longfellow --On Monday, January 08, 2001 12:29 PM -0800 Peter Van Epp wrote: > > > > > > OK, lots of us are seeing traffic 'from' this site, however > > is anyone seeing traffic leaving their site with a source > > address of 209.67.50.203 (or ... if you block source addresses > > which are not yours, have you seen it in the log of what has > > blocked?) While I am seeing requests coming in, I see no > > attempts at forged packets going out. > > > > Blocking traffic 'from' 209.67.50.203 does not eliminate the source > > of the problem. > > > > -- > > Pete Hickey | | VEIWIT > > Communication Services | Pete@mudhead.uottawa.CA | Makers of > > transparent University of Ottawa | | > > mirrors for Ottawa,Ont. Canada K1N 6N5| (613) 562-5800x1008 | > > dyslexics. > > > > > > I'm seeing the packets come in to me from our Internet connection and > no indication that it is being generated internally (which the antispoof > filters on the border would supress going out). I could try and trace > the MAC address back through our provider's transit routers (not a > trivial task, but at least I'm part of the technical team for our > provider as well so I could go that far), but past experience tells me > that it will die there because the transit providers aren't prepared to > trace the MAC address back through their networks (too much work) and it > is very likely coming in via one of the transit routes. Assuming (which > certainly used to be too much of an assumption, but perhaps things have > improved?) that the transit links are doing antispoof filtering on the > borders, this traffic should be orginating near the site being attacked > and that would probably be the place to catch it. If one or more transit > providers (which used to be the case) depend on their customers to filter > spoofed source addresses it will likely be pretty much untracable unless > someone has a new idea I haven't heard yet. > > Peter Van Epp / Operations and Technical Support > Simon Fraser University, Burnaby, B.C. Canada > From binzina@mbl.edu Mon Jan 14 11:06:43 2002 Date: Tue, 09 Jan 2001 13:58:58 -0500 Subject: Re: DNS requests from 209.67.50.203 From: binzina@mbl.edu To: Matt Fearnow , unisog@sans.org The DNS hits from 209.67.50.203 host are continuing here. We have blocked the address at our entrance. We are receiving about 10 or so packets per second, all of them doing a DNS query looking for an MX record for aol.com. Fascinating stuff. It is interesting to me that these packets are directed toward one of our addresses that has not been a DNS server here in about a year. The address is "retired", since the machine here at that address, from 1996 to 1999, was severely hacked over a long period of time. (This was before I started working here. :-) This means, I think, that whoever/whatever is doing this has some very old information about what addresses DNS servers have. Could it be that the only way to find and/or eventually stop this kind of thing is for the ISP's and backbone providers to be involved? Cheers to all, Barbara Inzina Network Manager Marine Biological Laboratory Woods Hole, Mass. Matt Fearnow wrote: > > I'm just curious if anyone can tell me any more information about what is > going on here? Is the traffic still happening? Gene, you said you > contacted Mathew Zito, have you heard any more from him? Any information > would be great. From matt@sans.org Mon Jan 14 11:06:43 2002 Date: Tue, 09 Jan 2001 13:56:48 -0500 Subject: Fwd: Information about Register.com DoS From: Matt Fearnow To: intrusion@sans.org, unisog@sans.org Follow up with Matthew Zito. They are recommending that sites put in an acl with the ip 209.67.50.203 port udp 53, as most of you have done already. Chris B/ other GIAC handlers, Is there anything else that can be done? I'm not savvy on bind, but to me it would seem that there is some type of misconfiguration? Thoughts? Matt Fearnow SANS GIAC Incident Handler matt@sans.org *** Not for posting *** >Delivered-To: giacmatt@iquest.net >From: Matthew Zito >To: matt@sans.org >Subject: Information about Register.com DoS >Date: Tue, 9 Jan 2001 11:40:34 -0500 >X-Mailer: KMail [version 1.1.94] > > >Hey, > >Basically, Register.com has been under attack since Weds. at 5:30 pm. The >attack is detailed in CERT incident notes 2000-4. Someone is sending dns >requests for aol.com's mx records to various dns servers around the net with >a spoofed source address of 209.67.50.203. The 25-byte request results in a >500 byte response coming back into our network. We are not currently being >affected adversely by this attack - we're just trying to contact as many isps >as possible whose dns servers are being used as amplifiers to notify them >that the attack is a spoofed attack, and not legitimately originating from >us. Anyone who is being affected by this can block the traffic by denying >udp packets with a source address of 209.67.50.203 and a destination port of >53. If you have any questions or concerns, please feel free to contact me as >noted below. > >Thanks, >Matt > >-- >Matthew J. Zito >Systems Engineer >Register.com, Inc., 11th Floor, 575 8th Avenue, New York, NY 10018 >Ph: 212-798-9205 >PGP Key Fingerprint: 4E AC E1 0B BE DD 7D BC D2 06 B2 B0 BF 55 68 99 Matt Fearnow SANS GIAC Incident Handler matt@sans.org From bdc1@humboldt.edu Mon Jan 14 11:06:44 2002 Date: Tue, 9 Jan 2001 10:56:01 -0800 Subject: Re: DNS requests from 209.67.50.203 From: Ben Curran To: Robin , Gene Runion , Matt Fearnow Cc: unisog@sans.org, Global Incident Analysis Center Yes. At Humboldt State Univ. Still seeing UDP (MX record) AOL.com queries to Campus DNS server. (No CPU MIPS impact, since we are dropping src IP 209.67.50.203 at server) Our ISP is opening ticket with QWest ("their" ISP) to begin tracking src. Ben Curran On 8 Jan 2001, at 19:15, Matt Fearnow wrote: > I'm just curious if anyone can tell me any more information about what > is going on here? Is the traffic still happening? Gene, you said you > contacted Mathew Zito, have you heard any more from him? Any > information would be great. > > Thanks > > Matt Fearnow > SANS GIAC Incident Handler > matt@sans.org > > At Saturday 1/6/2001 07:05 PM, Robin wrote: > >On Sat, 6 Jan 2001, Gene Runion wrote: > > > > > Have others seen a steady stream of DNS requests ( ~220/min/DNS > > > server) from 209.67.50.203? On Thursday I talked to Mathew Zito, > > > the admin for this address, who said they are not the source but > > > the target of a DoS attack. He claimed they are seeing 60 - 90 Mb > > > data stream toward 209.67.50.203. On Thursday I blocked > > > 209.67.50.0 at our boarder routers. After ~ 48 hours these DNS > > > requests are still coming. > > > >While we don't normally check for DNS requests in our NIDS, we added > >a rule to look for this address. We're also receiving a veritable > >flood of this traffic. > > > >We're going to ask our network folks to block this also. Thanks for > >the heads up. > > > >--- > >Robin Anderson UNIX SysAdmin, Specialist > >Office of Informational Technology University of MD, Baltimore > >County (UMBC) > > > >PGP fingerprint: (resumbc99) 1024/5B5A87A > >DA F3 7F 1E D3 75 28 9F 75 7D 6A 0C 10 8D CE 35 > > > >"Pulvis et umbra sumus." (We are but dust and shadow.) -- Horace > > Network Specialist Humboldt State University c/o Communications & Network Services 1 Harpst St. Arcata, CA 95521 Phone: (707)826-5000 FAX: (707)826-6161 Email: bdc1@humboldt.edu From matt@sans.org Mon Jan 14 11:06:44 2002 Date: Tue, 09 Jan 2001 14:07:39 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Matt Fearnow To: binzina@mbl.edu, unisog@sans.org Cc: intrusion@sans.org I agree that the only way to find and/or stop this is to get others involved. I have posted to GIAC to ask people to check their networks for outgoing packets. But this is a very limited resource. I will check to see about getting an alert sent out from SANS if need be. Matt At Tuesday 1/9/2001 01:58 PM, binzina@mbl.edu wrote: >The DNS hits from 209.67.50.203 host are continuing here. We have >blocked the address at our entrance. We are receiving about 10 or so >packets per second, all of them doing a DNS query looking for an MX >record for aol.com. Fascinating stuff. > It is interesting to me that these packets are directed toward one of >our addresses that has not been a DNS server here in about a year. The >address is "retired", since the machine here at that address, from 1996 >to 1999, was severely hacked over a long period of time. (This was >before I started working here. :-) > This means, I think, that whoever/whatever is doing this has some > very >old information about what addresses DNS servers have. Could it be that >the only way to find and/or eventually stop this kind of thing is for >the ISP's and backbone providers to be involved? >Cheers to all, >Barbara Inzina >Network Manager >Marine Biological Laboratory >Woods Hole, Mass. > >Matt Fearnow wrote: > > > > I'm just curious if anyone can tell me any more information about what is > > going on here? Is the traffic still happening? Gene, you said you > > contacted Mathew Zito, have you heard any more from him? Any information > > would be great. From grunion@NRAO.EDU Mon Jan 14 11:06:44 2002 Date: Tue, 9 Jan 2001 15:47:59 -0500 (EST) Subject: Re: Fwd: Information about Register.com DoS From: Gene Runion To: Matt Fearnow Cc: intrusion@sans.org, unisog@sans.org Hi, For a few minutes I enabled logging for 209.67.50.203 on one of our routers. Here is a snap shot of the log file from the router: Jan 9 13:07:27 est: denied udp 209.67.50.203(8886) -> 192.33.115.2(53), 1 packet Jan 9 13:07:32 est: denied udp 209.67.50.203(8399) -> 192.33.115.2(53), 1 packet Jan 9 13:07:33 est: denied udp 209.67.50.203(13303) -> 192.33.116.115(53), 1 packet Jan 9 13:07:34 est: denied udp 209.67.50.203(1173) -> 192.33.115.2(53), 1 packet Jan 9 13:07:35 est: denied udp 209.67.50.203(4331) -> 192.33.115.98(53),1 packet Jan 9 13:07:36 est: denied udp 209.67.50.203(28995) -> 192.33.116.115(53), 1 packet Jan 9 13:07:37 est: denied udp 209.67.50.203(4978) -> 192.33.115.2(53), 1 packet Jan 9 13:07:41 est: denied udp 209.67.50.203(9835) -> 192.33.116.115(53), 1 packet Jan 9 13:07:42 est: denied udp 209.67.50.203(22566) -> 192.33.116.115(53), 1 packet Jan 9 13:07:43 est: denied udp 209.67.50.203(3547) -> 192.33.115.98(53), 1 packet Jan 9 13:07:44 est: denied udp 209.67.50.203(9777) -> 192.33.116.115(53), 1 packet Jan 9 13:07:46 est: denied udp 209.67.50.203(8718) -> 192.33.115.98(53), 1 packet Jan 9 13:07:48 est: denied udp 209.67.50.203(27374) -> 192.33.115.98(53), 1 packet Jan 9 13:07:49 est: denied udp 209.67.50.203(25032) -> 192.33.115.98(53), 1 packet Jan 9 13:07:50 est: denied udp 209.67.50.203(5688) -> 192.33.115.2(53), 1 packet Jan 9 13:07:51 est: denied udp 209.67.50.203(19981) -> 192.33.116.115(53), 1 packet Jan 9 13:07:52 est: denied udp 209.67.50.203(7099) -> 192.33.115.2(53), 1 packet Jan 9 13:07:54 est: denied udp 209.67.50.203(16924) -> 192.33.115.2(53), 1 packet Jan 9 13:07:55 est: denied udp 209.67.50.203(8467) -> 192.33.115.98(53), 1 packet Jan 9 13:07:57 est: denied udp 209.67.50.203(8954) -> 192.33.116.115(53), 1 packet Jan 9 13:07:59 est: denied udp 209.67.50.203(18412) -> 192.33.115.2(53), 1 packet Jan 9 13:08:00 est: denied udp 209.67.50.203(8602) -> 192.33.115.98(53), 1 packet Jan 9 13:08:02 est: denied udp 209.67.50.203(14816) -> 192.33.115.2(53), 1 packet Jan 9 13:08:03 est: denied udp 209.67.50.203(29241) -> 192.33.116.115(53), 1 packet Jan 9 13:08:04 est: denied udp 209.67.50.203(27452) -> 192.33.116.115(53), 1 packet Here is a snap shot of the logs from one of the DNS servers. (I did not try to line up times). Jan 9 13:16:53 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:16:56 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:02 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:03 cv3 named[12433]: unapproved query from [209.67.50.203].11239 for "aol.com" Jan 9 13:17:04 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:06 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:06 cv3 named[12433]: unapproved query from [209.67.50.203].16840 for "aol.com" Jan 9 13:17:08 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:08 cv3 named[12433]: unapproved query from [209.67.50.203].19309 for "aol.com" Jan 9 13:17:11 cv3 named[12433]: unapproved query from [209.67.50.203].23622 for "aol.com" Jan 9 13:17:14 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:14 cv3 named[12433]: unapproved query from [209.67.50.203].8107 for "aol.com" Jan 9 13:17:14 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:16 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:16 cv3 named[12433]: unapproved query from [209.67.50.203].12247 for "aol.com" Jan 9 13:17:16 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:21 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:22 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:22 cv3 named[12433]: unapproved query from [209.67.50.203].2612 for "aol.com" Jan 9 13:17:23 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:26 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:27 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:28 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:28 cv3 named[12433]: unapproved query from [209.67.50.203].24182 for "aol.com" Jan 9 13:17:30 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:30 cv3 named[12433]: unapproved query from [209.67.50.203].20310 for "aol.com" Jan 9 13:17:30 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:31 cv3 named[12433]: XX+/209.67.50.203/aol.com/MX/IN Jan 9 13:17:32 cv3 named[12433]: unapproved query from [209.67.50.203].6333 for "aol.com" Starting with BIND 8.2 (I think) one can configure it not to resolve requests coming from external hosts trying to resolve a host not on your network. That is what we have done on one of our NDS server's. This configuration is what lead me to the constant stream of traffic from 209.67.50.203. I hope this is helpful. Gene Runion Ph: 804 296 0327 National Radio Astronomy Observatory 520 Edgemont Road Fax: 804 296 0278 Charlottesville, Va 22903-2475 From matt@sans.org Mon Jan 14 11:06:44 2002 Date: Tue, 09 Jan 2001 15:14:26 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Matt Fearnow To: Jose Nazario , binzina@mbl.edu, mzito@register.com Cc: unisog@sans.org, intrusion@sans.org, joem@email.nist.gov Also, can someone that is receiving the packets that isn't already dropping them (or who wouldn't mind changing their acl for a while) get me a tcpdump trace with -e option. Then we can hopefully start tracing it back. From mzito@register.com Mon Jan 14 11:06:46 2002 Date: Tue, 9 Jan 2001 15:30:20 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Matthew Zito To: Matt Fearnow , Jose Nazario , binzina@mbl.edu, mzito@register.com Cc: unisog@sans.org, intrusion@sans.org On Tue, 09 Jan 2001, Matt Fearnow wrote: > I'm not sure, but Matt Zito can probably answer this. > Hi all, > At Tuesday 1/9/2001 02:32 PM, Jose Nazario wrote: > >has the upstream been contacted? > >what was the response? > > > >has their uplink provider been contacted about an access control measure > >being taken to stop this? > > Okay, here is the current status: Exodus and Globix (our providers) are currently filtering the incoming responses upstream of us. Since the requests are spoofed from elsewhere, we've been trying to start tracebacks from the dns servers that are being used as "amplifiers" in the attack, so far with no useful information. So, we're not being harmed by this anymore (we haven't since about an hour after it initially started), we've verified that the packets are definitely not originating from our network, and we're trying to coax other people's upstreams into doing something useful about it. As an alternative measure, we're recommending anyone who is seeing traffic from futuresite.register.com (209.67.50.203) to their dns server (53 udp - being paranoid) access-list it off on their routers. There's no reason that ip address - which is just a virtual address on a load balancer- should be doing dns queries. Please don't access-list that whole ip address off, though - that is an ip address that gets a lot of valid web traffic. We appreciate everyone who's gotten in touch with us about this - and we want to make sure that everyone understands that we are not the source of this - rather, we're the target, and that our current strategy is to contact all the affected dns servers (which is a lot) and ask them to request tracebacks from their upstreams and to ACL the packets off from their network. > >how long has it been? a week? > > Yep. Since January 4 at abount 5:20 pm EST. If anyone has any useful information or questions, please feel free to contact me. Thanks a lot. Thanks, Matt -- Matthew J. Zito Systems Engineer Register.com, Inc., 11th Floor, 575 8th Avenue, New York, NY 10018 Ph: 212-798-9205 PGP Key Fingerprint: 4E AC E1 0B BE DD 7D BC D2 06 B2 B0 BF 55 68 99 From dittrich@cac.washington.edu Mon Jan 14 11:06:46 2002 Date: Tue, 9 Jan 2001 16:34:33 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Dave Dittrich To: Matt Fearnow Cc: Jose Nazario , binzina@mbl.edu, mzito@register.com, unisog@sans.org, intrusion@sans.org, joem@email.nist.gov On Tue, 9 Jan 2001, Matt Fearnow wrote: > Also, can someone that is receiving the packets that isn't already dropping > them (or who wouldn't mind changing their acl for a while) get me a tcpdump > trace with -e option. Then we can hopefully start tracing it back. I don't think that trying to figure out what ethernet address the packets are coming in on is going to help. You need to get your upstream providers to trace the packets through their networks. Nobody will be able to do hop-to-hop traces by MAC address past their own border. -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From dittrich@cac.washington.edu Mon Jan 14 11:06:46 2002 Date: Tue, 9 Jan 2001 17:15:23 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Dave Dittrich To: binzina@mbl.edu Cc: Matt Fearnow , unisog@sans.org Barbara, On Tue, 9 Jan 2001 binzina@mbl.edu wrote: > It is interesting to me that these packets are directed toward one of > our addresses that has not been a DNS server here in about a year. The > address is "retired", since the machine here at that address, from 1996 > to 1999, was severely hacked over a long period of time. (This was > before I started working here. :-) > ... > This means, I think, that whoever/whatever is doing this has some very > old information about what addresses DNS servers have. When I observed yesterday that traffic was going to two hosts on campus that are DHCP servers that don't even run BIND, one of our network engineers made the same observation you have: On Mon, 8 Jan 2001, Aaron Racine wrote: > I theorized that whoever is doing this harvested authoritative > nameservers for some set of domains a while ago and sent their > queries to that set of servers. Unfortunately for the attackers, > between the time they harvested our nameservers' IP addresses and > the time the attack was initiated, our authoritative nameservers > changed IP addresses. XXXXX's and XXXXX's IP addresses are now used > by XXXXXXXX and XXXXXX. XXXXXXX has been authoritative for > washington.edu for a while now. > > [snip] > > Aaron -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From andy@umbc.edu Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 13:49:51 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Anderson Johnston To: Dave Dittrich Cc: Matt Fearnow , Jose Nazario , binzina@mbl.edu, mzito@register.com, unisog@sans.org, intrusion@sans.org, joem@email.nist.gov On Tue, 9 Jan 2001, Dave Dittrich wrote: > On Tue, 9 Jan 2001, Matt Fearnow wrote: > > > Also, can someone that is receiving the packets that isn't already dropping > > them (or who wouldn't mind changing their acl for a while) get me a tcpdump > > trace with -e option. Then we can hopefully start tracing it back. > > I don't think that trying to figure out what ethernet address the > packets are coming in on is going to help. You need to get your > upstream providers to trace the packets through their networks. > Nobody will be able to do hop-to-hop traces by MAC address past their > own border. > I've put a call in to the admin of the U. Maryland System network. He's the first step for us. If he can point me to another router I'll let hte list know. > -- > Dave Dittrich Computing & Communications > dittrich@cac.washington.edu Client Services > http://staff.washington.edu/dittrich University of Washington > > PGP key http://staff.washington.edu/dittrich/pgpkey.txt > Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 > ------------------------------------------------------------------------------ ** Andy Johnston (andy@umbc.edu) * pager: 410-678-8949 ** ** Distributed Systems Manager * PGP key:(afj2000) 1024/F67035E1 ** ** Office of Information Technology, UMBC * 5D 44 1E 2E A6 7C 91 7A ** ** 410-455-2583 (v)/410-455-1065 (f) * C4 66 5F D5 BA B9 F6 58 ** ------------------------------------------------------------------------------ From dittrich@cac.washington.edu Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 16:34:25 -0800 (PST) Subject: Re: DNS requests from 209.67.50.203 From: Dave Dittrich To: unisog@sans.org, intrusion@sans.org FYI. I have to run right now and won't be around email for a few hours to perhaps tomorrow afternoon. Just wanted to say that the traffic should have stopped (sometime around 13:52:03 PST). If anyone is still seeing it, let me know. More details later as appropriate. -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From andy@umbc.edu Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 15:04:50 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Anderson Johnston To: Matt Fearnow Cc: unisog@sans.org I spoke with the network admin for the U.Md. system. When he looked, he saw packets swarming in to several campuses. There are also TCP packets, by the way. 209.67.50.203 is a web server. He suggested that someone may have gotten hold of an old version of the edu zone file froma root name server. He also confirmed that the packets were coming from our upstream ISP, Qwest. I'll try them next. Just for grins, he will extract the target IP's of the packets he's getting and send them along to me. I'll just check a sample to see if they match the current/former DNS profile. If this is a distributed attack, tracking it will be a real pain. - Andy ------------------------------------------------------------------------------ ** Andy Johnston (andy@umbc.edu) * pager: 410-678-8949 ** ** Distributed Systems Manager * PGP key:(afj2000) 1024/F67035E1 ** ** Office of Information Technology, UMBC * 5D 44 1E 2E A6 7C 91 7A ** ** 410-455-2583 (v)/410-455-1065 (f) * C4 66 5F D5 BA B9 F6 58 ** ------------------------------------------------------------------------------ From andy@umbc.edu Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 13:49:51 -0500 Subject: Re: DNS requests from 209.67.50.203 From: Anderson Johnston To: Dave Dittrich Cc: Matt Fearnow , Jose Nazario , binzina@mbl.edu, mzito@register.com, unisog@sans.org, intrusion@sans.org, joem@email.nist.gov On Tue, 9 Jan 2001, Dave Dittrich wrote: > On Tue, 9 Jan 2001, Matt Fearnow wrote: > > > Also, can someone that is receiving the packets that isn't already dropping > > them (or who wouldn't mind changing their acl for a while) get me a tcpdump > > trace with -e option. Then we can hopefully start tracing it back. > > I don't think that trying to figure out what ethernet address the > packets are coming in on is going to help. You need to get your > upstream providers to trace the packets through their networks. > Nobody will be able to do hop-to-hop traces by MAC address past their > own border. > I've put a call in to the admin of the U. Maryland System network. He's the first step for us. If he can point me to another router I'll let hte list know. > -- > Dave Dittrich Computing & Communications > dittrich@cac.washington.edu Client Services > http://staff.washington.edu/dittrich University of Washington > > PGP key http://staff.washington.edu/dittrich/pgpkey.txt > Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 > ------------------------------------------------------------------------------ ** Andy Johnston (andy@umbc.edu) * pager: 410-678-8949 ** ** Distributed Systems Manager * PGP key:(afj2000) 1024/F67035E1 ** ** Office of Information Technology, UMBC * 5D 44 1E 2E A6 7C 91 7A ** ** 410-455-2583 (v)/410-455-1065 (f) * C4 66 5F D5 BA B9 F6 58 ** ------------------------------------------------------------------------------ From cbrenton@sover.net Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 17:02:53 -0500 Subject: Re: Fwd: Information about Register.com DoS From: Chris Brenton To: Matt Fearnow Cc: intrusion@sans.org, unisog@sans.org Matt Fearnow wrote: > > They are recommending that sites put in an acl with the ip 209.67.50.203 > port udp 53, as most of you have done already. > > Chris B/ other GIAC handlers, > > Is there anything else that can be done? I'm not savvy on bind, but to me > it would seem that there is some type of misconfiguration? Yup, under options you can do a: blackhole {209.67.50.203}; This will cause the name server to completely ignore the address, thus prevent DoS packets from being sent. HTH, Chris -- ************************************** cbrenton@sover.net * Mastering Cisco Routers http://www.amazon.com/exec/obidos/ASIN/078212643X/ * Mastering Network Security http://www.amazon.com/exec/obidos/ASIN/0782123430/ From tep@SDSC.EDU Mon Jan 14 11:06:46 2002 Date: Wed, 10 Jan 2001 21:26:46 -0800 (PST) Subject: Re: Information about Register.com DoS From: Tom Perrine To: intrusion@sans.org, unisog@sans.org I keep seeing people report that they have blocked or blackholed the DNS DDoS to register.com. This is interesting, but for now, not that useful to the end victim, register.com, as at least one (and probably all) of their providers are currently blocking DNS responses to the victim host (possibly and parts of the victim subnet). This may be good for those sites that were seeing enough DNS traffic to consider themselves victims, but that's another issue... In short, the victim is not (at the last report I got) actually seeing much of the DDoS, or at least not the one we've been talking about. Obviously it would be trivial for the DDoS attacker to shift targets to other locations within the victim address space. The main problem is the same we have seen with open SMURF amplifiers, ping and other DDoS's already: people can raise their own shields, in most cases, but this does nothing to protect the infrastructure at large, and does nothing to actually find and stop the attackers. In fact, this makes the problem "go away" for most folks, allowing them the luxury of ignoring the actual problem. Until all stubs and some other parts of the infrastructure take responsibility for their network traffic and perform suitable egress filtering, the rest of us will have to keep raising our shields until the attackers get bored or move on. In fact its worse. We have to keep "adjusting the shield harmonics" :-) (re-implmenting different ingress filters) for each sepcific incident. This Does *Not* Scale. I'm not going to argue the merits/costs/difficulties of egress filtering, at least not in the stub or near-stub cases: if you aren't a backbone or near backbone, and you don't do egress filtering, you are operating irresponsibly. And the rest of us may have to resort to MAPS-style shunning. Run a known open relay and I won't take email from you. Run a known "open network" (allowing any forged source addresses) and I won't take packets from you. -- Tom E. Perrine (tep@SDSC.EDU) | San Diego Supercomputer Center http://www.sdsc.edu/~tep/ | Voice: +1.858.534.5000 "Libertarianism is what your mom taught you: 'Behave yourself and don't hit your sister."' - Kenneth Bisson of Angola, Ind. From tep@SDSC.EDU Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 07:38:31 -0800 (PST) Subject: Re: Information about Register.com DoS From: Tom Perrine To: tep@SDSC.EDU Cc: intrusion@sans.org, unisog@sans.org I don't usually respond to my own posts, but a night of reflection and consultation has begun to convince me that a MAPS-like registry of networks that don't do egress filtering may be a non-starter. Technically, its a good idea, but Paul Vixie privately pointed out that the legal issues make this a non-starter as a MAPS-like service. Since he has more experience running a MAPS service, I'll bow to his superior wisdom. I still believe that egress filtering is part of being a responsible network operator, though. --tep -- Tom E. Perrine (tep@SDSC.EDU) | San Diego Supercomputer Center http://www.sdsc.edu/~tep/ | Voice: +1.858.534.5000 "Libertarianism is what your mom taught you: 'Behave yourself and don't hit your sister."' - Kenneth Bisson of Angola, Ind. From ellidz@eridu.uchicago.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 12:51:45 -0600 Subject: Re: Information about Register.com DoS From: E. Larry Lidz To: Tom Perrine Cc: intrusion@sans.org, unisog@sans.org Tom Perrine writes: >I don't usually respond to my own posts, but a night of reflection and >consultation has begun to convince me that a MAPS-like registry of >networks that don't do egress filtering may be a non-starter. > >Technically, its a good idea, but Paul Vixie privately pointed out >that the legal issues make this a non-starter as a MAPS-like service. >Since he has more experience running a MAPS service, I'll bow to his >superior wisdom. > >I still believe that egress filtering is part of being a responsible >network operator, though. I'm always hesitant to think that a MAPS-like service for attacks is a good idea -- I think that the big loosers in such a setup would be us, the Universities. The fact is, because we don't have firewalls and because we have high-bandwidth pipes and are often used as middle men in attacks, I think we'll be the first onto the lists and the last off of them. If the lists were based off of certain practices (egree filtering, etc), it might work, but if it were based off of number of attacks seen or firewalls, or one of the popular corporate buzzwords, it could be a disaster. -Larry From benchoff@vt.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 09:24:24 -0500 Subject: Re: [UNISOG] Information about Register.com DoS From: Phil Benchoff To: unisog@sans.org On Wed, Jan 10, 2001 at 09:26:46PM -0800, Tom Perrine wrote: > I'm not going to argue the merits/costs/difficulties of egress > filtering, at least not in the stub or near-stub cases: if you aren't > a backbone or near backbone, and you don't do egress filtering, you > are operating irresponsibly. No argument from me. > And the rest of us may have to resort to MAPS-style shunning. > > Run a known open relay and I won't take email from you. > > Run a known "open network" (allowing any forged source addresses) > and I won't take packets from you. I like the idea, but this is difficult to test remotely (unless they also allow loose source routing). The ISP upstream of a near-stub site could enable RPF check on the interface going to the site. That would be fairly painless for a lot of links. Another approach would be to update the requirements for internet gateways to have an RPF check on by default. I doubt you could get enough support to do any of the above. What's left? Maybe a "Good Networkkeeping Seal of Approval" for ISPs that have a policy that says their downstream sites must do egress filtering, not be broadcast amplifiers. None of this stuff works unless the vast majority of other sites take these steps. Phil From matt@sans.org Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 11:59:16 -0500 Subject: Fwd: Re: DDoS From: Matt Fearnow To: unisog@sans.org, intrusion@sans.org Is anyone still seeing this? Could it be that an upstream provider is just filtering, or is it a lack of communication from exodus? Matt >Delivered-To: giacmatt@iquest.net >From: Matthew Zito >To: Matt Fearnow >Subject: Re: DDoS >Date: Thu, 11 Jan 2001 10:54:46 -0500 >X-Mailer: KMail [version 1.1.94] > >On Thu, 11 Jan 2001, you wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Matt, > > > > I was wondering if you had an update as to the DDoS attack that was > > happening? > > > >Well, I've put in a call to our security guy at exodus - since our upstream >is filtering, its impossible to know whether or not its still going on. >Someone claimed it had ceased, but others are still reporting traffic. >Perhaps someone's backbone pre-emptively blocked it, creating the impression >for some people it had stopped. Still no word on the traceback - we're >setting the lines on fire trying to get people to try to track it back. I'll >let you know if its stopped (you'd think Exodus would notify us if it had, >but......). > >Thanks a lot, >Matt > >-- >Matthew J. Zito >Systems Engineer >Register.com, Inc., 11th Floor, 575 8th Avenue, New York, NY 10018 >Ph: 212-798-9205 >PGP Key Fingerprint: 4E AC E1 0B BE DD 7D BC D2 06 B2 B0 BF 55 68 99 Matt Fearnow SANS GIAC Incident Handler matt@sans.org From jose@biocserver.BIOC.CWRU.Edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 18:53:28 -0500 (EST) Subject: Re: Fwd: Information about Register.com DoS From: Jose Nazario To: Dave Dittrich Cc: Chris Brenton , Matt Fearnow , intrusion@sans.org, unisog@sans.org has there been anything like the CenterTrack network being used for analysis? http://www.nanog.org/mtg-9910/robert.html if anyone has seen it in action, i would suspect it's you, dave. is it relavent to the current attack? ____________________________ jose nazario jose@cwru.edu PGP: 89 B0 81 DA 5B FD 7E 00 99 C3 B2 CD 48 A0 07 80 PGP key ID 0xFD37F4E5 (pgp.mit.edu) From dittrich@cac.washington.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 14:23:25 -0800 (PST) Subject: Re: Fwd: Information about Register.com DoS From: Dave Dittrich To: Chris Brenton Cc: Matt Fearnow , intrusion@sans.org, unisog@sans.org On Wed, 10 Jan 2001, Chris Brenton wrote: > Matt Fearnow wrote: > > > > They are recommending that sites put in an acl with the ip 209.67.50.203 > > port udp 53, as most of you have done already. > > > > Chris B/ other GIAC handlers, > > > > Is there anything else that can be done? I'm not savvy on bind, but to me > > it would seem that there is some type of misconfiguration? > > Yup, under options you can do a: > > blackhole {209.67.50.203}; > > This will cause the name server to completely ignore the address, thus > prevent DoS packets from being sent. While this does solve the partial problem of taking one of M DNS reflectors out of service, this doesn't scale well (as pointed out by others on the list) and doesn't address the other N methods of reflection. I think a longer term solution will be better tools/techniques for early detection and traceback of these attacks, so they can be analyzed, understood, and fully resolved in less than a week. There are many promising new technologies on the horizon. -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From ellidz@eridu.uchicago.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 17:38:45 -0600 Subject: Re: Information about Register.com DoS From: E. Larry Lidz To: Russell Fulton Cc: unisog@sans.org Russell Fulton writes: >> I'm always hesitant to think that a MAPS-like service for attacks is a >> good idea -- I think that the big loosers in such a setup would be us, >> the Universities. The fact is, because we don't have firewalls and >> because we have high-bandwidth pipes and are often used as middle men >> in attacks, I think we'll be the first onto the lists and the last off >> of them. > >If I read Tom's comments correctly he was not proposing a MAPS like >system for attacks but for those who fail to take elementary >precautions to stop their networks from being abused, either by third >parties or legimitate members. Right -- but this is what I'm worried about. Who ends up determining what "elementary precautions" are? To us, it might be something like egress filtering, but we're probably not going to be the ones running the service. The person running the service might well have a more corporate sense of what "elementary precautions" are. >I strongly promote the use of moitoring traffic at the gateway with >argus and snort or somthing similar (yes, there are privacy issues but >these can be worked through at the policy level). I have been doing >this at Auckland for about 4 years now and I find I usually pick up >comprmomised machines before their administrators do and more >importantly before crackers manage to do major damage to others. Agreed. We have amazing levels of success monitoring Cisco flow's. -Larry From r.fulton@auckland.ac.nz Mon Jan 14 11:06:46 2002 Date: Fri, 12 Jan 2001 09:59:03 +1300 Subject: Re: Information about Register.com DoS From: Russell Fulton To: unisog@sans.org On Thu, 11 Jan 2001 12:51:45 -0600 "E. Larry Lidz" wrote: > > Tom Perrine writes: > >I don't usually respond to my own posts, but a night of reflection and > >consultation has begun to convince me that a MAPS-like registry of > >networks that don't do egress filtering may be a non-starter. > > > >Technically, its a good idea, but Paul Vixie privately pointed out > >that the legal issues make this a non-starter as a MAPS-like service. > >Since he has more experience running a MAPS service, I'll bow to his > >superior wisdom. > > > >I still believe that egress filtering is part of being a responsible > >network operator, though. > > I'm always hesitant to think that a MAPS-like service for attacks is a > good idea -- I think that the big loosers in such a setup would be us, > the Universities. The fact is, because we don't have firewalls and > because we have high-bandwidth pipes and are often used as middle men > in attacks, I think we'll be the first onto the lists and the last off > of them. If I read Tom's comments correctly he was not proposing a MAPS like system for attacks but for those who fail to take elementary precautions to stop their networks from being abused, either by third parties or legimitate members. > > If the lists were based off of certain practices (egree filtering, > etc), it might work, but if it were based off of number of attacks seen > or firewalls, or one of the popular corporate buzzwords, it could be a > disaster. > I strongly promote the use of moitoring traffic at the gateway with argus and snort or somthing similar (yes, there are privacy issues but these can be worked through at the policy level). I have been doing this at Auckland for about 4 years now and I find I usually pick up comprmomised machines before their administrators do and more importantly before crackers manage to do major damage to others. This helps mitigate the worst apsects of our open network policy. Argus is an open source IP audit tool, version 2.0 is about to be released rsn. It was written by Carter Bullard when he was at CMU's SEI and he has continued to maintain and develope it. Current version is available from ftp.qosient.com. Cheers, Russell. Russell Fulton, Computer and Network Security Officer The University of Auckland, New Zealand From dittrich@cac.washington.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 14:14:20 -0800 (PST) Subject: Re: Fwd: Re: DDoS From: Dave Dittrich To: Matt Fearnow Cc: unisog@sans.org, intrusion@sans.org On Thu, 11 Jan 2001, Matt Fearnow wrote: > Is anyone still seeing this? Could it be that an upstream provider is just > filtering, or is it a lack of communication from exodus? > > Matt I have received indications that the the flow is significantly lessoned, but not entirely stopped at this point. There were 53/tcp and 80/tcp flows being reported throughout this incident (register.com has a web page on that host, which is legit traffic), and some spurious ICMP port unreachable messages from systems that formerly were DNS servers, but now are not. I'm not clear on what is currently flowing in, but I think it still includes some 53/tcp traffic. My gut feeling from everything I've seen (on this list, INCIDENTS, and other private lists) is that there are a limited (perhaps as few as two) number of individual hosts running one of a handful of year+ old DNS reflector tools, not a neo-classic DDoS tool. (Typical DDoS networks I'm seeing/hearing about recently are in the 30-100 range.) -- Dave Dittrich Computing & Communications dittrich@cac.washington.edu Client Services http://staff.washington.edu/dittrich University of Washington PGP key http://staff.washington.edu/dittrich/pgpkey.txt Fingerprint FE 97 0C 57 08 43 F3 EB 49 A1 0C D0 8E 0C D0 BE C8 38 CC B5 From jtk@depaul.edu Mon Jan 14 11:06:46 2002 Date: Thu, 11 Jan 2001 18:59:37 -0600 Subject: Shunning [was: Information about Register.com DoS] From: John Kristoff To: unisog@sans.org This was talked about a few months back on the NANOG list and back then I had the same concerns as Larry. There are two approaches that might be used and one of them could be helpful in both solving the problem and addressing Larry's concerns. In the open mail relay world, two popular systems promote publicizing abusive systems -- ORBS and MAPS. In the case of MAPS, it is hard to get off, but hard to get on. In the ORBS case, its easy to get on and easy to get off. I think Larry is afraid of the ORBS case. You can follow this thread (long): http://www.merit.edu/mail.archives/nanog/msg03268.html John From rdump@river.com Mon Jan 14 11:06:46 2002 Date: Fri, 12 Jan 2001 15:51:12 -0700 Subject: Re: Shunning [was: Information about Register.com DoS] From: Richard Johnson To: jtk@aharp.is-net.depaul.edu, unisog@sans.org At 17:59 -0700 on 1/11/01, John Kristoff wrote: > This was talked about a few months back on the NANOG list and back then > I had the same concerns as Larry. There are two approaches that might > be used and one of them could be helpful in both solving the problem and > addressing Larry's concerns. In the open mail relay world, two popular > systems promote publicizing abusive systems -- ORBS and MAPS. In the > case of MAPS, it is hard to get off, but hard to get on. In the ORBS > case, its easy to get on and easy to get off. I think Larry is afraid > of the ORBS case. > > You can follow this thread (long): > > http://www.merit.edu/mail.archives/nanog/msg03268.html In the case of MAPS, it is actually easy to get off the RBL, and very difficult to get on. To be removed, you merely need to present a credible promise to stop the abusive (spam) behavior from your network in a reasonable amount of time. With ORBS (and MAPS RSS, for that matter), you actually have to fix the open relay problem to get off the list. I'm not sure why Paul Vixie would think a vaguely similar list of networks that allow spoofing of other netblock's IPs would be legally nonviable. Perhaps he was referring to a list of 'bad' networks that reportedly harbor script kiddies instead? Whereas, in cases like the register.com DoS, a list of nets that do not do egress filtering to block spoofing should suffice. If possible, denizens on the list would be convinced, via collateral damage, to fix their spoofing problem. A list of networks that don't block spoofing might be a non-starter for technical reasons, however. How would you actually determine which networks were allowing the spoofing past their border routers? And how would you draw the lines between where networks should block the spoofing, and where egress filtering shouldn't be required (keeping in mind ASNs with multiple links, legitimate transit traffic, backup links, and similar issues not generally apparent to outsiders)? Are there answers that would allow such a list to be useful on a reasonable time scale? Richard From TIHOR@acfcluster.nyu.edu Mon Jan 14 11:06:46 2002 Date: Fri, 26 Jan 2001 14:46:10 -0400 (EDT) Subject: [unisog] Re: [UNISOG] Information about Register.com DoS From: Stephen Tihor To: Phil Benchoff Cc: unisog@sans.org One step for everone to take aside from the obvious broder filters on people who can afford it computationally is to arrange to have your next level up supplier require all clients to be broder filtering or conenct only to border filtering clients. that will move us a good chunk of the way to solving things and then we can look at other steps once we have a mjor chunk of the civilized internet or at least Internet2 properly configured.