Forgot your password?
typodupeerror
Google The Internet Networking Apple Technology

Beware of Using Google Or OpenDNS For iTunes 348

Posted by timothy
from the speak-friend-and-enter-slowly dept.
Relayman writes "Joe Mailer wanted to download an iTunes movie recently and his Apple TV told him it would take two hours. When he switched his DNS resolver settings, the download time dropped to less than 20 seconds. Apparently, iTunes content is served by Akamai which uses geolocation based on the IP address of the DNS request to determine which server should provide his content. When you use Google or OpenDNS to resolve the Apple domain name, all the requests to Akamai appear to be coming from the same location and they're all directed to the same server pool, overloading that pool and causing the slow downloads. The solution: be wary of using Google or OpenDNS when downloading iTunes files or similar large files. Use your own ISP's DNS servers instead or run your own resolving DNS server."
This discussion has been archived. No new comments can be posted.

Beware of Using Google Or OpenDNS For iTunes

Comments Filter:
  • by eldavojohn (898314) * <eldavojohn.gmail@com> on Friday December 31, 2010 @12:24AM (#34718708) Journal
    First, I was able to verify this with the iTunes download. My Cox DNS was 20 seconds while my Google DNS was 2 minutes 10 seconds.

    But I just tested this on my own by using a different source that uses Akamai: Adobe.

    So I picked a file at this URL: http://ardownload.adobe.com/pub/adobe/reader/unix/9.x/9.4.0/enu/AdbeRdr9.4-1_i486linux_enu.bin [adobe.com]

    Sure enough, the initial server directed me to 72.215.224.16 with this partial tracert:

    4 12 ms 10 ms 10 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
    5 17 ms 14 ms 12 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
    6 12 ms 15 ms 12 ms 72.215.224.16

    Firefox told me this would take 3 Minutes and 35 Seconds.

    Then, I set my DNS to the 8.8.8.8 and 8.8.4.4 addresses and tried it again. This time I was sent to 72.246.30.19 with this partial tracert:

    4 11 ms 12 ms 14 ms mrfddsrj02gex070002.rd.dc.cox.net [68.100.0.145]
    5 13 ms 11 ms 13 ms ashbbprj01-ae0.0.rd.as.cox.net [68.1.0.220]
    6 17 ms 17 ms 13 ms ge13-1.br01.ash01.pccwbtn.net [63.218.44.125]
    7 21 ms 18 ms 12 ms akamai.ge13-4.br02.ash01.pccwbtn.net [63.218.94.142]
    8 17 ms 18 ms 13 ms a72-246-30-19.deploy.akamaitechnologies.com [72.246.30.19]

    Surprisingly, this second server that I was directed to using Google DNS only took 10 seconds to download the same file. I did it a second time and it took 30 seconds.

    Now after restoring my default DNS resolution that URL continually directs me to 72.215.224.40 and the download is as speedy as the Google DNS. If I switch back to Google DNS it now continually directs me to 72.246.30.32 so you can see that there's some load balancing going here that apparently can be divvied up by geographic location for some of their customers. Apparently Apple needs to investigate the same solution that Adobe is using from Akamai. Which doesn't consider everything from Google DNS being fulfilled from a west coast replication server?

    • by sumdumass (711423)

      I though Google used Anycast [wikipedia.org] just like the rest of the large providers. Perhaps it's a routing issue where Google's servers are separated a bit geographically from certain people and the servers they are wanting to connect to?

      • by rekoil (168689)

        The geographic information isn't the issue. It's the fact that there are a very large number of clients using the same pools of DNS resolvers. Akamai uses those resolvers' IP address to map the client to a cache pool; if there are too many requests from the same netblock, they'll all get sent to the same cache pool, overloading it.

        At some point, Akamai's load feedback system will notice this and direct users to a different pool, but it's a reactive measurement.

        • by TooMuchToDo (882796) on Friday December 31, 2010 @03:00AM (#34719572)

          The problem is Akamai is using DNS to determine location, when it should be determining the geolocation of the client IP and throwing an HTTP redirect to the proper server. You can't rely on a client using the "proper" geographically-located DNS server.

        • by icebike (68054) on Friday December 31, 2010 @03:49AM (#34719824)

          Well the point of the GP's Anycast comment was that simply using 8.8.8.8 as your dns server is not sufficient to pinpoint WHERE your dns comes from.

          8.8.8.8 will resolve to different physical machines depending on the load balancing that Google is doing.

          Your dns request might be served out of California on one hit, and out of Ireland on the next hit.

          Akamai, by paying attention to where the DNS request came from is doing it fundamentally WRONG, because they could actually deny service (for national licensing reasons) based on location of the DNS server when the actual user was in a totally different (and legal) location.

    • Re: (Score:2, Informative)

      by sleeper0 (319432)

      Two hours vs. instant streaming isn't a localization issue, you can easily stream 1-2mbps (or much more) from half way around the world. ~100ms in latency is nothing with a fat, non time sensitive stream like recorded video.\

      It sounds like the specific POP the google DNS server is being fed is overloaded with traffic. It should be fairly easy for Apple to resolve the problem on their end, by simply not resolving to overloaded pops (they shouldn't ever anyway).

      Other video cdn backed services (like netflix) d

      • by wagnerrp (1305589)

        It sounds like the specific POP the google DNS server is being fed is overloaded with traffic.

        That sounds exactly what was surmised in the summary.

        When you use Google or Open DNS to resolve the Apple domain name, all the requests to Akamai appear to be coming from the same location and they're all directed to the same server pool, overloading that pool and causing the slow downloads.

    • I'm not familiar with exactly what the options and price sheet are from Akamai, they aren't nearly as 'consumery' about it as Amazon EC2 is, more of a 'our rep will call you' sort of thing; but one wonders if Apple(who serves huge amounts of audio and video at relatively low prices, and presumably fairly low margins, is cheaping out in a way that Adobe isn't...

      Presumably, Akamai uses their geolocation trickery because local deliveries are faster and cheaper. No need to traverse numerous hops, possibly co
    • So here's an interesting element to this: corporate clients of hosted SAAS products which use CDNs for delivery. If I'm in the Munich office of company X which uses hosted software from company Y, and if my local DNS server is completely subordinate to the central nameservers in X's NYC datacenter (don't laugh; I've seen it happen), then chances are when I'm accessing Y's software via Akamai, I'm going across the pond to get to an Akamai POP in NYC, then going over the Akamai network to access Ysoft. Need
  • This is a very widespread practice now. Use your own ISP for DNS.
    • by jaymz666 (34050)

      yeah, cause comcast and at&t never have DNS outages... Last month Comcast had a huge DNS outage, I didn't even notice it since I have been using openDNS for years. My MIL called me up saying her internet was down, I had her ping some IPs and they worked, but DNS didn't. Changed her over to opendns and it worked fine after that.

      • by MachDelta (704883) on Friday December 31, 2010 @01:08AM (#34719018)

        MIL - I realized after a few seconds that probably stands for "Mother-In-Law", but the mechanic in me instantly interpreted it as "Malfunction Indicator Lamp."

        Shortly after that I had a chuckle upon realizing that they're both things no one likes to see.

    • by shentino (1139071) on Friday December 31, 2010 @12:34AM (#34718800)

      Only if I trust them not to fuck with it.

    • Too bad modern OSes only have a spot for a single DNS server. Otherwise you could add multiple.

      Add Multiple.
      Drop Timeout Time.
      Enjoy.

      If Comcast goes down, I'll fail over to Verizon/Google. If Comcast is up it knows my location.

      • by omglolbah (731566)

        Um, I know it is early (6am here for me...) but which modern OS only supports one server?...

        Win7 supports a long list if you so desire, as does linux...

        Or did you typo? :p

        • Um, I know it is early (6am here for me...) but which modern OS only supports one server?...

          Grab your morning coffee and fire up the sarcasm detector.

        • by jaymz666 (34050)

          if you're on a home network chances are pretty high your router is acting as your DNS server or setting the DNS servers for your machine via DHCP.

          Maintaining DNS primaries, secondaries, tertiaries etc. on more than one or two machines can be a bit of a PITA too.

      • by jaymz666 (34050)

        yeah, so using the DNS servers from comcast that are provided by DHCP never change for you, huh. Troubleshooting that mess is a pita.

    • Re: (Score:2, Troll)

      Umm no, I think I'll just pass on those services if they are that daft, thanks.

      Fuck akamai... if any software delivery system or service is slow for me because of content distribution tomfoolery, I simply won't use it. I would never have anything to do with iAnything in the first place, though.

      Most ISP's DNS servers suck... and the whole reason I started using OpenDNS is because the ISP's were slow to respond, and the primary was often out and there were delays until the resolvers queried the secondary.

      Hell

    • Use your own ISP for DNS.

      When you first get a Comcast account, before you've registered your modem's MAC address with them, they give you an IP address but the DNS server they give you always points you at their registration server. Trouble is, the database that the DNS server reads out of can sometimes get out of sync with what modems are actually registered, and there's nothing Comcast's first- or second-level techs can do about it other than to tell you how to set your DNS servers manually to something else (they'll give you th

    • by Z00L00K (682162) on Friday December 31, 2010 @01:58AM (#34719298) Homepage

      I already do, and since my ISP censors the internet through their DNS there is no alternative to go back to them.

      And a cleaned up version of my config. It doesn't involve the ISP at all but queries the root servers on the net instead.

      And as long as the ISP:s doesn't filter the DNS requests to the root servers this is the way to go right now.


      options {
                      allow-query {
                                      127.0.0.1;
                                      192.168.0.0/16;
                      };
                      directory "/var/named";
                      pid-file "/var/run/named/named.pid";
                      recursion yes;
                      dnssec-validation no;
      };

      key mykey. {
                      algorithm HMAC-MD5;
                      secret "** Secretas... ***";
      };

      zone "." {
                      type hint;
                      file "root.hints";
      };

      zone "int.anon.org" {
                      type master;
                      allow-update { key mykey.;};
                      file "int.anon.org.db";
                      notify yes;
      };

      zone "1.168.192.in-addr.arpa" {
                      type master;
                      allow-update { key mykey.;};
                      file "1.168.192.db";
                      notify yes;
      };

      zone "localdomain" {
                      type master;
                      file "localhost.db";
                      notify no;
      };

      zone "0.0.127.in-addr.arpa" {
                      type master;
                      file "0.0.127.db";
      };

      zone "0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN {
                      type master;
                      file "ip6.local.db";
                      allow-update { none; };
      };

    • Why use your ISP for DNS? Chances are their servers suck, and they will insert spam links for failed resolutions to add insult to injury for their horrible service. Find a server that is 1) geographically close and 2) measurably performs well. I personally use this tool for locating a DNS server that measurably works well with my connection: http://www.grc.com/dns/benchmark.htm [grc.com] .
    • With a little effort, you can set up BIND on your own system.

    • by hazem (472289) on Friday December 31, 2010 @03:03AM (#34719584) Journal

      Use your own ISP for DNS.

      Do you have any tips for keeping your ISP from directing a "server not found" to one of their crappy ad-ridden search pages? I think that's a major reason people choose DNS servers that aren't at their ISP.

    • This is a very widespread practice now. Use your own ISP for DNS.

      I prefer using a DNS provider who doesn't serve me a Yahoo powered by Bing search page if I try going to a bad URL - unlike my "own ISP".

  • There's some good technical discussion in the Hacker's News discussion [ycombinator.com] of this issue.

  • Namebench DNS tool (Score:5, Interesting)

    by maggotbrain_777 (450700) on Friday December 31, 2010 @12:30AM (#34718772) Homepage Journal
    This afternoon, I found a tool from Google Code called namebench [google.com] which tests response times against multiple DNS servers and give recommendations based upon a number of query types. The results returned when checking the 'censorship tests' were interesting. Seems a number of sites (wikileaks, isohunt, stormfront) returned 'incorrect' results across DNS servers. I'm going to try this over the next couple of days and see if any of my browsing speeds improves.
  • by xaoslaad (590527)
    I have to ask why they are playing games with dns rather than using some kind of LB solution to direct users to the closest server(s) based on the client ip address. Is this not feasible or is it cost prohibitive; the method theyre using seems crazy to me though i fully admit to not being up to speed on high level networking design.
    • by omglolbah (731566)

      The beauty of the DNS "trick" is that a user requesting say "yadiyadi.com/media/cheez.mp4" in Norway would get one IP and a client in say Australia would get a completely different IP. This makes the whole CDN implementation a whole lot easier as you avoid the whole negotiation issue by having the domain resolve to different IPs based on the source of the request.

      This is overly simplified of course.

      It works for the vast majority of users too.

      • by xnpu (963139)

        While it's arguably "prettier" I don't see anything wrong with old school redirects though. Either using 302's or "sourceforge" style.

        • You can anycast DNS with UDP and the client never knows the difference since it's handled with BGP and IP. HTTP redirects would be much more difficult to do from a reliability and scalability standpoint. Also, when it fails, it would be much less graceful.

  • I've used our university's DNS servers as primary for over a decade, with whatever my current ISP is as secondary. I haven't had any complaints.

  • I use to setup my own DNS at home and casually use forward zones when needed. I started this when ther was that issue with redirecting non existant names.

    Sure, not every one should do this as it stress load root servers and some ISP may redirect UDP/TCP 53 to their own servers. BTW, that's still my way of using DNS.

  • M$ does it too... (Score:2, Informative)

    by alanshot (541117)

    Microsoft does this too. After scratching my head over the past several weeks trying to figure out why I cant download M$ files worth crap half the time, this appears to be why.

  • So the moral of the story here is not that Google and OpenDNS services are bad, but that Apple's iTunes QoS methods are of "questionable quality" - at best.

    How did this make Slashdot's frontpage, again? Maybe this should be filed as a bug report to Apple (do they read those?) instead.

  • by Anonymous Coward

    Seems like it would be useful to use multiple DNS servers and then choose whichever one has the fastest download and abandon the other connections.

    Do any browsers/OSs/whatever have this feature? As I understand it, the secondary DNS feature only uses the secondary server when the primary server is down.

  • by xnpu (963139) on Friday December 31, 2010 @01:41AM (#34719202)

    This applies to tons of GEO-optimized services and has been this way since day one. Really, how is this news?

  • by macraig (621737) <(mark.a.craig) (at) (gmail.com)> on Friday December 31, 2010 @03:22AM (#34719682)

    Use your own ISP's DNS servers instead or run your own resolving DNS server.

    The first suggestion is just no longer an option, for so many reasons, all of them based on lack of trustworthiness in this climate of corporate dominance and machination. I was using OpenDNS for several years, but recently I started using TreeWalk [ntcanuck.com] to host my own modest DNS server. Seems to work fine, and I don't even notice it's there.

  • by louarnkoz (805588) on Friday December 31, 2010 @03:26AM (#34719702)
    Load balancing based on the DNS resolver is so 1999! Even when it works, it works by chance, and does not test the actual speed between your PC and the potential servers. Compare that to Bit Torrent, which actually tests the speed of the downloads. You really wonder why Apple, and Akamai, would not use some kind of torrent technology!
    • Re: (Score:3, Insightful)

      by Anonymous Coward

      You really wonder why Apple, and Akamai, would not use some kind of torrent technology!

      Because BitTorrent is a free open protocol which Akamai would not be able to charge money for.

    • by thegarbz (1787294)
      Some of us have metered uploads and our net also slows to a crawl when we send a mere 75KB/s upstream even though we can download at 2MB/s. I like bittorrent as much as the next guy, but I actively disable it where possible for services which charge me money AND leech off my bandwidth which also costs me money.

Receiving a million dollars tax free will make you feel better than being flat broke and having a stomach ache. -- Dolph Sharp, "I'm O.K., You're Not So Hot"

Working...