Google preparing for full roll-out?

Google shuffles around some more DNS records et voilá:

• mail.google.com now resolves to a Google-owned IP (one of the cpt01s01 series)
• mt.google.com now resolves to a Google-owned IP (as above, so below)

Was the hosting of the Google Maps tiles on the [presumed] GGC caching box just a fig-leaf, a ploy for respectability? Since only YouTube access still seems to be directed towards those caching boxen, perhaps that traffic was a quite-substantial chunk of the original Google traffic to TENET's network, the rest of which is now easily handled by the local peering arrangement.

It might allow easier filtering or shaping for the IT/ICTS departments so inclined, but why else keep the YouTube traffic on those boxes? Google's purported 'boxenertia'?

However, seeing that GMail is now locally hosted, that means the major services are being served from local machines. Perhaps this means Google is getting ready for a full-bore load test, in preparation for a proper roll-out?

Google local peering starting (slowly)

Google now serves [almost] all traffic to TENET sites directly from either:

• peering connection, or
• some sort of GGC-like nodes [Google Maps, YouTube videos]
  

Name resolution and tracerouting from UCT hosts confirms that all Google services are being served from local [South African] servers. This has huge implications in bandwidth savings, and quality of user interaction because of lower latency for content that is already available on the local datacenter and caches.

Issues
Strangely, while www.youtube.com resolves to a locally-hosted server (64.233.179.100 currently), youtube.com (with no leading www.) resolves to a 208.65.153.xxx address, seemingly in Richmond, VA.

Clarification
The caching nodes [presumably GGC nodes] currently serve only certain services (map tiles, YouTube videos) and are generally co-located within a client network. The other Google services are served from machines located in Cape Town (based on ping times) and that are administratively within a Google-owned IP block.

Further clarification
Assume this is a "small" trial run (limited number of users on TENET, maximum goodwill from helping the poor academics) before a full roll-out. The caching nodes already save TENET up to 45 Mbps international transport! There is no local visibility to Google's ZA IP block from IS's route server.

Google 'local peering' live?

From some traceroute data [target: www.blogger.com]:

9 unknown.uni.net.za (155.232.253.254) 2.341 ms 2.055 ms 2.237 ms
10 64.233.174.40 151.205 ms 151.276 ms 151.457 ms

Note the jump in RTT as we go from the TENET router to the Google router (64.233.160.0/19 belongs to Google).

Compare to traceroute data for a less aggressively-peering company [target: www.microsoft.com]:

8 v2750-tiger-brie.uni.net.za (155.232.145.226) 2.341 ms 2.254 ms 2.337 ms

9 unknown.uni.net.za (196.32.209.25) 153.050 ms 152.938 ms 152.223 ms

Where the next hop is a TENET-controlled router overseas.

So perhaps Google has started 'peering locally', and are transparently tunneling the data overseas to allow IS [and/or other ISPs] to avoid paying for each user's international bandwidth. Of course Google are therefore paying for it, but with the standard international bandwidth pricing feedback damping the usage, they probably aren't seeing a huge explosion in bandwidth demand.

Contrast this to the local YouTube traffic caching, where having locally-available video means that videos load faster: better user experience translates to higher usage, and the demand increases. For the user this is a virtuous circle, but to the various I[C]T[S] departments it must seem more like a vicious circle.

However, for organisations like TENET where the pipe is purchased old-school (i.e. 'per Mbps' rather than 'per GB'), this will be a boon, because up to 40 Mbps [peak] is now being handed-off locally. Interestingly the upstream/outbound traffic, presumably web crawling, is still going through the Ubuntunet point in London.

Some graphs:

The London peering session going [relatively] quiet late Tuesday.

A 'local peering' session [presumably with Google] in Cape Town going live late Tuesday:

And some of the local caching traffic seems strongly correlated (via Mk. 1 eyeball-based peak matching on other graphs) to the local peering session.

The takeaway: Google is probably peering locally, but is trying to avoid flooding ISPs with bandwidth-conversion economic problems.

Google to merge YouTube AS

Peeringdb.com always yields interesting nuggets [login:guest, password:guest] if you watch which companies are performing updates.

YouTube, although a Google company, still maintain a separate AS, but it looks like that is going to change [sourced from the above-linked peering record]:

Notes: Peering will be migrating to Google (AS15169) by EOY 2009.

Many have wondered how long YouTube would peer independently of Google, and many have also speculated on the technical complexity of merging their operations to allow peering via both AS. I assume Google will start announcing routes to AS 36561 (if they haven't already), and YouTube will start pre-pending [padding] their routes to make the Google routes preferable.

YouTube is peering in two exchanges that Google seems to be missing from:

• Equinix Newark
• PAIX Dallas

One could speculate on whether Google will keep these connections active for AS 15169, as they already privately peer at Equinix Dallas and PAIX New York (amusingly this is almost the opposite Company/Location matching).

User-generated video in South Africa

There are two major upcoming events in South Africa that will, both directly and indirectly, drive uptake of services like YouTube. Those events are:

1. The upcoming South African general election
2. The 2010 FIFA World Cup

The general election
A natural candidate [excuse the pun] for election advertisements, documentaries on parties and candidates, and other related propaganda. This will be more effective if one of the mobile operators does streaming for their users, perhaps in collaboration with YouTube.

The soccer world cup
Unless FIFA does something crazy with the streaming rights, this will be a tough one to get right. The SABC have all of the broadcast rights, and will probably moan at anything unusual or technically creative. As above, partnership with one of the mobile operators will be most important. Especially considering how many people have mobiles vs computers. Of course, since many of those mobile phones are really old, perhaps some aatv-style filtering will be necessary...ahem.