Ramping up Research Traffic on SEACOM

While TENET has enjoyed high-speed interchange of traffic with GÉANT before, it has mostly been from its nodes co-located in London (e.g. local mirrors).

However, it looks like things are heating up - traffic of up to 700 Mbps between the SANReN ring in Gauteng...

and GÉANT in Europe...

Interestingly, none of the TENET graphs show which client networks this is going to, so perhaps this is just an internal test or mirror population?

End of an era

TENET now runs all institutional traffic over SEACOM:

Bandwidth usage on the SAT-3 portions has dwindled to almost nothing; the SAT-3 connectivity should terminate sometime in the next month or so.

Interesting to see some BGP traffic on the Telia backup link - perhaps routing changes to some of the institutional ASs propagating through and settling down?

Google tuning

Even though Google's new Cape Town nodes are in limited production / test mode, users have been connecting to both the local data-centre, as well as to overseas datacentres [e.g. London?].

Presumably Google is still tuning the DNS based on usage while simultaneously expanding the datacentre capacity.

Interestingly it is only mail.google.com that returns a non-local host. So apparently mail is not amenable to normal caching modes or it would have been done through either the [presumed] GGCs or even transparently cached within the Cape Town datacentre.

Let's hope that Google get some cheap capacity on SEACOM even though is has higher latency than SAT-3 (~215 ms vs ~150 ms). Since most of the front-end logic (e.g. client javascript) would be served by local machines, interactivity should not be compromised too much.

TENET peering at LINX

More interesting traceroute data [target www.yahoo.com]:

8 v2750-tiger-brie.uni.net.za (155.232.145.226) 2.732 ms
9 unknown.uni.net.za (196.32.209.25) 152.780 ms
10 ge-1-1-0.pat1.the.yahoo.com (195.66.224.129) 154.808 ms
11 so-0-1-0.msr1.ird.yahoo.com (66.196.65.33) 163.932 ms
12 gi-1-1.bas-b1.ird.yahoo.com (87.248.101.1) 164.020 ms
13 f1.us.www.vip.ird.yahoo.com (87.248.113.14) [open] 165.543 ms


The TENET router in London (196.32.209.25) connects to the Yahoo! AS, and the 195.66.224.0/19 router address is a giveaway that it's through LINX.

Testing other networks [target www.facebook.com] yields more data:

9 unknown.uni.net.za (196.32.209.25) 153.603 ms
10 linx.br01.lon1.tfbnw.net (195.66.225.69) 153.336 ms
11 xe-x-x-x.br01.ash1.tfbnw.net (204.15.22.245) 229.906 ms
12 te-13-0.csw06a.ash1.tfbnw.net (204.15.23.55) 233.942 ms
13 www.11.06.ash1.facebook.com (69.63.186.12) [open] 262.342 ms

where hop 10's hostname confirms the LINX connection, and also shows we're still connecting to the Virginia servers.

The LINX looking glass also shows the routing to TENET through AS1299 (Telia) and AS2914 (NTT) and directly.

Traffic graphs confirm significant traffic through LINX


Several other autonomous systems are reachable via LINX peering sessions, including hosting providers Hurricane Electric and LeaseWeb and CDN Limelight Networks [ex YouTube providers, and 2009 Presidential inauguration streamers].

Clearly this is in preparation for the new SEACOM pipe, although TENET is saving on transit requirements.

Speculations on Google's peering timing

Take the following two events:

• Google should start peering locally "soon". [Events seem to have overtaken the original plans for implementation by the end of January, so perhaps mid-February?]
• SEACOM is coming online "in June".

These might seem unremarkable events individually but when viewed together they provoke interesting questions. At the root of all of these questions has to be some speculation regarding the timing of Google's technical entry into this market (ignoring their sales operation) prompted by this observation:
Local Google peering is very close to, but before, the SEACOM launch.

Some obvious simple questions, with possible answers:

• Q: Are Google using SAT-3 or SEACOM?
  
• A: Google is likely to be using SAT-3.
• It's unlikely that SEACOM have any segments finished and ready for operations yet.
• However Google probably don't need a full path from South Africa to London; the 'express path' to Kenya would help operations both in Kenya and in South Africa, so any costs involved in bringing content to Kenya or South Africa could be amortized over both operations.
• Google would also like some redundancy in their paths, especially considering the Egypt-to-France portion will be the responsibility of Egypt Telecom.
  
  
• Q: Does Google have a big pipe?
  
• A: Probably not.
• 10 Gbps IRUs are available on SEACOM, but SAT-3's total capacity is 120 Gbps. Even 1 Gbps is a large chunk of the available capacity.
• Pricing on SAT-3 circuits is also very high, although it'll be under pressure because of the imminent SEACOM launch.
• However, if Google have negotiated a long-term deal with Telkom, they may have secured better pricing.
• Presuming they buy a circuit on SEACOM, one assumes it will be a 10 Gbps IRU.
  
  
• Q: Why hasn't Google peered in South Africa before this?
• A: It was too expensive for the possible benefit to Google.
• ISPs must secure bandwidth to provide service to their customers. Those customers want connection to Google, so ISPs wanting to keep customers happy bought the bandwidth.
• Google started local [service / sales] operations a while back (witness the recent Yellow Pages / Entelligence / Google spat) and are ramping up their presence in the country. Further usage of YouTube, Google Earth & Google Maps, GMail and other bandwidth-intensive services would help drive traffic (and hence advertising earnings), and usage of Google Docs would help steal marketshare from Microsoft in a developing market that Microsoft itself recognises is vulnerable (hence Microsoft's pre-paid Office offering).
• Google can afford to run the service at a high marginal cost for a while, until SEACOM comes online.
• Also, it seems the development of the GGC has only recently come to fruition.
  
  
• Q: Why hasn't Google waited until June (e.g. for better pricing)?
• A: Competitive advantage.
• Come June, every Tom, Dick, Harry, Yahoo! and Microsoft will be presumably be buying large circuits to tie in local datacenters or extend peering.
• Moving first allows better publicity (even via word of mouth as opposed to above-the-line advertising) and can be used by their sales team as a competitive advantage.
  

Anything else?

ICTS vs users

For some reason there is an article praising UCT ICTS's War on Terror...er...Bandwidth:

[snip lots]
ICTS has implemented Packeteer Packetshaper
[snip lots more]

AFAIK this actually happened ages ago. So there's no real reason to trot this out now unless people are griping that GEN3 isn't doing much for them.

And there's no mention of what will happen once:

• UCT transitions to SANReN and has a 10 Gbps link to the local node, and
• TENET moves to the 10 Gbps overseas pipe due mid-2009

I don't see the Packeteer appliance scaling to multi-Gbps speeds without serious extra $$$ invested. And would it be worth doing so?

Consider this paper on web search clickstreams http://doi.acm.org/10.1145/1177080.1177110]:

The MWN [Munich Scientific Network] provides a 10Gbit/s singly-homed Internet connection to roughly 50,000 hosts at two major universities along with additional institutes; link utilization is typically only around 200–500 Mbit/s in each direction.

Now this might have been back in 2006; it doesn't really matter if the traffic demand has doubled since then. Also, since major local and international peering will then be in effect, actual transit pressure should decrease.

C'mon ICTS - plan ahead now!

Developments on GEN3 and SANREN

According to the latest Rhodes IT minutes, they expect to increase their bandwidth radically:
4.5 Mbps of international bandwidth x a 35 fold increase in ... Internet bandwidth yields 155 Mbps - an OC3. In 2008! At least South Africa isn't too far behind the times ;)

Does this mean UCT will be getting a similar capacity, or something a little faster? Since we'll be near one of the cores of SANREN, hopefully things will be a bit better.

Also, ITWeb thinks SANREN will be a 10 Gbps network (presumably per link, not in aggregate). Note also the proximity to JINX - that should offload some of the transit requirements to direct peering. Also, the projected operational date is end 2008! (a bit earlier than previously thought, and also overlapping with GEN3's mandate with TENET).

Also also, the Infraco weirdness with cabling up the west coast to Europe and South America should matter a little less with the redundant path on SEACOM that TENET has purchased the rights to use.

So, perhaps academic and high speed networking in South Africa has a bright future after all?

Government envelopes (v)

South African Communications Minister Ivy Matsepe-Casaburri told FMTech that she is hopeful that InfraCo and SEACOM will be incorporated into the new Nepad cable project.
[thanks FMTech]

It's amusing (and simultaneously heartbreaking) to see the Department of Communications making their bid to control all of South Africa's external communication networks.

However, the framing of the statement indicates that Neotel has the go-ahead with SEACOM, else why would they want this project folded into the NBIN?