Ramping up Research Traffic on SEACOM

While TENET has enjoyed high-speed interchange of traffic with GÉANT before, it has mostly been from its nodes co-located in London (e.g. local mirrors).

However, it looks like things are heating up - traffic of up to 700 Mbps between the SANReN ring in Gauteng...

and GÉANT in Europe...

Interestingly, none of the TENET graphs show which client networks this is going to, so perhaps this is just an internal test or mirror population?

Return of CINX

The ISPA has re-launched CINX (the Cape Town Internet Exchange). PR here.

Nothing on the old CINX and why it was discontinued, except for a rather oblique statement that there is now has "more than enough Internet traffic to justify an exchange for the city".

No graphs for the exchange yet, but the TENET folks are on the ball and you can view their port graphs here. 10 Mbps in Cape Town doesn't seem like much compared to the 100 Mbps of JINX (which looks suspiciously flat, although there is a 175 Mbps spike in there).

Excellent news for local providers.

End of an era

TENET now runs all institutional traffic over SEACOM:

Bandwidth usage on the SAT-3 portions has dwindled to almost nothing; the SAT-3 connectivity should terminate sometime in the next month or so.

Interesting to see some BGP traffic on the Telia backup link - perhaps routing changes to some of the institutional ASs propagating through and settling down?

Google preparing for full roll-out?

Google shuffles around some more DNS records et voilá:

• mail.google.com now resolves to a Google-owned IP (one of the cpt01s01 series)
• mt.google.com now resolves to a Google-owned IP (as above, so below)

Was the hosting of the Google Maps tiles on the [presumed] GGC caching box just a fig-leaf, a ploy for respectability? Since only YouTube access still seems to be directed towards those caching boxen, perhaps that traffic was a quite-substantial chunk of the original Google traffic to TENET's network, the rest of which is now easily handled by the local peering arrangement.

It might allow easier filtering or shaping for the IT/ICTS departments so inclined, but why else keep the YouTube traffic on those boxes? Google's purported 'boxenertia'?

However, seeing that GMail is now locally hosted, that means the major services are being served from local machines. Perhaps this means Google is getting ready for a full-bore load test, in preparation for a proper roll-out?

Mmmm mirrors

Wow.

1.2+ Gbps traffic from the mirror.ac.za front-end in London.

TENET has new transit providers

The TENET traffic graphs show NTT as a new provider:

And a 10G interface, to boot! This is likely in preparation for the arrival of SEACOM, as TENET's international bandwidth is the current bottleneck.

Traffic on Datahop has been halved, and traffic on Telias is at a quarter to a third of normal rates. So they are either being phased out or being used as backup providers.

UCT permanent upgrade to 43 Mbps

The bandwidth test was obviously successful as UCT has upgraded their connection to 43 Mbps. Graphs at the usual place [notice the jump around 12.30pm].

ICTS has issued a remarkably restrained notice. Did they not trumpet this to the whole campus because:

• it's a drop in the bucket compared to the demand?
• they don't want to stimulate further demand?
• they don't want any recognition?

Whatever the reason, we still look forward to the upcoming 1 Gbps temporary link for TSN 63 (UCT)

Wits trounces UCT

I'm not writing about sports now, but rather about Wits's recent climb to the number 1 position on the unofficial bandwidth rankings:

Their monitoring graph shows peaks above 60 Mbps, and TENET's order status page shows an upgrade of their backbone link to 57920 kbps.

UCT's monitoring page show the same old peaks, with an upcoming 43008 kbps link as a consolation prize. This is obviously what was under test over the last week or two.

And it's interesting to see a new 'five-thirds' formula mentioned. Taking UCT's international commit of 26 Mbps and multiplying by 5/3 yields about 43 Mbps. Fully 40% of traffic is expected to be local! TENET: how about showing some flow statistics to the IS Akamai clusters?

UCT's extra bandwidth

UCT seems to have gotten some extra temporary bandwidth today. Looking at the graphs we see a spike at 14h00 (GMT+2) in the traffic graphs.



Most of the increased bandwidth usage seems to have consumed international traffic and so that international portion of UCT's bandwidth allocation is finally pegged against the limit of 26 Mbps.



Which is interesting.

Because it means that until now, the international traffic has been crowded out by national traffic. Much of which will be to the Akamai cluster at IS, and the upcoming Google Global Caches (GGCs) and local servers.

So more international bandwidth isn't going to help unless the actual "last-mile" to UCT is drastically improved.

AMS-IX breaks 600 Gbps

The previous post about AMS-IX embedded the traffic graph instead of displaying a snapshot. At least I noticed that AMS-IX now pushes over 600 Gbps [peak] of traffic through the exchange!


But what's this? Number 2 exchange DE-CIX claims a 650+ Gbps peak:


Does that mean the Deutscher Commercial Internet Exchange is now #1?
Comparison of the mean throughputs (AMS-IX: 375 Gbps, DE-CIX: 299 Gbps), daily peaks (AMS-IX: 560 Gbps, DE-CIX: 460 Gbps), and number of interconnected members (AMS-IX: 311, DE-CIX: 264) shows AMS-IX still firmly in the lead. But DE-CIX is forging ahead - AMS-IX's lead doesn't seem as clear as it once was.

Science bandwidth requirements estimate

A recent e-mail from a Meraka engineer working on SANReN:

We know eVLBI would require a 1Gb all the way to Amsterdam, current projections for full SKA is 100GB international capacity, the MeerKAT project will probably require 10GB of bandwidth.

So something like the 'bandwidth glut' of Infraco would definitely be required.

It seems like the only people talking about having too many cables are those with a vested interest in having only a few [competing] cables.

Extra transit for TENET

Viewing the monitoring graphs for TENET's London node, a new transit circuit through DataHop [web?]is apparent:


Peak speeds are about 50 Mbps. Combined with the Telia peak of ~100 Mbps, this means TENET currently has ~150 Mbps peak transit.

Note the very small outbound. Mmmm, eyeballs.

ICTS vs users

For some reason there is an article praising UCT ICTS's War on Terror...er...Bandwidth:

[snip lots]
ICTS has implemented Packeteer Packetshaper
[snip lots more]

AFAIK this actually happened ages ago. So there's no real reason to trot this out now unless people are griping that GEN3 isn't doing much for them.

And there's no mention of what will happen once:

• UCT transitions to SANReN and has a 10 Gbps link to the local node, and
• TENET moves to the 10 Gbps overseas pipe due mid-2009

I don't see the Packeteer appliance scaling to multi-Gbps speeds without serious extra $$$ invested. And would it be worth doing so?

Consider this paper on web search clickstreams http://doi.acm.org/10.1145/1177080.1177110]:

The MWN [Munich Scientific Network] provides a 10Gbit/s singly-homed Internet connection to roughly 50,000 hosts at two major universities along with additional institutes; link utilization is typically only around 200–500 Mbit/s in each direction.

Now this might have been back in 2006; it doesn't really matter if the traffic demand has doubled since then. Also, since major local and international peering will then be in effect, actual transit pressure should decrease.

C'mon ICTS - plan ahead now!

Flatlining

With all of the negative feeling towards TENET and GEN3 because of the recent outages, I thought it might be quite amusing to show the reverse:



CPUT's main Bellville campus is obviously in need of more, More, MORE! bandwidth. Roll on, SANREN.

150 GB DSL caps?

From the latest DSL Prime:

My follow-up article will be unreliable sources, which I'm factchecking more carefully. For now, a few types to distrust:

• The FCC or anyone else who takes seriously a 200K definition of broadband.
• Anyone who defends a bandwidth cap below about 150 gigabytes per month, except where bandwidth costs are unusually high or the service selling for under about $20. (India, South Africa, possibly the U.K.). Ask mo [sic] for the numbers if you don't get it.

I guess for South Africa, he's specifically talking about high bandwidth costs, 'cos I don't know of any ISPs selling DSL for $20/R160 here.

GEN3 draft SLA

GEN3's draft SLA is available (and claims to have been posted on 27th February).
Interesting to see specific figures. I'm intrigued to see that 260 ms RTT 'average'.

5. Availability

The Providers commit to the following levels of service element availability:

Services provided by Neotel:99%

Services provided by Internet Solutions:

International services: 99.8%

National services: 99.95%

6. Response times (“RTT”)

The Providers commit to the following network RTT averages:

Services provided by Neotel: 150 ms

Services provided by Internet Solutions:

International services: 260 ms

National services: 35 ms (services provided out of Rosebank)

50 ms (services provided out of Cape Town)

7. Packet loss

The providers commit to confining packet loss to the following levels:

Services provided by Neotel: 1%

Services provided by Internet Solutions:

International services: 1.5%

National services: 2%

GEN2 to GEN3

Is UCT currently operating on both GEN2 and GEN3?

Compare GEN2 stats vs GEN3 stats:

The big view seems to indicate traffic starting sometime in Week 07, about Tuesday:


The uncommissioned orders page shows UCT's link to become active on 1st March, 2008. So maybe it's all testing traffic.

It's interesting to see the TENET-JINX graphs too:
Note that we're not seeing massive usage through JINX. This basically boils down to two costs as limiting factors:
1) The equivalent-line-charge
2) The ISPA category charge (small, medium, large)

Now TENET is an honorary ISPA member, so presumably doesn't have to pay the small [by comparison] R6100 pm. But there's an incredible step function in the Equivalent Line Charge going from R0 per month to R22k per month as you transgress 2 Mbps. So can we assume private peering in/near the JINX facility?

Google's local servers

I don't know how this slipped by me, but it's finally appeared in my Google Alerts:

Stafford Masie of Google SA at a MyADSL conference in November 2007 [thanks ITWeb]

“We are going to establish a point of presence in this country, which means the international bandwidth problem is solved. There will be a big announcement in a month,” Masie said at the time.

Google do host a lot of bandwidth-intensive services, and having these served locally will help a lot with latency (for the cached stuff, of course) and cheaper bandwidth, especially for consumers.

Those local-only ADSL accounts are starting to look great. Telkom's partial-compliance with the ICASA rulings on local bandwidth will help too. Perhaps using Stefano's DSL split-routing setup will become more commonplace.

Of course, a month from November 2007 was December 2007 or, charitably, January 2008. Still no word though. Google - what news?

---

Let's see, the more bandwidth-intensive Google services are:

• GMail
• Picasa and PicasaWeb and Image Search
  
• Docs (watch out Microsoft - your anti-piracy drive may backfire)
• Maps
• Google Video (and does this mean YouTube will be served locally too? What about Limelight?)
• Google Earth (mmmmmm)
• full list of all services
  

AMS-IX tops 400 Gbps

AMS-IX finally tops 400 Gbps (this happened a few days back, but it's taken me a while to get in front of a PC):


Now the daily graph does only show a 395 Gbps maximum, so this could be [partially] an aliasing issue, but I think it's safe to say that the milestone is passed.