Tuesday, October 16. 2007The Demands Of 'Legitimate' P2PJohn Linton Exetel recently signed an agreement with a P2P caching provider and last week we received the large number of caching servers and switches needed to implement this service. Over the next 4 - 5 weeks we will implement this new service on a trial basis and, if the trial's successful, we will put the service into production in early 2008. The decision to implement P2P caching was made almost a year after we implemented P2P 'filtering' to better balance the usage of our rapidly increasing ingress/egress and customer connectivity bandwidth. The results of our installation of the P2P 'filtering' technology was highly effective and met all of our objectives of moving approximately 400 mbps of peak bandwidth from the 8pm to 12 midnight time (which is the most heavily used period of each day) to 12 midnight to 8 am time (which is the most lightly used period of each day). This resulted in a monthly saving of expenditure on bandwidth of almost $100,000 a month and improved overall bandwidth usage efficincy by almost 15%. This was a really good financial result for Exetel (and without it I think we would have struggled to remain in business) but it was achieved at a cost of seriously annoying some customers who felt inconvenienced by their P2P downloads taking a longer time to complete. That in itself was not a total negative in that the, relatively, small number of customers who moved away from Exetel included almost all of the heaviest 'free time' users. As was obviously going to happen (whether those users who left us realised it or not) was that EVERY ISP in Australia who could afford the cost of a P2P filtering solution would do what Exetel did - our openness in telling our customers in advance about what we were doing only accelerated this event (I must ask our provider of P2P filtering solutions for commission on the 9 sales they made in Australia immediately following Exetel's successful implementation of their products). However, the P2P filtering 'solution' was only ever regarded as a first step in a three year plan to deal with the very significant increases in peak time bandwidth usage that had been escalating for the past two years and showed every sign of increasing. One of the most notable indications has been the 'jump' in the utilisation of the peering point connectivity between Exetel and PIPE - this has reached 350 mbps at its peak compared with less than 50 mbps two years ago. The most obvious reason for this rapid, and continuing, growth is that P2P provides download 'targets' that are increasingly based locally rather than internationally. The other reason was that Akamai established their Sydney download point in the PIPE data centre and Akamai provides software distribution for Microsoft and many other major software houses from a Sydney server rather than from a US based server. In the latest edition of New Scientist there is an article that, in part, points very strongly to the likelihood of companies such as Microsoft using P2P in the future to distribute their games programs and the updates to those programs: http://technology.newscientist.com/article/mg19626256.700-p2p-growth-creates-battle-for-bandwidth.html I also read an article last week in which the most popular of the P2P software developers, BitTorrent, was now actively pushing its products to Microsoft and other major on line distribution users. While we are not super-bright in looking in to the future, we saw (probably along with everyone else who has an interest in managing bandwidth resources) that this would almost certainly happen. The Akamai success around the world demonstrates that companies like Microsoft and Adobe and many other producers of software are prepared to pay big money to dedicated distribution companies. (just as, for instance, modem and other hardware companies outsource the delivery of their products to end users via fulfilment companies). However Microsoft et alia didn't get to be big companies by not controlling costs and making savings where they could. So......using some form of P2P is a very logical way, if the financial side of this methodology can be worked out - not very difficult, of distributing software titles and software updates including the two big items - operating systems and games. If this eventuates, and that appears to be a highly likely scenario, then a very high percentage of legitimate HTTP traffic will become legitimate P2P traffic used by commercial organisations as well as by individuals. I have no idea how to calculate what the percentage of file downloads via P2P will become once/if this happens but my guess would be well over 80%. We decided that Exetel would need a more effective way of dealing with a constantly increasing usage of P2P as a percentage of total traffic and, after almost a year of looking at the various options available to an ISP of our size we selected a P2P caching solution that, if it's successful, will increase the speed of delivery of P2P files and reduce the cost of delivery to Exetel. It will, if the BitTorrent press releases can be actually implemented to the complete satisfaction of companies such as Microsoft, Sony, Warner Bros etc, allow the signature detection of 'legitimate' P2P traffic. In other words the P2P traffic will carry an ability to tell the equipment used for identifying P2P traffic that the owner of the copyright of the data has authorised its distibution via P2P processes. All that, of course is in the future......Exetel's principal reason for starting this trial is to determine whether P2P caching will significantly speed the delivery of downloads to an end user and if that increased speed can be delivered at a lower cost to Exetel. Just one more complication to delivering what, in "the old days", was a pretty simple service. Monday, October 15. 2007"Naked" ADSL 2 (Where is Frank Drebbin When You Need Him?)John Linton With the already heavy use of mobile telephones with 'free' calls in off peak periods and the slowly growing use of VoIP by the more technically adept DSL users the issue of "why do I have to pay for a telephone line rental" is becoming more insistent. I read this, brief, article in the SMH this morning: http://www.smh.com.au/news/wireless--broadband/the-bare-facts-about-naked-dsl/2007/10/12/1191696156893.html which indicates that ISPs such as iiNet are getting quite close to offering an ADSL service over a standard telephone line without the end user being required to also pay a rental for the telephone line. iiNEt is quoted as providing a "naked" ADSL service at $44.95 per month including 4 gb of downloads which is a very attractive offer compared to what's available today. The article references other ISPs doing something similar early in 2008. There is little doubt that no competent user needs a telephone line to make conventional calls and either their mobile or a VoIP service is both lower cost and more convenient with many more features. For those users the perceived payment of $20 to $30+ to rent a service they have no need for is becoming more than irritating. Exetel has the opportunity of offering a "naked" ADSL2 service via either it's current two ADSL2 providers (Powertel and Optus). This could be done, literally, today but the cost structures currently being offered make it very difficult to structure a plan that would be attractive. Doubtless the current pricing will change over time, but what that time might turn out to be is not possible to predict. It seems to be one more complication in the assessment process of determining what future, if any, a 'land line' (as opposed to wireless) has in providing data services to end users in Australia. Right now the reduction in the land line price (for ADSL only) would seem to be a logical and obvious reaction to the 'threat' of wireless. This would undoubtedly be true if there were multiple providers of land lines who would naturally react to a technology threat to their revenue structures. But as there is only Telstra (who has forgiven itself all operating inefficiences and dealt with competitive pricing of other services over the past 20 years by raising the base telephone line rental from around $A10 a month to $A30+ a month) a reduction in line rental to its wholesale customers has taken a long time and wasn't done to combat wireless competition it was done by edict from the regulator. What Telstra now does to protect the inflated (and one suspects unsustainable) prices it charges its own huge base of retail users will be very interesting to see. As with every delay perpetrated by Telstra since 'de-regulation' was wished on it getting on for 20 years ago there will be no change for a long while - it wouldn't be financially possible unless there is something really clever that Telstra has cooked up over the twenty or so years this inevitable erosion began and therefore, presumably, so did the Telstra planning of what to do about it. No matter which way Telstra moves the financial downsides look insuperable. However, as I referenced yesterday, the most likely scenario is the bundling/price capping of ADSL, telephone line rental, call costs and, maybe mobile call charges plus the new wireless roaming abilities in to one 'super' communications bundle. A carrier that owns a wire line, mobile and wirelss network (and ones larger and more comprehensive than all of its competitors) would easily be able to do that. Sometimes I wonder how I ever got involved in this business. Sunday, October 14. 2007Sunday Paper DSL DelugeJohn Linton Two things struck me on seeing this ad: 1) Optus marketing must believe there is a market for people to spend $99.00 a month on broadband 2) Optus must have very low internal costs to make this offer work for themselves Exetel buys ADSL2 services from Optus but pays around $50.00 for the combined ADSL and telephone monthly rental plus around 60 cents per gb of traffic to the customer. On top of that Exetel has to pay aproximately 80 cents for traffic to and from the internet and our cost of a wireless router plus delivery would be around $60.00 - once only. We don't have anything like the number of telephone users that Optus would have to arrive at an accurate assessment of estimated call spend once there is no charge for the majority of calls but the average cost to us, for Optus telephone customers, is around $16.00 per user per month. So assuming a 24 month contract to recover the cost of the modem we would cost this offer as: modem recovery - $2.50 ($60.00 divided by 24) port/line cost- $48.00 Optus traffic charge - $12.00 (20gb x 60 cents) IP traffic cost - $16.00 (20 gb x 80 cents) Call costs - $16.00 (estimated on current charged spends) This would give us a cost of $94.50 plus GST = $103.90. Of course - maybe the average downoads would be less than 20 gb and that would make it margiinally profitable for Exetel. We think Exetel is a very efficient company, far more efficient than Optus, so we would have a lower 'on cost' than Optus and we certainly don't spend millions a month on advertising and those differences would, quite possibly, close the gap between our wholesale costs from Optus and their own internal costings to themselves. Last time I looked at our customer by spend analysis we would have had less than 5% of our total users spending more than $100.00 a month on broadband and telephony. It seems that Optus believes there is a market for such a product combination and, I assume, this is based on what they have achieved with their lower cost/lower download FUSION plans. I'm tempted to try out such a combination at prices lower than Optus is offering and offering with ALL calls bundled (not exclude mobile and international calls) - by using VoIP to reduce the call costs and using the current Optus Wholesale offer of zero install cost to reduce the 'pain' of the zero activation/no cost modem freebie. Perhaps also with a 12 month contract instead of Optus' 24 month contract as, from what I've seen, 24 months is a turn off right now. One of the reasons for experimenting with such an offer is that the new P2P caching servers. switches and routers arrived yesterday and that project is on time which, if the tests are successful, will reduce the IP bandwidth costs by around 40%. The attraction is to increase our VoIP minutes and to also see if we can increase the percentage of Exetel customers who spend more than $80.00 per month. Saturday, October 13. 2007Why Do So Many People Pay So Much For ADSL?John Linton We need some painting done on the outside of our house and have got a couple of quotes from recommendations from friends. Early this morning (before he drives to Newcastle to work on a "big job") 'Terry' dropped in to give us a quote. He, and his team, have done work for us before on three occasions and we have recommended him to several of our friends. He always turns up when he says he will, always completes the work faultlessly and always does the job quickly working long hours each day. He did his usual careful inspection, ensured he knew exactly what sort of paint was required and then gave us a price that was less than the other lowest quote by 80% (lower by 80% - not 80% of). He will do the job for us next weekend. That domestic situation put in to perspective this report commissioned by the Australian Internet Industry Association I read over breakfast: http://www.iia.net.au/images/stories/here%201.pdf It's a long report couched in a lot of words in the way expensive consultants always coat the sparse information they provide but if you skip to Figure 3 in Section 3 you will see a table of plans from Telstra, Optus, Primus and iiNet (also Unwired but that information isn't relevant to the thrust of this rant). Now I know that large companies can charge far more than small companies (due to their shareholder's money they spend on advertising and freebies) but that table shows prices and download allowances that are very, very uncompetitive with the rates charged by, effectively, every other ISP of any longevity in Australia. Why does any potential buyer of an internet service select plans from those four companies when they are so plainly over priced? Somewhere towards the end of the report, if you bother to read that far, you will find this 'gem' of an explanation of how they calcualte likely spends based on these plans: "For example, the monthly cost to a ‘Low’ user of a 0.2GB 12 month plan with a monthly subscription charge of $29.95, an initial charge of $189 for a modem and an excess data usage fee of $150 per GB would be: $29.95 + ($189/12) + ((0.5-0.2)*$150) = $90.70" This seems to imply that the research done indicates that users of these low download included plans actually incur the draconian excess charges levied by companies such as BigPond - if that's true then it appears that the majority of Australian ADSL users are wasting a huge amount of money each month on a slow ADSL service for which they are being massively overcharged. Must be time for Exetel to re-look at our pricing. (only joking). If you want the name and contact details of a really low cost, but very high quality, painter - let me know. If you want some advice on what ADSL service to select and don't think you know enough - ask a more knowledgeable friend. Friday, October 12. 2007Taking The Plunge?John Linton I'm not sure whether the timing is right, but then I doubt that anyone can ever be sure about timings, but we have pretty much decided that the time has come to recognise that we can't continue to just buy wholesale services from major carriers and 'add value' to those base infrastructures beyond the near future. The costs of building any sort of real communications infrastructure are terrifyingly large but I have been encouraged by recent enquiries into equity funding, leasing and a visit from our bank yesterday to be reasonably confident that the financial resources needed for a modest start would be available and affordable. To date, Exetel has been funded by Exetel's owners and, including a sizable bank guarantee to Telstra, those investments are approaching $A2 million which is more than we had planned for and we would be reluctant to go beyond the current levels so, if it turns out to be necessary, it's possible that we could obtain up to another $A3 - 5 million of capital if we needed to. Our current view is that we will base any 'Exetel' infrastructure on wireless services to business users in specific areas where we can see a likely acceptance and a, relatively, 'safe' investment return. The geographic areas we have in mind for the 'pilots' are: North Sydney Orange Bathurst and we would only provide business services (data, internet and VoIP). The reasons for selecting Orange and Bathurst are the lack of competition (only Telstra and Soul offer business solutions there) and our long term and fairly detailed knowledge of business and government users in those two cities and we can obtain relatively cost effective backhaul. We selected North Sydney because from the roof of our building we have line of sight to a very high percentage of buildings in North Sydney, Milsons Point and the North side of the CBD and we have direct fibre connectivity from our offices to our Sydney PoP. The capital costs of providing wireless coverage in these three areas are, relatively, modest while the number of potential users is quite substantial. So we applied for a wireless carrier license yesterday (and I seem to remember I said only recently that Exetel had no plans to obtain a carrier license (dynamic business this communications) and we will select the base station and end user equipment from the short listed providers by the end of October with the aim of having the service available in February 2008. We already have a substantial number of business users in the North Sydney and the Northern Sydney CBD area which provides us with an immediate 'market' for the services and, at the prices we believe our own infrastructure would allow us to offer we would expect the intake of new business, which is currently quite reasonable, to significantly increase. In the two regional cities we would work with one of our current agents in each city to market the services and, depending on the circumstances, could enter in to some formal joint ownership arrangement if that was seen as being beneficial - but our plans are not based on that and we will go ahead in those two cities ourselves if there is no mutual benefit in a joint venture. The reasons for taking this approach are pretty simple: 1) The pilot expenditure is available from our current retained earnings 2) The cost of adding base stations can be funded from either leasing or external capital injection if necessary 3) The chosen approach has a narrow focus in sensibly lucrative markets that have little competition 4) We have a ready source of 'prospective users' in Sydney, and later in Melbourne and Brisbane 5) We have an excellent 'converged product' business offering 6) It is the first piece of the 'jigsaw' in developing a viable Exetel owned comms network that 'fits' the next two steps There are, of course, a huge number of dependencies and decisions before we could get this project to a sensible pay off point but, in my view, Exetel can't continue to, solely, provide services based on other companies backbones beyond the next 12 - 18 months. Thursday, October 11. 2007The Challenges Of Wireless BroadbandJohn Linton Exetel continues to try and put in place a wireless broadband offering that will provide a wider coverage and lower cost of entry than the current Unwired service we have been offering for three years now. As with any investigation you tend to cast your net pretty wide in searching for information but I was taken aback by today's iTWire article that dropped in to my inbox this morning: http://www.itwire.com/content/view/14797/127/ While quoting a third party (in this case a spokesperson for GoTalk) isn't definitive/100% factual, it seems astounding to me that he appeared to say that there are " between 40,000 and 50,000 unused iBurst modems in Australia at the moment". If that were true then it seems that some people have spent over $A10 million on modems for a service that they don't use. Perhaps he meant 4,000 to 5,000 or was misquoted by the journalist? I have no facts but I would have thought that it was highly unlikely that iBurst would have sold 40,000 modems in total since it began offering the service - however let me repeat - I have no facts other than the published figures from Unwired in their successive annual and half yearly reports that put their total connected customer base (and those figures are unlikely to be 'under-stated') at around 70,000 after 4 years of intensive advertising and promotion. I would think it unlikely that iBurst has sold more modems than Unwired but have never seen any published figures for iBurst. Whatever the actual figure is, it seems very odd that iBurst would 'publish' it. From our own experiences of buying back and re-supplying Navini modems for the Unwired service and the eBay method of on selling unwanted modems it seems unlikely that there would be very many wireless modems not in use. If there were 40,000 to 50,000 (and I can't believe the quoted figures were correct) it would seem to point to a significant problem with the service itself - why else would someone pay $200 - $300 for a modem and then stop using it? I hope someone from iBurst contacts iTWire and gets a correction made. Exetel personnel have been doing some tests of the iBurst service around the country over the past few weeks with the pedictable mixed results (wireless coverage is always going to be highly variable at these early stages of network deployment). Our, not definitive testing, seems to indicate that where there is coverage the service is acceptable for the 'mobile' user but wouldn't suit a 'fixed' user - neither the speed nor the likely plan download allowance would meet such a user's requirements - in my opinion. The best wireless service we have tested to date (in terms of both speed, sustained connectivity and coverage) has been the Vodafone 3G service but, at least at the moment, the cost of data on that service is very, very expensive. The "3' service was fast and far more affordable but coverage was far more limited than the Vodafone service but better than either the Unwired or iBurst services. It seems that the only sensible wireless broadband avaialble in Australia right now is a "3" service bought directly from "3" providing your needs are met by their coverage areas. We will continue to consider all available options and have two more areas of opportunity to explore over the next 6 - 8 weeks but my view, at least right now, is that wireless broadband is a premium priced service for a relatively small number of user types. I hope to be proved wrong in the not too distant future and that the WiMAx future is a lot closer than it appears to be from what I've currently found out. In the meantime we continue to make some progress with a 'business' WiMax solution and, depending on the availability of internal resources we might activate a pioneer service early in the new calendar year - either on our own or in 'partnership' with another company. Wednesday, October 10. 2007Reducing Operating Costs By 50%John Linton I have to admit though that most of the possible reductions all come with 'strings' and some come with considerable capital investment that won't please someone involved in such decisions who tends to think that after 4+ years no more investment money should be required to grow Exetel but such investment should come from retained profits. Nice thought. I can't take much credit for many of the planned reductions as they come 'naturally' from our reaching the end of 2 or 3 year contract terms and now have greater volumes and a good track record with the various providers that almost automatically give us lower pricing. But it does make an amazing difference to the monthly P and L each time even a relatively small recurrent expenditure is reduced by 50%. We re-signed the first of the supply contracts today for a 50% lower cost than we had paid each month for the past three years which was a small, but significant, indication that an 'across the board' cost reduction process will produce very real results. Producing the same result with Telstra will definitely be a challenge. However, like all things that are considered to be impossible at first, or even second or third, consideration - all that's really needed to be done is to look beyond the road block and see if there's a completely different solution. The different solution in the case of Telstra ADSL1 is not to 'negotiate' with Telstra for a better price (and a 50% reduction on current tail costs and GigE connections seems highly unlikely to me at the moment) but to 'turn 180 degrees' and look at the issue completely differently. So if Telstra won't reduce their costs by 50% then the next alternative is to get 100% more revenue from the same cost. In the case of ADSL1 the suggestion was to use a wireless router to share a service with a neighbour as just an example of the concept of 180 degree thinking. Sure there's all sorts of issues with such a concept and that particular scenario may never work but it does start the process of not believing reducing the ADSL1 tail and connectivity costs by 50% is impossible - it can now be seen as quite possible. A different example, again with tail costs but this time with the tail costs for SHDSL services we pay to Optus and Powertel, is that there is, in my opinion, less than no chance of getting those two companies to reduce their current prices by 50%. However Exetel has a growing number of business SHDSL/Ethernet users and plans to treble those numbers in the next twelve months. The tail cost is already over $100,000 a month and is planned to grow to over $300,000 a month by the end of 2008 using the current services at the current costs. So we met with a wireless base station provider this morning to discuss how we could implement wireless Ethernet in the Sydney CBD and North Sydney where the majority of our business customers are at the moment. If all the difficulties of doing it can be resolved (and they dont, to me a non-technical person, look too difficult) then we could reduce an SHDSL/Ethernet monthly tail cost not by 50% but by 90%. I've previously referenced the ways of reducing ingress/egress bandwidth costs and the more progress we make with those current and possible suppliers the more likely it looks that the 50% savings on those costs can be made. Similarly with lesser costs such as premises and communications we have been able to hold those planned future costs at the current levels while the revenue and personnel double so they too will result in a 50% reduction in future months. So, I'm not sure whether to be really pleased that such a difficult project is turning out to be possible or to be seriously annoyed that I didn't start this process three years ago. Maybe we need a financial controller? Tuesday, October 9. 2007What A Sheltered Life I Have LivedJohn Linton As part of the investigation process of putting in place an Exetel calling card service/VoIP without VoIP equipment service I have been looking at the range of amazingly low cost calling card and VoIP options available to Australian users. I would be the first to admit that my knowledge of calling card service delivery costs and VoIP costs achieved by other VoIP providers is very basic. So I did what any sensible person would do - I started talking with people who had far more knowledge than I did and who were willing to share their knowledge. To say that I was surprised by many of the statements I heard would be an understatement. I had lunch late last week with a business acquaintance who has a good knowledge, based on long experience in the calling card business, of how to buy voice minutes and how to package calling card offerings to make them both attractive to the purchaser (versus the plethora of other cards on the market) and priced in ways that enabled a profit, albeit very small, to be made. His advice and knowledge was comprehensive and freely given and his analysis of the top 5 companies in the calling card business was precise and pointed. I was extremely grateful for his detailed picking apart of the various cards I had purchased to get a better understanding of how they actually worked and what sort of quality they provided. I was astonished at, what I perceived to be, the criminal/fraudulent ways that some of the cards actually operated to deliver their unbelievably low rates. I understood that most companies that provide calling cards build in to their cost structures the benefits of things like: 1) Expiry dates occurring before all minutes are used 2) Tourists seldom using up a card they buy before leaving the country 3) A percentage of cards getting damaged/lost before they are used up All reasonable assumptions and backed by experience of many years in determining those percentages. What I, in my naivety, had no idea about was that several of the low cost cards I had purchased to test achieved their unbelievably low 'face value' call charges because: 1) The 'billing system' used by the provider used a "loading" of between 50% and 300% on the minute counter (ie. 1 actual minute could be counted as three minutes) 2) The face value of the card might say $20.00 but the actual value of calls programmed in to the 'billing system' might be as low as $15.00 3) The card didn't spell out a connection charge but contained a statement in almost unreadable print that "some destinations may incur a connection charge" which actually meant that ALL destinations incurred a connection charge and that connection charge could be as high as 65 cents per call - on one card it was $1.00. There were several other even more scurrilous comments made but I'm not going to write them here. I had trouble believing that any company that operated in Australia could get away with such practices so I tested a couple of the statements by dialling the 'speaking clock' in three different countries which were at very low rates on three different $10.00 cards. Sure enough - my $10.00 cards that should have given me between 200 and 300 minutes behaved like this: 1) All three cards didn't connect for more than the time it took to incur the connection fee the first time I used each one. 2) I got 67 minutes instead of 280 minutes on the first card, 108 minutes instead of 200 minutes on the second card and 115 minutes instead of 240 minutes on the third card before the call dropped out and when I re-dialled I got a 'credit is used up' message. I subsequently tested the remaining cards for call quality and found: 1) 30% of the cards I tested failed to connect for at least three trys before getting through to the number 2) The call quality was at best difficult and at worst required a re-dial on all cards 3) Over a period of two hours in the late afternoon it took up to 8 re-dials to actually get a line to call out on (each time involving a local call charge to me) In marked contrast to these results were the results achieved on calling cards that had very low rates but, from what I knew of carrier call charges, would have made a profit on each destination. Each of these cards from the three biggest providers to the Australian market had near perfect, or perfect, quality and my 'speaking clock' test showed they all had the advertised minutes. However all but two of them did have a call connection fee that I eventually found by calling the service desks. An interesting exercise and, if my experience was a true indication of what the really amazingly cheap calling cards provide, I fail to understand how some regulatory authority or TV 'shame and blame' outfit hasn't done something to bring this to more people's attention. My test buyer went back to three of the convenience stores to complain that he'd been ripped off and was told "none of my business/never heard that before/nothing I can do" - as was inevitably going to be the case. When then asked for a recommendation for a better card, each time one of the more expensively priced supplier's card was offered. I must get out more. Monday, October 8. 2007Pricing Of IP (Connectivity) BandwidthJohn Linton There are relatively few providers of IP connectivity bandwidth and an even smaller number of actual 'cables' connecting Australia to the rest of the world. So when it comes to 'negotiating' with the relatively few providers there are the usual difficulties of the 'cartel' type pricing situations. Peering Points (such as those set up in WA and Victoria and the commercial ones set up by PIPE in every mainland capital city) have enabled the cost of delivering traffic from Australian locations (particularly those in the same city as the end user) to be delivered very, very inexpensively. Over the past ten years the amount of this traffic, as a proportion of total traffic, has slowly, but significantly, increased which has delivered some significant cost savings to small and medium, or even large, ISPs. This increase has been driven by many things but three, in my opinion, have made the most difference. These are: 1) The wide use of P2P in Australia where downloads are sourced from other Australian users rather than international web sites 2) The ever increasing number of Australian based games sites (scaled back now by the popularity of games such as WoW) 3) The delivery of major software 'titles' from peer points set up in Australia like the Akamai service in Sydney hosted by PIPE. Almost 20% of Exetel's total connectivity traffic is now sourced, at peak times, from PIPE with the peak last week reaching over 350 mbps out of a total theoretical bandwidth provisioning of 1.75 mbps. This has increased from less than 3% when we first put in a PIPE connection some two years ago. The average cost, at the moment, of international connectivity bandwidth is around $240 per mbps compared to the PIPE cost (to Exetel) of around $10 per mbps. A very significant cost reduction contribution. However the very major portion of the PIPE traffic is P2P with legitimate software downloads from Akamai and games traffic being less than 25% of the total derived from PIPE. Exetel has decided to trial a P2P cache solution which will begin sometime in November with the objective of providing a further 400 mbps of locally sourced data during peak times. This has, of course, yet to be proven and it may well not succeed. If it does the cost of that '400 mbs' will be around $100 per mbps; far more costly than PIPE but very significantly lower cost than the bandwidth we currently buy from Optus and Verizon - and even lower cost than the likely cost of bandwidth purchased directly from Southern Cross - in the event that proves feasible. It's an interesting set of difficult to conjugate scenarios as we begin the boringly predictable, but nevertheless long drawn out irrespective of the predictability of the result, processes of negotiating for extentions of current IP bandwidth contracts or seek to put in place new IP contracts with different providers. However the PIPE experience, the experience we now have with the processes of restricting P2P at various times and the as yet unknown results of the P2P caching trials means that it is no longer sufficient to seek the lowest price of IP connectivity (and ensure its quality) as the only method of delivering an internet service. It's far more complicated. The overall goal remains to reduce the cost of IP connectivity to around $120.00 per mbps which is going to take some doing but is going to be essential for the planned growth of the business and, if you want to look at the most pessimistic downside - simple survival as a commercial entity. Sunday, October 7. 2007Where Would Exetel Be Without It's Forum?John Linton Saturday, October 6. 2007Making VoIP 'Universal' - Used By Every Exetel CustomerJohn Linton One of the tasks I set myself in developing Exetel's planning for the next 4 - 5 years was to increase the VoIP minutes we deliver to our customers by a factor of ten before June 30th 2008. Currently Exetel switches around 3 million VoIP minutes each month with around 2,000 of Exetel's DSL users actively using the service. We have taken our time in refining our VoIP service and ensuring all of the back end systems and processes are in place and that our VoIP switch and the associated PRIs cope with the growing volumes and, much more importantly, the peak demands. Our VoIP switch has been in operation for almost 12 months now and we provided VoIP via a third party for around 12 months prior to installing our own switch so we have some reasonable understanding of what is needed to increase the usage of this very cost effective service. There are, of course, many reasons why less than 5% of our current broadband users also use our VoIP service but, in my opinion, the major reason is that it's seen as being technicaly 'clunky' with a relatively high technical competence required and also a difficult to justify expenditure on suitable VoIP connectivity equipment. I use VoIP at home and the whole Exetel office is VoIP and I wouldn't want to be without those cost savings either in my private life and certainly not in terms of Exetel's business. The call quality is indistinguishable from "toll" services. So......simple challenge really...........show/convince the other 95%+ of Exetel's current users to also use Exetel's VoIP service. I've thought about it for a while and included a lesser target in two Exetel people's job goals. We've made some progress but not enough. So I think I know how to make it very likely that the majority of Exetel's broadband users will also use the Exetel VoIP service now. As with many things that Exetel has introduced this somewhat 'innovative' concept came out of a discussion on a different subject with someone who doesn't work for Exetel. If the barriers to 100% adoption of VoIP are in fact: 1) Cost of VoIP equipment 2) Lack of technical knowledge by the customer 3) Fear of failure to make it work 4) Doubt, despite all evidence, about the quality 5) Reluctance to enter in to a contract - especially if there's a 'free' equipment component involved then all that has to be done is to remove those barriers which I think can be done. The answer is a calling card (or at least the same concept) which is, of course, in use all around Australia, and the rest of the world, by hundreds of millions of people. I don't mean an actual calling card in the case of Exetel broadband users - simply providing them with a number that lets them use their current home phone set up to call the Exetel VoIP switch via their current PSTN or ISDN line and then dial their interstate number or overseas number or mobile number to save money on each call. To allow this to happen only requires the user to register the telephone number they want to use or the service Of course this is slightly 'clunky' because the customer has to dial an access number before they dial the end user number but that is not too inconvenient if they are making a call the want to save money on - some more modern phones allow a speed dial to make this an 'automatic' function.. Of course they also incur the local call charge but that is, in almost all cases going to be far less than the "flag fall" imposed by their current provider on calls to STD, International and Mobile numbers. In this way every Exetel broadband customer could get the benefit of low cost VoIP calls without: 1) Incurring any cost of VoIP equipment - there is none 2) Needing any technical knowledge - they just continue to use their current telephone - they don't "set up" anything 3) Having any need to fear anything as they don't do anything new/different 4) Validating the quality by making one test call 5) Entering in to a contract - there would be none I might be wrong, it wouldn't be the first time, but I am of the opinion that Exetel will reach 30 million VoIP minutes by pretty close to June 30th 2008, Of course, the only question needed to be asked and answered is why Telstra don't already make this a standard option across their network......but we all know the answer to that don't we? No....not that they want to protect ther current call charges. Friday, October 5. 2007Providing WiMAx Using Exetel's Existing Australia Wide 'Network'John Linton Some ten days ago an Exetel customer started a thread on the Exetel Forum suggesting that "wireless hotspots" could be a useful future service offered by Exetel. The thread can be found here: http://forum.exetel.com.au/viewtopic.php?t=23546&postdays=0&postorder=asc&start=0 He referenced in a follow up post an initiative by iiNet and a small New Zealand company that was aiming at doing something quite innovative. His original posts sparked my interest and we did some preliminary investigation. Earlier this morning I read an article in IT Wire: http://www.itwire.com/content/view/14740/127/ which showed that British Telecom together with a Spanish company called FON was also starting to do a WiMax roll out based on some sort of 'co-operative' venture with their customers. In essence this simple concept is to 'empower/provide with the ability' any customer who wishes to participate in providing wireless access to their private DSL connection to casual/semi-permanent other users in their immediate vicinity who can connect via WiMAx. There are an awful lot of ifs and buts attached to this simple concept but it is really interesting. If you look at it in pure financial terms, Exetel already has an Australia wide network that costs us well over $A1,500,000 a month that connects our 50,000+ broadband users to our internet services. This network is leased by Exetel from Telstra, Optus and Powertel but it is used only by Exetel's customers and it's paid for by Exetel. (as of course is every other ISP's network who uses backhaul and last mile providers). Each of the end customers, or at least the majority of them, run some sort of small network within their own residence or business (many via wireless) which is used by their families or house mates. What's the difference between those customers who wished to do so providing wireless connectivity to their neighbours - especially in multi unit apartment blocks? (as an aside - I use a laptop in my lounge room or study at home that in theory connects to the internet via our own ADSL2 service via a wireless router. However when I power up the laptop I get a choice of my own in house network or three neighbouring networks. The same applies in the office where there is a choice of over a dozen wireless networks if you use your laptop in the ground floor coffee shop). The difference, of course, is going to be that Telstra certainly and Optus probably will find some contractual reason to make this as difficult as possible. There is also the base regulatory issues in the communications act centering round when and under what circumstances a carrier license is required. I dimly remember in the early BigPond days that there were specific exclusions of connecting the early ADSL service to a 'network'. Those exclusions, if they ever existed, have long disappeared as a huge proportion of all DSL services are shared. Personally I see some very real opportunities in this train of thought and, irrespective of how the final legalities turn out, it has significantly simplified another course of action we've been considering. So, my public vote of thanks to "Stealth" who has fired the enthusiasm of key people within Exetel to look at this WiMax problem from a 180 degree different angle. The Exetel Forum has provided Exetel with dozens of great ideas over the past 4 years and this may prove to be the best one yet. Thursday, October 4. 2007Being Very Efficient Can Result In Being Very InefficientJohn Linton When we started Exetel, having had a fair amount of experience in setting up and operating several other ISPs, we put a lot of emphasis on the automation of all aspects of the business. This was easy enough to plan (starting from zero transactions is a whole lot easier than re-developing systems that are already in place) as we had a very clear idea of where the business was going to go and the sorts of volumes of different transactions we would generate as the business grew. Our automation program has been very, very successful and allows Exetel to operate the business today with a fraction of the personnel that, as far as public information discloses, other ISPs are able to do. This improves the financial viability of the company as well as ensuring very precise information is provided to Exetel's customers as no 'human interpretation' is involved in a majority of those processes. Very pleasing......except....... Some 12 months ago we completed the automation of fault reporting which allowed an Exetel customer to register a fault experienced on their DSL, SHDSL, mobile, wire line or VoIP service 24 x 7 x 365. All the customer had to do was to dial the fault support number and use their telephone keypad to answer answer yes or no to a series of 4 - 8 questions (depending on the service). The automated system would then set up a fault ticket, advise by return automated telephone call what the fault ticket number was, register the fault with the carrier and then update the customer on progress by an automated voice message to their nominated contact number each 12 hours. This automated process was a brilliant success. The vast majority of customers appreciated being able to log a fault by telephone any time of any day (rather than during business hours) and Exetel actually reduced the number of support personnel required to manage a continually growing customer base. Everyone was happy - for a few months. The downside to this level of efficiency and responsiveness was not evident until we had a visit from a senior manager from Telstra Wholesale who, during the operational review, pointed out that over the previous 12 months the number of faults per 1,000 customers reported by Exetel had gone from the lowest of the top 20 Telstra Wholesale ISP customers to the highest by a factor of 6 over what we had been lodging 12 months previously and to be more than double that lodged by the average of all other ISPs. There was something clearly wrong and it was annoying our largest supplier and, when it was drawn to my attention, it seriously concerned me - firstly because I didn't want to annoy any supplier and secondly because it seemed to indicate that there was something going massively wrong with our network delivery services. At an internal meeting no-one could shed any light on what was happening and every analysis we did provided no insight in to what might be happening. Several people continued to try and find out what was happening over the next several days without any success at all. Logic finally prevailed. We were too efficient. We came to the conclusion that by allowing our customers to log a service fault 'immediately' they experienced what the saw as an "Exetel Problem" rather than having to wait until the next business day to telephone Exetel and speak to a human to log their fault we had allowed customers who were experiencing a short term outage: a) on their individual line b) their ADSL port group at the exchange c) the exchange itself d) the backhaul from the exchange e) or some CPE/their own operational problem which they would subsequently realise and fix themselves (or someone else in the household would return and fix) to log a fault which would be already known to Telstra and would be fixed within a few hours as part of planned or unplanned maintenance. To test this assumption we modified the process to hold all automatically logged faults logged for 8 hours before doing a simple, automated, test to see if we could see the user logged in and if we could we closed the fault. The result was that 70% of total faults logged by the customer for ADSL1 services were resolved either by their own subsequent actions or by the actions of Telstra maintenance before any action on a fault lodged by Exetel on behalf of the customer could have started to have been investigated. A great relief on both counts of concern. We will continue to work on refining this process - the actions we took had a benefit of actually lowering the average fix time of reported faults - and will give more thought to the implementation of any such processes in the future. Wednesday, October 3. 2007Exetel - Year Five And Beyond Plan(s)John Linton When we did the original planning for Exetel we didn't go beyond the first four years because it seemed to be fairly pointless to attempt to forecast the changes that would take place so far in to the future. We extrapolated some basic figures for the fifth year but 45 months ago that was not a sensible basis for doing anything other than providing a 'tidy' looking set of spreadsheets for a five year period. In hindsight, our predictions of pricing and product changes were pretty reasonable and our assumptions about what, we then perceived to be competitors, would do weren't very far out. However all of that is now in the past and we are a little late in starting preparing the plans for the next four years. Planning a commercial entity's objectives, actions and financing for the second phase of its 'life' is both exciting and daunting and, at least for me, unbelievably difficult. It would seem obvious that all that would be needed to be done would be to extrapolate the current fully documented and understood trends in the same way that the original planning was done. That would be easier because Exetel now has 45 months of known 'history' and all the guessing of what would happen over the past 45 months has now turned in to examinable reality. There is one, at least, major problem with that view in my opinion. That is - what is really going to happen with the wholesaling of base communications services by the current carriers with whom Exetel deals and what new opportunities now exist for a company like Exetel that didn't exist when we were a zero revenue/customer start up? Exetel's first four year's plans were based on buying wholesale services and reselling them in pretty much the same way as every other such entity did. That has been, to date, relatively successful in that: a) We still exist whereas so many other companies with similar objectives that were around before we started or started up after we commenced business have disappeared. b) We've made a little bit of money (major stress on "little") rather than losing money. c) We have a much better relationship with our bank and most of our current suppliers than we did when we started. But it's taken a huge amount of effort by a very few people to achieve this and, realistically, that effort can't be continued for another four years. Also the wholesale model doesn't hold a lot of attraction at any sort of size much beyond where Exetel is currently planned to be in 9 month's time. By that I mean - an awful lot of effort goes in to making each month's revenue, the vast majority of which is immediately paid to someone other than Exetel's employees or shareholders. So.......... ......................................that's why I find it so hard to make the judgements as to how we do the two/three things that need to be done, in my opinion, at this time of Exetel's 'life'. The things exercising my mind (as I continue to stare morosely at the blank spreadsheet cells in front of me) are how to cut the current costs of providing data and other services in half and what level of capital invested in what technology will allow that to happen. The third thing is - can/should Exetel do this by itself or should it find a partner(s) to do "it" - when "it" is defined enough to quantify the timeframes and risks. The two most obvious things that seem self evident in aiming at the cost reductions are, of course, buying ingress/egress bandwidth from Southern Cross (or some other entity) and connecting customers via wireless technology that is progressively deployed and owned by Exetel. All I need to do now is work out how to do that and, very importantly, how to fund it and Exetel's next four years will be very exciting (hopefully in a positive way) and might even produce some profits greater than "little". Tuesday, October 2. 2007The Tyranny And Deception Of FiguresJohn Linton I was looking at Exetel's recurrent billing figures early this morning (Exetel bills for all recurrent services on the 1st working day of each month). It's one of the most enjoyable activities of the month as Exetel's recurrent billing has increased each month, without exception, since we sent out our first recurrent monthly bills on March 1st 2004. We chose to bill in this way, rather than cyclically billing a customer on the day of each month that their service was activated, for a number of reasons which included the need to make the customer's awareness of DSL monthly usage and therefore any excess charges easy to display and therefore control. The new financial year has got away to a very promising start with the latest recurrent monthly revenues up 41% on the same month last financial year and the monthly figures displayed as a line graph have a pleasing unbroken Northward heading aspect to them. As I said, a pleasing way to start each monthly billing day. The pleasurable feeling isn't long lasting and the "fearful symmetry" (with apologies to William Blake for cheapening that phrase so crassly) of the last 44 months of that trend line inevitably introduces the thoughts of what has to be done to maintain it - which raises the further issue of "but - is it too late to be maintained?". There is an inbuilt tyranny of trend lines that continue Northward over a protracted period and, to a lesser extent - but no less true, there is also 'deception'. The tyranny is easily understood in that no responsible manager wants to front up to a board meeting and try and justify why revenue has fallen from the previous month/quarter. The deception is much harder to come to grips with. A continually upward trend is visually pleasing but it conceals the fact that the results for any current month are the results of the cumulative actions taken in past months, often years, with recent actions only marginally affecting the continually growing 'dead weight' of customer hopes, expectations and "punishment" accrued from all past experiences and actions. Like a giant container ship the actions needed to turn or slow it have to be taken long before any actual turning or slowing becomes apparent to an observer. Similarly the actions needed by the manager of a business to grow (or prevent slowing of growth) aren't those that can be taken 'now'; they're the results of a whole lot of actions taken in the past with 'current' actions not yet affecting the total business in any meaningful/measurable way in comparison. So I take little comfort that Exetel is being 'run correctly' at any point in time because, in my experience, the correctness or otherwise of today's decisions (and tomorrow's and the next day's and so on and on) won't be known for some months in to the future and sometimes far longer than that. Of course, a North trending graph is infinitely preferable to one going South or East! So the monthly cycle continues; first check point that the planned revenue and profit growth is on target over the past month and then endlessly think about what needs to be done to ensure every aspect of the business (for which there are no simple graphs) is being run to continue to allow that trend line to be maintained at its planned rate of increase on 1st November. The pleasure in reviewing the monthly billing results seldom lasts beyond the first cup of coffee before being quickly shoved in to the background by the considerations of buying better and developing new services. "Plugging" the actual results in to the business plan at the start of each month is the starting point to see how well/badly the assumptions in the overall plan were/are. Then the endless cycle of looking at each aspect of the business begins again. I take some comfort from the trend lines but I'm still unclear as to whether they run me or I run them. |
CalendarQuicksearchArchivesCategoriesBlog AdministrationExternal PHP Application |