In-Depth

From Edge to Edge: Measuring Performance for E-Business Success

• 05/01/2000

Attend any conference on e-business building, application service provisioning or Web hosting, and two central messages quickly emerge. The first is reminiscent of the exhortation of early prospectors in the Old West: "There’s gold in them thar hills!" Translated to more modern parlance, vendors of Web commerce-enabling software and hardware argue that lucrative opportunities await the innovators who develop the right formula for selling products and services over the Internet.

Analysts report that a one-trillion-dollar industry has formed around the efforts of both existing brick and mortar and newly-formed dotcom companies to stake their claims on the Web and pan for the precious metals believed to be flowing through its rivers and tributaries. The message touched off a 1990s "gold rush" that rivaled its 1890s namesake in size and scope.

However, there is a second message that essentially says: Delivering services via extensive public networks is hard work. Service levels must be established and maintained if customers, a fickle group quite willing to "click out" to a competitor’s Web site if they encounter difficulties with your site, are to be retained.

The message underscores the dilemma confronting e-business: Never before has the success of business been so closely tied to the performance of its enabling technology. At the same time, never has the technical infrastructure so directly responsible for business success been so totally beyond the control of IT managers.

In the absence of control, which is usually a function of measurement and tuning, companies can find life on the Web to be, in the words of Thomas Hobbes, "nasty, brutish and short." Some companies, in their efforts to compensate for lack of control, throw additional technology at the problem. They seek to identify any potential "chokepoints" for information flow through their Web-hosting infrastructure, and build in additional failovers or extraordinary spare capacity measures.

The costs of these measures are justified to management by the potential costs of not implementing them. The industry has already seen the effects of temporary outages: Prominent online brokers incurring the wrath of customers and the Securities and Exchange Commission following a temporary outage reported on the cover of The Wall Street Journal.

Highly-publicized disasters are only the tip of the proverbial iceberg. Performance delays resulting in lost customers and lost business, while not as prone to media attention, are just as important, and potentially devastating, to smaller e-commerce firms.

The primary reason for the lack of control over e-business infrastructure performance is that the technical infrastructure itself is usually a composite of services, often obtained from a number of service providers. Typically, there are at least five tiers of technology services involved in an e-business presence.

Tier 1: Transport Services. One or more vendors may be contracted to provide basic information transport, including lines, connections, on-ramps, gateways and other specific point-to-point and multipoint network interconnects.

Tier 2: Network Services. Providers of transport services may or may not be responsible for providing network services such as routing, configuration, traffic shaping, and network management. If the transport vendor simply allocates bandwidth, a second vendor may be engaged (by the e-commerce company or the transport vendor) to provide these network services.

Tier 3: Data Center Services. In some e-business configurations, data center or hosting services are completely outsourced to a third-party provider. In other cases, front-end systems are operated by a third-party and back-end systems are retained under the local control of the company. Links between front-end and back-end systems may be the responsibility of the hosting company, the contracting company or yet another third-party service provider. The data center service provider handles the provisioning of the customer’s application platform, its backup and its maintenance.

Tier 4: Application Services. Some e-business sites deliver value to customers by providing access to internal corporate applications or databases. Others deliver to subscribers information that is "enhanced" or manipulated by internal processes (for example, stock market tickers to which analyses of customer-preferred issues are added). These applications and processes may or may not be owned by the provider, raising potential issues of protracted downtime for problem resolution, upgrades, support, etc.

Tier 5: Business-to-Business Services. In a growing number of cases, the public network is being used to establish business-to-business links between supply chain partners, key vendors and other entities. Providing gateways and portals to support inter-application messaging, security functions, etc., may entail the use of one or more third-party infrastructure providers.

This multiplicity of infrastructure providers is at the heart of the lack of control that most organizations are able to exercise over their e-business processes. It impairs the ability of companies to meet or fulfill the four prerequisites for selling services to customers via the Web, specifically:

Establishing Service Level Agreements (SLA). SLAs constitute the contract between the company and its customers. They are the means by which the company "puts its money where its mouth is" – setting customer expectations, linking them to specific performance criteria, and establishing the means, both for reporting SLA compliance, and for redressing situations where performance falls short of promises.

With so many providers involved in delivering a service, it is extremely challenging to set a reasonable SLA or to measure and report SLA fulfillment. While some companies promise to return some portion of a purchase price or subscription fee to the customer in the event of an SLA shortfall, this may not be an adequate safeguard from the customer’s perspective. It also exposes the company to significant risk of lost capital and revenue.

Establishing Capabilities for Identifying and Resolving Breakdowns. Related to the SLA issue above, the multiplicity of providers involved in delivering an e-business process may obviate efforts to establish ways and means for tracking transactions through the morass of systems, networks and processes that affect it. Without a clear view of the details of transaction processing, the company may have little or no capability to identify the cause of a delay or breakdown. As a result, problem resolution can be significantly delayed.

Establishing Problem Ownership. To the customer, the company owns all problems that arise in the delivery of a service. In the case of e-business, as with any business, the company has ultimate responsibility for delivering services per the SLA. However, given the number of providers involved with an e-business process, the actual control that the business exercises over the execution of the process is rarely equal to the accountability and responsibility that the company bears. Resolving even minor problems can lead to finger pointing between service providers and significant delays, while the company itself is responsible to the customer directly. Few CEOs, CFOs or CIOs of traditional (not e-business) firms would abide such a situation.

Understanding Usage Patterns for Operational Growth. This final prerequisite for selling services represents a strategic safeguard for the company providing the services. The e-business needs to be able to collect, aggregate and assess usage patterns for its service over time, in order to identify future capability and capacity requirements. Complex service delivery systems, particularly those characterized by numerous third-party service providers, can make the collection of reliable usage information a daunting task.

In the absence of reliable data, one frequently finds e-business firms addressing capability/capacity issues in very unscientific ways, speculating where bottlenecks, slowdowns and breakages may occur in the future and adding redundancy or spare capacity without much solid evidence that these additional features will provide an adequate safeguard against downtime or contribute much of a safety net for handling usage growth. In some cases, companies may actually be slowing down their business processes or making them even more complicated and less amenable to monitoring and planning in the process.

The above portrayal of the current situation in e-business may seem to paint a rather grim picture of an industry that, by all accounts, is among the fastest growing segments of modern business. Based on numerous discussions with large and small e-business enterprises, however, it is clear that nearly all firms view the problem of inadequate performance measurement as a non-trivial issue.

E-businesses are dependent upon the performance of their information technology platforms to a greater extent than any business of the past. Moreover, the penalties for not performing per SLA are significantly greater than even a decade ago.

To address the complexities of e-business service delivery, advancements in performance measurement are clearly required. Initially, the industry has responded with solutions purporting to provide "end-to-end" management.

End-to-end management methods typically entail the placement of agents on a desktop client – that is, on the workstation of the end user accessing the e-business service site. These agents collect data about the amount of time that transpires between the initiation of a transaction by the end user and the return of results from that transaction. The re-painting of the end user screen with the results of the transaction is often taken as the conclusion of a transaction.

Such performance measurement methods, which derive from techniques employed within a traditional business enterprise, are severely limited within the context of modern e-business. One limitation is the use of a desktop agent. Given the diversity of client system types in the world of the Internet, not only is the definition of a universal agent a potential hurdle (in terms of compatibility with various operating systems, browsers, resource availability on the client, etc.), but also its implementation and operation.

Such an architecture is ill-suited to the stateless environment of the Internet, where disconnects, dynamic address assignments, and inconsistent message routing are the rule rather than the exception. Moreover, the fundamental premise of this approach – that end users initiate all transactions and that all transactions are completed when the end user receives acknowledgement from an e-commerce server – is essentially flawed. In many cases, e-commerce transactions are designed as multi-tiered processes that are triggered by an end user, but continue to process long after the end user has been provided a "feel good" response that his or her order has been taken.

Vendors of end-to-end performance measurement solutions suggest that their products provide "real information" about the end user’s e-commerce experience. Response time, they insist, is where the "rubber meets the road" in e-business customer satisfaction.

While this argument has some merit, it is not performance measurement in the strictest sense. Using a metaphor from healthcare, such an approach is tantamount to taking one vital sign for a patient. In isolation, however, this single vital sign is meaningless. A poor response time measurement provides little information about the cause of the delay.

An alternative to end-to-end management being pursued by several developers at present is "edge-to-edge" performance measurement. This approach is better suited to the complexities of e-business transaction processing.

With edge-to-edge architecture, a series of collectors are deployed throughout the system that supports the transaction life cycle. These intelligent programs perform service tests and collect key elements of information that, taken together, better reflect the "health" of the overall system.

With edge-to-edge architecture, performance data is collected from virtually any component – router, server, application, database, transaction response, etc. – where collecting data makes sense. It requires, first and foremost, an understanding of the transaction life cycle. Based on this information, specific collector placement ensures that those system components that should be and can be monitored are, and that those components that should be, but cannot be, monitored directly are "black boxed" or isolated. This approach facilitates quick problem detection and troubleshooting.

The allure of e-business is two-fold. First, it promises to deliver to companies unprecedented access to untapped pools of prospective consumers. Second, it enables orders-of-magnitude improvements in the efficiency of business-to-business transactions.

However, the realization of these potential benefits may be compromised by the complexity of infrastructure used by e-business firms to deliver services. The lack of effective performance measurement capabilities increases the likelihood that many businesses will discover the dream of the Internet is just so much Fool’s Gold.

About the Author: Julian Palmer is Chief Technology Officer for Datametrics Systems corp. (Fairfax, Va.), a supplier of performance management software and solutions for e-business.