Amazing Amazon.com

I have always been awestruck when I get to hear Jeff Bezos talk about the technology vision and rollout plans for Amazon.com. ACM Turing Award winner and Microsoft Technical Fellow Jim Gray interviews Amazon CTO Werner Vogels ans writes about learning from the Amazon technology platform. Excerpts with some edits and comments from the interview:

- Its now clear and out in the open - Amazon is first and formeost a technology company and not just an online bookstore.
- Amazon has grown from an online retailer (albeit one of the largest, with more than 55 million active customer accounts) into a platform on which more than 1 million active retail partners worldwide do business.
- Behind Amazon's successful evolution from retailer to technology platform is its SOA (service-oriented architecture), which broke new technological ground and proved that SOAs can deliver on their promises.
- The original system built during launch lasted till 2001 when it became clear that the front-end application couldn't scale anymore. The parts that needed to scale independently were tied into sharing resources with other unknown code paths. There was no isolation and, as a result, no clear ownership.
- In the new design, service orientation is seen as encapsulating the data with the business logic that operates on the data, with the only access through a published service interface . No direct database access is allowed from outside the service, and there's no data sharing among the services. Over time, this grew into hundreds of services and a number of application servers that aggregate the information from the services. The application that renders the Amazon.com Web pages is one such application server, but so are the applications that serve the Web-services interface, the customer service application, the seller interface, and the many third-party Web sites that run on our platform. If you hit the Amazon.com gateway page, the application calls more than 100 services to collect data and construct the page for you.
- The big architectural change that Amazon went through in the past five years was to move from a two-tier monolith to a fully-distributed, decentralized, services platform serving many different applications. A lot of innovation was necessary to make this happen
Some key lessons learned, in the transition:
- If applied, strict service orientation is an excellent technique to achieve isolation; it bestows a new level of ownership and control .
- A second lesson is probably that by prohibiting direct database access by clients, scaling and reliability improvements to the service state can be made without involving the clients.
- Other lessons are related to accessing services: To be able to aggregate services and to insert advanced infrastructure techniques such as decentralized request routing or distributed request tracking, a single unified service-access mechanism is needed.
- Another lesson learnt is that it's not only the technology side that was improved by using services. The development and operational process has greatly benefited from it as well. The services model has been a key enabler in creating teams that can innovate quickly with a strong customer focus. Each service has a team associated with it, and that team is completely responsible for the service—from scoping out the functionality, to architecting it, to building it, and operating it.
- Giving developers operational responsibilities has greatly enhanced the quality of the services, both from a customer and a technology point of view.
- Amazon has very diverse groups of customers. Besides retail customers there are retail partners. About a million small and larger businesses sell on the Amazon platform. Amazon.com is a very advanced e-commerce platform on which anyone can become a retail partner and instantly benefit from all the platform functionality that has made Amazon.com so successful. The Amazon.com technology and data is made available through the AWS (Amazon Web Services) e-commerce services. This is a free Web-services interface for developers, which they can use to build (and charge for) their own applications on top of Amazon. There are about 150,000 of these developers and we consider them important customers. Other important platform customers are the enterprise retail partners like Target, Bombay Company, and the NBA etc
- On the emerging technologies, he sees that there are three categories of interfaces :
- The first category is the services that make up the Amazon platform. There we use interface specifications such as WSDL, but we use optimized transport and marshalling technology to ensure efficient use of CPU and network resources.
- The second category is the interface with the retail partners, which has strict descriptions for XML feed processing, service interfaces, etc., and where Amazon leverages as many standard technologies as possible.
- The third category is the now public Amazon Web Services, which builds on the platform services and provides REST-like as well as SOAP interfaces.
An excellent interview and a must read for all those trying to understand innovative and scalable business platforms.