Overview

Download

People

Documentation

 

System Architecture

MyAmazonPlus service is a wrapper around Amazon.com. Clients talk to the MyAmazonPlus server through SOAP [1], which in turn talks to Amazon.com through SOAP, or more specifically, using Amazon Web Services 4.0 API (or AWS 4.0, now renamed to Amazon E-commerce Service 4.0) [7].


Figure 1: System architecture of MyAmazonPlus

MyAmazonPlus supports the following features:

  1. Keyword-based searching: Clients can perform keyword-based searches on one of the following categories: Books, Video, and Electronics. Clients can specify the price range of the products they are interested in. For each search request received from the client, MyAmazonPlus server issues two search requests to Amazon.com, one on items offered by Amazon.com itself, the other on items offered in Amazon.com's Marketplace. MyAmazonPlus will combine the first results returned from both searches and send the aggregate result back to the client, sorted by price. Additionally, some items' prices are hidden until they are added to the shopping cart. In this case, MyAmazonPlus server will first create a temporary shopping cart, add that item to cart, query price in the temporary cart and return the final price via the normal Web services interface.
  2. Shopping cart management: Using our GUI client, users can add items to and delete items from a locally maintained shopping cart. To place orders, the client first sends a coarse-grained AddItemsToCart request to MyAmazonPlus server, which in turn adds the requested items to a remote shopping cart maintained at Amazon.com using AWS. Then the user clicks on the "Place order" button to bring up a browser window showing the shopping cart page at Amazon.com. The user can continue his/her checkout from there.
  3. Price watch: MyAmazonPlus allows users to specify certain items on price watch. Periodically, our MyAmazonPlus client will issue a GetItemPrice request to the MyAmazonPlus server, which in turn gets the price from Amazon.com. This function is useful in at least two circumstances: 1) If you are planning to buy something from amazon.com in the near future but are not satisfied with the current price; 2) You bought something from amazon.com recently and would like to protect your purchase based on amazon.com's price protection policy (amazon.com will refund you the price difference if amazon.com lowered the price within 30 days of your purchase).
  4. Helping charity on the fly: For orders placed through MyAmazonPlus, 3% of the total order price will be donated to the China Overseas Education Foundation (www.ocef.org). We implemented this using the AssociateTag feature provided by AWS.

 

MyAmazonPlus Service

The MyAmazonPlus server is implemented on top of Apache Axis. We adopted the WSDL-first approach. That is, we designed the WSDL for the service first, and then used Axis' WSDL2Java tool to generate Java classes. The implementation of the service makes heavy use of Amazon's AWS 4.0 SDK. AWS exposes a comprehensive list of interfaces to Amazon.com, such as item searches, item lookups, and shopping cart management. We built simple wrapper interfaces around a small portion of the interfaces exposed by AWS.

MyAmazonPlus provides the following operations to the clients:

  • itemSearch: Clients specify keywords, minimum price, maximum price, and search index (Amazon's jargon for "product category") when invoking this operation. The return value of this operation is a list of items. Each item comes with its ASIN (a unique identifier for an item), price, name, author (if applicable), manufacturer, release date, description, editorial and customer reviews, image URL, detail page URL, etc. See WSDL for details.

  • getItemPrice: Clients provide the ASIN as input, and the output is the corresponding item's list price, price, tax, and shipping cost.

  • addItemToCart: Clients specify the item to add by providing its ASIN. Clients need also specify the quantity of this item to add. Clients can specify the cart by providing a cart Id and a HMAC, which is a unique security token that, when used with the cart Id, allows you to access and modify a specific remote shopping cart. If no cart Id or HMAC is specified, MyAmazonPlus server will create a new shopping cart and add the item to the newly created cart. A Success code will be returned to the client, to indicate whether the operation was completed successfully. When this value is false, a Reason string will also be returned to explain the cause for the failure. Finally, a cart object is also returned, which includes its cart Id, HMAC, and the ASINs and cart item Ids of the items in the cart.

  • addItemsToCart: This is the batch version of addItemToCart. Instead of specifying one single item's ASIN and quantity, clients provide an array of items.

  • deleteItemFromCart: Clients specify the item to delete by providing its ASIN. Clients need also specify the quantity of this item to delete. Again, clients specify the cart by providing a cart Id and HMAC, except that this time these arguments are required. Similar to the addItemToCart case, the result of this operation includes a Success code, an optional Reason string, and a cart object.

  • deleteItemsFromCart: This is the batch version of deleteItemFromCart. Instead of specifying one single item's ASIN and quantity, clients provide an array of items.

  • listItemsInCart: Clients specify the cart Id and HMAC. The return value is a list of items in the specified cart. 

A Subscription Id is required to invoke AWS services. The Subscription Id is obtained when you register with AWS. Thus all of the operations above require a Subscription Id as an additional argument.

We also deployed an instance of our service with authentication, called MyAmazonSecure. We implemented simple password-based user authentication using WS-Security. We consulted Apache WSS4J's source code, but didn't use their library in the end. We installed a UsernameTokenHandler on the MyAmazonPlus server, which will intercept the wsse:security header in the incoming SOAP envelope, extract the username and password from the header and put them into the current MessageContext as properties, and set the "processed" flag of the header to be true. When the message reaches the service implementation itself, the username and password will be read from the MessageContext and compared to a preset username/password pair. The authentication succeeds if and only if the username/password read from the SOAP header matches the preset username/password.

 

MyAmazonPlus Client

The client side was a GUI application written in C#. Figure 2 shows the primary user interface of the client side. The user interface is divided into two regions. The upper region provides research setting for users to locate the items they desire on amazon.com and a status summary of the current shopping cart. The lower region uses three separate tab windows to maintain the search result, shopping cart as well as the watch list on the client side. Users can use the search result list to find their desired item, read item reviews and product details (author, ISBN, weight, release date) and add selected items to shopping cart or watch list.


Figure 2: Client side screen shot of MyAmazonPlus

To achieve better usability, the keyword search function on the client side is implemented asynchronously. So the users can still browse the previous search results after a new search request has been submitted to the server. Other web services invocations, such as AddItemsToCart, RemoveItemFromCart, are implemented synchronously.

We implemented the web services publication and discovery on the client side by using Microsoft UDDI .Net SDK 2.0 beta [3]. We publish our services on http://test.uddi.microsoft.com with business name "Cal Berkeley SIMS web services course - Jingtao and Kai". UDDI publication/discovery functions can be accessed from Configure -> UDDI -> Publish / Discovery.

On the client side, we implemented the WS-Security enhancement by using Web Services Enhancements (WSE) 2.0 SP2 for Microsoft .NET [4]. We didn't choose WSE 1.0 because the name space of WSE 1.0 is "xmlsoap", which is not compatible with the "oasis" namespace used by WSS4J and WSE 2.0. We implemented the username/password mechanism for our web services. The user name/password can be specified by users from Configure -> WS-Security -> User Name / Password. The users also can enable (i.e. switch to MyAmazonSecure service on the server) and disable (switch to MyAmazonPlus service on the server)  the WS-Security enhancement by setting the Configure -> WS-Security -> Use Security checkbox.. A valid user name/password pair we chose is "webserv" and "sims04".

The pre-compiled binary package of the client side can be downloaded form [5], it requires Microsft .Net Framework 1.1 in the host operating system (Windows 2000/XP/2003). Additional libraries to support UDDI publish/discovery and WS-Security have been included in the binary distribution. The source code of the client side can be downloaded from [6]

 

Performance and Scalability Analysis

We employed several techniques to ensure our service is scalable and has good performance. First, we abide by the end-to-end design principle [12] and place most intelligence at the two end points (clients and Amazon.com). MyAmazonPlus server does not maintain any state (e.g., shopping cart) for the clients. Instead, clients maintain local shopping carts and send the final shopping carts to MyAmazonPlus server only when placing orders. As a result, the workload on the server will not increase dramatically when the number of clients increases. In addition, this stateless end-to-end approach, as proved to be compelling by the evolution of Internet, will make our service robust to temporary communication failure and network congestions.

Second, we provide coarse-grained operations, i.e., addItemsToCart and deleteItemsFromCart. The following figure illustrates how coarse-grained operations improve performance.

AWS provides minimum support for coarse grained operations (e.g., it allows to add up to two items to cart at a time). Thus although the interface of MyAmazonPlus is coarse grained, the second leg of, say, an addItemsToCart operation, is still fine grained. For this reason, we put our client behind a 56kbps dialup line in our experiments that generated the above diagram, just to amplify the effect of coarse-grainedness.

Finally, noticing itemSearch is the most expensive operation among all(each itemSearch operation needs 7-8 seconds to complete), we decided to cache results for recently requested search requests and serve itemSearch requests from this cache whenever possible. The cache size is fixed and we use the LRU (least recently used) cache eviction policy. To ensure the freshness of the results, the cache items must time out after a certain period of time. The parameters (e.g., size, timeout) of the cache are configurable.

 

Lessons Learned

We learned the following lessons while working on this project.
  • The designers of Axis did not have the WSDL-first approach in mind. The WSDL generated using the "?wsdl" query on the server differs from the original WSDL fed into WSDL2Java. For example, the original WSDL has a portType named MyAmazonPlusSOAPPortType, while the generated WSDL has a portType named MyAmazonPlusSoapPortType. Moreover, the code (albeit successfully generated by WSDL2Java without errors) generated from the original WSDL does not work. We had to re-feed the generated WSDL from "?wsdl" query to WSDL2Java to generate the new Java classes.

  • The Windows and Linux versions of Tomcat and Apache Axis server have slightly different behaviors when interacting with the .Net client. For example, the first character of all the supported method names in the generated C# proxy is uppercase if we use the wsdl from a Windows server, it becomes lowercase when we switch to the wsdl generated by a Linux server. 

  • Current .Net Web Services doesn't support xsd:NonNegativeInteger and xsd:PositiveInteger. Elements of these types are converted to strings in C#.

  • WSE 1.0 and WSE 2.0 use different namespaces for ws-security. The former uses "http://schemas.xmlsoap.org/ws/2002/04/secext" and the latter uses "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd". Apache's wss4j uses the same namespace as WSE 2.0.

 

References
  1. SOAP 1.2 specifications http://www.w3.org/TR/soap/
  2. Web Service Definition Language (WSDL) 1.1 http://www.w3.org/TR/wsdl
  3. Microsoft UDDI .Net SDK 2.0 beta
  4. Web Services Enhancements (WSE) 2.0 SP2 for Microsoft .NET
  5. Pre-compiled binary package of MyAmazon.com++ client.
  6. Source code of MyAmazon.com++ client.
  7. Amazon Web Services SDK http://www.amazon.com/webservices
  8. Apache Axis http://ws.apache.org/axis/
  9. Apache WSS4J http://ws.apache.org/ws-fx/wss4j/
  10. Source code of MyAmazonPlus service
  11. MyAmazonPlus service online documentation
  12. J. Saltzer, D. Reed, and D. Clark, "End-to-end Arguments in System Design". ACM Transactions on Computer Systems (TOCS), Vol. 2, No. 4, 1984, pp. 195-206.