Creating Shared Context with XML .. 9
Semantic Web Applications . 14
Trading Partner Coordination . 14
The explosive growth of the Internet during the last decade has been driven by two factors: a set of standardized protocols that enable previously unknown parties to automatically establish communications, and the tremendous commercial potential of a global, real-time mass communications network. The standardized protocols –TCP/IP, HTTP, and HTML – allow individuals, businesses, and automated software agents to retrieve and send information. The protocols form the basis for a communication framework for electronic commerce – of goods, services, and ideas. The continued growth and commercial viability of this medium depends on the ability of users to locate relevant information from a vast and rapidly growing volume of information, and to conduct increasingly more complex transactions in a decentralized environment with a variety of competing vendors and standards. The answer to this challenge is the development of the Semantic Web – a framework for representing data on the Internet. The Semantic web is based on XML and builds on existing communication and presentation protocols. This paper will focus on the Semantic Web’s potential to change how business is done. The paper will cover the basics of XML, Web Services, and then Semantic Web, and then show how the Semantic Web can be used for a variety of business purposes.
The W3C defines the Semantic Web as “the representation of data on the World Wide Web.” The protocols behind the Semantic Web are being developed by the W3C, the World Wide Web Consortium. The framework uses and extends XML syntax, runs over HTTP, integrates a wide variety of applications, and like XML, is designed to be machine-readable. To understand the nature of the Semantic Web, it is helpful to examine the origins and motivations for XML.
The World Wide Web was originally envisioned as a collection of static documents, hierarchically organized, and limited in volume. HTML, the Hypertext Markup Language was designed solely as a text-formatting and layout presentation language. The lack of features to describe the meta-data, or the content of documents presented a number of problems as the web grew.
Web searches of HTML documents can only search for particular keywords, without regard to context or content. For example, searching for the word “Aggie” yields very different results. “Aggie” is the name for a software program, a student of an agricultural school, a playing marble, or a student of a particular school – Texas A&M, part of the name of many organizations and clubs associated with it, and an adjective when associate with an agricultural school. It is currently impossible to specify whether one wants to search for software programs, organizations, people, or descriptions. The lack of context is a major problem for search engines: a search for a word or phrase that has multiple meanings will return different types of results.
Another problem with HTML is a lack of a data-description language. Suppose that a consumer wanted to find a digital camera to buy. He might find dozens of online stores that provide detailed information on digital cameras in dozens of different formats. Some categorize digital cameras under “multimedia,” while others will listed them under “video” or by brand name. The camera details themselves will certainly be organized differently, with different formatting and labels for the same features. With standard HTML, he could call up a dynamically-generated list of details from the web server. If he wanted to change the sort preferences, or show or hide certain features however, he would have to re-request the page from the server. Repeatedly reloading the page places strains on the server and client resources and limits the interactivity of the Web. If the user wanted to download the list to import it into a spreadsheet, he would have to manually copy and paste the cells, adjusting for formatting differences between different sites and losing the interactivity of the live web site. The lack of a data-description language also poses challenges when searching for vendors. A human being could eventually compile a spreadsheet comparing all the stores and camera models, but an automated comparison agent would have to be individually tailored for each site, and would break if the site formatting changed. Another problem with HTML is the lack of any support for intelligent search agents. If a consumer had a Macintosh computer, he would have no way to limit his digital camera search to models that supported his operating system.
A major issue for business is the lack of a standard electronic commerce language. If two companies wanted to conduct electronic commerce, they would have to agree beforehand on the data format for their messages and the rules by which to process orders. Developing software to handle such transactions would be costly, and a new agreement and communications protocol would need to be developed for each new interaction. A new company that wanted to offer its services would be out of luck until it could reach an agreement with existing enterprises on how to process messages. It’s possible to use EDI for electronic document transfers, but it is not suited for electronic messaging – for example, to check the status of an order. Additionally, providing electronic commerce services to consumer offers tremendous potential to businesses. Rather than using email alerts, companies could push product announcements and order information directly to consumer’s desktop.
The development of the Semantic Web offers solutions to all these challenges by transforming the web into a data-driven environment. The basis of structured information exchange is a metadata standard. Metadata is data about data. (XML Manager’s Guide.) The de-facto standard describing data is XML. XML creates a standard format for describing data. XML alone is not sufficient for describing content, however. Merely knowing that the “author” of a document is “John” does not tell a client the type of data (string, number, binary, etc) or how to use that data. To describe metadata, additional documentation is needed to create a shared context between the parties exchanging the data. DTD’s, XML Schemas, and function-specific XML-based standards provide this functionality.
XML, or eXtended Markup Language is accepted as the standard for data interchange on the Web. (SemanticWeb.org) Like HTML, it is a descendant/subset of SGML, the Standard Generalized Markup Language, the metalanguage that both languages are based on. XML was designed as a simple machine and human-readable data description language. The simple, <tag> based format allows XML files to be easily editable both machine and by hand, depending on the scale of the user. This allows small businesses to interact with the biggest enterprises using a single data format. For example, sole proprietors may manually edit a few orders written in XML per week, while an enterprise-scale company may develop an automated processing system to handle hundreds of files per hour.
As with HTML, editors add metadata to XML through tags. The syntax for XML tags is very similar to that of HTML. HTML-style open and close tags define elements, which must be arranged in a strict hierarchy. Unlike HTML, the tags can be any element, rather than a set of predefined formatting elements. Data is contained within a given tag or in a tag attribute. The structure of XML files can be easily seen in a simple example. Here is an XML file currently used as a database of favorite links that dynamically display on a website:
<?xml version="1.0" standalone="yes"?>
<LinksList>
<Link>
<Name>Texas A&M University</Name>
<URL>http://www.tamu.edu</URL>
<Category>Aggie</Category>
<Description/>
</Link>
<Link>
<Name>The Battalion</Name>
<URL>http://www.thebatt.com</URL>
<Category>Aggie</Category>
</Link>
</LinksList>
The document element of this XML file is a LinksList, which contains multiple Link elements with their own child elements. This well formed XML file observes the basic rules of the XML syntax: (Rules from XML: Manager’s Guide)
To describe the allowable structure of an XML document, DTD’s or Document Type Definitions were originally developed to constrain document structures. A DTD lists the elements within a document, their types, and any child entities they contain. A successor to DTD’s is XML Schema, which contains more specific and powerful rules for specifying the allowable content of an XML document. An XML schema allows a software agent to easily process or store data it encounters for the first time by referencing its schema.
While XML and DTD define the data and content of a document, XSL, or the Extensible Stylesheet Language defines its appearance, much as cascading stylesheets do for HTML. Used in conjunction, an XML document can separate data, metadata, and appearance by referencing external DTD’s/Schema’s and XSL style sheets. XHTML, the successor to HTML reformulates HTML by allowing XML-style extensions in additional to normal HTML tags.
While the XML standard is sufficient for many projects, it alone does not provide the shared context necessary for the semantic web. A standalone XML file merely provides a listing of data, without specifying its nature. For a machine to use the data contained in an XML file, it needs to know how to actually use it. A DTD or XML Schema provides a description of the XML document, allowing XML documents to be validated against a schema and for different parties to verify that they are storing their data in the same format. However, if a unique DTD or Schema was created for each new interaction or electronic-exchange agreement, much of the usefulness of XML would be lost due to conflicting schemas used by different groups. Two companies might arrange a common schema to be used to conduct transactions, but a third company, or another industry might use a totally different schema to present the same type of data. Even within a single enterprise, the schema used by different departments might vary. For example, the Sales and Manufacturing department might both want to measure inventory, but they could have very different notions of what data is important. An order received by sales might be stored and sent to Manufacturing as an XML document. The document created in Sales would probably focus on customer information and product features rather than fields important to Manufacturing – such as build specifications and production schedules. Sales and Manufacturing could compromise on a shared schema – or they could use XSL transformations, or XSLT to automatically transform Sales XML documents to Manufacturing XML documents. XSLT is an XML-based standard for converting one XML format to another. Like almost all XML standards, XSLT documents use XML to specify how data should be converted from one format to another.
While XSLT is probably sufficient for use within a single enterprise, where conversions are relatively few, stable, and predictable, in many cases it is impractical to use XSLT for cooperation between different companies. XSLT documents must be created by hand for each format a document needs to be converted into. For complex XML documents, this requires expertise equivalent to programming the conversion in a programming language. If a company wanted to deal electronically with many vendors, it would have to require each vendor to convert their orders and other relevant information into the company’s own XML format. New vendors into the market might face a challenge in providing this data- especially if they don’t use XML in their own in-house processes. To automatically provide a shared context between many individual businesses, some sort of standard is needed for common business documents. Such a standard would define standards for common business documents – such as purchase orders, invoices, etc. Unfortunately, such a standard has not yet emerged. However, several groups are working on it, including UBL, the Universal Business Language by the OASIS standards group. ( http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=ubl) Other groups are working on creating XML implementations of the EDI (Electronic Data Interchange) framework. (http://www.xmledi-group.org) These efforts to develop of a set of standards for specifying business documents are essential to the creation of an open, global, and easily accessible online marketplace.
While the creation of standard XML formats for conducting business over the web is a key step in the creation of a Semantic Web, static XML files alone are not sufficient for the creation of online services. It’s possible to exchange XML files by email, but this is inefficient, insecure, and fails to automate the data-exchange process. To automatically exchange XML-based transactions, a messaging protocol is needed to handle the sending and receiving of XML-encoded data between two parties. This functionality has been implemented by Web Services, a set of protocols and specifications enabling network communications between software.
Web Services is a vague term that refers to the use of XML-related technologies to link together information systems and business processes using a variety of (sometimes competing) protocols. The most fundamental building blocks of Web Services are:
All major Web Service initiates have decided on SOAP as their basic low-level messaging protocol. While this is an important achievement for the creation of a common messaging infrastructure, much of the popularity of SOAP lies in the fact that it leaves so much higher-level functionality up to individual implementations and extensions. (XML Manager’s Guide.) SOAP was designed to provide only very basic connectivity, and contains a mechanism for adding extensions for all other extended features. SOAP runs over HTTP, which provides a number of advantages over creating a communications protocol from scratch. It can rely on TCP/IP for addressing and session management, and can establish secure connections over SSL rather than trying to implement its own encryption model. Like a web server, SOAP uses the HTTP POST and HTTP 200 RETURN methods of HTTP.
Above the basic connectivity level, two major uses of Web Services have emerged: electronic document transactions and distributed messaging protocols. Electronic documents transactions are self-contained documents such as “Order.” These documents are based on the EDI model and are sent to another company or department without expecting an immediate response or follow up. The other kind of web service is as a messaging protocol, which sends short messages that expect an immediate reply. For example, one company may request an “Order Status” and will expect a quick reply. Unlike an order, the order status request and response will contain little information, but must be processed quickly and in high volumes. Another important difference is that the use of Web Services a messaging protocol usually does not involve human intervention, whereas EDI documents usually require human action (for example, approving the order) before a response can be made. Fortunately, the Web Services framework can be adapted to serve both needs. Several languages have emerged to define EDI-type web services, including BizTalk by Microsoft and ebXML by the UN/OASIS. Because the focus of this paper is on the Semantic Web, only the use of Web Services for distributed messaging will be addressed in detail.
One of the primary benefits of XML and XML Web Services is that they allow companies to present their services for consumption over the web by automated clients. This allows a company to use Web Service directories (such as UDDI) to automatically find suitable trading partners and engage in commerce with them. To provide an example of the benefits the Web Services provide, an idealized example of how they can be used to facilitate coordination between trading partners is useful:
Suppose that an automobile manufacturer wishes to allow customers to order custom-made cars. Customers would be able to order a car either from a website or at a car dealership (perhaps after test-driving a generic model.) The customer would be able to choose from a variety of options and styles just as one can order a customized computer from Dell today.
The order for a new car would be recorded either as XML or in standard relational database tables. It would initially go through Sales, where the credit of the customer is verified, and then sent to Manufacturing, perhaps as an XML document. Manufacturing would then be responsible for ordering the necessary components from hundreds and perhaps even thousands of individual vendors. Because the automobile maker has a JIT (Just-In-Time) system, it wants to order every single component for a new car when a new order is received, and it wants the component to arrive at the assemble line exactly when it is needed. The car maker could try to set up automated and synchronized ordering systems with each vendor, but with thousands of parts to track, it would need a bigger IT staff than it has assembly line workers. Not only would it have to signal each vendor for each new part, but the vendor’s manufacturing system would have to understand the order, make the part, and then deliver it in sync with the status of the destination car on the assembly line.
Rather than attempting to design communications protocols to integrate their assembly lines and track components, the car maker and the component vendor could create access points to their manufacturing systems using Web Services. The component vendor would use XML documents based on a standard such as UBL. For each new order, the car maker would send an order using the vendor’s published service specifications over SOAP and receive immediate confirmation. It would be able to check the status of each order and direct the vendor to deliver the component at exactly the right time. The transactions will be conducted over a secure connection running over HTTP, and before any connection is established, both parties will provide their digital certificates to validate their identity. A large manufacturer would probably run this service over a dedicated line, but a smaller company may conduct business over the Internet.
While Web Services can be useful once two parties have agreed to do business, they can also be used to help companies establish new business relationships. Suppose that there are many competing vendors for many of the car manufacturer’s low-level components, su ch as steel, leather, and oils. As long as a vendor meets certain quality, delivery time, and reliability criteria, the manufacturer wants to choose the lowest-price vendor for each batch of components. Without a live directory of suppliers, it would have to rely on existing contacts, search catalogs of existing suppliers, and negotiate a contract with each new supplier. With thousands of parts to buy, it would be impossible to negotiate a new contract on any kind of frequent basis, and many new suppliers would never be discovered.
A UDDI Web Service directory can automate this entire process. A trusted directory vendor would set up a general or industry-specific directory of Web Services. The directory provider would be responsible for verifying the identify and qualification of the vendors listing their services with the directory. It would provide a searchable index of how the services would be organized – actually, there are currently a number of standards of how web services directories should be organized, and most UDDI portals offer several of them. (For example, one can search Microsoft’s UDDI registry (http://uddi.microsoft.com/) by geographic location and by several industry organization schemes.) Each web service offered has a URI and a unique identification key. Using the web service directory, the car maker could find suppliers that provide the best price while matching geographical, delivery, or certification criteria. The search does not have to be done by a human – an inventory management system could automatically query and then purchase the correct quantity of goods according to preset purchasing criteria and the current inventory levels. The magic of the system is that e-commerce would go on automatically, giving purchasers the best price every week (or even day), and saving suppliers the cost of paying salesmen to wine and dine their customers.
Another major use of XML is to organize information using XML metadata. While XML is a powerful data-storage language in itself, it is also very effective for providing information about other data and documents. An example of a metadata use is a library catalog. Each book record has a number of public fields with the book’s title, author, date, and a number of private fields such as administrative information, where is book is, etc. A library will probably hold its data in a number of database fields and present it using an HTML interface. For a public library, this is probably sufficient. However, suppose that an online bookstore is evaluating how to build its catalog. It may sell a variety of products other than books -- such as audio books, music CD’s, and magazines – all of which have different metadata requirements. It will probably require many more metadata fields than a library, and it will probably process the books among multiple servers – such as between the purchasing, sales, and the product catalog. Additionally, suppose that the bookstore wanted to allow third parties to access its catalog. It may want to let potential book buyers set up a web service to check when a given book arrives. In addition to XML Web Services, using XML to store the product metadata would be advantageous to the business. Once book metadata is stored as XML, it would be straightforward to transform the data for display by a particular user – for example a book overview for the online catalog, the purchasing information for the sales staff, and processed XML files for web service clients. This may be done using a database system which stores XML natively, or by converting records to XML from standard relational database – which Microsoft’s SQL Server 2000 and many scripting languages provide native support for.
Methods which work for a bookstore will also work for a digital camera store – only the XML Schema would change. If a company chooses to provide access to the data behind its catalog, customers and autonomous web bots could easily access the data behind a webpage to provide price comparison and product information to consumers. A Semantic Web-aware version of Google could then accept a query such as “Show a price comparison for 10 megapixel digital cameras that works with a Mac” – and it would return those exact results, excluding camera reviews or products that do not fit to criteria. For this to happen, the search client would have to know that a “mexapixel” is a property of a “digital camera” and that a “Mac” is a “computer” that must be in the “compatibility” list of a particular digital camera. This functionality does not require artificial intelligence, but it does require a shared definition of “digital camera” to exist somewhere that both the online store and the search engine will agree to use. This description will need to contain a vocabulary of the valid properties and relation of the entity it is describing. It will also be hierarchical: a “digital camera” may inherit the properties of a “photo camera” which may belong to an “image recording product” which may belong to a top-level “device.” Perhaps a future Google will provide this common vocabulary in addition to search services.
Such a generic description language already exists: it is RDF or Resource Definition Framework. RDF is “a framework for describing and interchanging metadata.” (XML.com) An RDF file holds information in an abstract layer on top of XML to provide a number of advantages over base XML:
RDF and other Semantic Web solutions carry enormous potential for the future of the World Wide Web. The current Web consists almost entirely of plain HTML pages with virtually no metadata. This forces search engines to search documents by looking for strings entered into search engines. This is equivalent to searching for a book by looking for individual words without being aware of the title, author, or category. The problem is greatly compounded by the fact that the Internet is much bigger than any library that has ever existed. Adding metadata to web documents has the potential to completely change the way information is accessed on the web. The combination of HTML tags and XML data via the XHTML language is one of the steps in the process. The use of RDF metadata will allow a user to formulate an intelligent search that returns exactly the criteria he searched for, whether from web pages or online marketplaces.
RDF is currently in widespread use in one niche area of the web: blogging. A blog is an online journal that features frequents posts chronologically arranged by date. Blogs are usually updated by automated software that requires little technical knowledge from the users. Because virtually all blogs are auto-generated by content management systems of some sort, they are ideal candidates for structured metadata. While many blogs provide RDF versions of their content (many provide several alternative “syndication” URL’s) most also use RSS, “a lightweight multipurpose extensible metadata description and syndication format.” (According to the official RSS v1.0 specification.) RSS, or Rich Site Summary is an XML application that conforms to the RDF specification for describing data. Thus, XML forms the basic syntax for storing the data, RDF forms a generic metadata description framework, and RSS provides an application for the specific function of providing summaries of a website’s content for syndication. RSS itself supports extensions, so a fourth (or fifth, since XML itself is a subset of SGML) level of specialization is possible with RSS extensions.
According to UserLand Software, there are about 500,000 blogs on the Web, which hold significant influence relative to their number. (UserLand’s founder Dave Winer is the author or co-author of XML-RPC, SOAP, RSS, OPML, and Radio Userland, one of the most popular newsreaders and blogging programs.) Like XML, the RSS format is relatively simple. Here is a sample RSS entry from the author’s blog that was dynamically generated by the blogging software and placed within the XHTML webpage:
<!--
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:trackback="http://madskills.com/public/xml/rss/module/trackback/"
xmlns:dc="http://purl.org/dc/elements/1.1/">
<rdf:Description
rdf:about="http://rationalmind.net/david/archives/never_thought_it_would_happen_to_me_000721.html"
trackback:ping="http://rationalmind.net/cgi-bin/mt-tb.cgi/721"
dc:title="Never thought it would happen to me…"
dc:identifier="http://rationalmind.net/david/archives/never_thought_it_would_happen_to_me_000721.html"
dc:subject="My Life..."
dc:description="I discovered today that my credit card racked up $2,500 in charges during the last..."
dc:creator="David"
dc:date="2003-12-02T17:53:23-06:00" />
</rdf:RDF>
-->
The first URI in the sample (xmlns:rdf= http://www.w3.org/1999/02/22-rdf-syntax-ns#)
contains a reference to the XML Schema for the RDF protocol. It described references the all RDF tags. This is made possible by XML Namespaces, a method of referencing external schemas for a subset of the XML tags in a document. All tags prefixed with “rdf:” now belong to the RDF namespace described at the above URI. The second URI contains a link to the TrackBack extension of RSS, which provides web service functionality to let other blogs know when a post of theirs has been referenced . The third URI contains the schema for the blog-related tags (those prefixed with “dc:”) The RSS schema is not included because a separate RSS file is published each time the blog is updated. The RSS file contains a reference to the RSS file, while the XHTML document itself contains a reference to the XHTML 1.0 DTD that contains definitions for all XHTML tags.
Trying to learn all the XML DTD’s, and XML Schemas used in a blog can quickly overwhelm a non-technical user. Fortunately, there is no need to do so: if the blog and the RSS newsreader agree on the same schema, (which the newsreader can verify by looking the URI of the schema) the newsreader will be able to access the blog’s content.
There are a number of ways to use the syndicated content of a website. Websites like Syndic8 (http://syndic8.com/) and Weblogs.com provide directories of syndicated websites. Not just blogs are included: virtually all major media sources either provide their own syndication URI’s, or are syndicated by third parties, such as Moreover Technologies (http://moreover.com/) Some services (such as Moreover) offer both free and paid information aggregation services which customers may consume as aggregate RSS feeds. While many blogs are part-time hobbies of students and tech-savy web users, many are also run by Industry professionals or corporate portals, such as Microdoc News (http://microdoc-news.info/) Blog-like information aggregation services can also be valuable knowledge management tools within a company as a means of distributing news and creating company-wide and industry-wide newsletters.
Another growing use of XML technologies is for instant messaging. Many companies are recognizing that instant messaging can be a useful collaboration tool, but they want to retain control over how instant messaging services are used and what features are available to employees. Because sensitive business information may be discussed over IM, companies want to host their own instant messaging servers and to encrypt the conversations between clients. They would also like to be able to provide advanced features such as virtual group meetings, application sharing, and distribution lists integrated with the company’s employee database.
By providing an open and extensible messaging framework, XML applications can provide effective instant messaging solutions. For example, Jabber (http://www.jabber.org/) provides an XML-based instant messaging solution that uses streaming XML protocols. By leveraging XML Namespaces, Jabber clients can add functionality extensions on top of the core protocols and automatically negotiate a set of compatible protocols with other clients. As the Jabber website explains, extensions allow clients to do much more than send messages: “applications beyond IM include network management, content syndication, collaboration tools, file sharing, gaming, and remote systems monitoring.”
A similar application of XML messaging for group collaboration is as a calendar sharing application. SherpaView (http://www.sherpasuite.com/) is an XML-based calendar made by Semaview. It allows enterprises to share their calendars online as a web service and allow trusted parties to view and schedule appointments during free times no matter what scheduling software or operating system they use.
The potential of the Semantic Web has barely been explored today. The standards that make the Semantic Web possible are currently under active development. There are several competing initiatives for the creation of a universal business language – a key step towards the creation of online marketplaces. However the widespread adoption of Semantic Web solutions before the standards they are based on are finalized is a strong indication of public and business support the Semantic Web.
Future applications of the Semantic Web have the potential to fundamentally change how corporations and consumers interact with information systems and engage in commerce – online and off. Tim Berners-Lee, the creator of the World Wide Web and founder of the W3C is one of the major evangelists for the Semantic Web. He envisions a World Wide Web that allows devices online and in the real world to discover and intelligently interact with each other somewhat like how Microsoft’s Universal Plug in Play automatically detects and configures new hardware devices. Intelligent agents representing individuals and companies would interact with other agents to request information and news, find and purchase goods from vendors, and participate in distributed collaborative networks. Public and corporate search engines would use Semantic Web-aware agents to provide intelligent search results based on a common ontology. Web browsers would become interactive data-viewing clients rather than merely presenting static documents. The future of business and the Internet is definitely semantic.
6. W3C Semantic Web Page
http://www.w3.org/2001/sw
7. The Semantic Web lifts off
http://www.ercim.org/publication/Ercim_News/enw51/berners-lee.html
http://www.w3.org/TR/rdf-primer/#intro
http://www.semanticweb.org/
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=ubl
http://www.xmledi-group.org/
http://xml.coverpages.org/xmlAndSemanticWeb.html
http://www.w3.org/2002/ws/
https://uddi.ibm.com/ubr/registry.html
http://uddi.sap.com/
17. XML.com: What is RDF?
http://www.xml.com/lpt/a/2001/01/24/rdf.html
http://www.semaview.com
http://www.userland.com/
19. Jabber Software Foundation
http://www.jabber.org/
http://www.w3.org/People/Berners-Lee/