1 Introduction to Oracle XML DB

XMLType Storage

Figure 1-2 shows XMLType storage in Oracle XML DB.

When XML schemas are registered with Oracle XML DB, a set of default tables are created and used to store XML instance documents associated with the XML schema. These documents can be viewed and accessed in Oracle XML DB Repository.

Data in XMLType views can be stored in local or remote tables. Remote tables can be accessed through database links.

XMLType tables and views can be indexed using XMLIndex, B-tree, function-based, and Oracle Text indexes.

You can access data in Oracle XML DB Repository using any of the following:

• HTTP(S), through the HTTP protocol handler

• WebDAV and FTP, through the WebDAV and FTP protocol server

• SQL, through Oracle Net Services, including Java Database Connectivity (JDBC)

Oracle XML DB supports XML data messaging using Oracle Streams Advanced Queuing (AQ) and Web Services.

APIs for XML

Table 1-1 lists the reference documentation for the PL/SQL, C, and C++ Application Programming Interfaces (APIs) that you can use to manipulate XML documents and data. The main reference for PL/SQL, C, and C++ APIs is Oracle Database PL/SQL Packages and Types Reference.

Catalog Views Related to XML

Table 1-2 lists the catalog views related to XML. Information about a given view can be obtained by using the SQL command DESCRIBE. Example:

DESCRIBE USER_XML_SCHEMAS

USER_XML_SCHEMAS

ALL_XML_SCHEMAS

DBA_XML_SCHEMAS

USER_XML_TABLES

ALL_XML_TABLES

DBA_XML_TABLES

USER_XML_TAB_COLS

ALL_XML_TAB_COLS

DBA_XML_TAB_COLS

USER_XML_VIEWS

ALL_XML_VIEWS

DBA_XML_VIEWS

USER_XML_VIEW_COLS

ALL_XML_VIEW_COLS

DBA_XML_VIEW_COLS

Views RESOURCE_VIEW and PATH_VIEW

Oracle XML DB views RESOURCE_VIEW and PATH_VIEW provide SQL access to data in Oracle XML DB Repository through protocols such as FTP and WebDAV. View PATH_VIEW has one row for each unique path in the repository; view RESOURCE_VIEW has one row for each resource in the repository.

The Oracle XML DB resource API for PL/SQL, DBMS_XDB, provides query and DML functions. It is based on RESOURCE_VIEW and PATH_VIEW.

Overview of Oracle XML DB Repository

Oracle XML DB Repository is a component of Oracle Database that is optimized for handling XML data. The Oracle XML DB repository contains resources, which can be either folders (directories, containers) or files. Each resource has these properties:

• It is identified by a path and name.

• It has content (data), which can be XML data but need not be.

• It has a set of system-defined metadata (properties), such as Owner and CreationDate, in addition to its content. Oracle XML DB uses this information to manage the resource.

• It might also have user-defined metadata: information that is not part of the content, but is associated with it.

• It has an associated access control list that determines who can access the resource, and for what operations.

Although Oracle XML DB Repository treats XML content specially, you can use Oracle XML DB Repository to store other kinds of data, besides XML; you can use the repository to access any data that is stored in Oracle Database.

Oracle XML DB Repository Architecture

Figure 1-3 describes the Oracle XML DB Repository architecture. You can access the repository in SQL, for example, using the RESOURCE_VIEW API. In addition to the resource information, the RESOURCE_VIEW also contains a Path column, which holds the paths to each resource.

See Also:

• Chapter 21, "Accessing Oracle XML DB Repository Data"

• Chapter 25, "SQL Access Using RESOURCE_VIEW and PATH_VIEW"

Figure 1-3 Oracle XML DB Repository Architecture

Description of "Figure 1-3 Oracle XML DB Repository Architecture"

How Does Oracle XML DB Repository Work?

The relational model table-row-column metaphor, is accepted as an effective mechanism for managing structured data. The model is not as effective for managing semi-structured and unstructured data, such as document- or content-oriented XML. For example, a book is not easily represented as a set of rows in a table. It is more natural to represent a book as a hierarchy, book:chapter:section:paragraph, and to represent the hierarchy as a set of folders and subfolders.

• A hierarchical metaphor manages document-centric XML content. Relational databases are traditionally poor at managing hierarchical structures and traversing a path or URL. Oracle XML DB provides a hierarchically organized repository that can be queried and through which document-centric XML content can be managed.

• A hierarchical repository index speeds up folder and path traversals. Oracle XML DB includes a patented hierarchical index that speeds up folder and path traversals in Oracle XML DB Repository. The hierarchical repository index is transparent to end users, and lets Oracle XML DB perform folder and path traversals at speeds comparable to or faster than conventional file systems.

• You can access XML documents in Oracle XML DB Repository using standard connect-access protocols such as FTP, HTTP(S), and WebDAV, in addition to languages SQL, PL/SQL, Java, and C. The repository provides content authors and editors direct access to XML content stored in Oracle Database.

• A resource in this context is a file or folder, identified by a URL. WebDAV is an IETF standard that defines a set of extensions to the HTTP protocol. It lets an HTTP server act as a file server for a DAV-enabled client. For example, a WebDAV-enabled editor can interact with an HTTP/WebDAV server as if it were a file system. The WebDAV standard uses the term resource to describe a file or a folder. Each resource managed by a WebDAV server is identified by a URL. Oracle XML DB adds native support to Oracle Database for these protocols. The protocols were designed for document-centric operations. By providing support for these protocols, Oracle XML DB lets Windows Explorer, Microsoft Office, and products from vendors such as Altova, Macromedia, and Adobe work directly with XML content stored in Oracle XML DB Repository. Figure 1-4 shows the root-level directory of the repository as seen from Microsoft Web Folder.

Figure 1-4 Microsoft Web Folder View of Oracle XML DB Repository

Description of "Figure 1-4 Microsoft Web Folder View of Oracle XML DB Repository"

See Also:

Chapter 3, "Using Oracle XML DB"

Hence, WebDAV clients such as Microsoft Windows Explorer can connect directly to Oracle XML DB Repository. No additional Oracle Database or Microsoft-specific software or other complex middleware is needed. End users can work directly with Oracle XML DB Repository using familiar tools and interfaces.

LSNRCTL for 32-bit Windows: Version 11.1.0.5.0 - Production on 20-AUG-2007 16:02:34
 
Copyright (c) 1991, 2007, Oracle.  All rights reserved.
 
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=IPC)(KEY=EXTPROC1521))) STATUS of the LISTENER
--------------------------------------------------------------------------------------------
Alias                          LISTENER
Version                        TNSLSNR for 32-bit Windows: Version 11.1.0.5.0 - Beta
Start Date                     20-JUN-2007 15:35:40
Uptime                         0 days 16 hr. 47 min. 42 sec
Trace Level                    off
Security                       ON: Local OS Authentication
SNMP                           OFF
Listener Parameter File        C:\oracle\product\11.1.0\db_1\network\admin\listener.ora
Listener Log File              c:\oracle\diag\tnslsnr\quine-pc\listener\alert\log.xml
 
Listening Endpoints Summary... 
(DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(PIPENAME=\\.\pipe\EXTPROC1521ipc)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=quine-pc.us.mycompany.com)(PORT=1521)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=quine-pc.us.mycompany.com)
             (PORT=21))(Presentation=FTP)(Session=RAW)) (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=quine-pc.us.mycompany.com)
             (PORT=443))(Presentation=HTTP)(Session=RAW))
Services Summary...
Service "orcl.us.oracle.com" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
Service "orclXDB.us.oracle.com" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
Service "orcl_XPT.us.oracle.com" has 1 instance(s).
  Instance "orcl", status READY, has 1 handler(s) for this service...
The command completed successfully

Programmatic Access to Oracle XML DB (Java, PL/SQL, and C)

All Oracle XML DB functionality is accessible from C, PL/SQL, and Java. These are the most popular ways to build Web-based applications:

• Using servlets and Java Server Pages (JSP). A typical API accesses data using Java Database Connectivity (JDBC).

• Using Extensible Stylesheet Language (XSL) plus XML Server Pages (XSP). A typical API accesses data in the form of XML documents that are processed using a Document Object Model (DOM) API implementation.

Oracle XML DB supports both of these styles of application development. It provides Java, PL/SQL, and C implementations of the DOM API.Applications that use JDBC, such as those based on servlets, need prior knowledge of the data structure they are processing. Oracle JDBC drivers allow you to access and update XMLType tables and columns, and call PL/SQL procedures that access Oracle XML DB Repository.Applications that use DOM, such as those based on XSLT transformations, typically require less knowledge of the data structure. DOM-based applications use string names to identify pieces of content, and must dynamically walk through the DOM tree to find the required information. For this, Oracle XML DB supports the use of the DOM API to access and update XMLType columns and tables. Programming to a DOM API is more flexible than programming through JDBC, but it may require more resources at run time.

XMLType Data Type

XMLType is a native data type for XML data. It provides methods that allow operations such as XML schema validation and XSL transformation on XML content. You can use XMLType as you would any other data type. For example, you can use XMLType when you do any of the following:

• Creating a column in a relational table

• Declaring PL/SQL variables

• Defining and calling PL/SQL procedures and functions

XMLType is an object type, so you can also create a table of XMLType instances. By default, an XMLType table or column can contain any well-formed XML document.

XMLType Tables and Columns Can Conform to an XML Schema

XMLType tables or columns can be constrained to conform to an XML schema. This has several advantages:

• The database ensures that only XML documents that validate against the XML schema are stored in the column or table.

• Since the contents of the table or column conform to a known XML structure, Oracle XML DB can use the information contained in the XML schema to provide optimized query and update processing of the XML.

• You have the option of storing the XML document content using structured storage. This decomposes the document and stores it as a set of object-relational objects. The object-relational model used to store the document is derived from the XML schema.

XMLType API

Data type XMLType provides the following:

• Constructors, which you can use to create an XMLType instance from a VARCHAR, CLOB, BLOB, or BFILE value.

• XML-specific methods that operate on XMLType instances. These include the following:

  • extract()– Extract a subset of nodes contained in the XMLType instance.

  • existsNode() – Check whether or not a particular node exists in the XMLType instance.

  • schemaValidate() – Validate the content of the XMLType instance against an XML schema.

  • transform() – Perform an XSL transformation on the content of an XMLType instance.

XML Schema Support

Support for the Worldwide Web Consortium (W3C) XML Schema Recommendation is a key feature in Oracle XML DB. XML Schema specifies the structure, content, and certain semantics of a set of XML documents. It is described in detail at http://www.w3.org/TR/xmlschema-0/.

XML Schema Unifies Document Modeling and Data Modeling

In Oracle XML DB, you can use XML Schema to automatically create database tables and data types for storing XML data. You can thus use a standard data model for all of your data, whether the data is structured, unstructured, or semi-structured.

Create XMLType Tables and Columns, Ensure DOM Fidelity

You can create XML schema-based XMLType tables and columns and optionally specify that they conform to pre-registered XML schemas, maintaining DOM fidelity.

Use XMLType Views to Wrap Relational Data

You can also choose to wrap existing relational and object-relational data into XML format using XMLType views. You can store an XMLType object as an XML object that is based on an XML schema or not based on an XML schema:

• XML schema-based objects. These are stored in Oracle XML DB tables, columns, or views using binary XML storage, CLOB storage, or object-relational storage.

• Non-schema-based objects. These are stored in Oracle XML DB as LOBs only.

You can map from incoming XML documents to XMLType storage. The mapping is specified in an XML schema, which you register with Oracle XML DB. After it is registered, the XML schema can be referenced using its URL.

W3C Schema for Schemas

The W3C Schema Working Group publishes an XML schema, often referred to as the "schema for schemas". This XML schema provides the definition, or vocabulary, of the XML Schema language. An XML schema definition (XSD) is an XML document, that is compliant with the vocabulary defined by the schema for schemas. An XML schema uses vocabulary defined by W3C XML Schema Working Group to create a collection of type definitions and element declarations that declare a shared vocabulary for describing the contents and structure of a new class of XML documents.

XML Schema Base Set of Data Types Can be Extended

The XML Schema language provides strong typing of elements and attributes. It defines numerous scalar data types. This base set of data types can be extended to define more complex types, using object-oriented techniques such as inheritance and extension. The XML Schema vocabulary also includes constructs that you can use to define complex types, substitution groups, repeating sets, nesting, ordering, and so on. Oracle XML DB supports all of the constructs defined by the XML Schema Recommendation, except for redefines.

XML schemas are commonly used as a mechanism for checking (validating) whether XML instance documents conform with their specifications. Oracle XML DB includes XMLType methods and SQL functions that you can use to validate XML documents against an XML schema.

XMLType Storage Models

XMLType is an abstract data type that provides different storage models to best fit your data and your use of it. As an abstract data type, your applications and database queries gain in flexibility: the same interface is available for all XMLType operations. Because different storage (persistence) models are available, you can tailor performance and functionality to best fit the kind of XML data you have and the pattern of its use. One key decision to make when using Oracle XML DB for persisting XML data as XMLType is thus which storage model to use for which XML data.

You can change XMLType storage from one model to another, using database import/export (see Chapter 36, "Exporting and Importing XMLType Tables"); your application code does not need to change. You can change XML storage options when tuning your application.

XMLType tables and columns can be stored in these ways:

• Structured storage – XMLType data is stored as a set of objects. This is also referred to as object-relational storage and object-based persistence.

• Unstructured storage – XMLType data is stored in Character Large Object (CLOB) instances. This is also referred to as CLOB storage and text-based persistence.

• Binary XML storage – XMLType data is stored in a post-parse, binary format specifically designed for XML data. Binary XML is compact, post-parse, XML schema-aware XML. This is also referred to as post-parse persistence.

Unstructured and binary XML storage each provide two LOB storage options, SecureFile and BasicFile.

You can mix storage models, using one model for one part of an XML document and a different model for another part. The mixture of structured and unstructured storage is sometimes called hybrid storage. What is true about structured storage is true about the structured part of hybrid storage. What is true about unstructured storage is true about the unstructured part of hybrid storage.

XMLType has multiple storage models, and some models can be configured in more than one way. Each model has its advantages, depending on the context. Each model has one or more types of index that are appropriate for it.

The first thing to consider, when choosing an XMLType storage model, is the nature of your XML data and the ways you use it. A spectrum exists, with data-centric use of highly structured data at one end, and document-centric use of highly unstructured data at the other. The first question to ask yourself is this: Is your use case primarily data-centric or document-centric?

• Data-centric – Your data is, in general, highly structured, with relatively static and predictable structure, and your applications take advantage of this structure. Your data conforms to an XML schema.

• Document-centric – Two cases:

  • If your data is relatively structured, your applications do not take advantage of that structure. That is, you treat the data as if it were without structure.

  • Your data is generally without structure or of variable structure. Document structure can vary over time (evolution). Content is mixed (semi-structured): many elements contain both text nodes and child elements. Many XML elements can be absent or can appear in different orders. Documents might or might not conform to an XML schema.

Once you've located the data-centric or document-centric half of the spectrum that is appropriate for your use case and data, consider whether your case is at an end of the spectrum or closer to the middle. That is, just how data-centric or document-centric is your case?

• Employ object-relational (structured) storage for purely data-centric uses. A typical example of this use case would be an employee record (fields employee number, name, address, and so on). Use B-tree indexing with object-relational storage.

• Employ hybrid storage if your data is composed primarily of invariable XML structures, but it does contain some variable data; that is, it contains a predictably few mixed-content elements. A typical example of this use case would be an employee record that includes a free-form resume. Index the structured and unstructured parts of your data separately, using appropriate indexes for each part.

• Employ binary XML storage or CLOB-based (unstructured) storage for all document-centric use cases. In terms of indexing, we can distinguish two use cases:

  • If your data contains some predictable, fixed structures that you query frequently, then you can employ function-based indexes on those parts. A typical example of this use case would be a free-form specification, with author, date, and title fields.

  • Otherwise, for general indexing of document-centric XML data, use XMLIndex indexing. A typical example of this use case would be an XML Web document or a book chapter.

  In all document-centric cases, you can additionally use Oracle Text indexing for full-text queries.

These considerations are summarized in Figure 1-5. The figure shows the spectrum of use cases, from most data-centric, at the left, to most document-centric, at the right. The table in the figure classifies use cases and shows the corresponding storage models and indexing methods.

Figure 1-5 XML Use Cases and XMLType Storage Models

Description of "Figure 1-5 XML Use Cases and XMLType Storage Models"

See Chapter 5, "Indexing XMLType Data" for more information about indexing XML data. In particular, note that some types of indexing are complementary or orthogonal, so you can use them together.

The following list and Table 1-3 outline some of the advantages of each storage model.

• Structured (object-relational) storage advantages over unstructured storage include optimized memory management, reduced storage requirements, B-tree indexing, and in-place updates. These advantages are at a cost of increased processing overhead during ingestion and retrieval of XML data, and reduced flexibility in the structure of the XML that can be managed by a given XMLType table or column. Structural flexibility is reduced, because data and metadata (such as column names) are separated in object-relational storage; instance structures cannot vary easily. Structured storage is particularly appropriate for highly structured data whose structure does not vary, if this maps to a manageable number of database tables and joins.

• Unstructured (CLOB) storage enables higher throughput than structured storage when inserting and retrieving entire XML documents. No data conversion is needed, so the same format can be used outside the database. Unstructured storage also provides greater flexibility than structured storage in the structure of the XML that can be stored. Unstructured storage is particularly appropriate for document-centric use cases. These advantages can come at the expense of certain aspects of intelligent processing: in the absence of indexing, there is little that the database can do to optimize queries or updates on XML data that is stored in a CLOB instance. In particular, the cost of XML parsing (often implicit) can significantly impact query performance. Indexing with XMLIndex can improve the performance of queries against unstructured storage.

• Binary XML storage provides more efficient database storage, updating, indexing, and fragment extraction than unstructured storage. It can provide better query performance than unstructured storage — it does not suffer from the XML parsing bottleneck (it is a post-parse persistence model). Like structured storage, binary XML storage is aware of XML Schema data types and can take advantage of native database data types. Like unstructured storage, no data conversion is needed during database insertion or retrieval. Like structured storage, binary XML storage allows for piecewise updates. Because binary XML data can also be used outside the database, it can serve as an efficient XML exchange medium, and you can off load work from the database to increase overall performance in many cases. Like unstructured storage, binary XML data is kept in document order. Like structured storage, data and metadata can, using binary storage, be separated at the database level, for efficiency. Like unstructured storage, however, binary XML storage allows for intermixed data and metadata, which lets instance structures vary. Binary XML storage allows for very complex and variable data, which in the structured-storage model could necessitate using many database tables. Unlike the other XMLType storage models, you can use binary storage for XML schema-based data even if the XML schema is not known beforehand, and you can store multiple XML schemas in the same table and query across common elements.

  Binary XML storage is the closest thing to a universal storage model for XML data — you can use it effectively for a very wide range of use cases, from document-centric to data-centric.

When XMLType is stored object-relationally, the XMLType instances contain hidden columns that store information about the XML data that does not fit into the SQL object model.

XML/SQL Duality

A key objective of Oracle XML DB is to provide XML/SQL duality. XML programmers can leverage the power of the relational model when working with XML content and SQL programmers can leverage the flexibility of XML when working with relational content. This lets you use the most appropriate tools for a particular business problem.

XML/SQL duality means that the same data can be exposed as rows in a table and manipulated using SQL or exposed as nodes in an XML document and manipulated using techniques such as DOM and XSL transformation. Access and processing techniques are independent of the underlying storage format.

These features provide simple solutions to common business problems. For example:

• You can use Oracle XML DB SQL functions to generate XML data directly from a SQL query. You can then transform the XML data into other formats, such as HTML, using the database-resident XSLT processor.

• You can access XML content without converting between different data formats, using SQL queries, on-line analytical processing (OLAP), and business-intelligence/data warehousing operations.

• You can perform text, spatial data, and multimedia operations on XML content.

SQL/XML INCITS Standard SQL Functions

Oracle XML DB provides the SQL functions defined in the SQL/XML standard. This standard is defined by specifications prepared by the International Committee for Information Technology Standards (INCITS) Technical Committee H2. INCITS is the main standards body for developing standards for the syntax and semantics of database languages, including SQL.

The SQL/XML standard is evolving, so the syntax and semantics of its functions are subject to change in the future. The Oracle XML DB implementation of SQL/XML functions will evolve accordingly.

SQL/XML functions fall into two categories:

• Functions that you can use to query and access XML content as part of normal SQL operations.

• Functions that you can use to generate XML data from the result of a SQL query.

With SQL/XML functions you can address XML content in any part of a SQL statement. These functions use XQuery or XPath expressions to traverse the XML structure and identify the nodes on which to operate. The ability to embed XQuery and XPath expressions in SQL statements greatly simplifies XML access.

Rewriting of XQuery and XPath Expressions

SQL/XML functions and their corresponding XMLType methods use XQuery or XPath expressions to search collections of XML documents and to access a subset of the nodes contained within an XML document. In many cases, Oracle XML DB is able to rewrite such expressions to code that executes directly against the underlying database objects.

Unifying Data and Content

Most application data and Web content is stored in a relational database, a file system, or both. XML data is often used for data exchange, and it can be generated from a relational database or a file system. As the volume of XML data exchanged grows, the cost of regenerating this data grows, and these storage methods become less effective at accommodating XML content.

Figure 1-7 Unifying Data and Content: Some Common XML Architectures

Description of "Figure 1-7 Unifying Data and Content: Some Common XML Architectures"

Organizations often manage their structured data and unstructured data differently:

• Unstructured data, in tables, makes document access transparent and table access complex.

• Structured data, often in binary large objects (such as in BLOB instances), makes access more complex and table access transparent.

With Oracle XML DB, you can store and manage data that is structured, unstructured, and semi-structured using a standard data model and standard SQL and XML. You can perform SQL operations on XML documents and XML operations on object-relational (such as table) data.

Efficient Storage and Retrieval of Complex XML Documents

Users today face a performance barrier when storing and retrieving complex, large, or many XML documents. Oracle XML DB provides high performance and scalability for XML operations. The major performance features are:

• Native XMLType. See Chapter 4, "XMLType Operations".

• A lazily evaluated virtual DOM support. See Chapter 12, "PL/SQL APIs for XMLType".

• Database-integrated XQuery, XPath, and XSLT support. This support is described in several chapters, including Chapter 4, "XMLType Operations", Chapter 10, "Transforming and Validating XMLType Data", and Chapter 18, "Using XQuery with Oracle XML DB".

• XML schema-caching support. See Chapter 6, "XML Schema Storage and Query: Basic".

• Indexing – both full-text and XML. See Chapter 5, "Indexing XMLType Data" and Chapter 11, "Full-Text Search Over XML Data".

• A hierarchical index over Oracle XML DB Repository. See Chapter 21, "Accessing Oracle XML DB Repository Data".

Integrate Applications

Oracle XML DB enables data from disparate systems to be accessed through gateways and combined into one common data model. This reduces the complexity of developing applications that must deal with data from different stores.

Use XMLType Views If Your Data Is Not XML

XMLType views provide a way for you wrap existing relational and object-relational data in XML format. This is especially useful if, for example, your legacy data is not in XML format but you need to migrate to XML format. Using XMLType views, you do not need to alter your application code.

To use XMLType views, you must first register an XML schema with annotations that represent a bidirectional mapping between XML Schema data types and either SQL data types or binary XML encoding types. You can then create an XMLType view conforming to this mapping, by providing an underlying query that constructs instances of the appropriate types.