14 Expressions with Spatial and Text Predicates

14.1 Expressions with Spatial Predicates

The expressions stored in a column of a table may contain spatial predicates defined on SDO_GEOMERTY attributes. This section describes an application for spatial predicates using the Car4Sale example introduced in Chapter 11. For this purpose, the information published for each car going on sale includes a Location attribute in addition to the Model, Price, Mileage, and Year attributes. The Location attribute contains geographical coordinates for the vehicle's location, as an instance of the SDO_GEOMETRY data type.

Using the Location attribute, the consumer interested in a vehicle can restrict the search only to the vehicles that are within a specified distance, say half a mile, of his own location. This can be specified using the following spatial predicate involving the SDO_WITHIN_DISTANCE operator:

SDO_WITHIN_DISTANCE(
  Location, 
  SDO_GEOMETRY(
    2001, 8307, 
    SDO_POINT_TYPE(-77.03644, 37.89868, NULL), NULL, NULL
  ) , 
  'distance=0.5 units=mile'
) = 'TRUE' 

Note that spatial predicates are efficiently evaluated with the help of spatial indexes. Section 14.1.1 and Section 14.1.2 will describe how to specify spatial predicates in arbitrary expressions and how to ensure the predicates are evaluated using appropriate spatial indexes.

CREATE OR REPLACE TYPE Car4Sale AS OBJECT
                   (Model    VARCHAR2(20),
                    Year     NUMBER,
                    Price    NUMBER,
                    Mileage  NUMBER,
                    Location MDSYS.SDO_GEOMETRY);
/
BEGIN
  dbms_expfil.create_attribute_set (attr_set => 'Car4Sale',
                                    from_type => 'YES');
END;
/

INSERT INTO user_sdo_geom_metadata VALUES ('CAR4SALE','LOCATION',
     mdsys.sdo_dim_array(
          mdsys.sdo_dim_element('X',  -180, 180, 0.5),
          mdsys.sdo_dim_element('Y',  -90, 90, 0.5)), 8307);

Model = 'Taurus' and Price < 15000 and Mileage < 25000 and 
  SDO_WITHIN_DISTANCE (Location,
      SDO_GEOMETRY(2001, 8307, 
        SDO_POINT_TYPE(-77.03644, 37.89868, NULL), NULL, NULL), 
      'distance=0.5 units=mile') = 'TRUE'

Model = 'Taurus' and Price < 15000 and Mileage < 25000 and 
  SDO_RELATE (Location,
      SDO_GEOMETRY(2001, 8307, NULL, SDO_ELEM_INFO_ARRAY(1, 1003, 3), 
         SDO_ORDINATE_ARRAY(-77.03644, 37.89868, -75, 39), 
      'mask=anyinteract') = 'TRUE'

SELECT * FROM Consumer WHERE
  EVALUATE (Interest,
    sys.anyData.convertObject(
       Car4Sale('Mustang', 2002, 20000, 250000,
            SDO_GEOMETRY(2001, 8307,  
            sdo_point_type(-77.03644, 38.9059284, null), null, null)))
  ) = 1;

14.2 Expressions with Text Predicates

The Text predicates in the stored expressions are specified using the CONTAINS operator within the SQL-WHERE clause syntax. The text predicates include the document to be searched (through an attribute name) and a text query expression. The text query expression specifies the text pattern to be searched within the document and it is represented using the CONTEXT grammar (see Oracle Text Application Developer's Guide).

Model='Taurus' and Price < 15000 and Mileage < 25000 AND
    CONTAINS (InsReport, 'near(water, damage), 4)') = 1

The Text classification engine currently allows storing (text) query expressions in a column of a table and processing them for incoming documents. The incoming documents are matched with the stored expressions using a MATCHES operator within SQL queries. The metadata required to match the document with the stored expressions is obtained from the CTXRULE index defined on the column storing the query expressions. This metadata includes the information about the text preferences such as the type of LEXER to use and the list of STOP words. This metadata is assigned to the CTXRULE index at the time of index creation and hence the MATCHES operator is only operational when the CTXRULE index exists on the column.

The support for text predicates in stored expressions is provided through the integration of Text classification engine and the Expression Filter feature. In these stored expressions, the text predicates can be combined with other text predicates, scalar or XML predicates using the SQL WHERE clause syntax. The text predicates are specified using the CONTAINS operator, which identifies the text attribute on which the text query expression should be evaluated. At the time of expression evaluation, the text query expression is matched with the document assigned to the text attribute. The text preferences required for this matching operation are assigned to the text attributes as a form of metadata.

In an attribute set created for an expression column, a text attribute is identified as an attribute of CLOB or VARCHAR data type with associated text preferences. Such attributes can be used as the first argument to the CONTAINS operator to form text predicates within the stored expressions. The text preferences for an attribute are specified using an instance of EXF$TEXT type, which accepts the text preferences in string format, for example: (LEXER insrpt_lexer WORDLIST insrpt_wordlist). The preferences specified through this argument are used for parsing the document bound to the attribute and for indexing the text query expressions (using ctxsys.CTXRULE Indextype). Alternately, an EXF$TEXT instance with an empty preferences string can be assigned to use default preferences for a CTXRULE index (See Oracle Text Reference).

begin
  dbms_expfil.create_attribute_set(attr_set => 'Car4Sale');
  
  // create scalar attributes 
  dbms_expfil.add_elementary_attribute(
                                   attr_set  => 'Car4Sale',
                                   attr_name => 'Model',
                                   attr_type => 'VARCHAR2(30)');

  dbms_expfil.add_elementary_attribute(
                                   attr_set  => 'Car4Sale',
                                   attr_name => 'Price',
                                   attr_type => 'NUMBER');

  . . .
  // create text attribute
  dbms_expfil.add_elementary_attribute(
                                   attr_set  => 'Car4Sale',
                                   attr_name => 'InsReport',
                                   attr_type => 'CLOB'
                                   attr_type => 
                                     exf$text(
                                     'LEXER insrpt_lexer
                                      WORDLIST insrpt_wordlist'));
end;

For the attribute set previously described, the LEXER preference insrpt_lexer and the WORDLIST preference insrpt_wordlist should be created using the CTX_DDL package (see Oracle Text Reference).

Figure 14-1 Text Predicate in the Stored Expression Using the CONTAINS Operator

Description of "Figure 14-1 Text Predicate in the Stored Expression Using the CONTAINS Operator"

When the attribute set with a text attribute is used as the metadata for an expression column, the expressions stored in the column can include text predicates. The text query expression used within the text predicates use a subset of the CONTEXT grammar to support proximity searches, theme searches, and so forth, with the use of CONTEXT operators: ABOUT, AND, NEAR, NOT, OR, STEM, WITHIN, and THESAURUS. (See the Document Classification Section in Oracle Text Application Developer's Guide for complete syntax). The scalar predicates in the stored expressions are validated for syntactic and semantic correctness when the expressions are inserted into a column of expression data type. However, the text query expressions specified as the second argument to the CONTAINS operator are not validated until the time of index creation or index maintenance. Any text query expression errors identified during index creation are reported through CTX_USER_INDEX_ERRORS view as described next.

Unlike in the case of expressions containing purely scalar predicates, the expressions containing predicates on text attributes cannot be evaluated when the Expression Filter index is not defined on the column storing expressions. At the time of index creation, all the text attributes are added to the indexed predicate groups and CTXRULE indexes are created on the predicate table for each of these predicate groups. The CTXRULE index creation for the text predicate groups also identifies any errors in the corresponding text query expressions and these errors are reported in the CTX_USER_INDEX_ERRORS view (see Oracle Text Reference). However, the errors reported in this view refer to the rows in the Expression Filter index's predicate table structure. These errors can be mapped back to the rows in the user table using the USER_EXPFIL_TEXT_INDEX_ERRORS view.

Once the index is defined, the stored expressions can be evaluated using the EVALUATE operator in a SQL query. When the attribute set is defined with one or more text attributes, the data item passed to the EVALUATE operator consists of corresponding values (VARCHAR or CLOB) assigned to these attributes.

SELECT * FROM Consumer
WHERE EVALUATE (Interest, 
       AnyData.convertObject(
           Car4Sale('Mustang',19000,25000,2001,
                    '...4 star crash test rating ...') = 1

The previous query with the EVALUATE operator identifies all the expressions that are true based on the text and the non-text predicates.

When an Expression Filter index is defined on the column storing expressions, some of the most common and selective scalar and XPath predicates are identified by the end-user and the predicate table is created to accommodate such predicates. While selecting the predicate groups for an Expression Filter index, the predicates involving a text attribute are always included in the indexed predicate groups. The predicate table is created with a VARCHAR column to store the text query expressions specified for a text attribute and a CTXRULE index is defined on this column. These indexes along with the bitmap indexes defined on the other indexed predicates process the expressions for a data item efficiently.

When a query with EVALUATE operator is issued, the values from the data item passed in are bound into a query on the predicate table. The predicate table query uses a MATCHES operator on the columns storing text query expressions to match the incoming documents with the stored text query expressions. The MATCHES operator internally makes use of the CTXRULE index to process the text query expressions efficiently and return a subset of expressions that match the document. The results from matching the documents with text query expressions are combined with results from matching other scalar and XPath predicates to narrow down the working set to a set of candidate expressions that are true based on all indexed predicate groups. The processing of stored predicate groups and the sparse predicates remain unchanged with the inclusion of text predicates in the expression set.

Delayed DML Maintenance for Text Predicates

The modifications made to the data stored in Expression data type column are all transactional in nature. That is, any modifications made to the expression column are automatically reflected in the corresponding Expression Filter index. However, the modifications made to the text query expression within the stored expressions are not immediately reflected in the corresponding CTXRULE index. The index maintenance is forced by manually synchronizing the index and until the CTXRULE index is synchronized, the Expression Filter index may return incorrect results (based on the old text predicates and new scalar predicates). One or more CTXRULE indexes defined for expression sets stored in a table can all be synchronized using a DBMS_EXPFIL.SYNC_TEXT_INDEXES procedure call. The name of the user table with one or more expression columns is passed as the only argument to this procedure call.

begin
  dbms_expfil.sync_text_indexes (expr_tab => 'Consumer');
end;

You must have EXECUTE privileges on the CTX_DDL package for successful completion of the previous command. The call to the DBMS_EXPFIL.SYNC_TEXT_INDEXES procedure processes all the newly added or modified text predicates in the expression set and synchronizes the CTXRULE indexes accordingly. Any text predicate errors identified in this process are reported through the USER_EXPFIL_TEXT_INDEX_ERRORS view.