XTRAN Example — Interactive Data Query

Scenario — you are assessing the impact of changing the API for a PL/I function library.  You have mined the occurrence of declarations and calls and generated a cross-reference, using XTRAN rules , but you'd really like to explore the dependencies interactively.

XTRAN to the rescue!

The following example uses an XTRAN rules file comprising 141 non-comment lines of "meta-code" (XTRAN's rules language) to provide interactive query of information in the form of delimiter-separated name/value pairs (DSV format).

The rules took 2 hours to write and ½ hour to debug.  (That's right, only 2½ hours total!)

The rules used for this example are typically used to query information mined from code, data, or text using XTRAN.  Examples include:

• Module/function dependencies (as in this example)
• Symbol usage (global and/or local)
• Function calling trees
• Include/COPY file dependencies
• Locations of code quality problems
• Occurrences of data values in structured data
• Citations in text documents
• Programming language features used
• Results from forensic analysis of computer code, data, or text

More generally, the rules can be used to query any DSV name/value data, from any source.

Because the rules use XTRAN's in-memory data base facility, they can accommodate large quantities of data while providing quick response to queries.

You can optionally specify, via environment variables:

• What delimiting character is used in the data (defaulting to comma).
• What each entity is called (for instance, function, variable, or grammatical error).
• Which order the entities are in (entity/occurrence or occurrence/entity).  This allows you to provide the query access in either direction.

The rules accommodate qualifiers on occurrences that were detected by the code / data / text mining that generated the data.  For instance, in the case of declarations of, and references to, entities such as functions or data names, each query result is prefixed by a description of the occurrence, such as (in the case of a function) a declaration, a call, or a recursive call.

How can such powerful and generalized data manipulation be automated in only 2½ hours and 141 lines of XTRAN rules?  Because there is so much capability already available as part of XTRAN's rules language.  These rules take advantage of the following functionality:

• Text file input and output
• Text manipulation
• Text formatting
• Delimited list manipulation
• Environment variable manipulation
• Content-addressable data bases

The input to this example is from another XTRAN that uses XTRAN rules to extract module/function information from PL/I code.  Exactly the same information used in that example for reporting is used in this example for interactive query access.

The user interaction with XTRAN shown below is untouched.

---

Process Flowchart

Here is a flowchart for this process, in which the elements are color coded:

• BLUE for XTRAN versions (runnable programs)
• ORANGE for XTRAN rules (text files)
• PURPLE for text data files

---

Interaction with XTRAN (this means user input):

Enter function to do, or ? for list [done]:  ?<ENTER>

    proc1
    proc2
    proc3
    proc4

Enter function to do, or ? for list [done]:  proc3<ENTER>

    ExtD: demmdf1.pli(17)
    Call: demmdf1.pli(22)
    Call: demmdf1.pli(26)
    Call: demmdf1.pli(31)
    ExtD: demmdf2.pli(10)
    Call: demmdf2.pli(14)
    Call: demmdf2.pli(15)
    Call: demmdf2.pli(17)

Enter function to do, or ? for list [done]:  proc2<ENTER>

    ExtD: demmdf1.pli(12)
    Call: demmdf1.pli(21)
    Call: demmdf1.pli(24)
    Call: demmdf1.pli(28)
    Decl: demmdf2.pli(13)
    Recu: demmdf2.pli(16)

Enter function to do, or ? for list [done]:  proc5<ENTER>

<BELL>?Function "proc5" not seen!

Enter function to do, or ? for list [done]:  proc4<ENTER>

    NstD: demmdf1.pli(23)
    Recu: demmdf1.pli(25)

Enter function to do, or ? for list [done]:  proc1<ENTER>

    StaD: demmdf1.pli(20)
    Recu: demmdf1.pli(27)
    Recu: demmdf1.pli(30)

Enter function to do, or ? for list [done]:  <ENTER>

Interactive query like this can often provide insights into data that are difficult to achieve otherwise, and can also permit a stepwise refinement of the investigation.  Here are some observations drawn from the session shown above:

• proc1's code is in demmdf1.pli, where it recursively calls itself twice.  Note that its declaration is marked as "static", which explains why we don't see it called in any other analyzed module.
• proc2's code is in demmdf2.pli, where it recursively calls itself.  It's also externed and called in demmdf1.pli.
• We know that proc3 is an external function, since the only declarations of it in the analyzed code are external (in demmdf1.pli and demmdf2.pli, and both of them call it).
• proc4 is a nested function in demmdf1.pli, and it recursively calls itself.
• The rules detected that the call to proc1 at line 27 of demmdf1.pli is a potentially recursive call, even though the potential recursion occurs in proc4, a function nested within proc1.