Ticket #437 (closed task: fixed)

Opened 4 years ago

Last modified 4 years ago

Ref-contraction in OMDoc1.2

Reported by: cmueller Owned by: dmisev
Priority: major Milestone: Release v0.1.3
Component: System Implementation (SI) Version:
Keywords: Cc: vzholudev, kohlhase, nmueller, clange, frabe
Blocked By: Blocking:
Due to close: Include in GanttChart: no
Dependencies: Due to assign:

Description (last modified by cmueller) (diff)

Ref contraction is the replacement of ref elements with OMDoc elements. It is a two step process including the (a) loading of the referenced OMDoc elements and (b) replacement of the loaded content with the respective ref.

The method implementing ref contraction should be available to any OMDoc element.

loading

Implement a method that takes an URI as input and returns the referenced element. The method loads the OMDoc element from the respective target source.

For now we only support the following URIs:

  • absolute paths to documents in the filesystem (the URI can reference any element in the document)
  • relative filepaths, where we only allow paths relative to the current doc (no backward edges), e.g. we can parse "subfolder/doc1#el1" but not "../folder/doc1#el1"

For any other URI the method should return a warning (or error) saying "Sorry, the processing of URIs such as X is not yet implemented.". In particular, this relates to more complicated relative filepaths (talked to Normen if you want to understand why this can be tricky) and references to web sources (starting with http or https).

replacement

The loaded element replaces the respective ref element. Note that this process should be constructive, i.e. a replacement does not modify the original source but modifies a copy.

Change History

  Changed 4 years ago by cmueller

  • description modified (diff)

  Changed 4 years ago by cmueller

  • owner changed from nmueller to dmisev

  Changed 4 years ago by cmueller

  • summary changed from Ref-contraction in OMdoc1.2 to Ref-contraction in OMDoc1.2

in reply to: ↑ description ; follow-up: ↓ 5   Changed 4 years ago by clange

Replying to cmueller:

For now we only support the following URIs: * absolute paths to documents in the filesystem (the URI can reference any element in the document) * relative fielpaths, where we only allow paths relative to the current doc (no backward edges), e.g. we can parse "subfolder/doc1#el1" but not "../folder/doc1#el1"

In Java, one gets URI resolution almost for free, so we can actually afford supporting everything (even remote URIs).

in reply to: ↑ 4   Changed 4 years ago by cmueller

Replying to clange:

Replying to cmueller:

For now we only support the following URIs: * absolute paths to documents in the filesystem (the URI can reference any element in the document) * relative fielpaths, where we only allow paths relative to the current doc (no backward edges), e.g. we can parse "subfolder/doc1#el1" but not "../folder/doc1#el1"

In Java, one gets URI resolution almost for free, so we can actually afford supporting everything (even remote URIs).

Normen, can you please comment on this?

follow-up: ↓ 7   Changed 4 years ago by dmisev

Agreed, why would you want such restriction on the URIs?

in reply to: ↑ 6   Changed 4 years ago by cmueller

Replying to dmisev:

Agreed, why would you want such restriction on the URIs?

If Java provides URI resolution for free we should of course not impose and restrictions on URIs, ergo the ref contraction should be able to resolve any URI to the filesystem and load the referenced content, respectively.

  Changed 4 years ago by cmueller

see discussion in  OMDoc trac

  Changed 4 years ago by cmueller

  • description modified (diff)

  Changed 4 years ago by cmueller

Please consider cycle detection to keep track of all the elements that have already been loaded for replacement.

  Changed 4 years ago by cmueller

see Normen's email from July 17, 2009:

I am totally confused by all this discussion regarding ticket:437 and ticket:438.

Have you already solved this issues?

If yes, then we should meet or you send me detailed instructions on how to initiate a couple of examples from the command line in order to test your solution(s). That is, I don't wanna run any (JUnit) test cases but perform "live" test from the command line. So what I would prefer is sth. similar to this:

  1. Update JOMDoc WC
  2. run "ant xy"
  3. ensure JOMDoc bin is on your PATH
  4. download the following sample OMDoc documents:  http://.... (a),  http://.... (b), ... (n)
  5. initiate the following command line statement(s): jomdoc refcontr (a) ..., jomdoc refintro (a) ...
  6. the output should be ....
  7. valid the output to be valid OMDoc by: jomdoc validate (a') ...

I'll then perform all this steps and if everything is fine we're gonna close these tickets. By the way, these detailed instructions will then be put into the JOMDoc manual. So if you prefer you can even send me a LaTeX document but the main emphasize lies on the term / detailed/ (not on the representation) so that every 3rd party user would understand.

follow-ups: ↓ 13 ↓ 18   Changed 4 years ago by dmisev

  • status changed from new to assigned

loading

This was already implemented as  JOMDocURIResolver.java

replacement

This constructive would be quite expensive, because deep copy of an OMDocNode is done by serializing it and then parsing it into a new object again. Thus I think it's better if the method caller takes care of providing a copy or the original object. But if you think it should be constructive by default I'll do it that way no problem.

ref-contraction-ref-normalization

  1. Update JOMDoc WC
  2. ant jar
  3. ant install as root to put link in /usr/bin to jomdoc script
    • if that doesn't work use trunk/build/lib/jomdoc
  4. download  these sample OMDoc documents
  5. initiate the following command line statement(s):
    • jomdoc transform --contract-refs narrative.omdoc -o narrative.out
      • the output should be a ref-normal form of narrative.omdoc, i.e. exactly narrative.exp
    • jomdoc transform --contract-refs circular_ref.omdoc
      • the contraction should stop because of detected circular references

implementing code

 Contractor.java, ant doc if you want to see javadoc in build/apidocs

in reply to: ↑ 12   Changed 4 years ago by cmueller

Replying to dmisev:

1. Update JOMDoc WC 2. ant jar

I get

BUILD FAILED
/home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build.xml:99: 
Compile failed; see the compilererror output for details.

  Changed 4 years ago by dmisev

Please attach this compile error output.

  Changed 4 years ago by cmueller

sorry, here you go:

Buildfile: build.xml       

javac:
    [mkdir] Created dir: /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build/classes
    [mkdir] Created dir: /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build/test   
    [javac] Compiling 315 source files to /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build/classes                                                                                               
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/jomdoc/org/omdoc/jomdoc/util/xml/XMLUtil.java:19: warning: com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl is Sun proprietary API and may be removed in a future release                                                        
    [javac] import com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl;                       
    [javac]                                               ^                                            
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/jomdoc/org/omdoc/jomdoc/util/xml/XMLUtil.java:86: warning: com.sun.org.apache.xerces.internal.jaxp.SAXParserFactoryImpl is Sun proprietary API and may be removed in a future release                                                        
    [javac]         factory = SAXParserFactoryImpl.newInstance();                                      
    [javac]                   ^                                                                        
    [javac] Note: Some input files use unchecked or unsafe operations.                                 
    [javac] Note: Recompile with -Xlint:unchecked for details.                                         
    [javac] 2 warnings                                                                                 
    [javac] Compiling 45 source files to /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build/test                                                                                                   
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:44: cannot find symbol                                                                             
    [javac] symbol  : method existsFile(java.lang.String)                                              
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         if (!IOUtil.existsFile(GLOBAL_DIR))                                                
    [javac]                    ^                                                                       
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:45: cannot find symbol                                                                             
    [javac] symbol  : method createDir(java.lang.String)                                               
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]             IOUtil.createDir(GLOBAL_DIR);                                                  
    [javac]                   ^                                                                        
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:46: cannot find symbol                                                                             
    [javac] symbol  : method getFile(java.lang.String)                                                 
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         File fdir = IOUtil.getFile(GLOBAL_DIR);                                            
    [javac]                           ^                                                                
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:49: cannot find symbol                                                                             
    [javac] symbol  : method writeToFile(java.lang.String,java.lang.String)                            
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         IOUtil.writeToFile(GLOBAL_DIR + fileName, content);                                
    [javac]               ^                                                                            
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:58: cannot find symbol                                                                             
    [javac] symbol  : method readFile(java.lang.String)                                                
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         return IOUtil.readFile(GLOBAL_DIR + fileName).getContents();                       
    [javac]                      ^                                                                     
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:67: cannot find symbol                                                                             
    [javac] symbol  : method getFiles(java.lang.String)                                                
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         return IOUtil.getFiles(GLOBAL_DIR);                                                
    [javac]                      ^                                                                     
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:75: cannot find symbol                                                                             
    [javac] symbol  : method getFile(java.lang.String)                                                 
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil                                           
    [javac]         return IOUtil.getFile(GLOBAL_DIR + fileName);                                      
    [javac]                      ^                                                                     
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:79: cannot find symbol
    [javac] symbol  : method fileOutputStream(java.lang.String)
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil
    [javac]         return IOUtil.fileOutputStream(GLOBAL_DIR + fileName);
    [javac]                      ^
    [javac] /home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/src/test/org/omdoc/jomdoc/Storage.java:88: cannot find symbol
    [javac] symbol  : method fileInputStream(java.lang.String)
    [javac] location: class org.omdoc.jomdoc.util.etc.IOUtil
    [javac]         return IOUtil.fileInputStream(GLOBAL_DIR + fileName);
    [javac]                      ^
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 9 errors

BUILD FAILED
/home/cmueller/vc/svn/omdoc.org/jomdoc/src/jomdoc/trunk/build.xml:99: Compile failed; see the compilererror output for details.

Total time: 10 seconds

follow-up: ↓ 17   Changed 4 years ago by dmisev

Are you sure the svn up went fine? Because that Storage.java is not even in the repository.

in reply to: ↑ 16   Changed 4 years ago by cmueller

Replying to dmisev:

Are you sure the svn up went fine? Because that Storage.java is not even in the repository.

oops, svn update went fine as well as ant jar... so now I'm going to test for real ;)

in reply to: ↑ 12   Changed 4 years ago by cmueller

Replying to dmisev:

== loading == This was already implemented as  JOMDocURIResolver.java

implemented and tested? if so, that's fine with me and we can close this ticket.

== replacement == Thus I think it's better if the method caller takes care of providing a copy or the original object.

That's fine with me as well.

== ref-contraction-ref-normalization == 1. Update JOMDoc WC 2. ant jar 3. ant install as root to put link in /usr/bin to jomdoc script * if that doesn't work use trunk/build/lib/jomdoc 4. download  these sample OMDoc documents 5. initiate the following command line statement(s): * jomdoc transform --contract-refs narrative.omdoc -o narrative.out * the output should be a ref-normal form of narrative.omdoc, i.e. exactly narrative.exp * jomdoc transform --contract-refs circular_ref.omdoc * the contraction should stop because of detected circular references

I just ran your test cases, they work fine.

== implementing code ==  Contractor.java, ant doc if you want to see javadoc in build/apidocs

I will (and hopefully all other participants) have a glance at it.

Thank you very much. You did a great job. Sorry again for the above "bug report". I wait till tomorrow, if there will be no response from you on this ticket, I will (eventually ;-)) close it.

  Changed 4 years ago by cmueller

  • status changed from assigned to closed
  • resolution set to fixed
Note: See TracTickets for help on using tickets.