Error: Macro BackLinksMenu(None) failed

compressed data is corrupt

Error: Macro TicketQuery(summary=GROUP_PERSISTENCE_R0, format=table, col=summary|owner|status|type|component|priority|effort|importance, rows=description|analysis_owners|analysis_reviewers|analysis_score|design_owners|design_reviewers|design_score|implementation_owners|implementation_reviewers|implementation_score|test_owners|test_reviewers|test_score|) failed

current transaction is aborted, commands ignored until end of transaction block

Analysis

Overview

This group of tasks is about improving the base.persistence module and providing the new R3 resource format. After this task is complete, the following should be true:

• The base.persistence module will provide all the commons needed for persistence.
• The R3 format will be a Resource format, not just a Book format. This means that any kind of Resource (Book, Page, Frame etc..) could be saved in this format.
• Functionality for saving Immutables and ProObjects (Resources, in particular) will be present.
• An intermediate storage format will exist.
• Extensions and extension points for simplifying the usage and improving extensibility of the format.

Task requirements

The R3 format replaces the R2 format. These are the goals of these task:

• The R2 format is a file format for saving a given Book. Compared to it, the R3 format is a Resource format - it should be capable of saving any given Resource.
• This Resource-oriented format gives features that should be taken in mind when implementing it:
  • In future it should allow saving/loading any Resource - Frame, Page, Annotation. This will be very useful if we consider Frame/Page editor for example. It would be also useful for saving templates.
• Improve machine and human readability of the format:
  • XML aspect - redesign the used XML tags to be both human and machine readable.
  • File structure - create a good and consistent structure inside the archive.
  • Data standards - what goes where:
    • XML aspect - XML should contain Immutables and metinfo. Define where does it belong in the structure
    • Binary data - define location of the binary data for each ProObject (Resources in particular).
    • Text data - define how to handle text contents; where to store them and in what form.
• Define how a Resource is located and accessed (ID, local path, URL).
• Support caching and offline content - currently cache mechanism doesn't exust. The file format needs to support cache and ability to save online resources.
  • Caching however should be limited. In future releases we may consider GUI input for the limit, for example "100MB"
• Make the R3 format backward compatible with R2:
  • The application should be able to load a book in R2 format and save it to R3 format.
  • Optional - achieve saving a book in R2 format. This doesn't need to have GUI for saving in R2 but there may be a mechanism for converting R3 to R2.
• Support of saving online content. This may not be implemented now, but in future we may have to support downloading online Resources and saving them into the book.
• Also in the future, we need to refactor the Storage in order to implement lazy loading of Resources (and Resource preview, respectively).

Task result

• Source code
• Wiki content

Implementation idea

Define a tree structure and divide the book into folders.

• Each resource has its own folder and contains sub-folders of its subresources.
• Each folder contains an XML with properties (immutables) and information for the specific resource, and information about the subresources. The folder should also contain the binary data of the resource.
• The book's folder has a subfolder for cache. Consider should resources have own cache directory or all should use the book cache directory.
• Implement the annotations that were discussed - @Persist, @Immutable (see the video of the discussion for more information on these):
  • An annotation for entities not to be persisted might be needed.
  • (Optional) Provide a class that handles all Immutables coming from the JDK.
• Design and implement a format registry that keeps track of all formats and their features and limitations:
  • see BASE_PERSISTENCE_FORMAT_REGISTRY_R0 for more ideas on that.
• Ensure backward compatibility.
• Consider forward compatibility in the design.

Related

GROUP_RESOURCE_R0
GROUP_CHANGE_R0?
​http://asteasolutions.net/videos/

How to demo

• Explain the ideas of the format.
• Demo saving and loading of various Resources.
• Run the tests/demos.
• (Optional) Show the created wiki page.

Design

R3 Format Structure

The R3 format is not just a Book format but a Resource format. It represents a given Resource with its subResources, binary data, cached data, external Resource references etc.
Since Resources form a hierarchy (for instance Book -> Page -> Frame), the R3 format structure follows the same hierarchy.
The R3 format is a zipped file with some directory structure inside and some files:

• each Resource described inside (either the root Resource or any of its subResources) has a corresponding directory with some helper directories and files.

The exact structure description of the format is described below. In this example a Book is saved, though any kind of Resource can be also saved:

• The name of the zip file is something like "ExampleBook.s2b". This name without the file extension (namely "ExampleBook") is the name of the Book Resource.
• The root directory of the file contains:
  • _cache - a directory for caching remote Resources.
  • _data - a directory for storing binary and text data. Instead of embedding unreadable binary data in an .xml file, it is stored here and just referred from other places (like the .xml files)
  • _resource.xml - an .xml file describing the structure, properties and subResources of the root Resource.
  • other directories:
    • for instance a "MeddleSubResource" directory corresponds to a subResource of the root Resource with name "MeddleSubResource". The structure of this subdirectory is analogical to this of the root Resource.
    • certain Resources can contain other directories describing templates. Templating is still in design so conventions for them are still not finished.

Resource names cannot start with an underscore so they won't conflict with _cache, _data and _resource.xml.

_resource.xml

Here's a sample _resource.xml for the root Resource in ExampleBook ADD-LINK:

<?xml version="1.0" encoding="UTF-8"?>

<resource kind="book" entity-id="e42f2de5-7463-439c-8eb3-25fa60424e7b" size="1254" mimeType="text/html" date-created="2009-04-13T16:15:00">
	<!-- All resource common features are persisted as attributes of the resource tag. All specific book features are persisted as children. -->
	<title> Meddle In Wonderland (: </title>
	<page-size>
		<width>400</width>
		<height>500</height> 
	</page-size>	

	<pages>
		<page index="1">
			<ref location="./MeddlePage" entity-id="5e0f117a-033a-4612-957c-3e0dab3b22a9" /> 
		</page>
		<page index="0">
			<ref location="./templatedPage" entity-id="dsfg34dfg41234" /> 
		</page>
	</pages>

	<last-current-page>
		<ref location="./MeddlePage" entity-id="dsfg34dfg41234" /> 
	</last-current-page>	

	<frame-templates>
		<frame-template>
			<ref location="./goodFrameTemplate" entity-id="3f92f3ca-f581-4e10-9353-856ab1c32f2g" /> <!-- local resource-->
		</frame-template>
		<frame-template>
			<ref location="/home/mrsackless/Desktop/TanyaMemoBook/ChildhoodPage/lovelyFrame" entity-id="1e0f235a-033a-2314-957c-4e2dab6b47a1" /> <!-- outer resource-->
		</frame-template>
	</frame-templates>

	<page-templates/>		

	<comment> A book eveyone should read </comment>
</resource>

Code Structure

Storage

Persistor <S, T>

Loader <T>

PersistorRegistry

Core Persistors

• MasterPersistor
• ProObjectPersistor
• ImmPersistor

Annotations

• @Persist
• @PersistBy
• @PersistList

Implementation

^{Describe and link the implementation results here (from the wiki or the repository).}

Testing

^{Place the testing results here.}

Comments

^{Write comments for this or later revisions here.}