Sommaire
|
Objectives for Maarch framework 3.0 are to enhance the framework-style structure of Maarch, by making easier implementation of new modules. We also want to have a more structured architecture, and be able to reuse as much as possible existing objects. At the same time, many specific objects and services developed in business applications are put back into the framework to make it more complete.
We have mainly:
The framework is now matched and tested with the 3 following SQL servers:
The class class_request.php is used for all db requests in the framework and in the modules.
The framework has a new tree structure, allowing cutting in modules and business applications:
/maarch_framework_v3
/apps
/sample_app
/admin
/class
/css
/export
/img
/js
/lang
/tmp
/tools
/xml
/core
/class
/js
/sql
/mysql
/postgres
/mssqlserver
/xml
/modules
/module_id
/class
/css
/img
/js
/lang
/sql
/mysql
/postgres
/mssqlserver
/xml
/portal
/css
/img
/tmp
Maarch Framework design follows one main guideline: providing a structured framework with all the common functionalities used in document management, in such a way that building a concrete business application becomes an easy task.
Business application is the final products. The core and the modules are not enough to make things work. They need a "glue", to make it run and answer an user's needs. We call this glue an "apps". It brings specific functionalities, display rules, rules particular to an industry, etc...
Maarch Framework 3, however, provides the framework a bunch of reuseable functions, components and objects.
Some of Maarch Framework possibilities are shown in a sample application, "Maarch Sample". It runs quite every component of the framework. It is a good way to get in touch with the framework, as a demo app.
<ROOT> <CONFIG> <businessappname>maarch_sample</businessappname> <databaseserver>127.0.0.1</databaseserver> <databaseserverport>5333</databaseserverport> <databasetype>postgres</databasetype> <databasename>maarch_sample</databasename> <databaseschema>public</databaseschema> <databaseuser>db_user</databaseuser> <databasepassword>db_password</databasepassword> <databasesearchlimit>500</databasesearchlimit> <nblinetoshow>10</nblinetoshow> <limitcharsearch>2</limitcharsearch> <lang>en</lang> <adminmail>dev@maarch.org</adminmail> <adminname>maarch</adminname> <xmlpath>xml</xmlpath> <debug>false</debug> <applicationname>Maarch Sample</applicationname> <css>css/styles.css</css> <css_ie>css/style_ie.css</css_ie> <css_ie7>css/style_ie7.css</css_ie7> <helpcss>../css/help.css</helpcss> <defaultPage>welcome</defaultPage> <tmppath>tmp/</tmppath> </CONFIG> <TABLENAME> <departments>departments</departments> </TABLENAME> <RESOURCES> <tablename>res_x</tablename> <comment>Accounting documents</comment> </RESOURCES> <WORKBATCH> <size>100</size> </WORKBATCH> <HISTORY> <usersdel>true</usersdel> <usersban>true</usersban> <usersadd>true</usersadd> <usersup>true</usersup> <usersval>true</usersval> <doctypesdel>true</doctypesdel> <doctypesadd>false</doctypesadd> <doctypesup>true</doctypesup> <doctypesval>true</doctypesval> <doctypesprop>true</doctypesprop> <resadd>true</resadd> <resup>true</resup> <usergroupsdel>true</usergroupsdel> <usergroupsban>true</usergroupsban> <usergroupsadd>true</usergroupsadd> <usergroupsup>true</usergroupsup> <usergroupsval>true</usergroupsval> </HISTORY> <MODULES> <moduleid>autofoldering</moduleid> <comment>_AUTOFOLDERING</comment> </MODULES> <MODULES> <moduleid>stats_sample</moduleid> <comment>_STATS_SAMPLE</comment> </MODULES> <MODULES> <moduleid>workflow</moduleid> <comment>_WORKFLOW</comment> </MODULES> </ROOT>
<ROOT> <MENU> <id>admin</id> <libconst>ADMINISTRATION</libconst> <url>none</url> <style>admin</style> </MENU> <MENU> <id>search_adv</id> <libconst>SEARCH_ADVANCED</libconst> <url>index.php?page=search_advanced</url> <style>search_advanced</style> </MENU> <MENU> <id>search_folder</id> <libconst>SEARCH_BY_FOLDER</libconst> <url>index.php?page=search_by_folder</url> <style>search_by_folder</style> </MENU> </ROOT>
Services are defined in the services.xml. Here are some examples:
<root> <SERVICE> <id>admin_workflow</id> <name>_ADMIN_WORKFLOW</name> <comment>_ADMIN_WORKFLOW_COMMENT</comment> <servicepage>admin_workflow.php</servicepage> <servicetype>admin</servicetype> <style>wf_admin_workflow</style> <enabled>true</enabled> <WHEREAMIUSED> <page>admin.php</page> <nature>list</nature> <width></width> <height></height> </WHEREAMIUSED> </SERVICE> <SERVICE> <id>choose_workflow</id> <name>_CHOOSE_WORKFLOW</name> <comment>_CHOOSE_WORKFLOW_COMMENT</comment> <servicepage>choose_workflow.php</servicepage> <servicetype>use</servicetype> <style></style> <enabled>true</enabled> <WHEREAMIUSED> <page>indexation.php</page> <nature>frame</nature> <width>300</width> <height>80</height> </WHEREAMIUSED> <PROCESSINBACKGROUND> <page>file_index.php</page> <scripttoprocess>choose_workflow_result.php</scripttoprocess> </PROCESSINBACKGROUND> </SERVICE> <SERVICE> <id>view_sarbanes_oxley_form</id> <name>_VIEW_SARBANES_OXLEY_FORM</name> <comment>_VIEW_SARBANES_OXLEY_FORM_COMMENT</comment> <servicepage>view_sarbanes_oxley_form.php</servicepage> <servicetype>use</servicetype> <style></style> <enabled>true</enabled> <WHEREAMIUSED> <page>indexation.php</page> <nature>frame</nature> <width>460</width> <height>200</height> </WHEREAMIUSED> <WHEREAMIUSED> <page>details.php</page> <nature>popup</nature> <width>800</width> <height>600</height> </WHEREAMIUSED> </SERVICE> </root>
We identified for the moment three pages potentially using the services modules: the indexing form, a document's detail page and the home page.
There are two kinds of services: administration or use.
The nature of a service accepts following values (in the <nature> </nature> tags of the service:
You can also define tasks being executed in background with the tag <PROCESSINBACKGROUND> and defined where you want to execute the script (<page>) and setup the script to launch (scripttoprocess).
Its pages will be waiting for services of various modules.
We have some examples of services that may appear on these pages:
The core is composed of files essential with the good performance of Maarch business applications and its modules. In the core there is no CSS and image, they must be treated by business applications (maarch_sample in this case).
Some features are managed by the core (classes and data model).
The main philosophy is to manage the framework mechanics in the core. Display is not managed by the core but by modules and business applications.
config.xml:
NB : The config file must be in utf-8 !
<ROOT>
<CONFIG>
<corepath>/var/www/maarch_v3/</corepath>
<tmppath>/var/www/maarch_v3/tmp/</tmppath>
<unixserver>true</unixserver>
<defaultpage>/var/www/maarch_v3/welcome_framework.php</defaultpage>
<defaultlanguage>en</defaultlanguage>
</CONFIG>
<BUSINESSAPPS>
<appid>maarch_letterbox</appid>
<comment>Maarch LetterBox</comment>
</BUSINESSAPPS>
<BUSINESSAPPS>
<appid>maarch_peoplebox</appid>
<comment>Maarch PeopleBox</comment>
</BUSINESSAPPS>
<BUSINESSAPPS>
<appid>maarch_sample</appid>
<comment>Maarch Sample</comment>
</BUSINESSAPPS>
<TABLENAME>
<arboxes>arboxes</arboxes>
<arcontainers>arcontainers</arcontainers>
<authors>authors</authors>
<docservers>docservers</docservers>
<doctypes>doctypes</doctypes>
<extdocserver>ext_docserver</extdocserver>
<fulltext>fulltext</fulltext>
<groupsecurity>groupsecurity</groupsecurity>
<history>history</history>
<param>parameters</param>
<resgroups>resgroups</resgroups>
<resgroupcontent>resgroup_content</resgroupcontent>
<security>security</security>
<usergroups>usergroups</usergroups>
<usergroupcontent>usergroup_content</usergroupcontent>
<usergroups_services>usergroups_services</usergroups_services>
<users>users</users>
</TABLENAME>
</ROOT>
A common issue with standard DMS is that they put access control lists (ACL) on a document per document basis. This is very consuming in terms of resources, and not versatile enough for an archive. Indeed, if you need to modify access control rules on your dcument repository, it's important that it could be done easily an quickly, without modifying all the resources indexes.
Maarch security schema is suited to archives. It is based on filtering on a user group basis : each group has the right to see a set of the archive, because each result set is filtered with documents that can be viewed by the group. Filters are applied each time one has access to a Maarch collection.
Here is described the data structure (DB tables and configuration files) used to apply Maarch security mechanisms.
First of all, users needs to be declared into users table :
At the same time, groups are declared into usergroups table, and then users are attached to those groups using usergroup_content :
usergroups :
usergroup_content :
Now let's see how resources are described.
In Maarch, electronic resources indexes are placed into res_x table. However, from the user and administrator point of view, resources are known as Maarch 'Collections'. A Collection is an abstraction for a resource repository; it is composed of a table where data is stored, a view from where the data is retrieved, as well as a description and id.
Collections are defined into <application>/xml/config.xml :
<COLLECTION> <COLLECTION_ID>COLL_INVOICE_2008</COLLECTION_ID> <COLLECTION_TABLE>res_invoice_2008</COLLECTION_TABLE> <VIEW>view_invoice_2008</VIEW> <LABEL>Invoices year 2008</LABEL> </COLLECTION> <COLLECTION> <COLLECTION_ID>COLL_INVOICE_2009</COLLECTION_ID> <COLLECTION_TABLE>res_invoice_2009</COLLECTION_TABLE> <VIEW>view_invoice_2009</VIEW> <LABEL>Invoices year 2009</LABEL> </COLLECTION>
In this example, Collection 'COLL_INVOICE_2008' refers to data stored into res_invoice_2008 table. Because data for a resource can use external references to display labels, searches and result lists use SQL views that solve all references (for example, get the CUSTOMER_NAME from CUSTOMER_ID). When defining the SQL view, always put in the Select list a column called 'COLLECTION_ID', set with the id of the collection needed to access and update the resource. This column is used to know which table is to be updated, in the case of a multitable collection.
A multitable collection allows to browse and select resources across distinct primary collections. In the definition file, tag COLLECTION_TABLE is not filled, and the view definition implies more than one table.
If we had to make a search on all invoices, collection definition is defined as :
<COLLECTION> <COLLECTION_ID>COLL_INVOICE_ALL</COLLECTION_ID> <COLLECTION_TABLE></COLLECTION_TABLE> <VIEW>view_invoice_all</VIEW> <LABEL>All invoices</LABEL> </COLLECTION>
view_invoice_all is a union of res_invoice_2008 and res_invoice_2009 :
SELECT "COLL_INVOICE_2008" as COLLECTION_ID, DOCDATE, ... FROM res_invoice_2008 UNION SELECT "COLL_INVOICE_2009" as COLLECTION_ID, DOCDATE, ... FROM res_invoice_2009
Now that we have defined our resource repositories, we just have to filter group access, through table security :
This internal schema is mostly managed through administration screens :
Collections definition is made through xml config file.
A module provides packaged functionnalities pluggable to Maarch Framework. It comes with its own SQL tables, classes, scripts, and configuration files.
Using modules, you can add functionnalities like workflow, baskets management, autofoldering, etc. You can also build your own modules following Maarch structure, and make it share by the community.
Each module exposes its features as services, declared in the module configuration file. Using the core, you can give the users access (or not) to those services, depending on the "role" the user has on your resource repository.
/workflow
/class
wf_class_admin_workflow.php
wf_class_tasks.php
wf_class_workflow_engine.php
wf_class_workflow_show.php
/css
/img
/js
/sql
/mysql
wf_structure.sql
wf_data.sql
/postgres
/mssqlserver
/xml
wf_workflow.xml
wf_menu.xml
workflow.xml:
NB : The xml files must be in utf-8 !
<root>
<CONFIG>
<name>workflow</name>
<comment>_WORKFLOW</comment>
<path>/var/www/maarch_v25/modules/workflow/</path>
<file_prefix>wf</file_prefix>
<loaded>true</loaded>
</CONFIG>
<TABLENAME>
<wfactors>wf_actors</wfactors>
<wfevents>wf_events</wfevents>
<wfgroupworklist>wf_groupworklist</wfgroupworklist>
<wfinsts>wf_insts</wfinsts>
<wftasks>wf_tasks</wftasks>
<wfworklist>wf_worklist</wfworklist>
</TABLENAME>
</root>
menu.xml: (will be used for the generation of the menu)
NB : The xml files must be in utf-8 !
<root>
<MENU>
<id>exploit_workflow</id>
<libconst>_EXPLOIT_WORKFLOW</libconst>
<url>index.php?page=exploit_workflow&module=workflow</url>
<style>exploit_workflow</style>
</MENU>
</root>
Services are defined in xml/services.xml
NB : The xml files must be in utf-8 !
<root>
<SERVICE>
<id>admin_workflow</id>
<name>_ADMIN_WORKFLOW</name>
<comment>_ADMIN_WORKFLOW</comment>
<servicepage>admin_workflow.php</servicepage>
<servicetype>admin</servicetype>
<style>wf_admin_workflow</style>
<enabled>true</enabled>
<WHEREAMIUSED>
<page>admin.php</page>
<nature>list</nature>
<width></width>
<height></height>
</WHEREAMIUSED>
</SERVICE>
<SERVICE>
<id>choose_workflow</id>
<name>_CHOOSE_WORKFLOW</name>
<comment>_CHOOSE_WORKFLOW_COMMENT</comment>
<servicepage>choose_workflow.php</servicepage>
<servicetype>use</servicetype>
<style></style>
<enabled>true</enabled>
<WHEREAMIUSED>
<page>indexation.php</page>
<nature>frame</nature>
<width>300</width>
<height>80</height>
</WHEREAMIUSED>
<PROCESSINBACKGROUND>
<page>file_index.php</page>
<scripttoprocess>choose_workflow_result.php</scripttoprocess>
</PROCESSINBACKGROUND>
</SERVICE>
<SERVICE>
<id>view_sarbanes_oxley_form</id>
<name>_VIEW_SARBANES_OXLEY_FORM</name>
<comment>_VIEW_SARBANES_OXLEY_FORM_COMMENT</comment>
<servicepage>view_sarbanes_oxley_form.php</servicepage>
<servicetype>use</servicetype>
<style></style>
<enabled>true</enabled>
<WHEREAMIUSED>
<page>indexation.php</page>
<nature>frame</nature>
<width>460</width>
<height>200</height>
</WHEREAMIUSED>
<WHEREAMIUSED>
<page>details.php</page>
<nature>popup</nature>
<width>800</width>
<height>600</height>
</WHEREAMIUSED>
</SERVICE>
</root>
We identified for the moment three pages potentially using the services modules : indexing, details and welcome page.
There is three type of service : administration, menu or use.
The nature of service execution may be a frame, a popup, include (will retrieve visible results or not), listelement (admin), or a button (put button in nature) to launch the service.
You can also define tasks being carried out in background with the tag <PROCESSINBACKGROUND> and defined where you want to execute the script (<page>) and setup the script to launch (scripttoprocess).
Its pages will be waiting for services of various modules.
We have some examples of services that may appear on these pages:
To make a resource circulate from one user to another, there is mainly two methods : 1. Procedural workflow 2. State or index based views presented as independant objects (what we called baskets in Maarch)
Solution 1 is suited for controled validation routes featuring numerous steps, and pre-determined actors. It is CPU and data consuming, as an instance is created for each run of the workflow, and every step is logged. Maarch Framework also provides procedural workflow in a dedicated module.
Solution 2 (here we are: baskets), is a good choice when your organization treats
This module allows to add new resources to the application and to make searches on them. Although generic, it might be customized for each application, as fields and control rules are highly dependent on business requirements. In the sample application provided with the download package, we demonstrated two different process for adding a resource :
Indexing & searching module gives guidelines for customs developments, but doesn't mean to be a fully customizable "do-it-yourself" interface design tool.
This module has no dedicated tables, as it is highly dependant on the application. It is absolutely required for any application.
Indexing & Searching comes with the following services:
Autofoldering is a new Maarch concept (and tool) that provides hierarchical views to the archive.
Is is composed of two parts :
Using Maarch Autofoldering module, you can browse your archive using different trees designed by your administrator. For example, on a Customer Invoices collection, you could browse your documents by Year/Customer, and/or by Customer/Year. As this is pretty CPU consuming, trees should be built when server is not heavily loaded. Once the tree is built, browsing it takes few resources.
Maarch sample appllication illustrates this principle using an invoice collection.
Trees content is stored in SQL table autofoldering_node_item. Like in Security table, the user group is granted access to a tree. A where clause allows filtering on tree nodes.
See table definition for autofoldering_node_item and autofoldering_security
<tree> <id>view_customer</id> <desc>_TREE_VIEW_CUSTOMER</desc> <view>res_view_invoices</view> <node> <node_id>customer_first_letter</node_id> <comment>1ère lettre client / Customer 1st letter</comment> <select_key>substring(doc_custom_t4, 1, 1)</select_key> <datatype>STRING</datatype> <select_label>substring(doc_custom_t4, 1, 1)</select_label> <whereclause>status <> 'DEL' and status <> 'ATT' and status <> 'REP' </whereclause> <orderby>asc</orderby> <node> <node_id>customer</node_id> <comment>Client / Customer</comment> <select_key>doc_custom_t4</select_key> <datatype>STRING</datatype> <select_label>doc_custom_t4</select_label> <orderby>asc</orderby> <node> <node_id>year</node_id> <comment>Année / Year</comment> <select_key>year(doc_date)</select_key> <datatype>STRING</datatype> <select_label>year(doc_date)</select_label> <whereclause> year(doc_date) is not null </whereclause> <orderby>asc</orderby> <script>show_doc_folders</script> </node> </node> </node> </tree>
In this example you see how trees are described :
Sample application shows 2 kinds of trees for browsing the 'Invoice' collection.
Maarch Autofoldering comes with the following services:
It also comes with a batch script for building trees.
Maarch Core provides documents types that belongs to a 3-level fixed structure. Indeed, some complex DM applications use extensive lists of documents types (several hundreds), and it's much more easy to select and see those document types through a structured tree. Maarch supporting multiple collections, document types depends on the collection.
If your application makes extensive use of folders, like in customer management, you would want to use different folder types, associated to their own well defined structure. The Maarch Folders module add some great functionnalities to easily manage fixed-structure folders : for example, a loan company will want all documents related to a customer sorted as in the paper folder :
Folder Management is different from AutoFoldering ! Whereas autofoldering builds automatic trees based on resource descriptors, Folders module manages links between resources and user-defined folders.
Maarch Folders adds foldertypes depending on the collection, as illustrated below :
Deep-blue tables are coming with the Folders module, light-blue tables are part of the Core.
Post indexing defines the way that a document is qualified after being imported in the DMS. This module goes with the basket module, to which it adds the capacity to mass update a basket. Compared to former Maarch 2.1 postindexing module, this one relies on a basket to get the list of documents to check, through the basket SQL clause. Postindexing can be used on the same basket by more than one operator.
Total number of documents in basket / Nb total de documents pour la requête Operator / Opérateur Operation date / Date Postindexing batch id / Lot de videocodage en cours Total number of documents in batch / Nb total de documents pour le lot Current resource id / Identifiant du document en cours Number of processed documents / Documents validés dans le lot de travail
Given a basket, a parameter gives the number of documents to be taken for postindexing. For example, a basket can have 3000 resources in it, and the first postindexing batch will get 100 of them. Each resource is pre-flagged with the postindexing_batch_id and postindexing_timestamp.
The general menu of the application is generated automatically via xml files (menu.xml) of each module. By default the first menu item is link to the admin board of the application
The pages of Maarch are assembled like lego bricks.
We have a general structure created since the index.php page from the business application.
This page from the html header and footer are created automatically by the core, as well as links to the stylesheets css files and languages of the business application and modules loaded.
The automatic inclusion pages from index.php has somewhat changed, we have added a page from the parameter "module=". Here we can identify the affiliation of a page to a module or in the core.
Example :
The Beginning of the files name: by module name and nature: class, frame, popup… Names of tables will be prefixed by module. Put all names of tables and all names of column in lower case.