The SOFTWARE principle

I wanted to copyright my new mnemonic and was looking for an easy way to do so.  Since google indexes EVERYTHING and my blog posts are date-time stamped I thought I could copyright it by simply posting

So here is is.

“Use the SOFTWARE principle in your development!

S eparate API from implementation
O SGi over J2EE
F ind simplest/smallest solution
T est drive your development
W ork out your alternatives
A ssertions and checked exceptions
R euse dont re-write, and refactor often
E ngage others early”

Documentum Development: Past, Present and Future

 

The Past

Using DAB the mode de jour was to develop against a Docbase; live, connected, whatever you want to call it.  You nominated a "System of Record" Docbase.  You then went about the business of defining all the artifacts; types, lifecycles, workflows, relations and so on and so forth, that collectively made up your Docapp (a DOCbase APPlication).

Your application(s) code would never execute against these particular artifacts, in this particular Docbase; because they really were just the definitions of resources that your application requires to be present in a production Docbase in order to run correctly. Meanwhile, the rest of your application artifacts; java classes, servlets & jsps, XML, etc, were all managed out of a source control system.

To prepare a content-rich application for distribution. Release engineering (releng) would checkout all the application artifacts, build and package them as an installer or as a war or an ear.  They would then also export the associated Docapp as a Docapp Archive and place with the application package.  These resources were then placed on a download site where customers could find them.

To install the application the customer would use the installer or deploy the war or ear. He would then use DAI to install the docapp archive into one or more production Docbases.

So, that is how it was.  The System of Record for your source artifacts was the Docbase itself.  But what was so wrong with that I hear some of your ask?  Well a few things as it happens:-

1. Having different Systems of Record is just plain wrong, and what made us so special anyway?
2. Versioning semantics for Docbase artifacts is not always the same as they are for source control, complicating development, histories, labelling and snapshoting
3. Releng needed to have and to know Docbases and to construct Docbases processes (not all of which were that automatable) which was hugely burdensome for them

With reference to the versioning semantics.  It is important for us all to recognize that the Docbase is just plain bad at development versioning.  For example, types in a Docbase cannot be versioned but code certainly depends on particular versions of types.  Now if we could have reliably versioned artifacts then we can support explicit versioned dependencies, even when the target environment does not support it. Explicit versioned dependencies allow us to bind code to specific versions of Docbase resources and have those dependencies validated.  The upsides of reliable versioning are hopefully evident.

The Present

Recognizing that content management would soon standardize through efforts like CMIS.  Recognizing that this would foster and entire industry of organizations building content-rich applications we knew we would have to address these shortcomings and move more mainstream.

So we built Composer and a system of linked XML-based artifacts and an install process that can transform these XML-based artifacts into Docbase objects, preserving the links by converting them to Docbase links; i.e. OIDs.

No longer is there a need for a "System of Record" Docbase.  Docbase artifacts can now be developed in the same eclipse IDE right alongside all your other application artifacts.  They can also be managed out of the same source control system, giving us reliable version semantics and greatly simplifying the releng process.

The Future

OK so we have ironed out that little wrinkle.  Docbase artifacts are now analogous to other source artifacts and therefore we are all now followers of the same development model.   In this respect we’ve become a bit more standard and hopefully removed a big barrier to entry for release engineering departments.  We believe this will help promote EMC|Documentum as the platform for developing content-rich applications.

So what’s next?

In a word "OSGi".

A phenomenon is sweeping the java community in the form of OSGi, a dynamic module system for Java. Most application servers are OSGi-based already.  And this phenomenon will undoubtedly impact the ECM community too. 

OSGi is mature.  It is over 10 years old and has grown out of embedded systems world.  So this is not new technology by any stretch.  It may just be new to some of us.  It is also standards-based. 

OSGi promotes a world of modules (bundles in their parlance) that can be dynamically inserted (or removed) into any live, running system with no interruptions in the services it provides.  Each bundle has versioned dependencies that the system will honour and enforce these.  And a system can support multiple versions of a module at any one time.

For all these reasons (and many others) I believe that OSGi is the perfect fundamental building block for next generation ECM solutions and solution frameworks.

It is also very important to recognize that in this future world of solutions and solution frameworks we may well also see direct, application hosted, manipulation of our XML systems of record.  As well as continuing to support the more traditional offline development model, both will co-exist quite happily side-by-side.  This direct manipulation will be supported by a family of web-based tooling built on the central core of the Composer tooling platform.

But this is still a little way in the future.  Because OSGi will effect how applications are delivered the most noticeable change you will see in the short-term will be to our friend the dar (Documentum Archive) which I believe is already set to become an OSGi bundle.  This will prepare the way for it to be just like every other piece of the solution.  Developed the same way, delivered the same way.  Depended upon, dependent upon others.  Really binding into the application as the vital constituent part that it really is. 

Conclusions

So what conclusions should you draw from this? 

Well if you are considering developing a content rich application (or larger) on Documentum then you need to be following Composer’s lead and adopting model-driven development.  Making XML models a central tenet of your development practices is the right thing to do.  Leverage Composer, the tooling platform, and in particular its EMF-based modelling and your not going to go too far wrong. 

Also getting more involved in Composer, the tooling platform, by extending it for your purposes by adding your own Docbase artifacts would be an excellent way to introduce yourself to the wonderful world of OSGi.

Happy Composing

CMIS Webcast

Yesterday my colleague Craig Randall, assisted by David Choy, gave a webcast on CMIS.

It was a good introduction.  What it is.  What it is not.  Some of the use cases we think it will facilitate.  The domain & data models.  Relationship to DFS and a whole lot more.  In case you missed it you can get the materials from here.

Content Management Interoperability Services

Finally, CMIS is here.

I’m a little late to this particular party as it all happened early PST which was a little later GMT by which time I was unfortunately otherwise occupied.  And there is already a plethora of good, initial write up out there in the blogosphere from my colleagues; Craig Randall, Chuck Hollis, Cornelia Davis, Andrew Chapman. From former colleagues; John Newton.  Others in the industry; Bex Huff, Laurence Hart.  And of course from the analysts; CMS Watch, 451 Group, AIIM.  I encourage you to check out Craig’s interview of Kyle McNabb who is the principal analyst at Forrester.

Content is were most of your organization’s high value information is kept.  Just think about the thousands, if not millions of documents, slide-decks, spreadsheets, forms, PDFs, not to mentioned emails that your organization tries to manage.  It is the same the world over and it is getting worse.  We are generating content faster now than at any time before and this trend continues a pace.  Also legal and regulatory concerns are placing more and more constraints on this content requiring it to be managed ever more intelligently, not to mentioned, requiring it to be discoverable.

But everyone knows this and has been attempting to solve the problem for a number of years.  Fast forward to the present day and take a look in any typically medium to large enterprise.  What you will find is a number of different types of content repository.  Each of which was introduced to solve a particular piece of the content management problem; DM, RM, Web Publishing, etc.  By and large they solve the problem in their domain but obviosuly each leave us with a silo of information in a repository with yet another interface.  This in itself is a big problem.  But with each new interface comes another skillset and therefore there is also a not an insignificant cost of ownership attached to each as well.

In short we need to start treating our content more strategically.

To date there have been quite a few efforts to standardize the ECM space but each has had its Achilles heal and none has really achieved what they set out to do, become the standard.

But I am excited about CMIS.  It’s based on web services and is therefore agnostic to your client architecture.  Something that JSR170 fell fowl of.  It doesn’t really favour WS* or REST, it specifies both, so you can use whichever suits your needs.  It is backed by us (i.e. EMC), Microsoft and Oracle as well as Alfresco, OpenText and SAP.  It only exposes features already present in these vendor’s repositories.   So you can have confidence that real implementations of CMIS are imminent and CMIS as a whole will have longevity.

A word of caution though.  We should be realistic about the present situation.  Even when adopted it’s no silver bullet in the sense that it won’t solve the silo’s problem overnight.  But it could be a real enabler.  Using CMIS, as the common interface for navigating content and its metadata, we can envisage solutions that allow us to index content in all of our repositories.  In turn allowing us, as users, to search and retrieve that content from any repository within the organization.  Or even across organizations.

It also makes the future look a lot more exciting.  In the same way SQL facilitated solutions like Documentum & SAP.  We are hoping CMIS will do the same for content-centric applications and solutions.   This is ultimately great news for us all as knowledge workers within content-driven organizations.  We will have real solutions to real problems based on the repsitories that are right for us.

Now, to help build those solutions our ISVs are going to need tools…like Composer.

Standards in ECM

I notice there have has been an upsurge in volume of posts discussing standards in ECM here, here, here and here.

In summary, some seem to believe standards add little or even nothing to the ECM space. Others seem to think standards are impossible to achieve in ECM. And some others still believe that standards loose you functionality but gain you flexibility.

From all of this two questions resonated with me;

“What problem are we trying to solve?”, and

“Might we want to consume content management on platforms other than Java?”

I think part of the confusion around the question “what problem are we trying to solve?” is historic. Enterprise Document Management (EDM), where we managed black-box files with the services such as metadata, versioning, categorization, workflow, access and collaboration, still exists but has been embellished with Enterprise Content Management (ECM) that also focuses on creating and managing of chunks of content (most likely XML) with the same services plus publishing.

My guess is that, as this discussion is found on ECM-based forums and blogs, we really are debating a case for standardization of this entire space of features and services, not one or the other. But I wonder if everyone is in alignment here? Or will my simple definition cause controversy? Which in itself will tell us something.

I also find the discussion on JSR170 et al, WebDAV and Atom interesting. I would hazard that we are in broad agreement that none of these existing standards in isolation constitute an ECM API. (I think this talks to one of Craig’s points a little). And in today’s world where we want the standard to be as widely applicable as possible surely any standard would itself have to be based on the WS-* set of standards. The others under discussion; JSR170, WebDAV, etc could then be used to implement a version of the ECM standard. However, this is of no consequence to the consumers of the ECM standard itself.

Interesting. Let the discussions continue.