Harnessing Conceptual Structures to Expose Organizational Dynamics

Alex and I have just had a paper published in the International Journal of Conceptual Structures and Smart Applications (IJCSSA), where we discuss how the practical application of Conceptual Structures lags far behind the theory. We introduce how Essential toolset can address this and provide a platform that can be used to support the real-world application of Conceptual Structures to explore and expose the dynamics of any organization.

The article is available at: http://www.igi-global.com/journals/abstract-announcement/119007


Securing the Viewer

The Essential Viewer can be configured to require that any user first authenticate themselves before accessing the requested View.

In the open-source release, the access control is relatively coarse-grain – can the user access the Viewer or not? – but this helps to keep things simple to configure and manage. We can connect Viewer to an LDAP directory such as Active Directory so that users can use their normal login and password to access the Viewer.

Continue reading “Securing the Viewer”

Essential Upgrade

Following on from Sarah’s last blog that announced what’s coming, I wanted to let you know that we’re in the final stages of packaging Version 3 of Essential Architecture Manager.

The big focus in this version is a complete overhaul of Essential Viewer to make it more easily customised in terms of look and feel – e.g. to tailor it to your intranet styling, improve performance, error handling and to make it simpler to control.

However, version 3 introduces a much extended and improved meta model – based on our own practical experience of using Essential in the real world and of course the contributions of the Essential Project Community. Many of the ECPs that we have been running have now been enhanced and included in the version 3 baseline meta model.

How do we upgrade to version 3?

We’re really excited about the power that version 3 provides but how do we upgrade all those users who have already been enjoying earlier versions?

Upgrading Essential Viewer is quite straightforward as it is delivered as a WAR package. Obviously, any custom Views that have been created will need to be migrated but it is easy to deploy the Version 3 Viewer alongside existing version 2 instances as we transition any custom views.

Migrating the meta model is a little more complex but we have created a number of tools to help make this consistent and more straightforward.

Protege manages the meta model and the model via the Classes and Instances, respectively. These are managed separately (e.g. the PONT and PINS files) as are the form configurations which control the layout, widgets, labels and tool-tips.

This is generally a good thing but it means that the safest and most consistent way to make automated changes to the repository is via the Protege API and doing so means that all the checks and balances that the GUI applies can also be exploited. This also means that we (and you) can make changes and enhancements very easily via the Protege GUI and then encode those as a script of API calls that can be applied to any repository. In layman’s terms, this means we can build upgrade packs to help ease the migration.

We are currently in the process of completing the upgrade pack that will take existing repositories based on v1.2 of the meta model to version 3 and this will be released soon after the Essential Architecture Manager version 3 is launched.

Preparing to upgrade

The idea and intention of the upgrade packs is that they will gracefully upgrade any existing repository. Any extensions that you might have made to the baseline meta model, e.g. additional slots / attributes on a class, additional classes will be preserved as part of the upgrade.

We always recommend that you take an ‘extend’ approach to any customisations or additional features that you might want. Changes to the names or purpose of an out-of-the-box slot – or changing the name of a class may be lost or have unexpected results. If you’ve lost track of what you might have changed, it might be worth comparing your repository to the version 3 baseline repository using the ‘diff’ function of the “Prompt Tab” in Protege.

Many of the forms have had a re-layout to improve the user interface – and in particular to move slots that we have deprecated (served notice of retirement) to the bottom of the form.

Where possible, we have tried to migrate the forms sympathetically but given that one of the powers of Protege is that it is very easy to customise the form layout, we cannot really make any safe assumptions about what a form might look like. There will be cases where the upgrade ‘resets’ the layout to the version 3 baseline form. If you have made changes, you will need to review the layout of such forms – especially if you have additional slots on the class.

It is common to find that you are working in specific areas of the model (rather than the whole model) for particular modelling activities and this means that reviewing the form layouts can be done on an ‘as required’ basis.

How do apply the update packs?

I have described how the update packs take the form of scripts that drive the Protege API but how do we apply them? Protege does not have a plugin for managing this sort of programmatic update of the classes, instances and forms.

Currently, we provide packs as Python scripts that are executed in the “Script Console Tab” that is provided in the standard Protege install. We have tried to package this up as simply as possible, but this is still a bit technical and requires some care when running the scripts.

To improve and simplify this process, we are developing a new Protege Tab Plugin to make updating the repository more like updating your iPhone or iPad!

We’ve seen how the use of the installer has (generally) simplified the install process and we want to provide the same capability for adding new packs of classes – and importantly instances (e.g. standard enumerations) to the repository. So, this plugin will have applications beyond just upgrading the meta model.

Options for migrating to Version 3

Are there any options for how to migrate your existing Essential repository to the version 3 platform? It is possible to make use of the Essential Integration Tab and import an XML snapshot of your repository (the same snapshot that Essential Viewer uses) into an empty version 3 baseline repository. I have used this approach myself to migrate one of our demo repositories before applying further update packs.

However, if you have made any changes to the meta model, this may not be straight-forward and the instance migration packs would still need to be applied.

For this reason, I would recommend using the upgrade pack to apply all the changes required to your existing repository, classes and instances. As with any toolset like this, you should take a backup (using the Protege Project->Archive capability) of your repository before starting to apply the upgrade. And of course, the Essential Project Team will be happy to help with any tricky migrations, perhaps due to some customisations, and the option of commercial support is always available.

Version 3 has so much that is new and improved, that we’re sure that this will be an Essential Upgrade! 😉



Essential Information and Data Pack

It’s been a couple of months since we released the Information and Data Pack and I thought it would be useful to take a more detailed look at what is in this extension to the Essential Meta Model and what it can do for us.


Firstly, thanks to our community members who have given us some feedback and found a couple of small bugs in there. We’ve started a forum thread to catch any more issues as we find them but will be releasing a patch and new version of the pack in the coming weeks – we wanted to make sure we had as many issues addressed in a single update as possible. If you’ve found any issues in there, please let us know in this forum.


This optional extension pack is a major extension to the Information layer of the meta model but although there are some tweaks of the core meta model elements, this is mostly an extension to what was already there.


We have added a number of important meta classes for managing Data elements and the relationships that these have to Application, Information and Business elements, enabling the resulting models to support Enterprise Data Management and Enterprise Information Management activities. More about that later in this blog.


One of the most important concepts in the pack is the separation between Information and Data. We have defined strong semantics for what is Information and what is Data, so that there is a clear and consistent framework for capturing and managing these elements of your Information and Data architecture.


We’ve based this on the commonly used “Information is Data in context” definition. Data elements have the same meaning and value regardless of the context in which they are used, whereas Information elements differ depending on the context. e.g. Product data means the same things regardless of whether we are looking it in the context of Stock, Sales, Manufacturing. That’s not to say that we only have one Product data element in our architecture, we can define as many variants of how Product data is managed in our organisation as we need – in terms of the attributes and relationships. e.g. Product Item data might have a different set of attributes to a Bill of Materials data object.


In contrast, Information takes data and uses it in a particular context. e.g. Stock Volume by Location might use data about Products at particular locations in the organisation and would have a different value for a single Product depending on the Location.


This separation of the Information and Data fits neatly into how we need to manage these things in the real world. Data combined and used to produce the Information that we need.


Naturally, we’ve used the Conceptual, Logical and Physical abstractions for the new Data metaclasses, covering the WHAT, HOW and WHERE for data elements. In addition to adding these new meta classes, we have created powerful relationship classes to them that enable us to clearly understand how Information and Data is being used by Applications and Business Processes. Some of this might seem a bit complex at first glance but this is due to the contextualised nature of these relationships. What we’ve created are constructs that enable us to understand what Information Applications use and create and in the context of that, which data elements are used, specifically, to deliver that Information – and in that context what are the CRUD for the Information and the Data. We believe that having these types of contextual relationships is a uniquely powerful capability of Essential Architecture Manager but we haven’t added these just because we can but because without them we cannot accurately or even reliably understand what is really going on in our organisation with respect to Information and Data.


So, what kind of things can we do with the Information and Data Pack?

We have designed the meta model to support Information Management and Data Management activities managing all the dependencies that exist between information and data elements and also how these are used by Application elements and how they are produced by Applications.


More specifically, in combination with the Views that are supplied out-of-the-box, we can understand where issues exist with our Master Data Management solutions, e.g.

  • where are we mastering Product data?
  • how is this data being supplied to our Applications?
  • how does this compare to how data should be supplied to our applications (according to our MDM policies)?


As you will have come to expect, the supplied Views are easy-to-consume, hyperlinked perspectives that are designed to give valuable insights to a wide range of stakeholders – not just Information and Data professionals.


We can browse the Data catalogue, drill into individual subjects, see what Objects we have in each subject and how each of these objects is defined. Further drill-downs can take us to views that should how this Object is provided to the relevant applications in the enterprise and from where.


Based on the detailed CRUD information that we can capture in the model, we can easily produce CRUD matrices from a wide variety of perspectives, e.g. CRUD of Data Subjects by Business Capability, CRUD of Data Objects by Business Process – both of which are derived from the details in the model, which means that these are automatically updated as the content of the model is updated.


One of the most powerful Views is also one of the simplest. We find that providing a browser-based, easy-to-access place to find accurate and detailed definitions for all the Information and Data elements – and most importantly in easy-to-understand terms – is a capability that is quickly valued, in particular by non-technical stakeholders. Deliberately, there are no ERDs or UML class diagrams in these definitions. Rather, we can explore the catalogue of the Information and Data elements in a highly hyperlinked environment, providing an automatically generated data (and information) dictionary to the organisation.


In that context, we’ve introduced some nice little features that will be included across all the Views such as the ability to identify the content owner of particular elements or groups of elements. This means that if we see that the content on a View is incorrect, we can click a link and send an email to that owner to let them know that things have changed or that there’s an error in the content.


We recognise that while most of the meta model concepts for the Information and Data pack are straightforward, there are some more complex areas, in particular in the definition of relationships. As always, we’ve worked hard to keep these as simple as possible but the reality of how Information and Data is used and produced is (or at least can be!) complex and we need to able to capture and manage these complexities. However, the capabilities and results are worth the investment in understanding how to use these. For example, the way the Information to Application (and in that context, to Data) relationships work, enable us to understand how a packaged application has been configured in terms of how the out-of-the-box data structures are being used. This means that we can understand where Product Codes are being stored in Location Code tables, for example, and this is the kind of scenario where the ‘devil is in the detail’ and fine-grained things like this can become the source of a lot of larger-scale issues in the architecture.


We’ve already had some excellent feedback on the pack and based on demands from real-world use of Essential Architecture Manager, we are now looking to extend the granularity of the application-to-data relationships to include data attributes, rather than just data objects. You might not always need to go to that level of detail but if you do, the capability will be there – and this will be designed to enable you to model at asymmetrical levels of granularity.


Although it’s currently an optional extension, the Information and Data pack will be incorporated into the next baseline version of the meta model. We think that the game-changing capabilities of the Information and Data Pack are a vital part of the enterprise architecture model and so it is natural that this extension become part of the core meta model.


Welcome to the ‘View Store’

Our approach to Views in Essential Architecture Manager is the same as Apple’s approach to apps for the iPhone / iPad. Essential Viewer provides a platform for you to easily assemble the views that you need.

It occurred to me recently that our approach to Essential Viewer, and the Views in particular, is just like what Apple have done with the iPhone / iPad and the Apps.

Where Apple were particularly disruptive in the smart-phone market was their approach to the Apps. Yes, they provided a number of useful Apps with the iPhone, but by providing a platform for people to create their own Apps backed by the App Store where these Apps can be shared, they totally changed the game of what it was to be a smart-phone.

In contrast, the established manufacturers had been creating smart-phones and all the ‘apps’ that you might want for them and it was difficult (compared to iPhone) to get additional applications for your phone. Even with Symbian-based devices, which of course is an open-source mobile device platform, it appeared to be harder for people to create their own applications, when we compare those to the sheer volume of 3rd party Apps that are available for iPhone.

Clearly Apple had the benefit of many years of early-adopter hindsight as to what does and what does not work on smart phones. However, I think that when they combined the open (if somewhat governed) platform with the infrastructure to easily acquire Apps and an innovative approach to what it was to be an App, then it all came together. Most Apps are very focussed on specific tasks and can therefore be small, easy to develop and easy to maintain and so if the App doesn’t do what someone needs, rather than extend it and bloat, they create a new App that just does that new function.

This is very analogous to the approach that we have taken with the Essential Viewer and the Views. I’m not sure that describing the Viewer as a toolkit (as I did until recently!) does it justice. The Viewer is a platform that enables focussed, often small and lightweight, Views to be rapidly assembled to meet the specific requirements that you have for presenting analysis, decision support, insights etc. about your architecture in a way that is clear and makes sense to your stakeholders.

Typically, we find that ‘architects’ Views do not make the impact you would expect on business users. We need different Views for different stakeholders.

Like Apple’s approach with the Apps (compared to the established manufacturers), rather than try to build and deliver all the Views that our Community could possibly want, The Essential Project provides the platform for you to assemble your own Views using standard web-development tools of your choice, based on HTML and XML/XSL.

Of course, we will continue to produce Views that will be included in the growing suite that is bundled with the Essential Viewer. But, we don’t want to be holding any one back in terms of their View requirements! One of the founding principles of The Essential Project is that it’s about providing capability.

Continuing the iPhone App analogy, we want the Community area of the Essential Project website to be the ‘View Store’. We’ve had some great contributions on the software components side and the goal is that members of the Essential Community can share, contribute and download new Views via this website.

We’re always looking at how we can improve the Viewer platform to make it easier to assemble views. We are increasingly finding that we’re creating little template components that do specific tasks (e.g. render the name of an element when given its Instance ID) and I think that such View components would also be very valuable when shared in the ‘View Store’.

So, if the Views are Apps, the Essential Viewer is an iPhone or iPad, then the Essential Project Community website is the App Store.

Happy View building!

Do I have to use Tomcat to use Essential?

The Essential Project takes advantage of a number of free, open-source components, including Apache Tomcat. However, do you have to use Tomcat to use Essential Architecture Manager?

This is a question that we realise potential users of the Essential Project tools are asking. In particular when the architecture team has spent many years trying to get to grips with technology diversity, it can be somewhat embarrassing – or even not an option – to have to go off-piste to use an enterprise architecture management tool!

The simple answer to this question is, however, “no”.

You do not have to use Apache Tomcat to use Essential Architecture Manager.

From the outset, we designed the Essential Architecture Manager software so that it can run on any platform – both operating system and application server – to provide as much flexibility in the prerequisites as possible. This means that Essential can fit into as many existing technology architectures as possible, taking advantage of existing technology platforms wherever possible.

The same question can be posed to the choice of relational database software for multi-user installations. Again, as we can see from the software architecture model, where a range of JDBC-compliant databases can be used, Essential can fit into your existing technology architecture, taking advantage of your standard database platform.

Essential Architecture Manager Software Architecture
Essential Architecture Manager Software Architecture

If I don’t have to use Apache Tomcat, what else can I use?

You can use any Java Servlet engine or J2EE application server as the runtime platform for Essential Viewer, which is packaged as a standard Java WAR file. This gives you the freedom to deploy and run Essential Viewer on a range of application servers such as Oracle AS, IBM WebSphere or JBoss.

Since the Essential Project is a free, open-source toolset, we have focussed on application servers and database platforms that are also available free and open-source. The available documentation covers these free platforms as we find that most of our users – certainly the initial stages of using Essential – use these platforms. However, in the spirit of open-source and applying this to our documentation we would welcome any documentation contributions from users who have experience to share about deploying the Essential Viewer WAR to alternative Java application servers.

What about the installer?

The Essential Project takes advantage of a number of other open-source toolkits and the installation process involves adding plugins or components to these other open source tools, e.g. Protege and Tomcat. The first releases of Essential did not include an automated installer but we provided detailed documentation for the installation process – including troubleshooting – and this documentation is applicable to other Java application servers.

In order to make the installation process as simple as possible, the automated installer checks the locations of the Protege and Application Server that you specify to help ensure that the Essential components are being installed in the correct places. Although the installer does not assume that Tomcat is being used (despite the messages on the installer windows!), it does assume that the target application server holds its web applications in a folder called ‘webapps’ – which is what Tomcat does.

This means that if the installer cannot find a folder called webapps in the location that you’ve specified for the Application Server, the installer will not proceed.

It is not really practical to cover all the variations of Application Server deployment approaches, so currently a work-around for anyone who is not using Tomcat is to create a ‘webapps’ folder in your Application Server and install the Essential Viewer WAR to there.

Perhaps the other approach is to relax the error-checking in the installer and allow the Essential Viewer WAR to be installed to any folder on the target environment? This places more responsibility on the user of the installer to install the WAR to the correct location but also provides them with more flexibility.

Either way, if you have any problems installing any of the Essential Project components, please contact us. Free support is always available through the forums.

I would be very interested to hear any views on the how to strike this balance in the installer.

What about OWL?

OWL is W3C standard for defining ontologies for the Semantic Web. When people find out that we’re using Protege in Essential Architecture Manager, they often ask why we haven’t used OWL. So, what about OWL?

When people find out that we’re using Protege as a part of Essential Architecture Manager, a common question is whether we will, or why we don’t currently, use OWL.

OWL is an open-standard language from W3C for describing ontologies – the Web Ontology Language. This uses RDF (another XML language for describing resources that is often used to capture the contents of a repository) to store the Protege repository – both classes and individuals (as instances are known in Protege OWL).

There’s a lot of work currently going on in the OWL space, developing standard ontologies for particular domains, producing advanced reasoners to use the ontologies and so on.

The Protege Project is heavily involved in supporting these ontology initiatives and indeed, currently Protege 4 only supports OWL ontologies. I had some concerns about this but have been assured by the Protege Team that Protege Frames is still very much part of the Protege roadmap and the continued development of Protege 3 demonstrates this.

So, why not use OWL for Essential Architecture Manager?

This is a question we asked ourselves several years ago and we took some time to understand which approach best supported our requirements. The advice on the Protege Project website was a key part of making this decision – our requirements very neatly matched the description of the Frames-based ontology.

Some seem to contend that unless you are using OWL, you are not working on an ontology. This is not the case. It’s the semantics, the types (classes) of things, the actual things and the relationships between the classes and things that are what makes an ontology.

With Essential Architecture Manager, we are not trying to create a universal or standard ontology for the Enterprise Architecture of all organisations. Rather, by using Essential Architecture Manager, you are building an ontology about your organisation. We find it helpful to think about the Essential Meta Model as a partially-populated ontology, like the trunk and branches of a tree, and by modelling your enterprise in Essential, you are adding the leaves to the tree.

In this way, everyone who uses Essential is creating a different ontology. However, they are using the same language to build or complete that ontology. The Essential Meta Model provides a Domain Specific Language for Enterprise Architecture that users of Essential are able to use to build the ontology of their enterprise architecture.

Using the same DSL means that details of these ontologies can be shared, exchanged, reasoned on, etc. using a shared set of tools and shared semantics.

It’s the instances of the ontology that are specific to each ‘enterprise’ but the classes and types – the meta model – that are ‘standard’ across all Essential deployments.

We realise that many existing users of Protege are getting great benefits from using OWL but in addition to the better ‘fit’ of Frames to our knowledge base requirements, the simplicity and ease of use of Protege Frames is very important to the Essential Project.

By supplying an extensive class hierarchy in the Essential Meta Model, users of the Essential Project are somewhat different from many typical Protege users in that they do not need to start with Class modelling. Rather, they pick up the Class hierarchy and start creating instances. The Protege Forms make this very clear and straight-forward, so users are able to get on with ‘adding the leaves’ to the ontology. There is no need to know anything about ontologies or ontology development to use and get the benefits of Essential Architecture Manager.

This also means that we can clearly and easily separate the development of the Essential Meta Model class-hierarchy from the modelling activity – creating instances of those classes. Indeed, we normally hide the Classes tab in a multi-user configuration to reinforce this separation. The main activities in using Essential are about creating and relating instances. Designing and creating new classes is something to be approached when the out-of-the-box meta model needs to be extended – not as part of the day-to-day use of Essential.

By focussing on describing the details (the Instances) of the organisation – current, planned or future – rather than on how to describe (the Classes) the organisation, the benefits of enterprise architecture can be realised very quickly. During our consulting engagements, we’ve seen clients spend months defining their own detailed meta model before getting started on using this meta model. By picking up the Essential Meta Model – and extending it where and when required – we’ve seen clients gain real benefits of EA and EA tools in the same timeframe that others are still defining their meta model.

For those who are interested in OWL and Essential, Protege provides the capability to export the Essential Architecture Manager repositories in OWL format, preserving the class-hierarchy and the properties of these classes. This export could provide a starting point for those who are interested in using the Essential Meta Model ontology as a starting point for OWL-based activities.

I’d be really interested to hear about anyone’s experience doing just this.

Exploiting information that you already have about your organisation

Whether you are just starting out with a new EA modelling solution or already have a powerful repository, everyone has information about their organisation in many other places in many other forms. Exploit this existing information.

Many organisations embarking on EA modelling already have a wealth of information in a variety of disparate forms, such as presentations, drawing tools and most commonly in spreadsheets. Rather than re-key, or re-model all this work, shouldn’t we be able to exploit this existing information – lifting it directly – by importing it into Essential Architecture Manager?

Over the last few years, we have explored a variety of options for importing existing information into Protege (and therefore Essential Architecture Manager) and we realised that the only safe and reliable approach was to do this using the Protege API to add, remove and update information in the knowledge base in effectively the same was as the graphical user interface and let Protege take care of all the integrity issues as it does when you work with the forms. Using the Protege API is at the heart of the Essential integration tools.

The new Essential Integration tab plugin for Protege and Essential Architecture Manager simplifying the one-off ‘data load’ of things like your spreadsheets but also the on-going synchronisation of important data that is maintained in other systems. We’ve had some very good experience of doing this sort of synchronisation with configuration management databases and virtual server environment configurations.

This new tab is an evolution of the ‘skunk-works’ release of the Integration Server. We have made it into a tab to make it easier for both stand-alone and multi-user use (both modes are supported) and to make it easier to install and add this capability to your Essential Architecture Manager. The process of running an import or synchronisation is now much simpler than it was with the Integration Server.

The concept of the integration server will remain but will be re-worked to provide a means of automating imports and on-going synchronisations from other sources, e.g. on a scheduled daily basis. However, the new integration tab will be the most commonly used mechanism for importing information into Essential en-masse.

Getting information out of Essential to share with other systems is simply a matter of creating a suitable view for Essential Viewer, maybe an XML document or a CSV etc. With the contribution from Essential Project Community member, Mathwizard, this can be saved directly as an export file from Essential Viewer, using your browser.

So how to do you go about importing your spreadsheets, XML documents etc. using the integration tab?

As you might expect, XML is the ideal format for integrating the source information and the integration tab expects source data to be in XML format, so it is often easiest to work with an XML export of your source information.

The first step is to work out how this existing information will be represented in Essential. Unfortunately, there is no magic solution to this. You need to define how your source information maps to the Essential Meta Model and with this understanding define the transform that is required.

By the way, creating relationships and relationship classes during imports can be particularly tricky as the source information often does not represent the relationships in a way that can be mapped to the relationships in the Essential Meta Model. In many cases, it can be best to focus on importing the instances and then completing the relationships in Protege via the forms.

There are two approaches to defining the transform that I’ll explore in a moment. We intend to produce and share a library of transforms from common source formats into the baseline Essential Meta Model.

Currently, we have a transform for importing the XML representation of an Essential repository that Essential Viewer uses. This means that you can import elements from other Essential repositories, out of the box. Also, our previous experience with certain configuration management databases and virtual server environments mean that we can easily share transforms to import Technology Nodes and Technology Instances into Essential.

The two approaches to transforming your existing source information into the Essential Meta Model are:

  1. Transform you existing source to XML using the Essential Viewer XML schema. Then import this resulting XML into Essential using the out-of-the-box Essential Repository transform. This approach is useful for once-off data loads from existing sources such as spreadsheets where it’s most important to get the main instances into Essential rather than complex relationships.
  2. Define your own transform file and use this to have the integration tab transform your source information and import it into Essential. Although this may seem to be a more complex exercise than approach 1, you have more flexibility in defining how your source information maps to the Essential Meta Model. A how-to guide for writing transforms is being written at the moment that explains how the transforms work and the supporting tools that are available, e.g. a library of script functions that wrap the Protege API to support things like on-going synchronisation of instances in Essential with instances in other, external repositories.

We would be very happy to help anyone who needs to construct their own custom transform via the forums or even to undertake commissions to build custom transforms as required.

And of course, if you’ve built some transforms that you’d like to share with the Community, we would really like to add these to the Share area of the site.

Happy exploiting of your existing information!

Getting the Graphical Views

Modelling in Essential Architecture Manager is focused on capturing knowledge rather than drawing diagrams. However, graphical views can bring a lot of value. Here, we explore new graphical capabilities of EAM.

A couple of months ago, in my blog article ‘Where have all the graphical models gone?‘ I described our approach to capturing knowledge about the enterprise using the forms in Protege rather than drawing diagrams. However, as the saying goes, a picture speaks a thousand words, so I would now like to highlight some new graphical features that are now available in Essential Architecture Manager and to explore some more of the background to our approach to capturing knowledge.

In Essential Architecture Manager, many of the elements that we capture are modelled so that they have a Definition of what the element is. This is then elaborated by an Architecture that describes how that element is composed. A useful way to think about this is that we black-box every element. The Definition is what we see on the outside of the box. We can still use that element in the overall model even if we know nothing more about how it works or how it is composed. However, if we do know more about the element – or we find out the details later on – we can then open the black-box and describe the Architecture, which tells us how the element is composed or how it works.

In fact, the Definition-Architecture approach means that we can define multiple architectures for an element. e.g. an Application Service or Application Provider can have both a Static Architecture and a Dynamic Architecture. It’s certainly more manageable to be able to separate these.

The Definitions are very naturally captured using the standard forms in Protege. We need to capture textual descriptions, relate the element directly to other elements in the model and so on. All of which is very productive, quick and straight-forward using the forms. This is why much of the input in Essential is form based.

In contrast, the Architectures add a contextual dimension to the relationships and dependencies that we are capturing between elements. We quickly found that the basic forms made this rather complex. Fortunately, the GraphWidget of Protege makes capturing Architectures much more straight-forward and we use this graphical tool in combination with the basic widgets for the capturing the Architecture.

The diagrams that are produced to capture these Architectures are focused on utility. Visually, they are basic and agnostic to any particular notation. However, whilst recognising that these diagrams may not be something you would hang on the wall, it would still be very useful to have these diagrams appear in the relevant analysis reports of Essential Viewer. This would be in addition to, not instead of, producing views in specific notations or other ‘graphical reports’.

To provide this capability, we have just released an update to the Essential Widgets and Essential Viewer that takes a snapshot of each architecture diagram during the repository publishing process. These snapshots are then presented in the relevant reports, such as Business Process Definition, Application Module Summary, Technology Product Details and so on. The update makes it very easy to bring in a relevant architecture diagram to any custom report. These updates to Viewer and Widgets have been packed into the latest version (1.3) of Essential Architecture Manager and all are available now to download.

But that’s not the end of the story for getting graphical views of your architecture model. Within the Protege environment, there is the Jambalaya SVG tab that provides a wealth of graphical reporting capabilities. Although we have been focusing on reporting within the Viewer environment – to open the analysis and view of the architecture to as wide an audience as possible in the organisation – there could be some value in sharing Jamabalaya reports with the community.

I would also like to draw your attention to Clint Cooper’s recent contribution – the Visio Export Tool. This produces a rendering either of selected areas or of the whole repository in Microsoft Visio. The resulting Visio file provides a readily-shared, graphical view of the model that can be easily manipulated to provide the view that you need to share with the wider audience in your organisation. Many thanks to Clint for sharing this with the rest of the Essential Project Community.

Although we take a forms-based approach to capturing the knowledge about the elements in the enterprise, there are a range of options for producing graphical views of this knowledge. From the snapshots of the architecture capture diagrams, clickable SVG diagrams to the Visio exports, there are now a range of options for getting the graphical view of your architecture that you need.

Enterprise Architecture, Knowledge Management – Knowledge Representation

A very nice definition of Knowledge Management rings very true for Enterprise Architecture – modelling in particular. How you realise the capabilities in the definition all depends on how you represent knowledge about your organisation.

I was pleasantly surprised when I came across this definition of Knowledge Management by Dave Snowden.

I like this definition of KM but was struck by how it could equally apply to Enterprise Architecture, in particular when Dave mentions processes and technology, and especially when thinking about the contents of our Enterprise Architecture models.

Back in the 1990’s ‘boom’ days of Knowledge Management, there were a number of definitions of KM – many of which seemed to be tied to tools (hmmm, sounds familiar…) – but the thing that stuck in my mind was that knowledge has to be transformed from the ‘raw’ information about the organisation. This transformation is what makes the knowledge applicable to any relevant scenario, having captured information about specific instances of it. And this transformation of information into knowledge – to some extent analogous to the transformation of data into information – is what never seemed to happen in any of the early Knowledge Management solutions. Apparently, all that was required was an Intranet where anyone in the organisation could contribute their knowledge and then anyone who needs to apply that knowledge just searches the Intranet to find it. More latterly, Web 2.0 was going to solve those problems all over again using wikis and blogs and all the social tools, where people could contribute their knowledge so that others can find and apply it. For some areas, wikis can be very useful if notoriously hit-and-miss when searching.

And that’s the real trick that none of these solutions solve; how do you find the knowledge? How can a search engine make the link between a blog posting about a particular topic to a request for best practice on a process? Without the capability to do this, I don’t think any of these solutions are what I would call Knowledge Management. Without the ability to abstract the contextual, subjective information to build generalised knowledge we can’t apply these valuable contributions to other similar scenarios.

How do we transform the information into knowledge? Well, at its simplest, we need some form of ‘framework’ on which to hang the information that we have to hand, so that we can apply objective definitions to what the information is about and relate it to other information with strong semantics. We can then use this knowledge framework to query – not just search! – the information that has been captured from many different sources and in a meaningful way.

Wait a moment, that sounds just like an EA meta model. It certainly does and that’s when I start to wonder, is it that actually Enterprise Architecture and Knowledge Management should be effectively the same thing – it is just that their current guises are particular perspectives on the knowledge base? For me, Knowledge Management has to be more than a bunch of Intranet sites and wikis and similarly, Enterprise Architecture models have to be more than a bunch of diagrams and pictures if we are to truly apply the knowledge that people are trying to capture.

The meta model gives us a framework for representing knowledge about the enterprise that defines the types of information that we need to capture and how this information relates. The resulting model of the organisation enables us to apply the knowledge to the particular problem that we are dealing with right now, whether it is a large scale, strategic issue or an fine-grained query from the operational or tactical agenda.

And this is what Enterprise Architecture is really about. EA is not the modelling. Rather, EA is about the application of knowledge about the enterprise (ideally, represented in a model, because it rapidly gets very complex) to the problems at hand.

In taking a knowledge representation approach to EA and KM (and I believe we should), is Enterprise Architecture modelling the same as Knowledge Management?