Where have all the graphical models gone?

Essential Architecture Manager takes a different approach to modelling your organisation’s enterprise architecture.

Essential Architecture Manager takes a rather different approach to modelling your organisation’s enterprise architecture. There are many benefits of this approach but the way of working may seem unusual – or even limiting – to architects who are more familiar with more graphical modelling tools.

Knowledge capture

The key thing to get to grips with in Essential Architecture Manager is that modelling your organisation is about capturing information about it in a structured way that builds a detailed knowledge base that can then be used to answer questions about the enterprise.

The repository in Essential Architecture Manager isn’t a repository of diagrams or of diagram elements. Rather the repository is a knowledge base of instances of elements of the Essential Meta Model. To capture this information, you simply complete the forms in modelling environment, Protege, which adds this new information to the knowledge base.

Far from being a low-tech solution, the form-based approach to capturing the information is very useful. We often found that with purely-graphical tools, we were building simple spreadsheet forms or the like to load groups of new artefacts into the repository. The Protege forms give us this straight away.

Where it makes sense, to, simple graphical forms are provided to make the capture of the information about your organisation as easy as possible, e.g defining business process flows, or technology architectures. However, these use simple graphical elements that help focus the user on capturing the information rather than which colours to use or how to lay out the diagram.

The benefits of this approach are that we don’t need to worry about particular diagramming notations, e.g. UML, to capture information. Simply complete the forms. This means that in many cases, the information can be captured even without an understanding of how the meta model works or any particular notation.

Don’t model the report – separating capture from reporting and viewing

The Essential Meta Model has been designed to represent detailed knowledge about enterprise architectures so that we can ask questions of it – many of which are answered by derived or inferred information.

We’ve consciously separated the capture of the information – modelling – from the reporting and analysis of the model in the knowledge base. By doing this, we avoid common scenarios where models are created that answer only the specific question that the architect had in mind when capturing the information – what we call modelling the report.

The two components of Essential Architecture Manager, the Essential Modeller [Protege] and Essential Viewer make the separation of the capture from the reporting and analysis.

Notation agnostic

Essential Viewer gives us the capability to present views of the model in any notation that we need – potentially multiple representations for different stakeholders. However, the real power of it is the ability to perform complex analysis of the modelled enterprise architecture and present this in forms that can be consumed by stakeholders in the format that is best for them.

So, what about graphical models?

It’s in Essential Viewer that the “graphical models” are to be found, dynamically created in the right notation for your requirements.

Essential Project Launch Success

The launch of The Essential Project has far exceeded our expectations in terms of number of visitors and number of downloads.

To date we have had over 1700 visits, with nearly 800 unique visitors to the website, and there have been about 200 downloads of the Essential Tool. The most popular pages on the site, apart from the home page, appear to be the downloads page, getting started tutorial, meta model documentation and the forums.

We realise it is early days and many people may still be investigating the tool, but we would love to hear any views on it, and where we can improve either the tool, the website or the documentation we have provided. If anyone has time, please do send feedback through the forums.

Identify the problems your EA needs to solve

In my opinion, when introducing enterprise architecture into an organisation the most crucial aspect is the identification of the problem you need it to solve.  And to do this effectively I believe that you need to clearly understand the aims and objectives of the business at that time.

We often find the EA team sitting within the IT organisation, but the team still needs to ensure that the identification of need is business led and that the EA is not used simply to document technical IT issues.

For example, it may be that the most important objective for the business at a point in time is to reduce costs to remain competitive in its pricing.  Enterprise Architecture can meet this need in a number of ways depending on the company in question, but one way will be to capture the current state in the application architecture layer to allow consolidation of applications and thereby achieve cost reductions.

However, this would be a wholly inappropriate response if the business need were to expand into new markets.  In this case the Enterprise Architecture should be identifying common processes and services that could be shared across divergent businesses to aid speed of change and agility.

This is at the crux of the reason that, to date, many companies are not satisfied with their EA initiative.  All too often the EA will follow a prescribed, often IT based, root; lets capture the current state first; lets create an application catalogue; lets sort out our integration architecture; without first determining what the business needs it to do.

Once the link to business need is in place the EA can begin to provide what the business needs and positive results will be quickly seen.

Once the business need is identified, it’s time to consider the tools, people and processes needed to support it.  In terms of processes we have identified two major contributing factors to a successful EA, ‘understanding’ and ‘control’.

By ‘understanding’, we mean having a coherent view of the business and IT assets of your enterprise readily available and accessible in order to support well-informed tactical and strategic decision-making.  As the amount of information to pull together is often large, tools with the right information captured will give the understanding part of the equation.

By ‘control’, we mean having appropriate processes and techniques in place to manage and co-ordinate changes to these assets in line with these decisions.

In general, many of the problems that an EA will solve will be IT related, but the link with the business is imperative to ensure that the problem is the most relevant at that time and, crucially, that the EA Team are seen as making a real difference to the business in a relatively short time space.

How can we exploit existing architectural information?

Even if you are just starting out with your enterprise architecture initiatives, it is highly likely that you already have architectural information about your organisation captured in some form or other. How can we exploit the existing architectural assets with Essential Architecture Manager?

In a bit of contrast to Jason’s last posting, I’ve taken a more technical perspective this time.

Even if you are just starting out with your enterprise architecture initiatives, it is highly likely that you already have architectural information about your organisation captured in some form or other. How can we exploit the existing architectural assets with Essential Architecture Manager? Can we automatically load these assets into the Essential Architecture Manager repository? And if we can do that, can we get information out of Essential Architecture Manager to be loaded into some of our existing systems?

These are questions that we quickly encountered when we started using Essential Architecture Manager in real organisations.

Potential Approaches

Since Essential Architecture Manager is built on Protege, we started to explore the tabs that were already available for bringing external data sources into your Protege project, e.g. the very useful DataMaster.

It is often the case that existing sources of information lack the structure of a formal meta model such as that in Essential and when they do, there are bound to be meta concepts that do not directly map to the Essential Meta Model.

We therefore find ourselves with a typical application integration mapping and transformation scenario. In order to import the information from the existing source, we may have to combine or split elements from the source information in order to map it to the target. We may also need to create inferred elements from the source information.

The existing plugins for Protege lacked this mapping and transformation capability, and so at first, where an import was only needed as part of the initial start up of the initiative, we found that we could construct transformation scripts, e.g. XSLT, to generate entries for the .PINS file of the Protege project and then paste them manually into the file – an approach known as ‘PINS hacking’ in the Protege community.

While this solved some immediate problems, we identified a number of problems with this approach, such as importing into projects using a database backend (there’s no .PINS file to hack!), creating new, unique instance IDs in Protege, and of course on-going synchronisation between the Essential repository and updates to the external information sources.

The Solution

It was clear to us from this point that in order to reliably and predictably import information into Essential Architecture Manager, we needed to be interacting directly with Protege through its API and not through its underlying data store – good practice for any data integration really. This way, Protege could take care of creating unique instance IDs,  defining relationships between imported artefacts etc.

We found that through the Script Tab, we could “drive” Protege, via its API, in a way that effectively automated the steps that you would do if you were using the front-end GUI. This way, we knew that Protege could properly manage the integrity of the repository. We just needed an effective way of turning the source information into Protege scripts that we could run and we’d have the basis of our solution.

We’ve now been using what we’re calling the Essential Integration Server for several months to synchronise the Essential Architecture Manager repository with an external configuration management suite. You may have noticed that every class in the Essential Meta Model has a slot called ‘external_repository_instance_reference’. This is used by Essential Integration Server scripts to synchronise individual assets between Essential Architecture Manager and one or more external information sources. An instance in Essential can have multiple external references and we can combine information from multiple sources to build a more complete picture of an architectural asset in the Essential repository.

The mapping between the existing information source and the Essential Meta Model is put together by defining an XSLT (or any similar approach) that writes the import script. We are building a library of useful Python script functions that help with things like creating instances (or returning a reference to it, if it already exists in the repository) or building certain more complex relationships. Using the script language is fairly straight-forward and is almost as productive as a more graphical integration tool – mainly because we have the power of the full Protege API plus a rich scripting language that enables us to handle any import / integration scenario. The Essential Integration Server provides a web-based user interface for running these XSLT scripts on the source data and producing the resulting Protege scripts automatically. This is particularly useful when you are running the import on a regular on-going basis.

However, the Essential Integration Server is not quite fully automated and that’s why we haven’t released it yet in the same way as the other components. Currently, it does everything except run the scripts for you in Protege’s Script tab. That’s the manual step that we are working to automate at the moment.

Getting information out of Essential

I’ve described in some detail how we recommend that existing information is imported or synchronised into Essential Architecture Manager. How about getting the information that is in Essential out to be used by other systems?

Essential Viewer already provides that capability. Rather than producing a ‘report’ that renders HTML, you can simply build a report that produces, XML, CSV files or whatever your systems need. We’ve used this with a lot of success and because it is run from within Essential Viewer, your target system (or your integration environment) can request this extract via HTTP as a REST-style web service.

Complete solution coming soon

The solution for getting information into and out of Essential Architecture Manager is there. It needs a little more work to make importing fully automated but what we have today is being used in anger on a regular basis in a real organisation right now.

If you need to get your existing information into Essential right now, let us know and we’d be more than happy to supply the current version of Essential Integration Server and to help you build the mapping to the Essential Meta Model.

PowerPoint and Excel for Architecture Modelling; Why Not?

Whether planning, designing or executing IT change, there are plenty of situations that require discovery and analysis of architecture related information.

Often, when a piece of work is viewed as a one-off exercise (e.g. IT solution architecture design, architecture reviews), we see a prevalence of architecture information captured using any combination of Word documents, Excel spreadsheets, presentation slides and Visio diagrams as project outputs.

In the short term, these formats appear to serve their purpose; they provide a means of capturing architecture elements and relationships for both analysis and communication. But why is it that over time, they ultimately become inefficient and cumbersome when used as a means of retaining architectural knowledge?

What You See is What You Get: If you want a new perspectives on the information captured, then you will usually find yourself drawing another diagram or designing another spreadsheet. Over time, maintaining consistency across these different views of your architecture becomes increasingly difficult.

The Nth Dimension: Here, we’re referring to the challenges associated with capturing complex multidimensional inter-relationships using office productivity tools. Even when armed with the rows, columns, formulas and scripts afforded by a spreadsheet, it would certainly be a non-trivial exercise to map the applications, underlying technology, information exchanged and business processes involved in, for example, an organisations’s global integration architecture?

Manual Meta-Model: Even if you do your best to ensure that common terms are used across your spreadsheets, documents and diagrams, the fact is that these formats lack any meaningful ability to share fine-grained information elements without some form of programming. In other words, it is left up to the project or architecture team to manually enforce a standardised, shared meta-model for consistent semantics. Not a very scalable approach.

Repository-based tools to the rescue then? Well, they are certainly capable of addressing most of the maintainability and scalability issues associated with using office productivity and drawing tools for architecture modelling. However, I still often find myself asking the question:

“Why do even the most experienced architects (with access to sophisticated repository-based tools), fail to resist the temptation to go back to good old PowerPoint, Excel and Visio”

I don’t believe there is a simple answer to this question, but I ‘m pretty sure that beyond familiarity, it is simplicity and ease of use that draws us back to them; characteristics that are not always the first to spring to mind when working with repository tools.