How can organisations afford the luxury of enterprise architecture in the middle of a global recession?

The current downturn is forcing companies and government agencies across the world to prune their IT budgets and shelve major business change projects. In this adverse economic climate enterprise architecture will often be regarded by management as an overhead activity of little relevance to the cost-cutting challenges they face. There are two main reasons why this attitude can prevail:

  1. Enterprise architecture is frequently associated with the design of the future operating model that will result from planned major changes in business processes. When times are hard, such long-term interests usually take a back seat and quick-fix initiatives take priority.
  2. By its very name ‘enterprise architecture’ implies building a comprehensive view of the entire business operating model. Organisations in survival mode will be more interested in surgical measures focused on selected areas of the business, than in the contemplation of elegant, all-embracing business models.

As a consequence, enterprise architecture activities may be seen as unimportant and easily expendable. Skills and knowledge built up over many years may then be lost as expert practitioners are laid off or else redeployed to take on work seen as having greater urgency.

The Essential Project Team believes such an outcome can be counterproductive, as enterprise architecture has a potentially vital role to play in enabling organisations to handle the challenges of an economic downturn. The key to this is to ensure that enterprise architecture activities are focused on what the business regards as high priority.

To address adverse trading conditions, the enterprise architect first needs to concentrate on the current state, rather than the future business operating model. Then the focus should be specifically on those areas where rationalisation of existing assets and business process simplification are likely to pay dividends in the short term. The Essential Architecture Manager has been designed to enable promising areas to be selected and modelled without the need to cover the entire spectrum of business and IT capabilities. And that modelling should be carried out only to the level of detail that is necessary to enable appropriate management decisions to be made.

We know that many organisations could make significant short-term savings simply by rationalising their technology platforms – for example, by eliminating redundant servers. The media company that provided the original beta test site for the Essential Architecture Manager during 2008 opted to use the package in the first instance to model and record its current technology assets. This exercise revealed inefficiency, duplication and obsolescence in a number of areas, and it highlighted possibilities for early savings to be made.
A similar case can be made for organisations using tools such as the Essential Architecture Manager to record details of current IT applications and the business activities they support. This can reveal scope for rationalisation and simplification, which can lead in turn to significant benefits – for example, savings in licence fees and software maintenance charges. While working with a financial services company, one member of the Essential project Team used such a targeted architectural approach to demonstrate that a range of new financial products could be supported by the firm’s existing systems: this obviated the need for a proposed new suite of applications.

In short, now is the time for smart enterprise architects to lift up the drain covers, rather than scan the distant horizons. Those who are sensitive to current business priorities will hopefully then still be around to contribute to and support a more strategic agenda when better times return.

Training in the Essential Project

We’ve been asked several times about training.  Obviously training can cover a broad range of topics.  In considering what type of training we might offer, we have thought about what would be useful to the people that are using, or will be using, the Essential Architecture Manager and tried to stick to our aim of offering things that are effective and add value.

We don’t think we should offer training that covers best practices in Enterprise Architecture or modelling techniques, but short courses, with a hands-on practical approach, that show how to use the Essential Project tools to achieve specific business goals, with clear value.  We’ve had a quick brainstorm of ideas based on what people have asked for and where we see Essential delivering value in the current economic climate.  The courses we think would be of use are:-
1.    Business Operating Model Planning and Control
2.    Application Portfolio Management
3.    Technology Infrastructure Management
4.    Managing Project Architecture Dependencies within Programmes
5.    Strategic Enterprise Architecture Definition and Transition Planning
6.    Essential Viewer Custom Report Development
7.    Essential Architecture Manager Administration

We think these would be 1 or 2 day courses, with a clear business case for each course to help justify the training costs.

We are keen to gauge interest in training, both in the types of courses that we have identified and also in the delivery method and location.  Are the courses the right ones or are there others we should be looking to deliver?  What would be useful?  If you think we’ve got it wrong or can give us more details on your views, please feel free to add a comment to this blog.

Thanks in advance for taking the time to do this.

Developing The Essential Project Through Community

One of the main hopes we had for the project when we launched was that the community of Essential users would become part of the development process for the project. In this blog, I explain how you can contribute.

Developing The Essential Project Through Community

It’s only been just over a month since we launched the Essential Project and we’re very pleased and excited in the response. One of the main hopes we had for the project when we launched was that the community of Essential users would become part of the development process for the project. Already we’ve had some some very interesting requests, through the forums, for things that we haven’t encountered before but can handle through small extensions to the Essential Meta Model and Essential Viewer reports.

Why Contribute?

As the Essential Project tools are open-source, anyone is free to make any extensions that they require. We are very keen that as many extensions as possible are put back into the Project and shared with the community – to help make sure that the Essential Project has the coverage that is required.

By contributing your extensions and enhancements back to the Project, it not only drives the Project forward but means that any extensions that you need will be brought into and managed by the Project as it moves forward.

How will this work?

Obviously, we will need some sense of order to how the contribution process works. The Essential Project Team provide governance of the Essential Meta Model and co-ordinate requests for enhancements, taking other similar requests into account before designing and delivering them. Likewise, contributions of completed components, e.g. a new analysis report or meta model extension, can be shared by contributors through the Essential Project website. When a meta model extension is received, the Team will produce the update pack to make it simple for other community members to introduce the extension to their Essential installation.

However, we are really keen that requests for additional capabilities (modelling, reporting or software extensions) are developed with the community to make sure that as many different perspectives on the same requirement are considered. To make this work, we would like to operate a process in the spirit of the approach taken for the Java Community Process and run this via the forums, in the Enhancements and Suggestions forum.

What is the process?

The main thrust of the process – called the Essential Community Process (or ECP) is about capturing requirements and reviewing the proposed solution with as wide a group as possible. Although in many cases (especially in the early stages of the Project) the Essential Project Team will be the ones developing the solution, the process does not assume this  – it could be any member of the community.

The process can be iterative and will operate as follows:

  • Request
  • Discuss and gather further requirements from community
  • Design proposed solution
  • Circulate to the community for defined period of time
  • Implement
  • Test within the community
  • Review tests and approve
  • Release update

Responsive

Clearly there’s a balance to strike between taking a controlled approach and the time it takes to produce the answer to the request. However, I think that we should be able to run the process such that the time taken is proportionate to the scope and complexity of the requirement being met (in some cases, perhaps only days). The important thing is to open up the gathering of requirements, design and testing activities to the community while making sure that the results fit in with the overall Essential Meta Model, can be taken forward and are easily shared.

We’re already looking at how we should capture software (or any other) licensing models and instances of these models, and we’d like to know more about requirements for organisation policy and compliance. If you have any thoughts on any requirements that you have in these areas, please let us know by taking part in this forum.

Sample Repository Coming (Very) Soon…

If you’ve been keeping an eye on the forums you’ll know that we’re aiming to get the sample repository complete by this Friday 24th April. Sorry for the delay, we had some EAS client work we had to finish which meant we couldn’t dedicate as much time to this as we’d hoped. So, it should be Friday and it is a very simple example based on a fictional financial services company. We’ll look to extend it over time and also will look for people to give us ideas on where to extend it to help show how things are modelled.

On another note, we’ve reached the milestone of 500 downloads which is great to see. It’ll be nice to see how people are using it. If you’d be willing to share a bit about what you’re doing, how you’ve found using the tools, then let us know and we’ll publish some experiences.

Manual Processes and ‘Non-Information’ Technology

How do we ensure that the enterprise architecture truly covers the enterprise? Many architects find that many EA frameworks are very oriented towards managing the IT aspects of the architecture and do not provide enough support for the non-IT things.

How do we ensure that the enterprise architecture truly covers the enterprise?

Many architects find that many EA frameworks are very oriented towards managing the IT aspects of the architecture and do not provide enough support for the non-IT things. I suspect that there’s an element of “the squeaky wheel gets the oil” here as managing and delivering IT systems has been and continues to be one of the most tricky activities that organisations do. However, I don’t think it helps to separate IT-focussed enterprise architecture initiatives from the broader EA discipline, rather the former is a part of the latter. We designed the Essential Meta Model to provide the constructs for capturing knowledge about all aspects of the enterprise (including the extended enterprise), where Information Technology is just one class of technologies that are used to support how the organisation functions.

Enterprise Architecture is about understanding and managing how the organisation works. This understanding enables us to make the right decisions both tactically and strategically in whatever issue is at hand. What information about the organisation do we need to capture? At the core of this understanding are people; processes; information that people need to perform the process; applications to support or automate the execution of the processes; and technology to support those applications. Pretty much all other knowledge about the enterprise refers to or operates on these core concepts. The last two things I mentioned (applications and technology) do not have to be Information Technology things. I’ll come back to these in a moment.

Is the knowledge we need just about what we can see in the enterprise? No, we need 3 main views of this knowledge, which I’ll describe in the terms we use in Essential Meta Model:

  • Conceptual. This defines ‘The What’, enables us to capture the requirements and very importantly, gives us a semantic grounding for all of the other elements that we will capture at the following views that operate at a lower level of abstraction.
  • Logical. This defined the ‘The How’ and is where we design how we see the enterprise working – either now or in the future.
  • Physical. This defines ‘The Where’ and ‘The Who’ and is where we capture what’s actually there in the organisation, such as individuals, teams, data stores, deployments of applications and technology instances (e.g. servers).

With these three views, we can then understand things like: how many ways do we do the same thing? Is there a good reason for that or should we be standardising?

OK, so this sounds all very much IT-oriented, and in many ways, the physical is particularly tricky for IT architecture for a number of reasons. First a start, physical applications don’t really exist as such in the physical world. The digital media arena demonstrates this, where physical instances of the product can be produced instantly and for no cost. Many applications – especially mash-ups and composites are run-time configurations of a distributed set of software components and other applications. What are the physical ‘bits’ (pardon the pun) of one of these?

We’ve had a lot of success with capturing and managing the physical technology architecture in the last year or so, taking clustered, virtual and distributed server architectures into account. However, we had to make sure we didn’t take any shortcuts with how these things really exist in the physical world.

So information technology things are well catered for but what about manual processes or processes that are performed by machines or by people using machines? How do we capture these in our meta model? Well, a manual process is performed by people we’ve captured in the physical view of the business layer. It’s likely that information is required to perform the process and we can capture that. (Note, ‘information’, not ‘data’). If it’s a manual process, surely it has no supporting applications defined for it. Or does it? Actually, the lack of information here cannot be used safely to make assumptions like this. Rather, we take exactly the same approach with ‘manual’ processes as we take with any other and capture the supporting applications and the technology used to support/deliver that application.

Processes can still be supported by ‘applications’ that are not delivered by Information Technology. For example, we might have a paper-based form that is used to support the process of capturing customer information. This form is the application and the paper is the supporting technology, which is stored in a filing cabinet instead of a database. Similarly, if we need to capture that a milling machine is involved in the process of manufacturing a widget, that machine is part of the technology architecture of the application (the behaviour of the control system of the machine) that supports that manufacturing process.

If someone described doing a manual process to me, I would ask “and what do you use to do the process?” Even if the answer is pen and paper or “I talk to the customer, using a script”. From the answers to these questions we get an understanding that we can choose to characterise as “manual”.

I don’t think that these examples are stretches of the terms ‘application’ and ‘technology’ if you think conceptually about what these are doing. As I mentioned earlier, the organisation is about people performing processes to meet the organisation’s objectives – to produce products, provide services etc. – and there are applications that support the doing of the processes (or could fully-automate the process) and these applications are supported by technologies. It doesn’t matter if this involves Information Technologies or any other Technologies.

The point is that manual is a function of what supporting things you use to do the process. With the exception of fully-automated processes, most processes -including those supported by applications – are driven manually. Indeed, many so-called “manual” processes are still supported by office productivity tools.

Although this involves more modelling, it’s far more accurate to capture processes in this way than adding a “this is manual” attribute to the process. This gives us the objective knowledge in the model on which to later on base inference and derive knowledge from, long after the process was captured.

There are many other elements of knowledge that architects of all types need to capture about the enterprise in order to support their architectural processes. We are already working on expanding what we call the EA Support layer of the Essential Meta Model. The things in here refer to or operate on the elements in the Core meta model and so far we’ve found that this approach has elegantly handled all our requirements to date. But we’d love to hear about the sorts of things that you need to have in there.

What are the Essential Project Team up to?

If you’ve been keeping an eye on the forums you’ll know that we’re in the process of putting together a sample repository.  The sample will take a fictional investment management company as an example and show a thin slice through all the meta-model layers to demonstrate the linkages between the different architecture concepts.  We chose an investment management company as it’s an area we know well, but also because we think it is broad enough to be understood without having to be a domain expert.  We’re currently populating the model and are expecting to get it out in the next couple of weeks – time permitting.
Something we’ve been asked a number of times via the forums is for a pictorial representation of the meta model, showing the links within and between layers.  We have been working on this and it is nearing completion so we hope to have it up on the site within a week or so, again, time permitting.
With many people trying out Essential now, it would be great to hear views on it, and ideas as to where the tool, the website or the documentation could be improved.  Please do let us have your views on the Forums.

Where have all the graphical models gone?

Essential Architecture Manager takes a different approach to modelling your organisation’s enterprise architecture.

Essential Architecture Manager takes a rather different approach to modelling your organisation’s enterprise architecture. There are many benefits of this approach but the way of working may seem unusual – or even limiting – to architects who are more familiar with more graphical modelling tools.

Knowledge capture

The key thing to get to grips with in Essential Architecture Manager is that modelling your organisation is about capturing information about it in a structured way that builds a detailed knowledge base that can then be used to answer questions about the enterprise.

The repository in Essential Architecture Manager isn’t a repository of diagrams or of diagram elements. Rather the repository is a knowledge base of instances of elements of the Essential Meta Model. To capture this information, you simply complete the forms in modelling environment, Protege, which adds this new information to the knowledge base.

Far from being a low-tech solution, the form-based approach to capturing the information is very useful. We often found that with purely-graphical tools, we were building simple spreadsheet forms or the like to load groups of new artefacts into the repository. The Protege forms give us this straight away.

Where it makes sense, to, simple graphical forms are provided to make the capture of the information about your organisation as easy as possible, e.g defining business process flows, or technology architectures. However, these use simple graphical elements that help focus the user on capturing the information rather than which colours to use or how to lay out the diagram.

The benefits of this approach are that we don’t need to worry about particular diagramming notations, e.g. UML, to capture information. Simply complete the forms. This means that in many cases, the information can be captured even without an understanding of how the meta model works or any particular notation.

Don’t model the report – separating capture from reporting and viewing

The Essential Meta Model has been designed to represent detailed knowledge about enterprise architectures so that we can ask questions of it – many of which are answered by derived or inferred information.

We’ve consciously separated the capture of the information – modelling – from the reporting and analysis of the model in the knowledge base. By doing this, we avoid common scenarios where models are created that answer only the specific question that the architect had in mind when capturing the information – what we call modelling the report.

The two components of Essential Architecture Manager, the Essential Modeller [Protege] and Essential Viewer make the separation of the capture from the reporting and analysis.

Notation agnostic

Essential Viewer gives us the capability to present views of the model in any notation that we need – potentially multiple representations for different stakeholders. However, the real power of it is the ability to perform complex analysis of the modelled enterprise architecture and present this in forms that can be consumed by stakeholders in the format that is best for them.

So, what about graphical models?

It’s in Essential Viewer that the “graphical models” are to be found, dynamically created in the right notation for your requirements.

Essential Project Launch Success

The launch of The Essential Project has far exceeded our expectations in terms of number of visitors and number of downloads.

To date we have had over 1700 visits, with nearly 800 unique visitors to the website, and there have been about 200 downloads of the Essential Tool. The most popular pages on the site, apart from the home page, appear to be the downloads page, getting started tutorial, meta model documentation and the forums.

We realise it is early days and many people may still be investigating the tool, but we would love to hear any views on it, and where we can improve either the tool, the website or the documentation we have provided. If anyone has time, please do send feedback through the forums.

Identify the problems your EA needs to solve

In my opinion, when introducing enterprise architecture into an organisation the most crucial aspect is the identification of the problem you need it to solve.  And to do this effectively I believe that you need to clearly understand the aims and objectives of the business at that time.

We often find the EA team sitting within the IT organisation, but the team still needs to ensure that the identification of need is business led and that the EA is not used simply to document technical IT issues.

For example, it may be that the most important objective for the business at a point in time is to reduce costs to remain competitive in its pricing.  Enterprise Architecture can meet this need in a number of ways depending on the company in question, but one way will be to capture the current state in the application architecture layer to allow consolidation of applications and thereby achieve cost reductions.

However, this would be a wholly inappropriate response if the business need were to expand into new markets.  In this case the Enterprise Architecture should be identifying common processes and services that could be shared across divergent businesses to aid speed of change and agility.

This is at the crux of the reason that, to date, many companies are not satisfied with their EA initiative.  All too often the EA will follow a prescribed, often IT based, root; lets capture the current state first; lets create an application catalogue; lets sort out our integration architecture; without first determining what the business needs it to do.

Once the link to business need is in place the EA can begin to provide what the business needs and positive results will be quickly seen.

Once the business need is identified, it’s time to consider the tools, people and processes needed to support it.  In terms of processes we have identified two major contributing factors to a successful EA, ‘understanding’ and ‘control’.

By ‘understanding’, we mean having a coherent view of the business and IT assets of your enterprise readily available and accessible in order to support well-informed tactical and strategic decision-making.  As the amount of information to pull together is often large, tools with the right information captured will give the understanding part of the equation.

By ‘control’, we mean having appropriate processes and techniques in place to manage and co-ordinate changes to these assets in line with these decisions.

In general, many of the problems that an EA will solve will be IT related, but the link with the business is imperative to ensure that the problem is the most relevant at that time and, crucially, that the EA Team are seen as making a real difference to the business in a relatively short time space.

How can we exploit existing architectural information?

Even if you are just starting out with your enterprise architecture initiatives, it is highly likely that you already have architectural information about your organisation captured in some form or other. How can we exploit the existing architectural assets with Essential Architecture Manager?

In a bit of contrast to Jason’s last posting, I’ve taken a more technical perspective this time.

Even if you are just starting out with your enterprise architecture initiatives, it is highly likely that you already have architectural information about your organisation captured in some form or other. How can we exploit the existing architectural assets with Essential Architecture Manager? Can we automatically load these assets into the Essential Architecture Manager repository? And if we can do that, can we get information out of Essential Architecture Manager to be loaded into some of our existing systems?

These are questions that we quickly encountered when we started using Essential Architecture Manager in real organisations.

Potential Approaches

Since Essential Architecture Manager is built on Protege, we started to explore the tabs that were already available for bringing external data sources into your Protege project, e.g. the very useful DataMaster.

It is often the case that existing sources of information lack the structure of a formal meta model such as that in Essential and when they do, there are bound to be meta concepts that do not directly map to the Essential Meta Model.

We therefore find ourselves with a typical application integration mapping and transformation scenario. In order to import the information from the existing source, we may have to combine or split elements from the source information in order to map it to the target. We may also need to create inferred elements from the source information.

The existing plugins for Protege lacked this mapping and transformation capability, and so at first, where an import was only needed as part of the initial start up of the initiative, we found that we could construct transformation scripts, e.g. XSLT, to generate entries for the .PINS file of the Protege project and then paste them manually into the file – an approach known as ‘PINS hacking’ in the Protege community.

While this solved some immediate problems, we identified a number of problems with this approach, such as importing into projects using a database backend (there’s no .PINS file to hack!), creating new, unique instance IDs in Protege, and of course on-going synchronisation between the Essential repository and updates to the external information sources.

The Solution

It was clear to us from this point that in order to reliably and predictably import information into Essential Architecture Manager, we needed to be interacting directly with Protege through its API and not through its underlying data store – good practice for any data integration really. This way, Protege could take care of creating unique instance IDs,  defining relationships between imported artefacts etc.

We found that through the Script Tab, we could “drive” Protege, via its API, in a way that effectively automated the steps that you would do if you were using the front-end GUI. This way, we knew that Protege could properly manage the integrity of the repository. We just needed an effective way of turning the source information into Protege scripts that we could run and we’d have the basis of our solution.

We’ve now been using what we’re calling the Essential Integration Server for several months to synchronise the Essential Architecture Manager repository with an external configuration management suite. You may have noticed that every class in the Essential Meta Model has a slot called ‘external_repository_instance_reference’. This is used by Essential Integration Server scripts to synchronise individual assets between Essential Architecture Manager and one or more external information sources. An instance in Essential can have multiple external references and we can combine information from multiple sources to build a more complete picture of an architectural asset in the Essential repository.

The mapping between the existing information source and the Essential Meta Model is put together by defining an XSLT (or any similar approach) that writes the import script. We are building a library of useful Python script functions that help with things like creating instances (or returning a reference to it, if it already exists in the repository) or building certain more complex relationships. Using the script language is fairly straight-forward and is almost as productive as a more graphical integration tool – mainly because we have the power of the full Protege API plus a rich scripting language that enables us to handle any import / integration scenario. The Essential Integration Server provides a web-based user interface for running these XSLT scripts on the source data and producing the resulting Protege scripts automatically. This is particularly useful when you are running the import on a regular on-going basis.

However, the Essential Integration Server is not quite fully automated and that’s why we haven’t released it yet in the same way as the other components. Currently, it does everything except run the scripts for you in Protege’s Script tab. That’s the manual step that we are working to automate at the moment.

Getting information out of Essential

I’ve described in some detail how we recommend that existing information is imported or synchronised into Essential Architecture Manager. How about getting the information that is in Essential out to be used by other systems?

Essential Viewer already provides that capability. Rather than producing a ‘report’ that renders HTML, you can simply build a report that produces, XML, CSV files or whatever your systems need. We’ve used this with a lot of success and because it is run from within Essential Viewer, your target system (or your integration environment) can request this extract via HTTP as a REST-style web service.

Complete solution coming soon

The solution for getting information into and out of Essential Architecture Manager is there. It needs a little more work to make importing fully automated but what we have today is being used in anger on a regular basis in a real organisation right now.

If you need to get your existing information into Essential right now, let us know and we’d be more than happy to supply the current version of Essential Integration Server and to help you build the mapping to the Essential Meta Model.