Data Lens

You may have noticed from our site that the Data Lens is in beta.  It’s a lens that we’ve developed because we’ve been continually told that people don’t have control of their data.

In our EA consulting, we have seen:

  • Organisations that were unwittingly reporting incorrect MI figures because data was inaccurate or incomplete
  • Projects that intended to master and duplicate data that already existed in the organisation
  • Inconsistency in what people thought certain data was
  • Differing views on where data was sourced from
  • Projects repeating the same data collection work, asking the same questions again

The Data Lens looks to address this by bringing transparency and coherence to your data estate.  It is aimed at supporting the demands of people wanting to use data, such as:

  • Data Lake or Analytics efforts, which need to know information such as where data is sourced from, what terms are used for the same data, e.g. client and customer, how good the data is in terms of quality and completeness, etc.
  • Platform projects need to know where data masters exist, where data flows, how data is transformed, etc.
  • Any data rationalisation project needs to know where master sources of data exist, where duplication exists and how data is used.
  • Plus, Data Scientists need to understand the sources of data available for their analysis

The lens addresses these needs by providing a number of views and tools.

The Data Definition views provide data definitions, summaries and dynamically produced data models.

The Data Architecture Analysis views are geared towards you understanding sources of data, data flows, where duplication exists, etc.

Data Management is where the lens excels.  You are able to understand data quality across a number of criteria and see sources of data.  The Quality Dashboard shows the quality of the key data required to support your strategic objectives and business capabilities, and also the initiatives impacting that data.  This allows you to identify where your data initiatives may need to be focused to improve your business data output and enable your strategy.  The Data Quality Analysis page lets you pick the data you need and it then shows you where to source it from, plus the quality, completeness and accuracy of that data. This is really useful if you are using the data for other purposes, e.g. MI reporting or analytics. The data dashboard provides and summary view of your data which you can drill down into.

We see the Data Lens acting as the bridge between the tools that are more focused on the physical data layer, and which typically meet the needs of the technical teams but not the business users or the data scientists.  Equally, where you have conceptual data in a tool, the lens can act as the bridge to the physical data, removing the gap between the conceptual and physical layers, bringing context and meaning to the data.

The lens is currently in beta but we are allowing organisations to register an interest and we would love to get any feedback on the lens.

IRM UK EA Conference – Outsourcing and EA

I presented a session on Outsourcing and EA at the IRM EA conference last week; specifically how, as Enterprise Architects, we are in a prime position to ensure that outsourcing deals are both created and run effectively as we are in the unique position of having the knowledge and understanding of both the business and IT across the entire enterprise.  We likened EA’s to the Spartans in the battle of Thermopylae who held off an army of (allegedly) a million men for seven days with only 300 warriors – primarily because they understood and had a map of the landscape.  (Unfortunately they were betrayed and slaughtered after a week – hopefully the analogy doesn’t stretch that far!).

Research by both Gartner and AT Kearney suggests that around 1/3rd of outsource initiatives fail.  We discussed how use of our architecture knowledge and artefacts can mitigate the risks of failure and how EA can be used to bring greater success.  We touched on our work to help organisations use EA and Essential together to reduce the outsource transition time (from idea to completed transition to a new provider) from a typical 18-24 months to 6-9 months, which addresses a key concern raised by the FCA.  We showed some examples of how Essential has been used to support such initiatives across a number of organisations.

The conference itself was very interesting and it seems to me that EA is really coming of age – there were many talks showing how EA is used in organisations to provide real and concrete benefit to senior managers.

If you would like a copy of the presentation then drop me an e-mail at the info at e-asolutions.com address.

Surf’s up! Essential catches the Forrester Wave™

Enterprise Architecture Solutions Ltd (EAS), sole sponsor of the Essential Project and developer of the Essential Architecture Manager software, has been identified by Forrester Research as one of the ten most significant providers of enterprise architecture management suites (EAMS).  In this year’s Forrester Wave™ covering EAMS, the research firm has reported extensively on those selected software providers based on their breadth of EA coverage (business/technology orientation and strategy/projects focus), as well as on their ability to serve four common EA objectives:

  • Manage cost and complexity
  • Ensure high-quality and efficient business solutions
  • Set technology strategy, innovate and govern
  • Business/IT planning and alignment

Based on a thorough evaluation of the Essential Architecture Manager Version 4, Forrester rated EAS as a “strong performer”, which they define as being able to demonstrate strong functionality in supporting most or all EA objectives.

For further information please contact Forrester Research Inc., and request the report entitled “The Forrester Wave™: EA Management Suites”, Q2 2013.

The Essential Project team are thrilled to be included in the latest Forrester Wave™. We have long believed that the Essential Architecture Manager offers a compelling EA solution for all sizes and types of organisation, but inclusion in the Forrester Wave™ is a great reflection on the maturity of our offering. We have some powerful innovations planned for 2013, so rest assured that we are not resting on our laurels.

Finally, we would like to say thank you to the international Essential Community for helping us to evolve and shape the Essential software platform.

The Clash between Enterprise Architecture and Project Management

Most organisations experience tensions between the central enterprise architecture team and the solution architects assigned to projects. While the enterprise architects are keen to assure the integrity of the overall business operating model and its IT support, project managers and architects are motivated by the need to deliver local solutions in line with agreed project budgets and schedules. When these interests conflict, the project managers often win the day. This is because change projects draw upon strong business support, while the enterprise architects are seen as serving a worthy but less tangible cause. The predictable result is a proliferation of incompatible solutions.

We have observed several practices used by companies to address these conflicts of interest:

  • Provide strong governance mechanisms
    An organization’s enterprise architecture should be derived directly from its business and IT strategies. Governance mechanisms with executive-level support should be set up to ensure that the business change portfolio is aligned with these strategies and the architecture. Effective governance will ensure that enterprise-level architectural principles are developed and observed. It should also provide an escalation process for handling challenges to these principles.
  • Keep the enterprise architecture up to date
    Regular updating of enterprise architecture models and standards is needed to reflect changes in business strategy or in available technology options. A common criticism of enterprise architects is that they focus on the re-use of existing IT assets at the expense of new technologies or innovative uses of IT. We know of one company that counters this by linking its enterprise architects with centres of excellence in areas such as portal development. When change projects draw upon the expertise of these centres, the solutions they present already conform to the organisation’s architectural principles. In another company the enterprise architects form ‘skunkworks’ teams for evaluating new technologies, and project team staff are then co-opted into these teams. In this way, the enterprise architecture function is seen as being at the forefront of innovation.
  • Build enterprise architecture into the change management process
    A powerful way of avoiding conflicts is to integrate enterprise architecture into the project delivery process. This starts by involving enterprise architects in the formative stages of any new change project well before any decisions on solutions have been made. The enterprise architects can then guide the selection process to encourage use of standard offerings. During the course of the project they can help resolve design problems and facilitate discussions of issues concerning the operating model with the business – they will thus be seen to add value, rather than impede progress. Moving staff between development projects and the enterprise architecture team can also create greater awareness of the broader architectural context.
  • Help project teams document their solutions
    Even the best run systems development shops struggle to ensure that projects are adequately documented. By providing projects with standard ways of recording the metadata that spans business and IT capabilities, the enterprise architecture function can reinforce the necessary disciplines. Note that the Essential Architecture Manager enables such metadata to be easily recorded by non-specialist project staff in a standard way. This means that much of the routine metadata capture process can be handed off to the projects, thus providing more time for the skilled enterprise architects to concentrate on the more challenging modelling and analysis tasks.
  • Establish sanctions for non-conformance
    Some organisations have established sanctions on change projects that fail to conform to enterprise architecture standards. In one case, divergent projects are required to plan and budget for subsequent remediation remedies. In another company the financial consequences of non-conformance are notified to the corporate CFO, who then adds the resulting amounts to the contributions that business units are required to deliver on a quarterly basis. Such sanctions can be effective, but we believe that for the most part positive reinforcement measures will work best.

Finding the right approach for your particular organisation can help build credibility and success for your EA initiative. Whilst the tension between EA and Project Management is inevitable, we have shown that it is something that can be managed and even leveraged to deliver a positive outcome for your organisation.

Essential Affiliate Training Course

EAS, sponsors of the Essential project, recently held its first Essential training course for people looking to become affiliates of EAS, specialising in Essential. The delegates were from a wide range of geographical locations and so web based training seemed like the obvious format, to allow everyone to attend without incurring prohibitive travel and accommodation fees. There was some concern over the format as, although we’ve completed many different class room based courses and webinars over the years, we have not been involved in a full five day web based training course, either as presenters or delegates. We decided to split the five day course over a three week period as we had a gut feeling that five days of online training on the trot could prove difficult in terms of concentration and focus. I’m pleased to say that this format worked well and that the course was a resounding success!

Delegates commented that the web based format did not actually seem very different to being in a classroom. The sound was clear and the screen sharing not only allowed delegates to see the presenters screen for slides and demo’s, but also allowed them to share their own screen when completing exercises, if there were things they wanted to discuss or point out. They also found that it was just as easy to ask questions or have a discussion during the web training as it is in a classroom. As a trainer it did at times feel a bit odd – rather like you are talking to yourself! However, the course is designed to be interactive, with discussion points and exercises, so there wasn’t much time of just trainer talk, which is crucial in any training environment, but especially over the web!

The course covered all layers of the architecture and meta model, with the focus being on understanding the details of the meta model, our interpretation of EA and how we use Essential to support that. There is a lot of material and concepts to cover in a relatively short time, and so the delegates were pushed quite hard. They found that the split of the course over three weeks not only helped with their geographic location and availability, but it also proved extremely useful as it gave the time to fully digest the materials and contextualise the content to their own individual situations before moving on.

The feedback was very positive, both in terms of the quality of the training and also relating to the attendees views of Essential. The affiliates (as they are now) found that the course brought Essential to life and they were very impressed with its capabilities, particularly in the new information layer – which will be released shortly.

We are now advertising for our next Affiliate training course, which should take place in June/July and will again be web based. If you are interested we’d love to hear from you.

Of course training is not just for affiliates. If you are an organisation that has been using Essential, or if you’re new to it, you might find that a little training goes a long way in helping you and your team get up to speed with Essential and push your architecture efforts forward at a faster pace. If you are interested in receiving information on Essential training, please complete the Training Enquiry form.

Strategy Management and Enterprise Architecture

We have noticed that many organisations are currently looking to their EA to support their strategy management, whether that be business and IT or just IT focused.  This is quite a shift for some organisations in moving EA up the stack and out of project or domain focused architecture to one that provides a broader, higher level view.

Our Strategy Management release is, therefore, very timely.  It has been in development for some time and was the focus of ECP 4, we would like to acknowledge and thank the community for their contributions to the release.   We have been using it at one of our global clients for some time and it is, therefore, released with us knowing that it actually works in real life situations – in a global organisation that has distributed businesses with differing regional and organisational objectives. 

A very brief overview of the key elements in this release is given below, but for full details see the release documentation:-

  • Architecture States – represent the different states of your architecture.  Sometimes these are referred to as ‘current state’ and ‘future state’, but we think it is best to avoid these terms as your current state is always evolving and will eventually (one would hope!) become your future state, which makes everything somewhat confusing.  In our opinion it is better to actually refer to the state you are in (politely of course!) and the state you want to be in, including the steps in between.  So, an example of the type of naming we would suggest is ‘manual invoicing’, then ‘automated payment – UK’, followed by ‘automated invoicing – UK ’ and finally ‘fully automated invoicing’ , which allow you to understand exactly what each architecture state is referring to. 
    An architecture state can relate to one or all of the layers of the architecture, and this will depend on the type of project that is being transitioned. 
    We would expect there to be many of these very specific architecture states that reflect different areas of focus, rather than having a very few states that represent the entire EA at specific points in time.  We think the idea of lots of ‘smaller’, more specific architecture states is very powerful for managing the complexity of the progression of the architecture; break it into manageable chunks that deal with specific programmes or areas of interest and use lots of specific, focussed roadmaps rather than having one unwieldy uber roadmap.  However, if that is what you need to manage the transition of your EA then you can, of course, do just that! 
  • Roadmap Model – is the pictorial representation showing how the architecture(s) will transition between the various different states, which are shown as roadmap milestones on the model.  A timeline for this transitioning can also be included in the model.
  • Strategic Plans – hold the detail of how the organisation plans to transition from one state to another and are implemented by Projects, which can be grouped into Programmes.  Strategic Plans also hold the detail of what Issues are being addressed and Objectives are being met by the implementation of the Strategic Plans. 

There are a number of standard views released with the pack, but in true Essential spirit we expect users to develop their own views to meet the needs of their organisation.  The type of views we envisage being used are, for example, a view that maps Projects to Issues and Objectives, so you can see which projects support which Objectives and resolve which Issues.  This can be useful in allocating resources to the areas providing most benefit to the organisation. 

We feel that this is a key area of EA, and additionally it is one that is, or should be, the focus of many organisations at the moment as they move forward following the recession.  Using EA to assist with the strategy management within an organisation is really where the big pay back from EA can start to be seen.  Of course, all the underlying activities and effort are also crucial and can provide benefits on a project or region or domain basis, but using EA to support strategy management is the activity that can provide the overall view of where the organisation is now, where it wants to be, the plans it has for moving there and, crucially, can be used to ensure the organisation actually gets there.  It can do this by analysing information and then providing the insights, as stakeholder specific views, that highlight potential pitfalls and areas of opportunity and allowing these to be used to aid decision making and ensure the strategic plans are achieved.

We are always keen to hear your views, please let us know if you have any comments.

Review of 2010

It’s the end of the first full year of operation for Essential and ‘every day and in every way, it’s getting better and better’ – sorry, couldn’t resist a bit of plagiarism from the Pink Panther film I watched last night!

Back to the review of 2010…..

For Essential, 2010 built on the success of 2009 with almost 3,500 downloads of the toolkit, some 17,500 unique visitors from across a wide range of geographic locations from the UK and Europe across the US and Canada to Australia and India, really a worldwide reach.  The total number of times the toolkit has been downloaded since launch is now around the 6,000 mark.  We are delighted that the community is now really starting to take off; with around 1200 members we are beginning to see an active forum with ideas and suggestions as well as queries, and members starting to post answers to queries from other members, which is great.  Essential has also been noticed by all the major analyst firms with positive written reviews and mentions at various conferences, again, all good news.

Away from Essential, and looking at EA in general, the year has been a mixed one.  Obviously the recession had taken its toll on budgets and whilst some forward thinking organisations were looking to their EA teams to lead the way into the future, others had followed the more traditional route of cuts. However, although it looked like 2010 might just be the year of the recovery – the squeezing of government spend, particularly across Europe, will surely have a significant effect on the public sector and their EA initiatives.  This is possibly one of the reasons that Essential has been so successful, organisations around the world are really needing to make the most of the budget they have available and a decent, free tool is too good an opportunity to ignore.

In the more general world of Enterprise Architecture we have noticed a number of areas being discussed, perhaps more than others.  In terms of activities, there have been two that have been mentioned many times, one looking at application portfolio rationalisation and cost savings and one looking at the role of business architecture in EA.

The first is always a popular route in for an organisation new to EA – display some decent cost savings in a short time period and you will get people’s attention and, hopefully, the buy-in and visibility to move onwards and upwards with your EA; especially applicable in these times.

The second, music to my ears!  As a business architect I have long been frustrated by the IT tag that EA has had and the prominence of the application and technology layers.  It has always been my belief that the business has to be an integral part of the EA and so I was delighted to see so much discussion at conferences and in blogs and discussion threads about the importance of business capabilities, understanding the business strategy and ensuring this is linked into the EA.  This has to be a good thing and I will be happy when all EA teams are sitting under the CIO and not the CTO – although I accept this may not be in 2011?

Looking at EA trends, it has been suggested that specific frameworks will become less important as organisations become more fluid.  That is not to say that EA does not need structure, just that there will be less reliance on being aligned to one particular framework.  We also noticed that TOGAF seems to be rising in prominence since the launch of TOGAF9, partly, I suspect, as it focuses more on the business than the previous versions – although it is still not business focused enough in my opinion…

In terms of tools, there was much discussion about a ‘new generation of tools’ that would support more mature EA objectives such as strategic planning, IT road mapping and risk management.  This would be a move away from the objective of some of the current tools, which is simply to provide documentation support.  There was also some discussion about the use of ontology’s in EA Tools.  Both of these are areas of particular interest to us.  We think we have the most advanced use of an EA ontology in an available tool at present – and thanks to all those who have championed us in various blogs! From our perspective, the main value of an EA tool is not purely in modelling but of being able to interrogate the information that has been captured in the model to aid decision-making.

All in all an interesting year, and one that leaves us looking forward to 2011 when we have a number of exiting events planned.  Our first training course will take place in January, to enlarge the EAS network of Essential trained affiliates, and we have two major releases planned.  Firstly the Strategy Management piece has just been released in late December and secondly a complete update to the information and data layer is nearing completion and will be released early in the New Year.  Keep your eye on the site for these as they move the capabilities of Essential Architecture Manager on a good deal.  And finally an apology for all those that signed up for the webinar last year, we had to move this down our list of priorities as we simply ran out of available bandwidth to complete it in time.  We hope to be able to pick this up in 2011 once the training and the release of the update packs are completed.

All that’s left to say is thank you all for your support, have a very Happy Christmas and we look forward to collaborating with you in 2011!

Do I have to use Tomcat to use Essential?

The Essential Project takes advantage of a number of free, open-source components, including Apache Tomcat. However, do you have to use Tomcat to use Essential Architecture Manager?

This is a question that we realise potential users of the Essential Project tools are asking. In particular when the architecture team has spent many years trying to get to grips with technology diversity, it can be somewhat embarrassing – or even not an option – to have to go off-piste to use an enterprise architecture management tool!

The simple answer to this question is, however, “no”.

You do not have to use Apache Tomcat to use Essential Architecture Manager.

From the outset, we designed the Essential Architecture Manager software so that it can run on any platform – both operating system and application server – to provide as much flexibility in the prerequisites as possible. This means that Essential can fit into as many existing technology architectures as possible, taking advantage of existing technology platforms wherever possible.

The same question can be posed to the choice of relational database software for multi-user installations. Again, as we can see from the software architecture model, where a range of JDBC-compliant databases can be used, Essential can fit into your existing technology architecture, taking advantage of your standard database platform.

Essential Architecture Manager Software Architecture
Essential Architecture Manager Software Architecture

If I don’t have to use Apache Tomcat, what else can I use?

You can use any Java Servlet engine or J2EE application server as the runtime platform for Essential Viewer, which is packaged as a standard Java WAR file. This gives you the freedom to deploy and run Essential Viewer on a range of application servers such as Oracle AS, IBM WebSphere or JBoss.

Since the Essential Project is a free, open-source toolset, we have focussed on application servers and database platforms that are also available free and open-source. The available documentation covers these free platforms as we find that most of our users – certainly the initial stages of using Essential – use these platforms. However, in the spirit of open-source and applying this to our documentation we would welcome any documentation contributions from users who have experience to share about deploying the Essential Viewer WAR to alternative Java application servers.

What about the installer?

The Essential Project takes advantage of a number of other open-source toolkits and the installation process involves adding plugins or components to these other open source tools, e.g. Protege and Tomcat. The first releases of Essential did not include an automated installer but we provided detailed documentation for the installation process – including troubleshooting – and this documentation is applicable to other Java application servers.

In order to make the installation process as simple as possible, the automated installer checks the locations of the Protege and Application Server that you specify to help ensure that the Essential components are being installed in the correct places. Although the installer does not assume that Tomcat is being used (despite the messages on the installer windows!), it does assume that the target application server holds its web applications in a folder called ‘webapps’ – which is what Tomcat does.

This means that if the installer cannot find a folder called webapps in the location that you’ve specified for the Application Server, the installer will not proceed.

It is not really practical to cover all the variations of Application Server deployment approaches, so currently a work-around for anyone who is not using Tomcat is to create a ‘webapps’ folder in your Application Server and install the Essential Viewer WAR to there.

Perhaps the other approach is to relax the error-checking in the installer and allow the Essential Viewer WAR to be installed to any folder on the target environment? This places more responsibility on the user of the installer to install the WAR to the correct location but also provides them with more flexibility.

Either way, if you have any problems installing any of the Essential Project components, please contact us. Free support is always available through the forums.

I would be very interested to hear any views on the how to strike this balance in the installer.

TOGAF 9

My colleague, Jon Carter, and I have just come back from attending a TOGAF 9 training course – and I was pleasantly surprised on two counts.

Firstly, the course was excellent; it really brought TOGAF 9 to life as it was focused on how to apply it in different organisations and scenarios, rather than a dry run through of its concepts with the aim of passing a test at the end – which was what I was, somewhat unsportingly maybe, expecting!

Secondly, I found TOGAF 9 to be a very useful framework in which to apply EA, obviously couched with all the usual warnings about having the right skills and knowledge to apply it, however the new focus on the business elements makes the framework eminently usable, and hopefully future versions of TOGAF will continue to expand the support for business architecture.

Whilst on the course Jon and I were also thinking about the compatibility between TOGAF 9 and Essential, and how effectively Essential can support organisations that are using TOGAF 9 as their EA framework.  I’m pleased to say that TOGAF 9 and Essential are, in fact, very compatible.  There are a few concepts within Essential that are not supported by TOGAF 9, and vice versa, as well as a few concepts that have slightly different meanings, primarily in the newer TOGAF 9 business layer.  For example, what we have described as a Business Capability in the Essential meta model is very closely aligned to what TOGAF 9 describes as a Business Function, however, these differences are very easily explained and in essence it would be very straightforward to use Essential to support an organisation using TOGAF 9 as its framework.

An area that struck both Jon and I where we are very closely aligned with TOGAF 9 is the definition and use of views and viewpoints.  TOGAF describes understanding each stakeholder’s concerns, defining the view of the architecture that they need to understand to satisfy their concern, and then building the viewpoint to produce this view.  In Essential we have always been very clear that information captured is done so to give the ability to provide decision support to all relevant roles. We have not provided an exhaustive list of pre-defined and coded reports that we think will satisfy all needs, as we are clear that each organisation not only has a different set of problems to solve, but that it will have different types of people solving the problems and therefore their needs and desires in terms of views required to provide decision support are different.  We have, therefore, designed the Essential knowledge base (the ontology) in such a way that once the information is captured, Essential can be used to rapidly define required views – and, in addition, we have developed Essential so that creating new views is relatively easy.   A new view will typically take 1-3 days to build depending on its complexity, and it then provides a template which can be used across various data sources, as opposed to a one off report.  We believe this supports TOGAF ideals well.

We do have plans to complete an Essential/TOGAF 9 mapping to cover how Essential could be used to support TOGAF 9, and propose to produce a mapping, rather than any alterations to Essential, as we do want to remain framework independent.

All in all a very valuable four days spent.

Business Capabilities

We have been struck recently by the volume of articles and blogs regarding Business Capability modelling, many seemingly of the view that this is a new concept that will resolve the old business IT alignment issue.

Whilst we don’t concur with the view that the concept of business capabilities is either new or capable of resolving the alignment issue alone, we are in agreement that business capability modelling is a key aspect of the business architecture.

We view business capabilities as the ‘services’ that the business offers or requires.  In Essential, these capabilities are modelled in the Business Conceptual layer and represent what the business does (or needs to do) in order to fulfil its objectives and responsibilities.

A business capability is at a higher level than a business process.  It represents a conceptual service that a group of processes and people, supported by the relevant application, information and underlying technology, will perform.  The capability represents the what, whereas the process, people and technology represent the how.  Business Capabilities can themselves be broken down into supporting capabilities, if this is useful.

Defining your business capabilities is extremely useful as it allows you to take a step back and focus on the key elements of your organisation.  You can avoid getting bogged down in the details of ‘how’ things happen and concentrate on ‘what’ does (or needs to) happen.  Once you have done this it is possible to identify your key capabilities, for example, the ones that will differentiate your business and you can use this information to ensure that you focus on the areas of importance in your business, whether this is in defining new projects or ensuring business as usual delivers appropriately.

You will find that your business capabilities are relatively static because you are defining the ‘what’ which rarely changes whereas, for example, your business processes will constantly be evolving as the ‘how’ things are done changes all the time with the advancement of technology and of customer demand.  A very obvious example is retail – twenty years ago the internet did not exist so there were no online sales channels; but the capabilities of a retail channel have not altered, Sales, Fulfilment and Billing are still capabilities, however the process of ‘how’ they sell, dispatch and take payment has altered dramatically.

In reviewing our tutorials we noticed that we already have tutorials on Capturing the Business Value Chain (a subset of the capabilities) and Business Process Modelling, but we don’t have a tutorial that focuses solely on business capability modelling.  In view of the current interest we aim to address this gap as soon as we can and a new tutorial will be available shortly.