Are open source EA tools as good as commercial tools?

One of our users/advocates this week pointed us at the FAQ of one of the commercial EA Tools where they state that free and open source EA Tools are only useful for rudimentary EA requirements, whereas commercial tools, like theirs, are much more advanced.  It struck us that this is either a poorly informed organisation, unaware of the competition, or lazy writing by someone who does not understand much about open source.  We normally ignore the competitors and let them worry about us, but in this case, we do feel it is important to respond to something that is clearly misleading.

They state that only commercial tools can provide business outcomes and decision-making insights through a central shared repository, an integrated set of views covering strategy, business, applications, information and technology, to provide the insights necessary to optimise, rationalise and transform.

There is, however, nothing in that statement that can’t be delivered by Essential Open Source – a free EA tool.  Far from supporting only ‘rudimentary’ EA requirements, Essential Open Source is, in fact, a very sophisticated EA Tool that is focused on providing decision support and “what if” analysis to EAs and CxOs.  In addition, Essential has a world leading, ontology-based meta model and has been designed to be exceptionally flexible – the meta model can be extended in minutes and new views can be created in hours by users.

The only areas where Essential Open Source falls short is that it is not Cloud based, it has to be hosted on site, and it has basic user permissions.  Enter Essential Cloud, the low-cost Essential Cloud solution that is, as the name suggests, cloud based and includes full user security permissions across both data entry and the viewer.  Oh, and did I mention it covers unlimited users.

I’m inclined to think the traditional EA Tool vendors, with their high costs and inflexible models, are trying to muddy the waters with this kind of article.  Have a look at what Essential has to offer and make your own decision.

Essential Key Points:

  • Over 120 out of the box views – have a look at the Technology Product Selector, the Application Rationalisation Analysis and the IT Asset Dashboard as just a few examples where the traditional commercial tools can’t compete
  • A free launchpad to get you started, with a simple data capture mechanism, to allow the population of 20 key foundational views across business, application and technology
  • Open Source or Cloud – Docker coming soon
  • Low Cost
  • Flexible
  • Built by architects for architects

Enterprise Architecture Firm of the Year 2018

We are delighted to announce that, for the second year running, EAS has won top place in Corporate Live Wires’ Innovation & Excellence Awards.  Following on from receiving the award for Innovation in Enterprise Architecture 2017, this year we have been named as Enterprise Architecture Firm of the Year 2018 based on Essential and our consulting work around it.

The Innovation & Excellence Awards are judged by a panel of industry experts following nominations from the wider community.  The Essential Project, the world’s most popular open-source EA Tool, is now available as SaaS through Essential Cloud.  EAS, Essential’s founders, are all practising architects and we continue to extend and improve Essential by listening to our clients and our community and reacting to their needs.  We were first to exploit the connection between the information required for EA and GDPR and launch a GDPR support pack, bringing massive benefits to both the Compliance and EA Teams within an organisation.

We are continuing to develop Essential to support our customers and our community; look out for our innovative data capture and maintenance solution which will ease the burden of data management and revolutionise EA!

Essential Version 6.1.1 Released

Essential v6.1.1 is now available from the downloads section of the website.

We’ve promoted some of the views that we’ve been trying out in the labs, including the NIST mapping and a couple of Business Capability to Technology views.  We’ve also enhanced some of the OOtB views, such as the Business Process Model, the Business Capability Model and the Technology Platform Model.  Meta Model extensions include a new class of ‘Business Process Family’ to make it easier to model and analyse standard and non-standard processes.

Full details of all the changes can be found in the release notes.

Data Lens

You may have noticed from our site that the Data Lens is in beta.  It’s a lens that we’ve developed because we’ve been continually told that people don’t have control of their data.

In our EA consulting, we have seen:

  • Organisations that were unwittingly reporting incorrect MI figures because data was inaccurate or incomplete
  • Projects that intended to master and duplicate data that already existed in the organisation
  • Inconsistency in what people thought certain data was
  • Differing views on where data was sourced from
  • Projects repeating the same data collection work, asking the same questions again

The Data Lens looks to address this by bringing transparency and coherence to your data estate.  It is aimed at supporting the demands of people wanting to use data, such as:

  • Data Lake or Analytics efforts, which need to know information such as where data is sourced from, what terms are used for the same data, e.g. client and customer, how good the data is in terms of quality and completeness, etc.
  • Platform projects need to know where data masters exist, where data flows, how data is transformed, etc.
  • Any data rationalisation project needs to know where master sources of data exist, where duplication exists and how data is used.
  • Plus, Data Scientists need to understand the sources of data available for their analysis

The lens addresses these needs by providing a number of views and tools.

The Data Definition views provide data definitions, summaries and dynamically produced data models.

The Data Architecture Analysis views are geared towards you understanding sources of data, data flows, where duplication exists, etc.

Data Management is where the lens excels.  You are able to understand data quality across a number of criteria and see sources of data.  The Quality Dashboard shows the quality of the key data required to support your strategic objectives and business capabilities, and also the initiatives impacting that data.  This allows you to identify where your data initiatives may need to be focused to improve your business data output and enable your strategy.  The Data Quality Analysis page lets you pick the data you need and it then shows you where to source it from, plus the quality, completeness and accuracy of that data. This is really useful if you are using the data for other purposes, e.g. MI reporting or analytics. The data dashboard provides and summary view of your data which you can drill down into.

We see the Data Lens acting as the bridge between the tools that are more focused on the physical data layer, and which typically meet the needs of the technical teams but not the business users or the data scientists.  Equally, where you have conceptual data in a tool, the lens can act as the bridge to the physical data, removing the gap between the conceptual and physical layers, bringing context and meaning to the data.

The lens is currently in beta but we are allowing organisations to register an interest and we would love to get any feedback on the lens.

Essential Cloud – Available Now

Today marks a step change in the life of The Essential Project as we move to Public Preview of Essential Cloud, the final step before General Release. A cloud offering has been at the top of the Essential Community request list for some time and we have combined the best of the Essential Project with a cloud based service to provide additional enterprise capabilities. As well as all the benefits of Essential Open Source, Essential Cloud offers a comprehensive security interface covering both the instances in the repository and the viewer, a user-friendly, browser-based data capture interface extended to include tablet and mobile access, an enhanced viewer environment and single sign-on support via SAML. As this is a cloud service, technical support is automatically included as are platform updates, to ensure that you can keep up with the latest Essential developments with none of the hassle.

In line with our focus on value, Essential Cloud will be a low-cost option, with an annual subscription covering access to both the modeller and the viewer for unlimited users. We are not utilising a seat-based license model as the feedback from the Essential Community and our clients is that the key to an effective architecture initiative, one that provides value to the business, is to enable the users to own and update key aspects of the architecture, i.e. those areas that do not require modelling expertise, such as dates, ownership, governance models and so on. This spreads the load of keeping organisational information up to date and enables architects to focus on business value rather than being distracted with managing routine updates. A seat-based license model does not fit with this approach as the costs quickly become prohibitive; we would rather an organisation’s investment in EA is used to build out their architecture than pay for licenses.

To support this new model further, we are working with our user groups to design new data capture mechanisms that will provide business users with easy access and enable them to update information without having to understand the detail of the meta model or architecture modelling techniques. We already have some early prototypes, and we see this as an important way of enabling EA to continue to provide value to the business.

This is an exciting step in the broadening of the Essential platform, but we do want to assure you that we remain fully committed to Essential Open Source. This will continue to evolve in parallel with Essential Cloud and, crucially, the meta model will remain shared so both platforms will benefit from all advancements as well as the ability to move easily between Cloud and Open Source. Going forward we see the Essential Community consisting of both Open Source and Cloud users. We greatly value the contribution made by the community and we will continue to look to them to help us evolve the Essential proposition to ensure it remains at the forefront of knowledge driven decision support.

We have created an overview video showing Essential Cloud’s capabilities and we will also be holding a series of Webex’s where we will provide a demonstration of Essential Cloud and hold a Q&A session.

If you are interested in the Webex or Public Preview sign up here.

The Public Preview benefits are, of course, in addition to the existing benefits that are provided across both Essential Cloud and Open source:

  • Over 100 out of the box views focused on analysis, road mapping and decision making
  • Ontology based meta model for an entire organisation, with the ability to support other EA Frameworks
  • Import and export of data via unique excel import utility, with fast start view loaders, and APIs to integrate with existing data sources
  • Access to business focused lenses providing dedicated support for key areas such as Application Portfolio Management, Data Management, Strategic Resource Optimisation

If you are new to Essential then, Essential Cloud aside, one of our most exciting recent developments is the addition of add-on business focused lenses.

The lenses have been in our mind since 2010 when Jon Carter wrote a blog titled ‘Welcome to the View Store’, suggesting the concept of an app store for Essential. We were staggered by the interest and up take of Essential and so our early focus was on developing the tool functionality, but now we have made our earlier vision a reality.  Our business outcome focused lenses consist of a series of dashboards and views that respond to specific business problems, supported by everything you need to light up the views – data capture spreadsheets, import specifications, process documentation and videos.

The lenses have provided an ideal means for us to partner with organisations outside of the usual EA arena, allowing us to extend the use of Essential to cover different aspects of an organisation. For example, we have partnered with a strategic resource specialist to create a strategic resource optimisation lens which enables organisations to ensure they have the right staffing resources in place to meet the future demands of the business, such as the skills to enable digital business expansion. We have a couple of packs on offer now: Application Portfolio Management and Strategic Resource Management, and we will be expanding the offering shortly to include Data Management, and then, over time, we plan to move into many additional areas such as M&A, Outsourcing Support, Financial Regulations, etc.

We also have a set of low-cost View Loaders that provide the templates to do bulk data capture into Essential Cloud or Open Source. So, if you want to get bulk data into Essential quickly, the loaders can speed this up.

If you would like to develop your own pack to put on the view store, or talk to us about an idea for joint development of a pack, please contact us.

Version 3.0 – it’s here!

There have been a couple of blogs recently about the imminent release of Essential Architecture Manager Version 3.0, and I am pleased to announce that it has arrived!

We think that Essential Architecture Manager v3.0 represents a significant move forward for Essential, encompassing not only a number of significant enhancements and additions to the meta-model but also a new and improved Viewer, with 100 out-of-the-box views and an updated development framework to make it easier to add new views, plus some key enhancements to the platform itself.

New tutorials and videos will follow, plus an upgrade pack for existing users – these won’t be long but in the meantime please take a look around the site at the information on version 3, which includes an overview article, release notes and some example screenshots of views.

We have added a new forum topic for Version 3, not only to capture any bugs and issues but also for queries and questions and any feedback you have. Please make use of the forums, as your feedback is crucial to the on-going development of the Essential toolset.

Why are you giving Essential away free?

This question commonly pops up when we discuss Essential, either with our community, analysts or people in the EA business generally.  I think, in many cases, the underlying question is ‘is Essential any good if you are giving it away for free?’.  Well, based upon our experience in applying it as well as feedback from customers and Essential Project community members, we think so, but I still think it is worth explaining our rationale.

Essential began life as something we needed to support us in our early client engagements back in 2000/2001.  Initially we created a meta model to support our EA work and either implemented it in client’s existing tools or, where there was no tool, we tried implementing it using UML (Rational Rose).   We found that often, although better than PowerPoint or Excel which tends to end up as shelf ware, neither of these implementation approaches really gave our clients what they needed and so we looked to develop our own tool, around 2006. 

We were still not thinking of developing a tool as a commercial offering, we simply wanted something that would make our lives as EA consultants easier.  We also felt strongly that the true value of an EA repository was beyond purely documentation.  We wanted something to allow us to interrogate the EA ‘knowledge’, to allow us to make the valuable insights that make EA essential to an organisation, and this didn’t exist at that time in most EA tools – and especially not in organisations that had no tool at all!  Luckily for us, we stumbled upon Protégé, an ontology editor and knowledge-based framework developed by Stanford University (many thanks go out to them by the way), whilst actually looking at a different area of EA capability, and found that this met our needs completely, saving us much time and, I’m guessing, considerable heartache! 

So, Essential was born and it has served us well in many client engagements.  In fact we often found ourselves implementing it alongside traditional EA tools where the client had a need that the incumbent tool could not support without considerable time and money, but that Essential could support quite easily.  Also, when engagements came to an end, most organisations were able to keep Essential up and running to continue to provide value, be it a consolidated view of applications and their links to processes, or as a simple technology standards catalogue etc.  During this time we enhanced the tool and came to realise that, without really planning to, we had developed an EA tool that was not only easy to use and reliable, but that took a different approach to most other EA tools, moving away from the traditional documentation based approach to a more leading edge knowledge representation approach which suited our needs and the needs of our clients.

It was at this time that we came up with the idea of launching Essential as a free, open source EA tool for a number of reasons:-

  • The objective of Essential was never really to directly compete with commercial EA tools.
  • We had seen the value of it in our own engagements and we could see that others would find it equally useful, especially organisations that:-
    • were at the beginning of their EA journey and could not cost justify the expense of a commercial tool
    • did not yet understand the requirements of a tool, but that had a lot of information that they could capture that would provide vital decision support if only they could interrogate it.
    • were unlikely to ever have the budget to cost justify the licensing fees associated with commercial EA tools, but that could definitely benefit from the decision support it offers.
  • We felt that there was an opportunity to develop the tool (requirements and actual software contributions) in collaboration with a broader EA community.  
  • There were no English-language open source tools available at that time.

But obviously EAS is a business, we didn’t do this solely to benefit the EA community!  The commercial opportunity for us was in providing services that would help organisations take full advantage of the tool (EA capability development) as well as raising our profile to the general EA community and potential customers.  

So, to summarise, we couldn’t find a tool to meet our needs, so we developed our own based on our principles and philosophy towards EA (taking what we believed to be an novel approach) and launched it as a free, open source tool.
As a result we hope to:

  • Aid those that may not yet, or ever, be in a position to justify buying a commercial EA tool
  • Give the EA Community the opportunity to contribute their requirements, and extensions (if they so desire)
  • Not just tell, but show everyone what EAS as an organisation is all about

Review of 2010

It’s the end of the first full year of operation for Essential and ‘every day and in every way, it’s getting better and better’ – sorry, couldn’t resist a bit of plagiarism from the Pink Panther film I watched last night!

Back to the review of 2010…..

For Essential, 2010 built on the success of 2009 with almost 3,500 downloads of the toolkit, some 17,500 unique visitors from across a wide range of geographic locations from the UK and Europe across the US and Canada to Australia and India, really a worldwide reach.  The total number of times the toolkit has been downloaded since launch is now around the 6,000 mark.  We are delighted that the community is now really starting to take off; with around 1200 members we are beginning to see an active forum with ideas and suggestions as well as queries, and members starting to post answers to queries from other members, which is great.  Essential has also been noticed by all the major analyst firms with positive written reviews and mentions at various conferences, again, all good news.

Away from Essential, and looking at EA in general, the year has been a mixed one.  Obviously the recession had taken its toll on budgets and whilst some forward thinking organisations were looking to their EA teams to lead the way into the future, others had followed the more traditional route of cuts. However, although it looked like 2010 might just be the year of the recovery – the squeezing of government spend, particularly across Europe, will surely have a significant effect on the public sector and their EA initiatives.  This is possibly one of the reasons that Essential has been so successful, organisations around the world are really needing to make the most of the budget they have available and a decent, free tool is too good an opportunity to ignore.

In the more general world of Enterprise Architecture we have noticed a number of areas being discussed, perhaps more than others.  In terms of activities, there have been two that have been mentioned many times, one looking at application portfolio rationalisation and cost savings and one looking at the role of business architecture in EA.

The first is always a popular route in for an organisation new to EA – display some decent cost savings in a short time period and you will get people’s attention and, hopefully, the buy-in and visibility to move onwards and upwards with your EA; especially applicable in these times.

The second, music to my ears!  As a business architect I have long been frustrated by the IT tag that EA has had and the prominence of the application and technology layers.  It has always been my belief that the business has to be an integral part of the EA and so I was delighted to see so much discussion at conferences and in blogs and discussion threads about the importance of business capabilities, understanding the business strategy and ensuring this is linked into the EA.  This has to be a good thing and I will be happy when all EA teams are sitting under the CIO and not the CTO – although I accept this may not be in 2011?

Looking at EA trends, it has been suggested that specific frameworks will become less important as organisations become more fluid.  That is not to say that EA does not need structure, just that there will be less reliance on being aligned to one particular framework.  We also noticed that TOGAF seems to be rising in prominence since the launch of TOGAF9, partly, I suspect, as it focuses more on the business than the previous versions – although it is still not business focused enough in my opinion…

In terms of tools, there was much discussion about a ‘new generation of tools’ that would support more mature EA objectives such as strategic planning, IT road mapping and risk management.  This would be a move away from the objective of some of the current tools, which is simply to provide documentation support.  There was also some discussion about the use of ontology’s in EA Tools.  Both of these are areas of particular interest to us.  We think we have the most advanced use of an EA ontology in an available tool at present – and thanks to all those who have championed us in various blogs! From our perspective, the main value of an EA tool is not purely in modelling but of being able to interrogate the information that has been captured in the model to aid decision-making.

All in all an interesting year, and one that leaves us looking forward to 2011 when we have a number of exiting events planned.  Our first training course will take place in January, to enlarge the EAS network of Essential trained affiliates, and we have two major releases planned.  Firstly the Strategy Management piece has just been released in late December and secondly a complete update to the information and data layer is nearing completion and will be released early in the New Year.  Keep your eye on the site for these as they move the capabilities of Essential Architecture Manager on a good deal.  And finally an apology for all those that signed up for the webinar last year, we had to move this down our list of priorities as we simply ran out of available bandwidth to complete it in time.  We hope to be able to pick this up in 2011 once the training and the release of the update packs are completed.

All that’s left to say is thank you all for your support, have a very Happy Christmas and we look forward to collaborating with you in 2011!

BCS Presentation

The BCS kindly invited us to present a session on Open Source Enterprise Architecture Toolkits, covering their effect on the market place, at their meeting on Tuesday.  Alex Mayall led the presentation, which focussed on the two major open source offerings, Essential and Iteraplan, as well as touching on a few others.  The feedback from the audience was extremely positive, and the slides can be downloaded from the BCS website if anyone is interested.

There were a couple of questions from the audience on the use of Essential and I thought I would cover them here in case they are issues that are encountered often.  Firstly, one member advised that they had successfully downloaded Essential and were able to add data using the forms, but that they had a lot of data in excel spreadsheets and they had trouble finding the relevant documentation on the site to enable them to successfully run the import, limiting their use of the tool.  We have had a number of queries regarding this issue on the forum also and so we have recently added a tutorial – How to Write Integration Transforms – which is found in the Getting Started Tutorials.  If you have experienced problems in this area it may be worth a look.

Another member reported making constructive use of Essential, but finding creating new reports quite taxing.  There is a tutorial on this also – Creating New Reports – in the Reporting and Analysing Tutorials, however, it is fairly complex and we are considering writing a training module in this subject.  It would be good to gauge interest in this, so if you think you would be interested in attending such a course please let us know by commenting on this blog.  If you are having trouble in this area, it might be worth checking the forums to see if anyone esle has posted about a similar problem, or posting a question yourself.

The vast majority of large enterprises do not use commercial EA planning tools. Why is that?

You may not be surprised to learn that I have been following this discussion with some interest!  I agree with the reasons given by various members of the thread for the lack of tool uptake; an organisation should have a process and methodology embedded before it can select a tool; the tool needs to support the objectives of the organisations EA, therefore these must be known before tool selection; many EA tools are either poorly constructed or were designed for something else, i.e. BPM, and have been adapted to EA; the organisation itself is not sufficiently mature; to name but a few, and I was very pleased to see that The Essential Project received a couple of mentions.

Firstly David Baker pointed out that using free, open source tools as a starting point would seem like a good idea and he wondered why more organisations didn’t do this, speculating that data migration at a later date might be the issue.  To deal with this particular point, Essential does have a data export option, so if an organisation starts with Essential and decides that it doesn’t suit its needs once it has defined its exact requirements, the data can easily be migrated to the tool of choice.

Later in the discussion Ric Phillips noted that ‘A good enterprise architecture will allow flexible modelling (primary ontology) that can allow architects to build rich (hyper-connected) models of the actual organisations in which they work – which do not always conform to the grammar and lexicon of the big frameworks’, and that uncoupling the presentation and analysis layers from the data layer is possible and desirable.  He points out (thanks very much!) that The Essential Project is the only EA Tool currently to take this approach.  He also notes that the ontology is relatively easy to modify, again something we felt was important, however, personally I would be cautious about his idea to throw away the ontology and build your own from scratch.  Obviously this is doable (we have done it!) but it is by no means a trivial exercise and in the best traditions of EA and reusability it seems hasty to throw it away and start again!

The tool debate is an interesting one in terms of when an organisation should start to use a tool, as obviously it is crucial to understand the business problem that your EA is to solve and what is required of your tool before you make an investment.  However, in practice a tool would often be useful fairly early on in the process, before any such investment would be prudent, and this is precisely where and why Essential was born, to give many of the benefits of a traditional tool, much beyond the capability of Visio or Excel, without the investment required of a traditional tool.