XML System Design Principles

Mark Norton  11 October 2011 10:41:52 a.m.

XML System Design Principles

This post highlights the benefits of using XML as the core of a systems design philosophy.

XML

Idiom promotes XML centric design approaches. These approaches underlay the Idiom Decision Manager and Idiom Forms.

The design impact of an XML centric approach can be significant and positive.

What do we mean by an XML centric approach?

An important characteristic of an XML document is that with an appropriately designed schema (that is, an xsd) it can carry a complete and substantial (hundreds of thousands of nodes is not uncommon) transaction context within a single document. This means one data artifact for the transaction – not tens, hundreds or thousands of tuples that might need to be designed, captured, and otherwise managed in a traditional relational database representation of the same transaction.

In an XML centric approach, this transaction XML becomes a single data artifact that can be stored in a database. Contrary to popular belief, this blending of XML and relational technologies is not necessarily detrimental to database search performance. Modern databases like SQL Server allow for database supported XML datatypes and blended XPath and SQL queries. Performance is enhanced by the huge reduction in database reads required to acquire the transaction data. Similarly, bloat that is sometimes presumed to occur with XML syntax is more than compensated for by the reduction in the number of tables, and their associated indexes, that are otherwise required to hold the transaction data.

The database

The bottom line is that for a business with complex transactional data (Idiom customer examples of this sort of transactional data include: finance, insurance, logistics, health admin, clinical health, Telco servicing and billing, government policy) an XML based approach can reduce the number of tables in the database by an order of magnitude. A rule of thumb suggests that 10 dependent entities are required to support each primary transaction entity – it is these dependent entities that are being collapsed into the XML transaction document.

It is important to recognize the no denormalization is implied here – the XML schema should be fully normalized, and the document that complies with the schema will be placed in its proper context in the database. The entire fabric of normalization is maintained.

The data that remains explicitly declared in the database includes the full set of reference data, and all other data that lives outside of and supports the core business transactions. For an extended discussion regarding the primacy of this transactional data, please see our article on Modern Analyst  or our original 2006 decisioning article published by the Business Rules Journal.

On the database tables which store the transaction XML we should retain sufficient database defined attributes to support the following:

·        Locating and identifying the transaction;

·        Managing transaction state;

·        Workflow.

We will assume that supporting activities such as accounting and reporting will be managed by subsidiary systems, and that transaction data will be transformed into appropriate views for these systems as required. This clear separation of function leads to clean system boundaries with well defined and transaction agnostic interfaces.

Transaction Components

The discussion above has outlined a design philosophy that isolates the business transaction inside an appropriate database – the transaction itself (e.g. an insurance policy, an airline reservation, a loan, a patient episode, etc) is a payload. We can now think of it as CONTENT.

The benefits of this design approach are already significant. By designing systems that treat the transactions as ‘black box’ content we simplify system design, which in turn encourages better engineered, faster and more robust systems at lower cost. These systems are more agile because we can introduce new and different types of content, potentially even repurpose the system, without changes to the underlying system.

We now need to turn our attention to managing this transaction ‘content’. By using declarative, XML aware tools we can significantly extend the benefits already outlined.

There are typically three types of functions that need to be ‘content aware’ – that is, to manage the internal state of the transaction content, and to manage the interaction of this content with the outside world.

1.        Decisioning: for managing internal state and for transforming the content to meet the needs of external system consumers.

2.        Forms: for managing interactive human involvement.

3.        Documents: for managing passive human involvement.

All non human users can be supported by XML that is generated by decision models, so that (1) above can be used to provide interfaces for multiple external system consumers.

Idiom has built a suite of tools and capabilities that support the management and operation of XML based content within XML centric systems as described above.

These tools provide a comprehensive capability to declaratively define the processes that are required to manage a transaction. For the sake of clarity, this means that we can now build a host system and plug in new transaction types by defining new decision models, and if required, forms and/or documents. By way of example, a new transaction type might be a new insurance product, a new billing regime for a hospital, a new loyalty points calculation for a new business partner, etc.

Idiom Decision Models

Idiom Decision Manager is the heart of the capability. It provides an easy to use tool for analysts or SME’s to define the various transformations that create value for the business transactions. For example, approving and rating insurance policies, approving claims, calculating health billing, calculating loyalty points, etc.

This includes decisions regarding validation, acceptance, value (cost and/or price), state, and workflow.

Decision models can also act as proxies for external consumers. For instance, generating accounting records that comply with the rules of the accounting system; or transforming data for use in generic reporting systems; or generating transaction updates for business partners that comply with partner system requirements.

The Decision Models can also be used to drive webforms and human readable document generation.

Idiom Forms

Idiom Forms is not a universal forms development tool. It is targeted at a specific design goal – that is, the integration of ‘full strength’ decision models into dynamic webforms. Note the use of the word ‘dynamic’ in the preceding sentence – if the decision model is only applied on submission of the form, then use of Idiom Forms is not mandated. Idiom Forms provides significant development advantages when binding is required between server side and browser side data images – the decision models execute against the server side images, and the webform interacts with the browser side image. Idiom Forms automatically manages the synchronization of this effort.

That being said, Idiom Forms does allow rapid forms development (faster than equivalent native forms) and it does support a wide range of events and custom controls.

The downside is that extending beyond the available events, controls, and other forms components already supplied by Idiom Forms is a programmer task.

Therefore, when integrated decisioning is not required and/or there is a significant degree of bespoke programming required in the form, then native forms development is indicated. Things that would indicate the use of native forms include sustained interaction with a database (use of one or more cursers to manage search results for instance), sustained interaction with external services, or maintaining multiple transaction contexts within one session.

Idiom has recently developed and used a hybrid forms model, which allows Idiom Forms to be managed within native coded WebPages. This hybrid approach means that native code development can be used to manage workflow, orchestration, database interaction, and other complex transaction requirements, while Idiom Forms can be plugged in to manage the decision intense transactions within the workflow.

Documents

The Idiom Decision Manager provides a range of capabilities that can be used to help produce documents for human consumption within an otherwise generic document generator.

In this context Decision Models can be used to:

·        Calculate new variables;

·        Transform calculated variables into print substitution variables - i.e. strings for printing;

·        Calculate the presence of circumstances that control the inclusion of text into the document:

·        Create the actual text to be included in response to the presence of circumstances.

When the above are applied to a pre-written document template, a transaction specific document can be produced with relative ease using a generic reporting program.

Summary

The use of XML centric design and development supported by appropriate tools leads to more agile systems at lower cost, time, and risk. Idiom is currently developing systems using these techniques and tools for multiple insurers, financial intermediaries, airlines, billing systems and Telcos.

Call us for a discussion on whether we can help you too.

Mark Norton

+64 21 434669

mark.norton@idiomsoftware.com

• Comments [0]

’InHousing’ Idiom development

Mark Norton  12 August 2011 01:49:31 p.m.

This discussion arose from an Idiom client who is looking to transition from an Idiom services dependency to full in-house development and support. As the client is keen to remind us, we do say on our website that ‘we will never lock you in!’

The project in question is a turnkey development that will perform loan screening for multiple banks. With a total of 2-3 people involved in the development from the Idiom side, it is relatively small in traditional development terms, and is managed directly between the Idiom decision designer and the client buyer.

It includes 8 decision models (generating ~60,000 lines C#) and a similar number of Idiom Forms, some quite complex.

The project has three notable development threads, with approximate relative effort shown below.

·        Idiom Decision Manager - 42%;

·        Idiom Forms - 16%;

·        ASP.NET application - 42%.

The Idiom Decision Manager [IDM] percentage includes all project management, analysis and design overhead - that is, all analysis, documentation, meetings, etc. This is likely to be 50% of the total ‘IDM’ effort. This bundling of requirements and solution design with decision design and automation is not unreasonable – Idiom sees itself primarily as a business requirements specification environment rather than a development tool, even though the resulting decision models are all generated as ready-to-run computer code. As Idiom users will be aware, the decision designs explicitly drive forms development (in the case of Idiom Forms) and implicitly drive all other technology requirements and designs. Therefore, the IDM development thread is the senior design and development stream in the overall development.

The ASP.NET component is building infrastructure to service the decision models. Each new project for the client should require less additional infrastructure as we go forward.

It is important to note that the very great majority of the client 'product specific' development occurs in the Idiom Decision Manager thread.

Taking the above comments into account, we expect the relative resource consumption percentages to change on a go forward basis. Therefore, a likely 'next project' resourcing  could look like the following in terms of relative resource time:

• Analysis, design, planning: 20% [C]
• Idiom Decision Manager: 20-30% [B]
• Idiom Forms: 10-15% [E]
• ASP.NET: 20-30% [D]
• Testing and Production: 15-20%  [A]

Our discussion point is the transition of skills and responsibilities to the client. Note that project management is not included – with the typical small project that Idiom encourages (1-3 people) project management does not need dedicated resourcing.

The letters A-E indicate a likely sequence of skills transition between Idiom and the client.

'A' is where we suggest the client focuses resource development in the first project delivery. Somebody who is doing in-depth testing will develop a good understanding of what the system does and should do. This is an essential foundation for B.

'B' is next off the rank. The knowledge developed by 'A' will provide good background for transitioning to 'B'. Development of the Idiom Decision Models does not require technical skills - it requires good domain knowledge and a detailed, analytical mind, similar to the tester but extending into more creative territory (Excel development is a comparable task).

The 'B' resource will have the best perspective of the overall system from a business delivery perspective - that is, what it does for the business. To put it more emphatically, they are likely to become the key knowledge resource as well as the custodian of the core IP of the organization. This person should therefore be a competent, ‘self actualized’ person with an eye for detail and a delivery focus.

Given the focus and aptitude of the ‘B’ resource, they will become a key player in the ‘C’ resource effort – they will be the domain expert for the project AND the key delivery resource for business functionality.

If we now switch to the other side of the project, the technical side of the application represented by D. The ASP.NET (or Java if a Java application) delivery is a traditional technology driven development, although in an Idiom centric development the technical plumbing is much (up to an order of magnitude) smaller than in a traditional development.

The technology development is about building operational systems capability, whereas the Idiom development is about directing and controlling that capability (plus of course, capturing the core business IP in an executable form). However, once the capability is built, it stays built, so that there is a flush of development that is required to build the initial capability. The first few projects will all add to the capability ‘critical mass’, but once a critical mass of capability is reached, the incremental development requirement will slow down.

The point at which the ‘D’ skills interface with the A-C skills is at the analysis, design, and planning stages. New business requirements must be mapped to existing, modified, or new capabilities - the decision models can’t control what doesn’t exist. Therefore the D skills also participate in the ‘C’ skill area (the infamous analyst/developer). In a well formed, light-weight project, C is in fact represented by the B and D resources and has no resources of its own.

Finally, we have the E skills. In general, the Idiom Forms skillset is light weight and localised. Some forms development is quite technical, and some entirely declarative. Generally, the more technical parts of Forms development (for instance, bespoke behavior attached to buttons) will be easily addressed by the ‘D’ skills. This leaves the Forms developer with a need to acquire easy to learn ‘Idiom Forms Builder’ declarative skills plus technical styling skills – specifically CSS. This sort of skill set could be picked up by any of the above skill types, and/or by junior and lower paid staff like students, testers, junior developers etc.

In summary, in building out the client in-house support capability, we suggest that:

1.        Get the A + B skillsets in one person. This will deliver the biggest bang for the buck – immediate and  direct business control of the technology, and of the core business IP that is embedded in the technology. Allow 1 week working with an Idiom consultant to acquire adequate Idiom Decision Modeling skills.

2.        With the correct person in place above, they should readily extend to the ‘C’ skills? Look for competent technical design skills in the ‘D’ developer to supplement any technology weakness in the A/B/C resource.

3.        The D skills are the biggest concern and biggest risk. Because of the smaller development load that is inherent to the approach, this person is likely to be a sole charge developer, and needs to be competent across a number of technical areas (web, application coding, database, frameworks, design). This person will become a core technical resource if successful, or an expensive waste of lots of money in addition to their own salary if not. Obviously only the former is acceptable.

4.        The E skills tasks can be done by a combination of D and the A/B/C resources in the early stages. When the project has achieved its primary objectives, bring in a junior staffer to provide some degree of additional support and redundancy. This person can pick up the routine forms development and testing which is typically viewed as menial by the more senior A/B/C/D staff. Allow 1 week working with an Idiom consultant to acquire adequate Idiom Forms building skills.

Finis

• Comments [0]

Anatomy of a large scale Hospital Billing Decision Model

Mark Norton  21 April 2011 01:55:12 p.m.

The PDF attachment to this post describes in broad outline a decision modeling architecture to support hospital billing. The implementation context is a nationwide billing system currently being implemented for an AsiaPac country. The comprehensive billing infrastructure is provided by PowerHealth Solution's new "Power Billing and Revenue Collection (PBRC)" application. This is a large scale application supporting dozens of hospitals with detailed hour-by-hour patient billing.

• Comments [0]

The 5 Step ’Decision Centric’ Development Approach

Mark Norton  11 April 2011 08:32:54 a.m.

The 5 Step 'Decision Centric' Development Approach™

The 5 Step Decision Centric Management Approach™ is an approach that uses the specification and management of decision making to assist the development and implementation of business strategy, with particular emphasis on optimising the development of systems to support the business strategy. This approach leads to:

·        Tighter alignment between business strategy and the systems that implement it;

·        Direct, immediate hands-on control of automated decision making by the business owners;

·        More responsive, more agile, more versatile business products and services;

·        Accurate, timely, decision making provided within frontline systems;

·        Faster, safer, cheaper systems development and ongoing maintenance;

·        Increased auditability, transparency, and control of operational business decision making.

The approach can be implemented progressively and without duress into most organisations, starting with small targeted business areas, and progressing as the rewards become apparent and confidence builds. The minimal investment required provides immediate returns through lower systems costs and improved business effectiveness.

Step 1 – Business Concept Modeling

Purpose: To understand the business value proposition and to build a vocabulary that describes the value proposition in the business’s own terms (an "idiom", Defn: the characteristic vocabulary or usage of a specific human group or subject. http://www.thefreedictionary.com/idiom).

Outcomes: Entity Relationship Diagram (ERD); a list of important and related events that the business recognizes and responds to; a matching list of the key value transformations that the events trigger.

Duration: 1-10 days, depending on the size of the target domain.

Analyze and describe the business domain in terms of business concepts (‘entities’) using the founding documents of the business – business strategy, business policy and procedures, mission statements, etc. These documents define the core value proposition of the business. The concept model should reconcile 100% to these business documents so that there is no disagreement between them before proceeding any further.

Expand and confirm the model through analysis of existing systems, practice and procedures, and by direct reference to subject matter experts (SME’s). However, be careful that these sources do not describe outdated concepts that do not align with the current aspirations of the enterprise.  The inertia in legacy systems, and in manually supported practice, procedures, and expertise creates an environment that protects legacy concepts and slows the organization. The founding documents as described above are, by definition, forward looking and should not be contaminated by legacy concepts.

Work the set of business entities and their relationships until you have addressed the full scope of the target problem domain.

The most useful tool for this process is the whiteboard, followed by an Entity-Relationship diagramming tool to capture the results of the analysis.

Note that we have referred to an ERD, not a data model. At this stage we are interested in the existence of concept level or primary entities, and the relationships between them. We are not interested in their attributes or in entities derived by normalization. At best, note attributes if they help to clarify the meaning and intent of the concepts.

Examples: Insurance policy, risk, claim; hospital patient episode, clinical pathway; airline frequent flyer, ticket; superannuation fund member entitlement, fee; government service application, consent.

The business value proposition is defined in terms of the entities in the business concept model. Value is derived by changing the state of these entities – if there is no change, then by definition no value has been created.

Furthermore, the change in entity state must be the result of a decision by this organization. The value created by a state change originating elsewhere belongs to the originator (for instance, the increase in value of an asset does not create value for the organization until a decision is made to buy or sell the asset, or revalue its book value). Creating value without making decisions is a zero sum game – if it were not, it would be the equivalent of perpetual motion and we would all be wealthy!

Events trigger each state change. The concept analysis should capture the core value changes and their triggering events as a list of candidate decision models.

Examples: Insurance underwrite and rate risks, approve claims; hospital cost patient episodes, evaluate participant locations on a clinical pathway; airline calculate frequent flyer awards, price tickets; superannuation fund calculate monthly entitlement for member, charge fees; government service approve applications, set consent conditions.

Continue development of the model until the ERD, core value changes, and major events comprise a consistent whole, and that there are no unexplained discrepancies between them (eg major events with no value change, entities that do not participate in value change, value changes without event drivers, etc).

At this point we have sufficient information to ballpark size the systems that will be required to support it, at least for standard commercial profile systems. The entity model is the basis of this sizing as follows:

·        Database: likely to expand by an order of magnitude (10 concept entities give rise to 100 database tables);

·        Functions: assume 5 externally usable functions per database table;

·        Function points: 1-2000 per entity.

Step 2 – Decision Modeling

Purpose: To specify how this organization creates value in sufficient detail for empirical testing, and to then test and prove the value assertions.

Outcomes: Demonstrably correct and perpetually maintainable decision models, schema definitions, and test cases that collectively define (define, not describe!) the business domain irrespective of systems and implementation concerns. The decision models and schemas provide:

a)        a rigorous ‘idiom’ that explicitly defines this business for clear communication between all internal parties, specifically including business executives, IT, and subject matter experts;

b)        a provably correct foundation for all subsequent systems development activity;

c)        a workbench for analysis of the impact of strategy and operational changes against the current state of the business;

d)        and with the aid of a generator, the working set of decision models that drive business value, ready to implement without fingerprints!

Duration: 1-50 hours per decision model – this time is for the relatively prescriptive process of capturing and testing a known decision making process, not for deriving the process itself from original sources, for instance, if it is represented simply by the undocumented ‘skill’ of multiple SMEs.

A decision model is a reasonably large unit of work, and should not be confused with a ‘business rule’. For instance, underwrite and rate a complex business protection insurance product for a significant business; or calculate the claim for a multi-month patient episode on an hour-by-hour basis across multiple funding contracts.

A simple model that can be built and tested in a day will have 10’s of decisions representing hundreds or even thousands of underlying atomic ‘rules’, and will generate thousands of lines of computer code. A complex decision model will be an order of magnitude larger, 100’s of decisions generating 10’s of thousands of lines of code (or more).

It is likely that the analysis of decision making will cause some ‘deep thinking’ by the SME’s and the organization itself, leading to greater understanding of the business value proposition, and often, to changes that can reach right back into the strategy itself. This activity can extend the time frames noted above.

This step focuses on building decision models for the core value propositions from Step 1. This will require development of transaction data models; that is, data models that are context specific for the proposed decision making. These data models will extend the entities in the ERD, and will be fully normalized subsets of any eventual enterprise data model. The most appropriate standard for describing transaction data models is the XML Schema standard (.xsd), which can be described as a data model for a specific context.

Develop the schemas in concert with the decision models. A decision model is a tree structured set of atomic, single valued decisions (each of which embodies many underlying rules) that takes the business entity from one valid state to another, and which creates value for the enterprise in doing so. A decision model is the specification of how the organization creates value.

The decision models both create and consume data – expand the schemas to describe this data as required.

Develop business use cases to test that each decision model fully addresses the business requirement – these test (use) cases can be manufactured for new decisioning and for testing hypothetical business boundary situations, and/or they can be acquired from existing real world systems for regression testing and impact analysis.  Ensure that the test cases are a true and complete reflection of the business problem domain.

Test the decision models against all test cases to ensure completeness, correctness and consistency of the decision models, the schemas, and the test cases collectively. When complete, we have a precise and executable model of the business that is fully aligned with the strategic intent of the business, and which is entirely independent of any technology platform or system.

We have now proven the business value proposition, and the viability of automating it, and we have strengthened the sizing estimates. Together, these results have significantly de-risked any downstream development at a relatively small cost. Furthermore, the models produced are reusable by ALL projects that arise from the analysis, directly reducing subsequent project cost, complexity, and time.

Step 3 – Systems Planning  

Purpose: To identify suitable platforms to support the decision making of the enterprise.

Outcomes: A plan describing one or more systems to be accessed, updated and/or developed to support the decision models built in Step 2.

Duration: Days to weeks.

The concept and decision modeling has provided a proven specification of the data required to support the business’s value proposition, and of the events that will drive it. By implication, process requirements have also been defined as follows:

·        The events need to be responded to;

·        The data needs to be acquired and supplied to the decision models;

·        The decision models need to be executed;

·        The decisions made need to be responded to in a manner that is consistent with regulatory requirements, industry practice, and the internal standards of the organization.

The first step is to find the earliest corporate awareness of an event, and the location of any existing sources of the required data. In particular, consider acquiring the event and data from other party systems – if available and accessible, this can shorten process paths significantly.

If it is not available externally, work inwards from the enterprise perimeter, always seeking the earliest event awareness and lowest cost data sources.

This process should consider the full breadth of available systems and services, both internal and external, and all technologies and technical capabilities available to the organization, as it seeks to implement the lowest impact processes to support the decisioning. This process will inherently optimize processes and data reuse. It should not presume a system build – at this point we are still in business mode, seeking to reuse or purchase the required process capability if we can. Note that the core organizational IP is already captured in the decision models. Optimal processes will assist the business in lowering cost and increasing throughput, but they are unlikely to drive a fundamental change in the value proposition – they are an enabler, not the end goal (the exception relates to monopoly process providers).

When the internal and external environment has been searched for optimal event and data acquisition locations, create a development plan to integrate them into an overall solution that will support the full extent of the decision models, by acquiring, re-using, updating, and/or building system components as may be needed. This plan is a decision centric systems architecture.

If the plan requires one or more solution builds, then the organization’s technology development strategy is also consulted. This is the point at which business architecture meets technology architecture.

Step 4 Solution Development

Purpose: To use, modify, or build systems as per the plan according to current industry practice.

Outcomes: Execution locations and deployment procedures for the decision models.

Duration: 1 day to 6 months per system component (a decision model can be implemented and executing across an existing database in 1 day or less; at the other extreme, if not complete in 6 months, a project is too big and risks failure).  

At this stage traditional approaches come into play. However, the starting point for the systems development life cycle (SDLC) is now significantly advanced from the traditional SDLC starting point:

a)        The business critical data has been defined and is provable relevant. Overlaying the schema definitions describes to a high degree the enterprise data model (certainly they must not be in disagreement). Additional data should be added for relevant regulatory, best practice, and internal information requirements and no more.

b)        The important processes have been prescribed by the underlying decisioning requirements. Additional process requirements should be added as required for regulatory, best practice, and internal information requirements and no more.

c)        The optimal placement and architecture of processes has been assessed independently of any specific development project, to ensure that opportunities for process reengineering have been identified and included.

d)        The technical architecture was not limited or prescribed in any way by the approach, nor was the business modeling limited or prescribed in anyway by the technology architecture; the constraints and opportunities of the technology architecture are introduced to the solution planning at the most effective point in the evolution of the solution.

As the projects kick off, the ERD (Step 1) and Decision Models (Step 2) will provide a consistent and proven foundation for the project SDLC that is common across all of the projects, improving the likelihood of project success individually and collectively by an order of magnitude.

The decision models and attendant schema’s built in Step 2 are detailed and granular. As the projects extend the analysis, assume that they will revisit these models and that there will be changes, possibly substantial changes.  

But we must emphasize – at all times these models live in Step 2 – they are owned by the business analysis function that created them. They are used by projects, not owned by projects. They will exist for as long as the business exists, whereas both projects and systems will come and go.

Step 5 – ‘What-if’ Modeling

Purpose: To evolve the business operational decision making in pursuit of the business strategy.

Outcomes: Confirmed and/or updated strategy and decision models, with feedback to systems and operational development.

Duration: Perpetual.

When the decision models are in place within operational systems, they provide an assurance that operational decision making accurately reflects the business strategy, so that we can use the decision models to experiment with the strategy. For instance:

a)        Change the strategy, and test it against existing and proposed use cases. Accept improvements and reject the rest. Deploy when ready – if new data is required then system changes will need to be made, but these are now tightly prescribed.

b)        Acquire and modify operational data to represent potential changes in operational activity or market conditions. Update the decision making to mitigate negative changes and to accentuate positive ones.

This completes the 5 Step cycle.

The Idiom Decision Manager and related tools and approaches fully support the approach described herein.

Finis

• Comments [0]

What are the best BRMS solutions in the Market ? - Linked In Business Rules Engines Interest Group

Mark Norton  28 January 2011 10:26:06 a.m.

Idioms Reply:

The key question is – why use a BRMS at all. Most rules are perfectly adequately declared in Java or C# for instance, and the internal IT department is always the major BRMS competitor.

So why a BRMS – I think the primary objective is separation of the ‘rules’ from the underlying system. But let’s be clear – we don’t mean just any rules, we mean decisions. For instance, it would be nonsensical to detach the rules that define a valid date (the 30^th February?) from any system component that processes dates. The rules we want to extract and manage are those that are intrinsically defined by the business, and not those defined by the domain at large. For the sake of clarity, we refer to these exclusively business defined rules as decisions.

Separation of these ‘decision making’ rules from the host system takes the rules out of the SDLC and allows for a new ‘decision development life cycle’ – one that can implement changes in decision logic in minutes rather than weeks or months. It also means that the business’ decisions can be transparent and auditable. In the form of discrete decision models, the decisions can be cataloged, counted, valued and traded – they become assets. And these assets can be given to business domain experts for safe-keeping, control, and adjustment like any other asset of the business.

Better still, the underlying system is dramatically simplified and agility is increased. With rules as ‘pluggable content’, the system becomes a machine of much more predictable dimensions.

If the above is an acceptable position to take, then what sort of BRMS would achieve these objectives? The following come to mind as desirable attributes of the ‘best’ BRMS:

·        If the business is to own and manage them, the BRMS must be business operable (say, as easy to operate as Excel, a tool widely used by the business) – preferably graphical without any scripting or arcane knowledge requirements;
·        For the business to be able to assure completeness, correctness, and consistency of the rules under their control, the BRMS must allow system independent and business operated testing that is sufficiently rigorous to ensure the integrity of the decision models without reference or connection to any system – after all, the decision designer may not even know which systems will be running their rules;
·        If they are to be transparent and auditable, they must produce decision documentation that is intelligible to any business reader;
·        If they are to be pluggable, the must be independent of the system AND the system technology – treat Java, C# and others equally;
·        It should allow easy many-to-many deployments between the decision models and systems, including third party systems; it is an arbitrary and unrealistic constraint to assume that systems will only have one source and type of rules, or that a set of rules will only be relevant to one system;
·        The architectural constraint on the host platform should be as light as possible – the BRMS should allow the architects to freely choose language, execution platform, and integration method of choice.

As to the best price - only Idiom publishes its price!

Thanks and regards,

Mark

• Comments [0]

The IDIOM approach to business rules and the development process by Mr Robert Love

Mark Norton  20 January 2011 05:38:09 a.m.

Mr Robert Love wrote this excellent description of the Idiom development approach while working with Idiom partner Object Oriented Ltd in 2004. Idiom is pleased to publish the article with permission. The article uses Tony Morgan's book

"Business Rules and Information Systems: Aligning IT with Business Goals" (2002) for context, commenting and building on the book's propositions.

The article remains fresh and relevant and provides useful information to any readers who are embarking on a rules development project. Please note that this article was not sponsored by Idiom, nor was Idiom involved in its authoring or subsequent review. The ideas that Mr Love presents are his own, and in some cases the terminology in the article is slightly different from Idiom's current terminology.

The Zachman model is also discussed in some detail in the Appendix.

Mr Love is currently in a senior technical role with UBS in Switzerland.  

• Comments [0]

Carole-Ann Matignon and James Taylor discuss complexity - Idiom responds

Mark Norton  13 November 2010 11:03:41 a.m.

This post is in response to a thread running between Carole-Ann Matignon and James Taylor's respective blogs. These are two of the most authoritative commentators in the industry. You can access their blogs from our external links section on this blog.

Carole-Ann opened with this post.

To which James Taylor responded with this post.

Carole-Ann returned with this post.

Idiom's response:

Carole-Ann and James, I think you are both exactly right. Innovation is required to achieve simplicity, but how this innovation plays out in terms of product or approach or even the actions of market participants and their clients is uncertain.

Idiom sees the disabling effect of complexity on our customers on a daily basis. This is not just a tool issue - James has referred to our product as one of the most business centric rules products on the market.

My view is that much of the complexity is only perceived complexity. By this I mean that at the end of the decision modeling process, what was perceived to be a problem of great complexity is now seen as being far simpler. So the problem itself in such cases is not complex, only the pre-analysis perception of it.

We think that the decisioning approach is crucial to distilling the simpler essence from apparently complex problems. This approach is dependent on having a clear definition of a decision and then decomposing the problem into a series of atomic level decisions within a decision model. This is analogous to data modeling. Note that the decision (comparable to an attribute in a data model) and the decision model are both essential ingredients in this approach. This is the antithesis of the historical BR approach which has a very imprecise definition of a business rule (hopefully this is not up for debate) and a model that is usually opaque and often dynamically inferred at runtime.

Many of us are trying to solve the complexity problem. I am convinced that natural language is NOT the way – there is nothing simple about natural language unless you are a local native speaker who understands the ‘idiom’ – the idiom is all important and never simple for any other than a native speaker – the Navaho whisperers being an extreme example. In other words, introducing a new business rules paradigm to current generation SMEs is going to be complex no matter how we cut it, simply because it is not how they currently think – it is not their current ‘idiom’ by definition.

So one solution is to mimic how they currently think if we are going to make it less complex for them. For this reason we have found the approach outlined in http://bit.ly/c6YfcI is useful – asking recursive questions like ‘what decisions do you need to make in order to achieve xxx?’ This reflects how SMEs think, and they do find it relatively simple to respond. The drawback is that it usually requires a moderator – this is not easily ‘self- actuated’.

We are finding that this approach is helpful even for the SMEs because it helps them to organize their own thinking and approach. Just last week we worked with the author of a complex clinical pathway trying to codify their own textbook on the subject. By going through the above mentioned approach we collectively evolved the pathway to a new level of simplicity and clarity in 3 days, helping the expert to simplify, organize, and codify their own domain knowledge.

This is not ‘problem solved’ though – it is still a mentored process, scalable only by the number of mentors. The tool is not the critical factor – the approach and the mentor is.

Notwithstanding all of the above, I do think that we (being the business rules community in general) have simplified things by an order of magnitude for an entirely different audience – if they let us!

The audience is the IT development community. The simplification is achieved by removing decisioning entirely from the SDLC – treat decisioning as content, not as application ‘plumbing’. Frankly, even without any BRMS tool at all, this can still deliver an order of magnitude improvement in development performance. Remove all decisioning into stand alone, discrete components that are owned and managed by the business – period! This improves application design and reduces application complexity by a large margin, and as T Capers Jones [http://en.wikipedia.org/wiki/Capers_Jones ] has comprehensively demonstrated, reducing size and complexity generates a corresponding exponential reduction on development cost, risk, and time.

We are not there yet, and as you observe, there is still plenty of room for innovation to tame the ‘beast of complexity’. But where will this innovation appear – products, approaches, and/or market practice?

• Comments [0]

Demo FLEX3 application

Mark Norton  29 September 2010 03:15:05 p.m.

Attached is a simple Flex3 Application which has been constructed in order to show how a Flex based client application is able to make calls to a server component which in turn makes use of Idiom Rules.

On the Flex side GraniteDS is used so that Flex components can call java based service objects. GraniteDS with Spring was used rather than POJO's as Spring would provide for more robustness should the application need to be expanded upon in the future. Although it is not at all necessary for the Flex to Idiom demonstration, the services layer has also been exposed as a RESTful webservice. This provides an example of how additional integration points could be used in the future as well as being useful for testing.

1)  Please double click on the zip file to download.  

2) Open flex builder

3) File -> Import -> Flex Builder

4) Choose the archive file

5) Hit the finish button and the project will then be imported

Useful Links:

http://www.graniteds.org/
http://www.adobe.com/products/flex/

• Comments [0]

Using the Idiom Decision Centric Development Approach™ to build a brand new system with outstanding business agility in 80 days (Part 5)

Mark Norton  14 September 2010 10:53:22 a.m.

This post is the fifth and last in a series. We are now at day 100 of our 80 day challenge. It has been just over a month since our last post. The system has now been released to UAT, and while some adjustment and corrections are inevitable, the system is more or less complete. We have now exceeded the 80 working days elapsed target that we set ourselves. However, in terms of total development effort, our Part 1 post anticipated that the project would require 4 people for 4 months (say, 4*4*20 days to give 320 resource days). Total development effort to date is slightly over 300 days, so in terms of resource effort we are under budget at this time. By the time we go through UAT and into production, we are likely to slightly exceed this resource budget.

Within this budget, we have assimilated significant scope change and expanded the customer base.

The system is significantly more complex and comprehensive than was first envisaged by anybody involved, including the vendor and the angel customer. Fortunately, a scenario of increasing complexity is easily managed in an agile development environment.

And because the the system was in a semi-operational state very early in its life cycle, it was able to be presented to multiple prospective customers. As a result, a second customer is now also funding development activity, and is in negotiation to fund a significant 'next phase' extension of the system to cater for large scale industry funds.

You can read the earlier blog posts here: Part 1  Part 2  Part 3  Part 4

To recap, our goal was to use the Idiom Decision Centric Development Approach™ to build an industrial strength platform for investment and superannuation funds management that incorporates a substantial decisioning component as 'business content' within the platform. This decisioning content is the basis of outstanding business agility. It also allows decision models to be shared across many application instances on the one hand, and allowing bespoke customer decisioning on the other.  

Data Architecture

As noted in the previous posts, we first scoped the study area, and built a conceptual data model. This model was expanded to form the basis of the Plex generated database. It also provided a conceptual framework to guide the development of the decisioning schemas being developed by the Decision Designer. So the Plex and Idiom components were built up independently from a common conceptual framework.

The Plex defined database is now complete at 93 database tables, which compares with our original (Part 1) guesstimate of 100 tables.

Decision Models

A generic component has been implemented within the Plex application to dynamically integrate decision models. The Plex application simply passes a context key(s) and the name of the required decision model to the generic interface, and the new component does the rest. From a Plex perspective, adding a new decision model invocation to the application requires only that the Plex developer implements this simple call.  

From the decision model perspective, the decision designer has a little more work to do. Besides actually building the decision model, they need to map their decision schema to the database using the Idiom Mapper. Then they need to publish the mapping and the decision model into the new Plex component for the decision model to become operational. The actual decision model itself can be replaced at will (between calls if required).

Following the execution of the decisioning process, the component notifies the Plex application that made the call that the decision model has completed. All decision updates are posted directly into the database for subsequent use by the Plex application. This approach can be used synchronously or asynchronously - note that some decision models may run for an extended period, for instance, hours if they are processing all members of a large fund; others may process a single entity, in which case the execution time will be reduced to millisecond timeframes.

The final set of decision models includes:

Capital Contributions
Calculate taxes and over cap warnings.
Asset Trading
Calculate the realised gains per certificate and summarise data per investment and pool  
Pool Pricing
Revalue all investments (calculating unrealised gains ‐ pre and post 12 months) plus tax calculations
Calculate interest accruals income bearing investments fees
Produce summary records for pricing events, pool price history, and including data from the pool cash file
Calculate fees
Benefit Payments
Calculate benefit components and tax amounts
Produce summary payment details
Warnings of insufficient or illegal components.
Pension Set Up
Calculate min / max amounts, relevant numbers, and life expectancy

Our experience has been that the logic defined in the decision models that were built in the early stages of the project is very stable, with most subsequent changes being driven by schema (mostly terminology and a lessor amount of structural) changes. Only a small number of decision design changes have occurred because of a more advanced understanding of the problem domain. This tends to emphasise the value of decision modelling as an early stage project activity - the requirements as captured by decision models are quite stable within the overall set of project requirements because they do represent the 'essence' of the business - they are not inherently mutable. Understanding the decision model at this early stage also clearly assisted with formulating the later stage data and functional requirements, while at the same time segregating out considerable complexity - this simplified the overall development, with the effect of reducing cost, risk, and time at an exponential rate (according to Capers-Jones).

The above models have generated >150,000 SLOC of C#.

Plex Models
The Plex development has produced 103 user interface panels, mostly combined maintenance grids with sub-grids managing multi-valued details.

Total SLOC is >1.89M lines of code, including generated C++ client side presentation and C# back-end programs.  

Analysis and Conclusion

The project team ended up being 3 and a half  people rather than 4, including 1 Plex developer, an SME (also acting as Decision Designer), a process/systems analyst, and some part-time resourcing from Idiom decision consultants to give a total project resource cost of just over 300 person/days, including inception and startup. The total is expected to be less than 350 resource days by the time the system in production. This team was also split geographically across 2 countries and 1000 miles of the Tasman Sea.

The team has built a complete business system, including CRM, unit pricing, fund and superannuation account management, asset trading modules, general ledger, document management., and statutory reporting. The current function point count is >20,000fps.

In terms of development productivity, the project has produced one complex (multi-grid) user transaction per Plex developer day. Taking the whole resourcing cost into account, the project has demonstrated a sustained productivity ~10 function points per resource hour for the full development life-cycle.

The development agility made possible by model driven development has accommodated significant adjustment to the scope and complexity of the system as originally envisaged. Note that no formal requirements phase was undertaken - the detailed requirements we developed 'just-in-time' according to an over-arching set of concepts that were agreed in the inception stages of the project.  

Business agility has been retained through the decision models. These decision models implement customer specific requirements (for instance fees and pricing), regulatory requirements (for instance contribution cap and tax calculations), and industry best practice (for instance, benefit calculations, reporting, realised gains).

Each of these decision models can by dynamically adjusted and replaced into the system without interruption to the operational system, so that changes in customer requirements, regulation, or business practice can be 'managed as content'. Changes in system capability and user interaction remain in the domain of a traditional SDLC change, albeit with the speed and agility offered by the model driven development environment.

For more information on this project and the approaches used, please feel free to call us. I can be contacted as follows:

Mark Norton | Product Director | IDIOM Limited | Microsoft GOLD Business Partner |
Office +64 9 304 1199 | Mob +64 21 434669 | After Hrs +64 9 817 7165 |
22 Dundonald Street, Newton 1021 | PO Box 67-067, Mt Eden 1023 | New Zealand |
Email mark.norton@idiomsoftware.com | Skype Mark.Norton |

www.idiomsoftware.com | Business rules for business people

• Comments [0]

Trigonometry Operators

Marta Byrski  10 August 2010 10:34:45 a.m.

Trigonometry Operators

• Comments [0]