Tag: End to End BI

Posted on by — Leave a comment

End To End BI 1

End to End BI Recommendations

This is the first in a series of short articles elaborating end to end BI philosophies and methods.

In the end to end BI philosophy (referred to as Cornerstone Solution® by BI System Builders) the BI component of the data warehouse is considered to consist of four layers.  The data warehouse is not regarded as simply a set of physical tables and views on a RDBMS.

The four layers are the Staging Layer, the Data Mart Layer, the Semantic Layer, and the Presentation Layer. Collectively these can be thought of as a Business Intelligence Data Warehouse (BIDW). This is an important principle because this way of thinking engenders the beginning of an end to end BI mindset.

Achieving a successful BIDW commences with modelling the data mart layer according to business user requirements. Committing to undertake this activity first becomes a catalyst to all other necessary requirements to create an end to end BI solution:

  • The Business Architecture is considered (‘as is’ and ‘to be’) and the Business Event Analysis and Modelling (BEAM) methodology is used to understand the business requirements. This will result in the development of a set of logical dimensional models that will later directly translate to physical data warehouse tables. These physical models form the ETL target tables in the data mart layer.
  • Data profiling of source data should now be carried out and fields mapped from source systems/ source files to staging tables. This activity will enable the commencement of ETL job development to populate the staging tables. Note that a source for BI may be among others a flat file, third party data, a legacy system, an ODS, or a 3NF data warehouse.
  • Fields are now mapped through from staging tables to data mart tables enabling further ETL jobs to be developed.
  • The semantic layer is configured as an abstraction of the data mart tables for business use and querying purposes. It’s a good practice to start this early as it often illuminates any flaws in the dimensional design, thus de-risking the programme.
  • BI reports can be developed in the presentation layer for information consumer purposes.

The next article will be a short introduction to sizing, hardware and BI Platform for the BIDW.

Posted on by — Leave a comment

Death Of The Cube – Long Live The Cube!

OLAP Cubes

OLAP Cubes

The acquisition of BusinessObjects by SAP paved the way for a very welcome tighter integration between the two softwares. One of the challenges coming out of that tighter integration was the performance of Web Intelligence against an OLAP universe generated on SAP cubes and BEx Queries. The reality of SAP project implementations was that SAP Netweaver experts designed large cubes and large queries. And why not; after all this was the OLAP world?! Large SAP cubes and large BEx Queries make sense for OLAP.

However, Web Intelligence is not an OLAP tool, it builds a cache of data referred to as a ‘microcube’. Note the word ‘microcube’. Attempting to pass large volumes of data from an OLAP query to the microcube could cause the Web Intelligence engine to perform poorly or crash. BISB have observed this on numerous occasions when undergoing performance testing at client site. Problems with the version of Explorer dependent up on the Web Intelligence engine have also been observed for the same reason.

But failing to process large volumes of data was not a weakness of Web Intelligence. On the contrary, Web Intelligence was designed for smaller, fast, ad hoc queries. Users experiencing problems with large volumes of data and Web Intelligence could consider the use of Crystal Reports. Crystal Reports uses a different cache infrastructure to Web Intelligence.

The above mentioned data volume issues have made the SAP BI 4.0 road map very welcome. Using the new Data Federator connectivity through the SAP BusinessObjects BI 4.0 universe means that the SAP MDX engine (OLAP) is bypassed. This removes one of the big issues of the SAP OLAP data volumes, namely MDX crossjoins. Other development means that the BI 4.0 universe now has connectivity to SAP HANA. If you have the budget available this makes SAP HANA highly desirable for Big Data and Analytics.

Finally, ardent OLAP users that cannot live without a cube have not been left out in the cold. BI 4.0 ushered in the end of the Voyager OLAP tool, replacing it with the new Advanced Analysis for OLAP tool.

The view expressed in this article is from BISB and not necessarily SAP. Russell Beech was Senior Analyst in the BusinessObjects Analytic Applications Division for almost six years. Check out Web Intelligence In Under Three Minutes here.

Posted on by — Leave a comment

Business Intelligence Architecture

Creating BI Architecture That Stands Out

In essence Business Intelligence is about taking raw data and turning it into information and then using that information to do intelligent business. The process of both transforming the data and consuming it as intelligence occurs in the BI system. The effectiveness of the BI system is dependent upon the quality of the overall BI architecture. Planning the BI system architecture occurs very early on and there is an art to getting it right. BI System Builders recognise that it takes wide and in-depth knowledge and experience to effectively make consideration of the full BI system in the early design stage of the Business Intelligence architecture. At this very early stage thinking may be embryonic and the components do not yet physically co-exist in the system. The skill set to visualise at this level can be sparse on the ground but the failure to do so can lead to consequent sub-optimal BI systems being developed. At their worse these BI systems run the risk of becoming expensive white elephants. They have taken a lot of effort and budget to implement but in the final analysis the end user community cannot access the critical information that it requires.

So what are the key areas at the macro level that constitute the Business Intelligence architecture and hold interdependencies?

Continue reading Business Intelligence Architecture