Migration from Siebel Analytics to OBIEE
16/12/2010
The below text outlines the self-defined process and approach why and how we need to migrate Siebel Analytics to OBIEE . There might be some gaps in the entire definition of the process as the entire view is of mine and I did not face any typical issue while doing the migration , though there are couple of known bugs flying around in OTN discussion forum and in Metalink3 (Oracle Support) . So definitely there must be something !
If there are any specific points(pros/cons) somebody could like to add I will be happy to make the alteration !!!
Introduction
Oracle Business Intelligence (BI) is a portfolio of technology and applications that provides the industry’s first integrated, end-to-end Enterprise Performance Management System, including category-leading performance management applications, MIS applications, BI applications, BI foundation and tools, and data warehousing. Oracle Business Intelligence Enterprise Edition consists of components that were formerly available from Siebel Systems as Siebel Business Analytics Platform, with a number of significant enhancements.
Background
As the “Oracle First” procurement strategy has been defined by some big companies after a close tie up with Oracle hence it force them to decide,move and built the reporting platform on newly release BI component by Oracle called Oracle Business Intelligence(OBI) .This require migration of existing reporting platform from Siebel Analytics to OBI enterprise edition .
Why to Migrate
There are ample reasons to leverage the benefits of new OBIEE framework after doing migration of the existing platform. Some of the benefits have been described below:
User Interface Integration
1) OBIEE is an emerging, popular, enhanced BI system to provide the intelligent interface for both front and back office reporting
2) Oracle BI Enterprise Edition is improved to support the business and technology requirements of application integration, not only from a user and data perspective, but also in the light of business process integration: moving from reactive looking-back-to-what-happened reporting to a pro-active, guiding, alerting intelligent business requirement
3) Technically speaking lots of new feature with the enhancement of existing functionalities have been introduced for better reporting
4) For the benefit of end use business communities lots of flexibility that can be the reason to adopt this
5) Because of the 100% web-based and standards based architecture (SOAP) of the OBIEE platform it is very easy to integrate the OBI Presentation Services with a web based application like Siebel CRM or EBS .
Security Integration
1) Besides the critical requirement of single sign on and authentication, an important and time saving characteristic of a BI Application is the integration with the base application security and visibility rules. Typically large organizations have the requirement to secure company information and transactional data based on not only the user’s role or responsibility (Sales Rep or Sales Manager) in the organization but also on his or her position (CEO vs. CFO ). Based on the user’s responsibility, he should have access to certain analytical content, i.e. dashboards, reports, alerts, subscriptions etc.; based on the user’s position; he should have access to certain areas of the database, e.g. the data belonging to division A or B, team 1 or 2
2) Oracle BI EE fully supports this role and position based security rules using the technique of row-wise initialized session variables and initialization blocks. Upon login and after authentication, queries are being executed against the database in a certain order to retrieve the security information which is stored in session variables like GROUP or POSITION. These variables are then used to control access to objects in the OBIEE repository or web catalog but also to build business model filters to restrict access to critical records. This integration becomes critical in a volatile organization where user change role and/or position frequently
Data Integration
The pre-built Business Analytics Warehouse (BAW) is one single but modular data warehouse model to support one or a combination of the source systems mentioned. The BAW is compliant with the Ralph Kimball dimensional modelling methodology and supports slowly changing dimensions, aggregate tables, hierarchy tables and many others.
The pre-built OBIEE repository and web catalog are aligned with the Business Analytics Warehouse and contain the domain specific and end user facing dashboards, reports and KPI’s , metrics, drill down paths, guided navigation, alerts, action links, traffic light alerts, etc .
Supporting Technology
Without going into detail the most important characteristics of the Oracle BI EE Platform to support successful BI Applications are:
1) The caching technology which enables a significant performance improvement for standard dashboards and reports
2) The open repository ODBC API which enables other enterprise ODBC compliant tools to access the OBIEE repository presentation layer and therefore the pre-built KPI’s, dimensional attributes, preserving the security business rules
3) The ability to join multiple different data sources and data source types in the OBIEE repository
4) The multilingual capabilities of the OBIEE platform that gives the ability to use it across wide enterprise
5) Ability to use multiple RPD hosted on a single server and distributed across multiple presentation services in parallel which offload a large subject area
Scope
Scope of this document is to orchestrate for key steps while migrating existing Siebel Analytics platform to OBI .However this is very High level plan and lots of dependency and hidden low level strategy need to gradually define. This document is not intended to define the low level features, enhancement, flexibility that has been introduced in latest OBIEE version
Step by Step Activities
There are couple of activities that need to taken care of considering the roadmap of migration from existing Siebel Analytics Platform to Oracle BI EE platform.
Environment
1) Acquiring the latest OBIEE licensed version for client from Oracle . Confirm the version to be downloaded from eDelivery metalink provided URL
2) Removing existing Siebel Analytics installation across all development and test servers and carry out the fresh installation of OBIEE
3) Do the same installation at Performance test environment to check the Single-Sign on features
4) Finally install that into production servers by Oracle Architect and configure the related key business components
RPD Changes
1) Take the latest production copy of RPD at Dev Environment and do the version up- gradation.
2) Do the sanity and consistencies check across all Business model and rectify any repository consistency error, metadata warning etc.
3) Migrate it across test environment and Single Sign-On Performance test environment.
Webcat Changes
1) Take the latest webcat copy for the respective RPD at Dev Environment and do the version up-gradation.
2) Do a sanity check and sort out all the invalid objects and there references, broken links etc after running sawmigrate utility .This need to be taken care of recursively and in reactive approach.
3) Go through all the reports, prompts and links to see everything working properly.
4) Revisit Subject Area, Dashboard, Page, Shared folder, Objects level permissions.
5) Housekeeping for invalid, redundant dashboards/reports/objects.
6) Change the ‘Help Information’ HTML paths aligning with new OBIEE installed folder structures.
7) Migrate it across test environment and Single Sign-On Performance test environment.
Environment Configurations
1) Provides a choice to deploy the Presentation Services and Presentation Services Plug-in either standalone Oracle Containers for J2EE (OC4J) or in Microsoft IIS. If J2EE will be decided to be the container then IIS need to be removed from all environments and OC4J will be act as HTTP Web server to communicate with Oracle BI Web Client .However existing IIS can act HTTP Web server but this is subject to POC to remove the whole set prior to install the OBIEE copy afresh.
2) Couple of benefits are there using J2EE as Web Container rather IIS which is out of scope of the documentation.
3) Configure the HTTP Webserver using the existing dev/test environment URL and the respective port. Configuring this into existing URL reusing the port is subject to POC and might be altered.
4) Configure the same into Production like SSO enabled environment e.g. Performance test environment.
5) Check the SSO functionality for authentication and authorisation. Also verify the same for any data and object level of security.
6) Deploy the RPD-Webcat at Production environment having basic functionality in place.
7) Any URL Port change for J2EE HTTP Server configuration need to reflect to Order Gateway URL, BI Arena URL and need communicate across users.
8) Alternatively IIS Web server can be redirected to new HTTP Server configured URL which is of no impact to anywhere. This is subject to POC again.
9) Revisit production server hardware configurations to see the capabilities.
Additional Required Configurations
1) Change the Skin file path default configuration parameters as per the new windows folder structure of OBIEE deployment.
2) Move the customized CSS, JavaScript’s, images into the Skin directory path.
3) Make the respective changes on OBIEE Analytics and Presentation Server configuration files.
4) Take the Oracle best practice approach to define the performance parameters as per the current Production hardware environment setup and Production DB Configuration parameters.
5) Configure the clustered environment setup for BI Server at production for replication.
6) Define the strategy for failover/ load balance presentation services.
Additional Optional Configurations
Below additional components is not available in default setup and can be installed and configured depending on the requirement to leverage the other business benefits and experiencing the new version features at maximum.
1) Install & Configure OBIEE Scheduler’s for Delivers .Acquire the relay service and configure the details
2) Install & Configure OBIEE Publisher and the BI Publisher desktop plug-in
3) Install & Configure OBIEE Disconnected client
4) Install & Configure OBIEE Open Intelligence ODBC interface
5) Install & Configure OBIEE Briefing book reader
6) Configure the Caching Strategy.
merging
1. If you’re sure that no changes have been made in the production RPD that are not already reflected in the development one, you can just copy across the development RPD to production and have done with it. The safer way to update the production RPD would be to three-way merge the development one into it, as this will preserve the changes made in production or at least prompt you to choose between them and any subsequent updates in development that conflict with them.
To start a three-way merge, you’ll need the following three RPD files
i. A copy of the original RPD as initially put into production (this is known as the “baseline” RPD)
ii. A copy of the current production RPD (which may now differ from the baseline RPD)
iii The development RPD that you want to merge into the current production one.
Once you’ve got all these files together, we can start on the three-way merge of the RPDs.
2. To start merging, open up the BI Administration tool using an offline copy of the current production repository.
3. From the application menu, select File > Merge… and then select the baseline RPD as the original RPD.
4. The merge process will then parse the original and current production RPDs to identify and differences between them. If this is the first time you have updated the production RPD, and you’ve not made any changes to it in production, then this process will come up blank, otherwise it’ll point out any changes between the original and current production RPDs.
5. Now you can select the development RPD that you’d like to merge into the production one. Press the Select… button next to the Modified Repository area and pick up the development RPD. It’s most likely that there will be no conflicts (there may be though if you’ve applied hotfixes to the production one), and so you’ll need to press the Stats button to see what items will be added, deleted or updated during the merge process.
6. Now that the production RPD has been updated, it’s time to consider the web catalog. Again, if you don’t let users develop reports in production, or at least you don’t let them develop in the Shared area, you can just archive and then unarchive the Shared and/or the Users folders from the development web catalog to the production one, using the Catalog Manager, to copy across the new reports. If you just want to migrate a few individual reports though, there’s a new option in the form of the Content Accelerator Wizard that you can use for this purpose.
The Content Accelerator Wizard (CAF) is a plug-in to the Catalog Manager that migrates reports between two environments, and will also create any logical column calculations that the reports depend on but that aren’t found in the target RPD. Coupled with a standard RPD three-way merge this is quite a neat way to migrate individual reports from development to production, as you don’t need to have two copies of the Catalog Manager open at one time and it takes care of any calculated columns that you need.
After installing CAF and making sure that the Catalog Manager is pointing towards a JDK 1.6 or higher, open the Catalog Manager to point to the development environment, and select the report you wish to promote (or “clone”) into production.
7. When the clone process starts, you are prompted to select the target web catalog (this need to be an online catalog, and can be either local or a remote server), and the location of the source and target BI Server repository (RPD) file, which have to be connected to offline. The RPD files are required so that the cloning process can create any logical calculations that are needed by the cloned report (if the report requires full logical columns, you’ll need to add these to the target RPD using a three-way merge).
The wizard then prompts you to select the target subject area that the cloned report will be mapped on to.
8. Next comes the mapping process. The logical columns in the source web catalog need to be mapped to their corresponding columns in the target web catalog, together with dimension levels, something that you need to do manually but that can be saved, as a CSV file, to use again in later migrations. Any new calculated columns are added automatically by CAF, but if there isn’t a corresponding target column to map regular columns to then you’ll need to bring it in via an RPD three-way merge.
9. Finally, the migration takes place, and you can choose to perform a consistency check as well as view the log of migrated objects.
10. Once the migration has happened, you can take a look in the target web catalog and see the new report, ready to run and within the /Shared/Cloned folder. The target RPD will have any new calculated logical columns added, with “Autogen” as the name prefix.
So there you have it, one method to do incremental updates of OBIEE projects. In the next instalment, we’ll take a look at version control using Subversion and TortoiseSVN.
OBIEE 10g Web Catalog “Best Practices”
i was working with an Oracle Partner last week who asked me to QA their OBIEE project. One of the areas I took a look at was their web catalog setup, and I was asked to come up with some “best practices” around web catalog design and maintenance. As my speciality is more the server and data-side of OBIEE, I asked around some of my colleagues and pulled together this list of web catalog best practices.
1. When starting on a new project for a new client, make sure you create a new web catalog.
This goes for the RPD as well. As John Minkjan pointed out a few months ago, it’s not good practice to build off of the Paint or Sample Sales demo RPD, and the same goes for the web catalog. To create a fresh, new web catalog, first create an empty directory within the $ORACLEBIDATA/web/catalog directory where the BI Presentation Server is installed, like this: Then edit the $ORACLEBIDATA/web/config/instanceconfig.xml file to point to this new directory.
Then stop and start the BI Presentation Server service (from the Services applet in Windows, or using the run-saw.sh script under Linux/Unix). When the Presentation Server sees the empty directory, it will create the necessary folder structure within it, giving you a nice, clean web catalog to work with.
You can now create reports, dashboards and so on in here, without any of the demo items from Oracle cluttering things up.
2. Create catalog groups and folders for each area of analysis within the web catalog.
In terms of the folder structure within the web catalog, we tend to leave the users folder to itself (the Presentation Server creates a folder for each user that registers in the web catalog), the system folder to itself (the Presentation Server looks after this), and concern ourself instead with the shared folder.
How you structure sub-folders under the shared folder really a question of how the system will be used. If you’ve got just a single user community for your system, you might create one folder per dashboard, and place all the requests used by that dashboard within that folder, keeping things simple for when you want to apply security. Typically though, you will have a number of departments or functions that are using OBIEE, and so you will usually want to create web catalog “groups” and “group folders” per department or function. The simplest way to create these groups is to use the Add/Edit Group function with the web-based Presentation Services Administration screen.
According to the instructions on this screen, it should also create the corresponding group folder as well, but when I tried it the folder wasn’t there. Therefore you may need to separately create the group folder using the Catalog Manager, just under /Shared, to hold that group’s dashboards and requests.
3. Create a “Corporate MI” folder for “gold standard” reports
If you have a “Corporate MI” portal where individual departments publish and maintain data, you might want to create another group folder to hold these dashboards. Then, departments can develop their own “private” dashboards and requests in their own group folders, and promote them to the Corporate MI dashboard using an approvals/change control process. These dashboards and requests can then become “gold standard” or “kite marked” authoritative sources within the organization, whilst giving departments the ability to create reports specifically for their own purpose (this blog post by Phil Wright contains some good suggestions around report governance, user acceptance and BI portals in organizations). 
4. Enable drop-down menus for dashboards within each catalog group.
Each department can then set up it’s own dashboards, requests, alerts and filters within its own shared, group folder (or indeed, create subfolders for specific areas of analysis). If departments end up creating lots of dashboards, the Presentation Server will automatically show them in a drop-down list with the group folder as the menu name once the number of visible dashboards for a user is fifteen or more. You can control this setting by adding a<DashboardBeforeMaxMenu> tag to the instanceconfig.xml file; I typically set it to 1 on real projects so that all departmental dashboards are shown in drop-down menus.
Then when users view the dashboard, all of the dashboards are clearly grouped by function.
Note that dashboards can only normally be placed in these top-level, group folders (within the _portal subfolder), and you get the choice of which group folder in which to place your dashboard when it is initially created.
These group folders can then be used to hold the requests associated with each group’s dashboards. If you like, you can store saved filters either directly in these group folders, or in a special “Filters” sub-directory under the corresponding group folder.
5. Enable permissions and security within the web catalog.
If you followed the process above, you will have web catalog groups and corresponding group folders in which you will store their dashboards. reports, alerts and filters. To secure access to these groups, ensure that their names correspond to your groups in the RPD or your LDAP server, and then the presentation server will assign them to the corresponding web catalog groups when the user is authenticated using the BI Server. If you’ve organized your web catalog content into these group folders, applying security should be reasonably straightforward as you’ll assign the same permissions to all objects within each group folder. Use the Catalog Manager to set permissions, where you can select “Apply Recursively” to set these permissions on all objects within the group folder.
Make sure you only ever assign permissions against groups, rather than users, to keep things managable. Even so, setting up web catalog security can get quite complicated, especially once you get down to individual dashboard pages and the like. This posting by Kurt Wolff goes through some of the options and suggests some approaches to make it work. 6. Set subject area permissions using the BI Administration tool, not the Presentation Services Administration screen.
The Presentation Server Administration screen can be used to set access to subject areas, stopping users from creating reports using these subject areas or running reports that reference them.
My preference though is to set these permissions at the RPD level instead. This keeps all data access permissions in one place (row-level and subject/table/column-level security) and also makes it easy to control access through the groups in your RPD or LDAP server. When you change permissions on these RPD objects you may need to restart the Presentation Server before the restrictions pass through correctly to the Presentation Server – until you do this users who don’t now have permission may still see the subject area listed and may be able to see the table listing, but each of the tables will be empty of columns.
7. Control access to Presentation Services functions using the Presentation Services administration screen.
Subject area security excepted, the Presentation Services administration screen lets you control whether a user or group can access Answers, Delivers or much lower-level items of functionality, such as which particular views they can include in their requests.
Again, define groups in the catalog for profiles of users, and use these groups to assign privileges in the web catalog. You can also assign permissions on BI Publisher functionality using the Presentation Server administration screens. Also, as Jeff McQuigg points out, if you’re currently only providing dashboards to users and someone suggests turning Answers on for them, be aware that this might mean that your RPD might need some additional thought before you let users loose on it. 8. Back up your web catalog regularly, and use the Catalog Manager for archiving/restoring catalog entries
You can back-up your whole web catalog by zipping it up and storing it on a backup device. To restore it, unzip the backup and copy it back in its entirety. If you want to archive or backup individual requests, dashboards or group folders, use the Catalog Manager application and select the Archive / Unarchive options.
Whatever you do, don’t filesystem copy elements of the web catalog and then try and copy them back, as you may hit problems around permissions that only manifest themselves some time later. Also consider using the Content Accelerator Framework (CAF) plug-in to the Catalog Manager to copy reports, dashboards and RPD calculations from one dashboard to another, although as Kevin McGinley points out, this doesn’t make CAF an environment migration tool. 9. Implement the Usage Tracking subject area and reports to monitor web catalog usage
OBIEE 10g now ships with a set of scripts, a predefined RPD and web catalog that you can import into your project to provide statistics on dashboard and request usage. This Oracle by Example article sets out how to install Usage Tracking and once you’ve done so, you can get a good handle on what requests are used most and which users might be having problems. Combine the usage tracking data with ibots and time-series functions, and you can start to generate alerts when key reports are taking longer to run that normal, letting you fix problems with performance before users can report it to you. 10. Implement an “impact analysis” system for your web catalog entries
One thing that’s lacking “out of the box” with OBIEE 10g is any way of measuring the impact, on web catalog entries, from changes to the RPD. You can export a list of reports, subject areas, columns and tables from the Catalog Manager, which gives you a central list of what RPD items are used by what reports.
You can then view this list on screen, or output it to a file.
From this list you can then at least see what reports are going to be affected by an RPD change. Within Rittman Mead we have developed an ApEx application that takes this file as an input, parses the RPD and allows us to do an automatic impact analysis; when we get a moment we’ll post details on here together with details on how you can use it on your projects.
So, there’s my “starter for 10″ in terms of web catalog best practices. Do you have any others that you can share? Do you disagree with any that we have come up with, or have a better way to achieve their aim? Have we missed anything off? Just add a comment and we’ll incorporate it into the posting.
