Install and Configure Oracle RTI-BI (Real-Time Integration Business Insight)

Introduction:

Oracle Real-Time Integration Business Insight or Oracle Insight is designed for business users to model, collect and monitor business level metrics. It comes fully integrated with Oracle SOA Suite and Oracle Service Bus.

Insight helps the business analysts to control the content and format of metrics they need to know by simply identifying the key points within their business integrations. They will have access to detailed information and data in real-time quickly in hand without any deployments.

Installation:

In current release, Oracle Insight works on Oracle SOA Suite and Oracle Service Bus (may be it will be expanded to other products like BPM, MFT etc later). So, we need to have SOA / OSB 12.2.1 installed for Insight. Oracle Insight can be downloaded from here: http://www.oracle.com/technetwork/middleware/insight/downloads/index.html

The installed is a zip file with 3 patches:

  1. 22189824 – Contains Oracle Real-Time Integration Business Insight 12.2.1.0.0 product files.
  2. 22655174 – Contains updates to SOA and BAM 12.2.1.0.0.
  3. 22659236 – Contains updates to Service Bus 12.2.1.0.0.

Although the readme claims that they are marked as zero-downtime patches (i.e., they can be applied without stopping the server), the patch readme mentions to stop the server and then apply the patches (it could be a readme documentation bug, I am not sure J. Will try in my next installation).

The patches are very straight forward and simple to apply. They have an order to apply though:

If you are installing Oracle Insight in a SOA-only Home, install the patches in this order:

1. 22189824

2. 22655174

If you are installing Oracle Insight in a Service Bus-only Home, install the patches in this order:

1. 22189824

2. 22659236

If you are installing Oracle Real-Time Integration Business Insight in a SOA & Service Bus Home, install the patches in this order:

1. 22189824

2. 22655174

3. 22659236

To apply the patch, it is same way like any other patch using OPatch command. Source the environment variables and then use “opatch apply” command:

set oracle_home=C:\Installs\Oracle\FMW12c\12.2.1

set java_home=C:\Installs\Java\jdk1.8.0_51

set path=%oracle_home%\opatch;%path%

Note: We might encounter the below error while patching:

[Mar 23, 2016 9:37:49 AM] Prerequisite check “CheckActiveFilesAndExecutables” failed.

The details are:

Following files are active :

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\tracking-fabric.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\fabric-runtime.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\tracking-core.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\tracking-core.jar

C:\Installs\Oracle\FMW12c\12.2.1\soa\soa\modules\oracle.soa.fabric_11.1.1\tracking-core.jar

[Mar 23, 2016 9:37:49 AM] OUI-67073:UtilSession failed: Prerequisite check “CheckActiveFilesAndExecutables” failed.

[Mar 23, 2016 9:37:49 AM] Finishing UtilSession at Wed Mar 23 09:37:49 IST 2016

[Mar 23, 2016 9:37:49 AM] Log file location: C:\Installs\Oracle\FMW12c\12.2.1\cfgtoollogs\opatch\22655174_Mar_23_2016_09_37_47\apply2016-03-23_09-37-35AM_1.log

This is OPatch error. Searched but didn’t find any answer to this. Most of them are asking to restart the servers. On my Windows machine, I have killed conhost.exe and closed JDeveloper, sihost.exe and taskhostw.exe and again ran opatch apply and it worked. Not sure exactly why sihost and taskhostw will lock the above files. Please note that I am on Windows 10 and once we kill taskhostw, Windows will throw an error and force us to logoff. I moved that dialog box to a side and continued to apply the patch. The patches were successful. But ultimately, I had to restart my windows.

Post Patch:

After the patch installation, we need to expand our existing domain (or create a new domain if you need a fresh one) to include Insight components and SOA agents. Run config.cmd from %Oracle_HOME%\oracle_common\common\bin directory to launch Configuration Wizard.

In the below screenshot, I am updating/expanding my existing domain – e2_domain.

We need to select the below components from the available templates for Insight:

  • Insight SOA Agent 12.2.1 [soa]
  • Insight Service Bus Agent 12.2.1 [osb]
  • Insight 12.2.1 [soa]

Choose Next to continue:

There is NO separate schema needed for Insight. It will use SOAINFRA schema and BEAM tables for its metadata. So, continue clicking on Next until the end.

Click Next. No need to select anything here. I clicked on deployments to show the Insight application and its target:

As we see in the below screen, our domain will be expanded with the below applications and agents:

In the summary, we can see what is being updated in the domain:

Click on Update to update the domain.

Click on Next and Finish and start the servers.

Once the servers are started, we can see that the deployments are active and ready:

TO launch the Insight UI, use the below URL:

http://hostname:port/insight

Insight Console looks like below:

We will explore more on the Insight in my next post. Happy Patching J

BAM 12c Series – Problems and Solutions – Custom Styles

BAM 12c has got lots of flexibility in terms of usability. In BAM 11g, we had only couple of color themes to show our reports. We didn’t had any scope to add or modify these themes. In BAM 12c, we now have got a feature to add additional themes (color combinations) to represent our charts in the reports. However, we cannot delete the pre-defined or existing themes. We can only add or modified the ones which are added.

Please note: I will use styles and themes interchangeably. For me, both are one and the same.

For example, this is my custom dashboards with some reports(charts) shown:

If I click on Display Properties, I can modify the properties of the chart, including the Themes(colors):

As we see here, there are few predefined styles. These styles/themes might be liked by some users and might not be liked by others.

There should be definitely a way to change these styles. Until “Bundle Patch 1” (19707784), there was no way to add/modify the styles. With Patch Bundle 1, Oracle has provided a way to add new styles.

Note: Please apply “Bundle Patch 3” (20900599) for better results and performance improvements.

To add new styles/themes, goto Administrator console and open the data object – /oracle/bam/internal/ColorThemes. Click on Data tab and we will need to add our styles in this data object. After saving the data object, the styles entered in this data object will be picked up by the designer and the user can apply those to his graphs.

Click on Add Row to add the colors. There are 15 Color columns and we are free to enter as many as we want. In the below example, I have entered 5 colors (in Hexa format):

Now, we can use the style – “Pavan_Theme1” in our charts:

One more point to keep in mind is that the styles that are applied here are different than in 11g. In 11g, if a particular theme is selected, each bar in the chart, will have a different color. But here, each measure will have a different color. So, in the above example, since count(orders) is a measure, and thats the reason all the bars in the graph in one color. Similarly, in the below example, Sum(totalprice) is one measure and count(*) is another measure, so there are two colors shown. These colors are from the style that we created above. The green color is the value of “Color1” column (#5B0069) and red color is the value of “Color2” column (#E60053):

Now, keep enjoying the various color combinations in your reports J

Ready reckoner for SSL commands and debugging

I have been doing some R&D on SSL and FTPs and I countered this URL. Found this to be useful. Copying the useful scripts and commands here. You can find more conversation here, in this DL.

1. Command to test which certificates are there for a particular website:

$ echo -n | openssl s_client -connect host:port

 

2. A very good bash script to test cipher suites. It gets a list of supported cipher suites from OpenSSL and tries to connect using each one. If the handshake is successful, it prints YES. If the handshake isn’t successful, it prints NO, followed by the OpenSSL error text.

#!/usr/bin/env bash

# OpenSSL requires the port number.

SERVER=192.168.1.11:443

DELAY=1

ciphers=$(openssl ciphers ‘ALL:eNULL’ | sed -e ‘s/:/ /g’)

 

echo Obtaining cipher list from $(openssl version).

 

for cipher in ${ciphers[@]}

do

echo -n Testing $cipher…

result=$(echo -n | openssl s_client -cipher “$cipher” -connect $SERVER 2>&1)

if [[ “$result” =~ “:error:” ]] ; then

error=$(echo -n $result | cut -d’:’ -f6)

echo NO \($error\)

else

if [[ “$result” =~ “Cipher is ${cipher}” || “$result” =~ “Cipher :” ]] ; then

echo YES

else

echo UNKNOWN RESPONSE

echo $result

fi

fi

sleep $DELAY

done

 

3. We can use the online tool on SSL Labs’ website to query the Public SSL Server Database:

URL: https://www.ssllabs.com/ssldb/index.html

 

If I find more, will keep updating here J

BAM 12c Series – Problems and Solutions – Adapter Connection problem

As mentioned in Part 1, BAM 12c now has Process analytics. We DONOT need BAM adapter separately for sending the data to BAM data objects. Now, we still that there is a BAM 11g adapter in the Technology adapters. This should NOT be used for sending the information to BAM objects. This is ONLY for backward compatibility i.e., using SOA 12c, BAM 11g can co-exist. So, even if we upgrade SOA composites but still want to use BAM 11g, we can use this adapter to send the information to BAM (there are couple of other settings that needs to be done, we will see this in the coming parts of this series).

Now, if anyone is trying to use this adapter, the first thing it will ask is for a BAM connection. When we try to create a connection, it ends up with the following error:

Although it says that the JNDI connection failed, the connection will be created and we will be able to view the folders, but NOT objects (obviously because JNDI connection failed). But if we goto Application Resources (Windows menu > Applications), we will be able to see the Data objects as well:

Same is the case when we try to view the data objects while creating the sensor (as it would also need BAM server connection).

Finally, this ended up in bug and a patch has been released for this – 20559178. Close the JDeveloper and shut down all the servers and apply the above patch.

SOA Bundle Patch 1 (BP1) – 19707784 – is the pre-requisite for this patch. So, apply BP1 first and then this patch.

BAM 12c Series – Part 5 – BAM Business Queries

Business query is the actual query that we want to run against the data (i.e, again the data object columns). We don’t have an option to write or modify a query, but we can choose the columns and the query will be written for us. There are 4 types of queries that BAM 12c offers:

  1. Continuous Query
  2. Group SQL Query
  3. Tree model Query
  4. Flat SQL Query

  1. Continuous Query: A Continuous Query is an analysis of data collected in real time. It is based on templates and data objects. Each template has its own settings. Based on the template, the settings will change.

    This query is mostly used for generating reports on streaming data i.e, data coming from sensors, Hadoop, OEP, etc. Following picture shows various templates available. If one is aware of OEP, they can see that these are familiar templates.

  2. Group SQL Query: A Group SQL Query query is an analysis of one or more numeric data fields, called measures, grouped by non-numeric data fields called dimensions. This query can be used whenever we want to show graphs like bar/pie with group by fields and their values. For example, number of items sold group by country.

    This query is also based on data object. X-axis will take the dimensions and Y-Axis will take the measures.

  3. Tree Model Query: A Tree model Query is an analysis of one or more measures grouped by a hierarchy of dimensions. For example, if we want to display reports for sales grouped by country, state and city per year, this type of query can be used.

As we see here, the data object can be selected, but hierarchy is empty. That is because we have to define these hierarchies in data object. So, open data object in Administrator and click on Hierarchies and create one.

Two type of Hierarchy:

Once the Hierarchy is cerated in Data object, we can select that in Treemodel Query:

  1. Flat SQL Query: is a simple table of data fields and their values. You select the data object and then select the data fields.

Each Query type has its own significance. Please see here for details: https://docs.oracle.com/middleware/1213/bam/BAMUG/bam-query-creation.htm#BAMUG98223

These queries are linked to Business Views and not all queries can be executed with all the views. They have mappings.

Tip: If you don’t know which query to use, use Top to Bottom approach i.e., create a business view and in that we get an option to create a business query. Based on the graph that we choose, the appropriate query will be chosen for us.

BAM 12c Series – Part 4 – BAM Consoles – Designer tab

BAM 12c Designer tab has got a complete refresh.

If both BPM/SOA and BAM are deployed in the same domain, then by default, we get a project called as “Process Analytics”. If they are deployed in different domains, this project does NOT come and we have to do few additional steps to open this project.

The first change in BAM 12c we will see is Folders are converted into Projects. So, we have to first create a project and create our objects inside that project.

While creating the project, we will be given an option to create a project with process analytics or without process analytics.

We then have to create various objects first in order to create a report. A report is based on a business view, which is in turn based on a business query, which is inturn based on a data object.

So, for creating a report in a dashboard, we can follow one of the 2 approaches:

  1. Top to Bottom: Create a dashboard > create a business view > create a business query > choose the data object and filters.
  2. Bottom to Top: Create a data object > Create a business Query > Create a business view > choose the view in the report/dashboard.

We will each of these objects one-by-one:

  1. Data Object:

    As in BAM 11g, the designer cannot create a data object, but only allowed to use one, that is created by the Administrator. So, here also, we need to choose a dataobject from existing one:


  2. Business Queries:

    Business query is the actual query that we want to run against the data (i.e, again the data object columns). We don’t have an option to write or modify a query, but we can choose the columns and the query will be written for us. There are 4 types of queries that BAM 12c offers:

    1. Continuous Query
    2. Group SQL Query
    3. Tree model Query
    4. Flat SQL Query

Please see more about these queries in Part 5.

  1. Business Views:

    A view is generally a representation of the data output. So, a business view is a visual representation of data fetched by a business query or one or more KPIs. There are various visualizations through which the data can be represented in BAM 12c. All these visualizations are created in Business views.

    The underlying query determines the order in which a view displays multiple measures or dimensions. The below table helps which query to choose for which view (more info from here):

Business Query Data Object Visualization/Graph component
Flat SQL Query Any non-stream data object List, Action List, and Geo Map views only
Group SQL Query Any non-stream data object All views except List, Action List, Geo Map, Treemap, or KPI Watchlist
Tree Model Query A non-stream data object with at least one hierarchy Treemap views only
Continuous Query: Duplicate Detection Template A stream data object only Alerts only
Continuous Query: all other templates Any non-external data object Alerts only

We can see more information on Business View Categories here: https://docs.oracle.com/middleware/1213/bam/BAMUG/bam-view-creation.htm#BAMUG98243

We can select the business query in the business view and if the data is present in the data object, the chart would be rendered immediately. There is also a provision to change the chart to another visualization as shown above.

Properties – will help change the properties of the chart like colors, legends etc.

Runtime-Interaction – will make the chart user-interactive like creating drill-downs, displaying the details of the selected chart, making the chart active etc.

  1. Dashboards:

    Dashboards are a collection of business views that are displayed together as a single point of view to the end-user. While creating a dashboard, it gives us option to create a dashboard with multiple cells (regions) to add our dashboards.



    Also, we can edit the properties of dashboard from here, like changing the name of the dashboard, layout of the dashboard (like resizing the cells, add/remove cells etc). These dashboards can be run from Home tab (which was Active Viewer in 11g).

We will see the below 3 in the upcoming Parts.

  1. KPIs:
  2. Alerts:
  3. Parameters:

BAM 12c Series – Part 2 – BAM Consoles – Administrator tab

In continuation to Part 1, in this blog, we will see various consoles in BAM 12c. Unlike in 11g, we have only 1 screen here with 3 different tabs – Home, Administrator and Designer.

Administrator Tab:

Same the name suggest, this tab is mainly for the administrators and team leads – to create data objects, create enterprise message sources, and for monitoring CQL queries and Viewsets. Basically this is used to define the data structure (data model) for the reports. All the reports would be based on this data structures.

Background:

Unlike in BAM 11g, there is NO separate schema. So, all the BAM metadata is in SOAINFRA. The metadata tables are now prefixed with BEAM instead of BAM. So, where-ever we see BEAM, we can assume that they are BAM tables. The way that the data objects are created is totally different than in 11g. I will explain the details of this metadata in Part 3.

Data Objects:

The data objects have now been extended to 4 different types: Simple, Derived, Logical and External.

a. Simple Data object:

This data object is analogous to data object in BAM 11g. It is a simple data object with different columns in it. Administrator has to have an idea while creating the Data object i.e., how many columns it might have. While creating this data object, we will be given an option to specify number of columns in the data object as seen beside (please see Part 3 for more details)

  • The name can be anything and is internal name. Once given, it cannot be changed.
  • The display name also can be anything and can be changed. This is for readability. If we give the name using / then they will be create in a hierarchy. For example, if we give the name as /MyExamples/MyTest, MyTest data object will be created under MyExamples
  • The query type can be a relational or a stream. The relational query is where we have a table and we can write a query using joins to fetch data. The Stream query is continuous query where the data is read from a stream of data. We can specify the number of seconds to read the data from the stream.
  • If we select Continuous Query type as “Stream”, then we also have an option for Archive the old data.
  • We will see the below fields (Number of string columns etc) in Part 3.

b. Derived Data object:

This data object extends a simple data object or another derived data object. It inherits the columns and hierarchies of the parent data object and adds its own columns.

We can consider the following scenarios for using this object:

  1. In a scenario where the base objects columns and structure is fixed and doesn’t want to modify it and where we want to add few more columns to accommodate our requirements.
  2. In a scenario where a technical department fixes a generic structure and each department will add its own columns to make it more specific to its needs.
  3. In a scenario where Oracle might give few data objects and dashboards OOB in future and we might want to reuse this object (like currently, we have process analytics data objects – although we can modify them, it is recommended that we create a derived data object and add our columns or conditional columns or hierarchies etc., in this derived data object).

Creating a derived data object ensure base object is not disturbed and thus base object can be reused multiple times. It is more like a base object and child object in OOPs concepts.


c. Logical Data object:

As the name mentions, a logical data object is NOT physical object. It references data in other archived relation data objects. This object will be read-only and not a physical data object. However, it can do more than add columns. It can rename or hide columns, or add calculated columns. It can also join two or more simple or derived data objects.

A logical data object is read-only, is not persisted, references data in other data objects, and is most commonly used to join other data objects. We can imagine this as a View (and above two objects as tables).

d. External Data object:

External data object is analogous to external data object in BAM 11g. This object persistence lies outside of BAM schema. When a JDBC connection to the external data object is configured in Oracle WebLogic Server, Oracle BAM can access it. The external data object name may be changed from the actual table name, but the column names are unchanged. Some columns may not be selected for visibility in BAM.

But the active data will NOT work when the persistence is outside BAM 12c i.e., if we use this data object in the reports, none of the reports/graphs will be active.

Enterprise Messaging Sources:

This is again same as in BAM 11g. One of the ways to push the data into BAM data objects is through JMS. As we push the message into JMS queue/topic, we need to configure an Enterprise Messaging Source to read that data from JMS Queue and copy that to Data object. That mapping is done in EMS.

Again, the name is internal to BAM and cannot be changed once created. The Display Name is what we see here. If we use / in the name, we can see the hierarchy as show in the picture here.

Once the messaging source is created, we need to give the details like the queue name from which BAM has to pull the data, XML element names, namespaces and mapping of the data object fields to XML schema elements.

Continuous Queries Monitoring:

Continuous Query Monitoring page monitors all continuous queries across different projects in a BAM server instance. Continuous Query doesn’t mean that we have to use Streaming queries. Continuous queries can also be enabled in the report properties and make the report active. Please see here for more information.

In order to monitor these queries – to find out what queries are running, how much time they are running, in order to terminate the queries, activate the queries etc – we can use this page. It will show only when any of the queries are running:

Viewset Monitoring:

Viewset is a snapshot of data. It holds the query results that were executed previously. This will ensure that if the same query is fired again, the data is not fetched again and used for the snapshot in this viewset.

The first view opened in a dashboard for a given query creates the master viewset. Opening other views that use the same query creates slave viewsets. Viewsets minimize time-consuming query reexecution and ensure that views are incrementally updated using a push-based mechanism.

We will see other tabs in next parts.

Follow

Get every new post delivered to your Inbox.