I am back with a new BI piece and this time its on a big data environment. This is not a tool but is amalgamation of multiple technologies to provide seamless BI experience in real time and to take business decisions quickly.
Many companies today is looking for alternatives to store huge data and gather insights quickly rather than waiting for regular BI jobs to get completed and then see the reports. Also, moving entirely to a completely different stack is also not possible as the business users got already accustomed and comfortable with the existing reporting tool. Also, what about the huge investments company made on the Oracle products? ?
To answer these, I have created a demo that reflects how we can leverage the existing technology and combine it with big data products at the backend to provide seamless experience to the end users. Also, the demo will show how we can process the data in real time and visualize it live in obiee.
The entire data flow works as follow:
The data is fed from multiple sources systems/applications to kafka topics which passes it to the apache spark. Now apache spark processes the data and open two streams for data population.
First stream pushes the data to the data warehouse, in our case its kudu db maintained on hadoop cluster. Once the data is received in kudu, we can further transform/aggregates it or use predictive analytics, which will finally get visualized in obiee through rpd using apache impala as the odbc connector.
The second stream pushes back the data from spark to kafka in a different topic. Now this evenly timed data is sent to node server which is responsible of sending this data to each client window with the help of socketio. Finally, the visualization on the data happens with the help of highcharts.
Some OBIEE sample reports :-
more info @
https://www.linkedin.com/pulse/streaming-analytics-obiee-parikshit-agarwal/
